Most attacks on IT systems and networks manifest themselves in event logs, which are stored in many organization’s SIEM systems or other log storage solutions. This makes them a great place to detect intruders in your network. However, there are few challenges.
The first challenge is knowing and developing a list of signatures of attack techniques. This is often a time-consuming task because the attack has to be replicated in a lab environment and log records specific to the attack have to be identified in the logging background noise. Someone from a threat intelligence community might already have shared a signature, but in a format that is not compatible with the organization’s SIEM. Converting queries from a different query language and with a different field naming is an error-prone task.
Another challenge is that many organizations store logs in different repositories. This may have come about due to a merger and acquisition or may be an organization’s strategy. There might also be technical or simply licensing reasons, meaning only specific logs are ingested into a SIEM while the remaining logs go into the so-called data lake. Different logs repositories mean different query languages in which the log signature has to be written.
Sigma is an open-source project that tries to solve these challenges. It consists of three components:
- A language specification for the generic Sigma rule format.
- A repository with over one thousand rules for several attack techniques.
- A tool for converting Sigma rules into various query formats.
It’s all open-source and available in the SigmaHQ GitHub organization. It can be cloned locally with the following command line:
git clone https://github.com/SigmaHQ/pySigma.git What does a Sigma rule look like?
Let’s take a look at a Sigma rule for CVE-2020-0688 as an example (Figure 2). This was one of the recent Exchange Server vulnerabilities that were actively exploited by threat actors to gain a foothold within organizations that hosted a vulnerable Exchange instance. The detection of exploit attempts was described in different blog posts, which were the basis for this Sigma rule:
There are five components in this Sigma rule:
- Metadata, which includes a title, a unique identifier and some further information that allows an analyst to contextualize the detection rule.
- Log source definition, which ties the signature to a specific log, in this case, the webserver log.
- The detection rule itself is defined in definitions that associate log event fields with values specific to exploitation attempts.
- These detection items are linked together with a condition.
- Finally, the rule contains some additional information, such as event attributes that are interesting, including events matched by the rule, known false positives, severity level, and linking to MITRE ATT&CK tactics and techniques.
Figure 3 shows another Sigma rule for a different Exchange Server vulnerability, CVE-2021-26857.
The elements that make up this Sigma rule are the same as in the first example, however, the detection logic is completely different. This rule describes process creation events as emitted by Sysmon or many Endpoint Detection and Response (EDR) products (1). The Exchange Server’s unified messaging worker process usually doesn’t spawn processes, except sometimes Windows error reporting processes in case of a crash, which is defined in the detection rule and its condition (2).
Conversion of Sigma rules
In the next step, the Sigma rules are converted into some target query languages with the Sigma conversion tool sigmac. The tool can be installed with pip:
pip install sigmatools Or a virtual Python environment where all dependencies can be spawned with Pipenv from the cloned Sigma repository:
pipenv shell Let’s assume the resulting query should be used in an Elasticsearch-based SIEM, which uses the ECS field naming scheme. This can be simply done by running the following command from the root of a Sigma repository clone:
tools/sigmac -t es-qs -c ecs-proxy
rules/web/web_cve_2020_0688_msexchange.yml The -t parameter selects the backend of the Sigma converter, which is responsible for the conversion of the rule into a target query language. The backend es-qs converts into Elasticsearch query strings that can be pasted into Kibana. The -c parameter selects a configuration. The configuration ecs-proxy is provided with the Sigma converter and implements the ECS naming scheme for proxy logs. The resulting query will be:
(http.request.method:"GET" AND url.original.keyword:(*\/ecp\/* OR *\/owa\/*) AND url.original.keyword:*__VIEWSTATE\=*) Now let’s convert the second rule. This time we need some additional configuration to map the generic log source process_creation to a specific log source. Assume we’re in an environment where Elasticsearch is used as SIEM and Sysmon as a log source that generates events for process creations on endpoints. The conversion command is as follows:
tools/sigmac -t es-qs -c sysmon -c winlogbeat rules/windows/process_creation/sysmon_cve_2021_26857_msexchange.ymlTwo configurations were provided in the command line. The configuration sysmon maps to the specific log source, and event identifier. winlogbeat maps the field names from the Sigma rule to the ECS scheme used by Winlogbeat. The resulting query is:
((winlog.event_id:"1" AND winlog.channel:"Microsoft\-Windows\-Sysmon\/Operational") AND winlog.event_data.ParentImage.keyword:*UMWorkerProcess.exe AND (NOT (winlog.event_data.Image.keyword:(*wermgr.exe OR *WerFault.exe))))Only some command line parameters must be changed to convert the same rule for a Splunk SIEM in an environment where Windows audit logging is used:
tools/sigmac -t splunk -c windows-audit -c splunk-windows rules/windows/process_creation/sysmon_cve_2021_26857_msexchange.ymlIt results in this Splunk query:
((EventCode="4688" source="WinEventLog:Security") ParentProcessName="*UMWorkerProcess.exe" NOT ((NewProcessName="*wermgr.exe" OR NewProcessName="*WerFault.exe")))Contributing to the Sigma Project and outlook
Because the Sigma project is open-source it depends on contributions. The project is hosted on GitHub and new rules can be contributed via the web interface by navigating to the target directory and adding a new file:
The Sigma Wiki contains a rule creation guide with a template that can be used to create new rules. Don’t hesitate to contribute — the rules will be checked before the pull request is merged, and issues will be fixed or suggestions made.
Feedback is also valuable, for example, opening an issue if a detection rule causes false positives under certain conditions or small pull requests that change the state of rules from experimental to stable if it is already actively used as detection and can be confirmed as useful. Currently, most rules are still in experimental status and there is much room for improvement.
Another area where you can contribute is to the conversion tool. Currently, a complete rewrite is developed in the pySigma library. Further, a new CLI is under development but is yet to be released. In the future, backends will be maintained as separate projects and lots of the backends from the current Sigma converter need to be ported to pySigma, or some that are currently not maintained anymore, need someone who takes care of them.
Thomas Patzke is threat hunter, incident responder and open-source security tool developer.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.