Written by Ariel Ropek and Remy Kullberg.
Sigma rules emerged as a game-changer in the world of threat detection, offering a standardized, SIEM-agnostic framework for sharing detection logic. In a landscape where collaboration and rapid dissemination of intelligence are paramount, Sigma rules simplify the process of creating, sharing, and implementing detection rules across platforms.
This blog explains everything you wanted to know about Sigma rules, including how Sigma rules are structured, their predecessors and benefits, and how to integrate them into your Security Information and Event Management (SIEM) software. Let’s dig in!
Sigma rules are an open-source, generic standard for creating and sharing threat detection logic across different SIEMs.
They were first developed and open-sourced by Florian Roth and Thomas Patzke in 2017. Roth and Patzke were inspired by similar standards that already existed in the network detection and malware analysis spaces.
However, the Sigma rules standard needed to address the unique challenge of handling variable log schemas, as well as threat detection platforms which each have distinct rule formats. This is unlike network packets and files that have standard attributes that do not change between platforms. This standardization makes it trivial to develop and share “signatures,” unique patterns that identify malicious behavior.
So how did they do it? Each Sigma rule captures key information about the log source and detection logic using YAML, a human-readable data serialization language, and then relies on converters, called “backends,” to translate the detection into a SIEM-specific query language. This makes the Sigma rule format flexible, easy to use, and applicable to any type of log.
The creation of Sigma was originally rooted in the need for a “human markup language” for threat intelligence sharing, where rules might be recorded manually. But with the growing desire for automation, Sigma has since evolved into a machine-readable standard that supports automated deployments.
Today, Sigma Rules are a cornerstone of collaborative threat detection, with over 3000 threat detection rules backed by an extensive open-source community contributing to its growth.
Before getting any further into its history, benefits, and how to use Sigma rules, let’s look at an example of this generic detection signature format.
Take a look at the following Sigma rule that detects when a GCP firewall rule is modified or deleted using GCP Audit logs.
title: Google Cloud Firewall Modified or Deleted
id: fe513c69-734c-4d4a-8548-ac5f609be82b
status: test
description: Detects when a firewall rule is modified or deleted in Google Cloud Platform (GCP).
references:
- https://cloud.google.com/kubernetes-engine/docs/how-to/audit-logging
- https://developers.google.com/resources/api-libraries/documentation/compute/v1/java/latest/com/google/api/services/compute/Compute.Firewalls.html
author: Austin Songer @austinsonger
date: 2021-08-13
modified: 2022-10-09
tags:
- attack.defense-evasion
- attack.t1562
logsource:
product: gcp
service: gcp.audit
detection:
selection:
gcp.audit.method_name:
- v*.Compute.Firewalls.Delete
- v*.Compute.Firewalls.Patch
- v*.Compute.Firewalls.Update
- v*.Compute.Firewalls.Insert
condition: selection
falsepositives:
- Firewall rules being modified or deleted may be performed by a system administrator. Verify that the firewall configuration change was expected.
- Exceptions can be added to this rule to filter expected behavior.
level: medium
Code language: YAML (yaml)Sigma rules are written in YAML and information is contained in several key-value pairs. There are 13 top level fields, with a handful of subfields. Despite the number, these fields can be categorized into one of three rule components:
detection field, this defines the logical conditions that identify the suspicious activity. In this example, the detection logic simply lists a series of action names in selection and specifies condition: selection so that a match with any of the action names in a GCP Audit log will trigger this detection.All Sigma rules contain these three components which capture not only the threat detection, but helpful context that supports rule refinement. The Sigma standard also supports correlation rules and filters, which are also written in YAML, but have slightly different specifications.
With an idea of how Sigma rules are structured, let’s take a look at predecessors to understand how open standards in threat detection have progressed over the years.
Generic, open-source standards for threat detection began in 1995 with Zeek (formerly Bro) in the network detection space. Zeek provides a signature-based approach to detecting threats and anomalies in network traffic. Its rules leverage the uniform structure of network packets, making it easy to write and share detection logic without requiring conversion for different systems.
Snort (1998) and Suricata (2010) further advanced network detection. Snort introduced contextual metadata, such as network segments and traffic patterns, and Suricata introduced processing improvements like multi-threading.
The next evolution moved beyond network traffic into file-based detection with YARA rules. Released in 2013, YARA uses regex and bytecode analysis to identify patterns within files, such as malware or executable binaries.
With YARA rules, we saw more metadata like descriptions and severity levels, and incorporation of boolean logic for more complex alerting conditions. Today, we commonly see YARA rules used in endpoint detection and response (EDR).
Finally, Sigma rules came onto the scene in 2017. Sigma rules were originally developed for Windows Sysmon logs only. Since their introduction, they have steadily expanded to support Linux, MacOS, application logs, cloud services, and more. To this day, Sigma rules still use Sysmon field names as their common schema, and Windows rules comprise the majority of rules in the open-source repo.
In the Sigma generic standard, we saw an increase in metadata to be able to properly describe the threat detection and handle the variability in log data. As shown in the previous section, the metadata includes references, severity level, false positives, and MITRE ATT&CK tags.
Sigma rules continued the trend of using boolean logic, filters, and regex to produce complex alerting conditions. The following code block shows an example of a complex alerting condition for an Okta detection that detects when a user has entered their password into the username field, which causes the password to be retained in log files.
To alert on this scenario, the detection logic defines an event to match on in the selection field and a filter to exclude service accounts in the filter_main field. These fields are then combined in the condition field to define the exact alerting condition.
detection:
selection:
legacyeventtype: 'core.user_auth.login_failed'
filter_main:
# Okta service account names start with 0oa
# Email addresses are the default format for Okta usernames, so attempt
# to exclude alternateIds that look like valid emails
# If your Okta configuration uses different character restrictions, you
# will need to update this regular expression to reflect that or disable the rule for your environment
# Possible false negatives are failed login attempts with a password that looks like a valid email address
actor.alternateid|re: '(^0oa.*|[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,10})'
condition: selection and not filter_mainCode language: YAML (yaml)To use Sigma rules within your SIEM, you can set up a command line interface (CLI) toolchain to convert rules locally on your computer, or you can also convert rules one at a time using the Sigconvert.io website.
The CLI toolchain is made up of the sigma-cli library and one or more plugins for the conversion process.
A plugin represents the target SIEM platform that you want to convert rules to and it is made up of a backend and one or more pipelines. A backend drives the conversion process between the Sigma format and the SIEM-specific format. A processing pipeline defines how a specific log source should be converted.
For example, Panther has a backend called panther-pysigma-backend, and if you take a look at the repository, you’ll see a handful of pipelines defined for EDR and cloud logs.
Keep in mind that the extent to which you can use Sigma rules with your SIEM varies, and directly depends on the processing pipelines they support. The best way to find out what is supported is by going to your SIEM’s documentation for Sigma rule conversion.
Using the sigma-cli is simple and fast: install sigma-cli, then invoke it to install the plugin for the target format you wish to convert to; finally, convert rules following the sigma-cli usage instructions.
Here’s an example of installing and using sigma-cli to convert Sigma rules into Panther’s format:
brew install sigma-cli, or follow these installation instructions for other platforms.sigma plugin install pantherRule conversion is a fast process. The following gif shows how the Sigma ruleset for AWS CloudTrail—about 40 rules—takes less than a second to convert to Panther’s Python-based rule format.
The work isn’t done after rules are converted. Since Sigma rules are a generic rule format, they will always need to be tuned to your environment. Whether that’s updating a list of IPs or adjusting a list of actions to match on, this is a crucial step that mitigates false positives.
Once you have a threat detection use case, creating a Sigma Rule is fairly straightforward. Simply follow the Sigma specification to create a new rule, filter, or correlation rule in YAML. Developing detection logic does have a learning curve, and walking through the Sigma HQ docs is your best bet on learning the basics. For a detailed guide on developing a Sigma rule starting from a threat detection use case through to the final Sigma detection, check out this guide.
To submit your new rule or filter to the repository on GitHub, follow the Sigma contribution guidelines. You can also contribute to the open-source ruleset by reporting false positives, working on issues, or proposing new detection ideas.
The Sigma rules repository assigns rules to three different categories to track and maintain its rulebase. Every rule has a status field that indicates whether it is:
The SecOps community loves the Sigma standard for four main reasons: sharing threat intelligence, collaboration, avoiding vendor lock-in, and streamlining SIEM rule deployments.
Because Sigma Rules are widely used and highly regarded, most SIEMs support Sigma rule conversion. However, it is up to the community to develop backends and conversion pipelines, so the extent of support varies. As of this blog’s publish date, there are 28 backends, some of which are for more general query languages like Powershell and SQLite, rather than SIEM-specific query languages.
Here are some of the most common SIEMs that work with Sigma rules:
Sigma rules are backed by an ecosystem of tools that support the open-source standard and automated deployments:
Not only are there backends and pipelines, as we’ve covered, there are other projects that support the ecosystem:
For more information, visit the SigmaHQ documentation.
Eliminating vendor lock-in, community collaboration, streamlined deployments, and rapid dissemination of threat intelligence are the reasons the Sigma standard has grown so much since its inception.
And these reasons are exactly why we expect to see open standards in cybersecurity continue to gain traction. They support the interoperability and scalability that improve threat detection. This is why Panther’s own ruleset is open source, and why we’re fully compatible with Sigma rules.
See why Panther is the leading cloud-native SIEM offering code-driven workflows that produce high-signal alerts, backed by a serverless security data lake that provides 365 days of hot storage for rapid investigations. Request a demo to learn how Panther can solve your threat detection use cases.
