How to Evaluate a Security Detection Platform

A New Approach to Modern Threat Detection

Leading cybersecurity teams are transforming their operations to meet the demands of today’s evolving threat landscape. Organizations of all shapes and sizes rely on cloud services for unmatched operational efficiency and agility. While they have many benefits, these services also come with unique security challenges. 

The sheer volume and velocity of cloud event logs makes it very difficult to separate the signals from the noise when identifying and triaging incidents. Traditional Security Information and Event Management (SIEM) tools are not designed for the scale of cloud threat detection workflows, forcing tradeoffs between cost and performance, reducing visibility, and increasing risk. Security teams need transformational tools and processes that support the development and deployment of detection automation for cloud-scale threat monitoring.  

The solution lies in a robust cloud-native security detection platform designed for the scale and complexity of the cloud, with engineering principles at its core. 

Key Detection Platform Evaluation Criteria

When selecting a detection platform, a standard set of evaluation criteria helps ensure the chosen platform will meet your objectives. Below you’ll find an overview of key criteria, with detailed functionality considerations, to streamline vendor assessments.

Detections-As-Code: 

Detections-as-code (DaC) provides similar benefits to Infrastructure as Code (IaC) principles commonly used in modern DevOps processes. DaC supports continuous integration / continuous delivery (CI/CD) processes for writing detections with a unified set of processes and tools to quickly and reliably identify security threats, at scale.

Leveraging DaC enables fast detection creation, testing, and iteration. DaC leads to higher-quality alerts that quickly flag suspicious activity while avoiding noisy false positives that fatigue your team and prevent you from responding to actual incidents. 

Despite DaC’s rising prominence, not all detection platforms support it. Platforms that do may require proprietary scripting languages to code the detections. It’s important to understand these limitations and resulting tradeoffs. 

When reviewing vendors that support DaC, consider the following functionality:

• Detection Versioning: this provides a reference point for historical changes to detections throughout their lifecycle. The ability to revert changes using a source code repository such as GitHub ensures traceability and accountability for edits to detection configurations. 
• Change Management: this facilitates peer review workflows for changes to detections in pull or merge requests. Change management features ensure that edits to detections undergo thorough peer review, fostering team collaboration and adherence to best practices.
• QA & Testing: this plays a crucial role in maintaining the integrity of detection processes. Enforcing standardized testing allows your team to more easily identify technical errors and supports repeatable, scalable methods that increase efficiency.  
• Automated Deployment: this mitigates bottlenecks and errors from manually applying the latest detection set. Automating detection deployments creates an authoritative, single source of truth that reduces the likelihood of discrepancies that arise from manual processes. 

Data Ingestion 

Event logs containing key insights on adversary behavior can come in nuanced, non standardized formats. That means it’s essential for the Detection Platform to allow your security team to define the content, format, and model of incoming data. 

The platform should offer flexible data ingestion methods, supporting the configuration of the following data items: 

• Custom Parsers: this allows your team to format data from any source to a well-defined data model or schema. With a common data model, you can streamline detections that correlate activity across disparate data sources.
• Pre-processor Support: this enables detection engineering roles to configure new filters for inbound data, so detection logic can focus on specific fields of ingested data, increasing their efficacy. 
• Sources, Types, and Custom Fields: these features allow the team to enrich ingested data with additional context, helping drive more targeted, flexible detection and investigation workflows for complex attacks. 
• Enrichment: in addition to ingesting raw logs, the ability to enrich data with additional information from threat intelligence, asset inventory and other sources provides additional context for security teams to triage events and mitigate threats quickly quickly. 

Detections

While Detections-as-code is a crucial building block, the detections themselves need to offer powerful, flexible functionality. Security analysts and engineers should have a variety of options to apply observations from ongoing threat intelligence when creating new or tuning existing detections. 

Detection functionality should be capable of identifying nuanced, granular actions across multiple systems that indicate a security incident or a breach, including:

• Real-time Detections: Analyzing streaming log data to detect real-time attack behaviors can be the difference between a minor security incident and a massive security breach. Detections should run in line with the event log ingestion process to minimize mean time to detect (MTTD) and mean time to resolve (MTTR). 
• Historical Detections: identifying and triaging threats often requires correlating current behaviors with past activities. In addition to analyzing real-time events, detections should be capable of analyzing historical logs. 
• Event Aggregation: nuanced cloud threats often involve attacker activity across multiple threat vectors and lateral moves between systems that occur over time. To identify these threats, detections should be capable of spanning multiple sources across time, log types, and other data parameters. 
• Signals vs Alerts: by stacking or building complex detections based on results from other detections and events, analysts can identify and triage more complex adversary behavior.
• Reference Lists: detections can reference static and dynamic lists.

Search

When responding to an incident, fast and flexible search capabilities are crucial. The platform’s search functionality should support broad threat hunting workflows with targeted queries across disparate data sets to get to the ground truth quickly.

The platform’s search capabilities should have the following attributes: 

• Reliable: queries should return consistent and complete results. 
• Timely: results should be returned in a reasonably timely manner, even when the system resources are taxed by other searches and connected operations.
• Query-Building Options: the platform should offer a range of options for creating queries. Many platforms use SQL or proprietary languages that will suit more advanced practitioners, but it’s also helpful to offer console-based point-and-click search functionality to allow users who don’t know SQL to drive efficient investigations. 
• Analytical Flexibility: the platform should offer a wide range of built-in analytics capabilities to rack, stack, sift, sort, and visualize the search results. 

Integrations and Extensibility

For modern security operations, detection platforms are one key piece of a larger puzzle. They need to integrate with other tools across the extended security ecosystem for a cohesive defense strategy. Integrations via full API access should be available to execute primary operations in a cohesive manner. Pre-existing integrations created by the vendor are a bonus.

While your organization may have additional tools to integrate with, the highest priority integrations for most security teams include:

• Threat Intelligence Platforms: these platforms enhance your detection and investigation workflows by surfacing adversary behaviors that otherwise might go unnoticed. Seamless integrations between detection and threat intelligence platforms allow you to cross-reference internal data with external intelligence on emerging threats and adversary tactics, techniques, and procedures (TTPs) to stay one step ahead of attacks.   
• Security Orchestration, Automation, and Response (SOAR) Platforms: when detections identify a potential security threat, they create alerts that can quickly pile up and fatigue your team. Integrating detection and SOAR platforms helps coordinate and automate responses to security alerts, reducing the manual burdens of your team’s runbook.  
• Alert Workflows via Ticketing Systems, Collaboration Tools & Custom Webhooks: integrations with modern ticketing systems such as Jira and GitHub, as well as collaboration tools such as Slack and Teams, are becoming critical to increase productivity, as well as the ability to integrate with custom systems leveraging custom webhooks to send alerts anywhere. 

Scalability and Performance

The platform should be capable of ingesting, running detections on, and searching high-volume security telemetry data from complex ecosystems spanning cloud environments, SaaS applications, on-premise services, and more.  

To ensure the platform can maintain high performance at scale, consider the following: 

• Cloud-Friendly Licensing: The platform’s licensing models should not severely limit the organization’s visibility into high-volume cloud event logs. Cost models should accommodate growing volumes of existing log sources and the introduction of new sources as they’re identified and incorporated into security monitoring workflows.  
• Detection Management: The platform should be capable of managing thousands of detections that deliver consistently performant, reliable results.  
• Retention: while many traditional security monitoring platforms limit data retention to just 30 days (with the option to extend to 90 days at a high cost), this severely limits modern threat detection use cases. A full year’s worth (365 days) of searchable data retention should be supported to enable threat hunting workflows across historical data sets.

Infrastructure and Deployment

Most vendors provide both customer- and vendor-hosted infrastructure for deploying threat detection platforms. Whether it makes sense to host yourself or have the vendor host it for you will depend on your preferences for security controls, privacy, and relative ease of administration. 

Regardless of which option is right for you, keep the following considerations in mind to ensure the deployment aligns to modern DevOps best practices:

• Documentation: the deployment model and all of its standard upgrade and maintenance procedures should be thoroughly documented 
• Backup and Restoration: by default, data backups should be performed regularly (typically once a day but no less than once a week). Procedures by which the historical log data is backed up and restored in the event of an outage should be clearly documented, with support for automated action.
• Horizontal Scaling: the infrastructure should accommodate scaling out by adding new computing nodes to handle increased workloads.
• Infrastructure / Configuration as Code: in line with the detection as code functionality explained above, infrastructure as code streamlines DevOps with automated provisioning and server management, treating configurations as versioned artifacts. 
• Availability / Uptime: high availability is essential for the detection platform to be consistently reliable and performant. Most vendors offer a minimum monthly uptime of 99%, with some leading vendors going above and beyond this minimum. 

System Operations and Administration

In order to maintain a reasonable level of administrative flexibility and security, the following system-level configurations should be included: 

• System Logs Access: the platform should allow access to logging telemetry on system performance, usage, and status for authorized users and API keys.
• IAM and SSO Integrations: the platform should seamlessly integrate with standard  identity and access management and single sign-on (SSO) procedures and platforms for seamless, secure access and role-based privilege assignments. 
• Risk-based Access Control (RBAC): the platform should enable RBAC for both human users and API keys/accounts for a streamlined approach to assigning permissions based on existing roles
• API Keys: the platform should support the creation and management of API keys to enable administrators and other authorized users to configure additional external integrations

Future-Proofing Your Threat Detection Programs

As cybersecurity teams navigate the challenges of the evolving cloud threat landscape, the advantages of security detection platforms over traditional SIEM solutions become readily apparent. The complexity of identifying and mitigating security incidents in a sea of cloud event logs requires a transformative shift towards detection tools tailored for the scale and intricacies of the cloud. 

Finding the right security detection platform will empower your team to make informed security decisions while supporting core strategic cybersecurity objectives. As your team starts this transformative journey, the above evaluation criteria will ensure your platform of choice not only addresses your current challenges but paves the way for future threat monitoring success.