Editor’s note: This post was originally published in December 2019 and has been last updated for accuracy and comprehensiveness in February 2021.
Amazon Web Services offers many services, from virtual machines (EC2) to storage (S3) to databases (RDS). Each of these services adds a new point of entry for potential cyber-attacks. To stay vigilant, security teams must keep a close watch on all activity and changes in AWS accounts. In this article, we will walk through how to collect high-value security log data with AWS CloudTrail, and discuss the various options available to get the most out of this service.
Before we get started with implementation, let’s take a quick look at why CloudTrail could be the perfect Swiss Army knife to effectively analyze events that occur in your environment:
Before we begin, let’s make sure to have the following pre-requisites in place:
These tools will enable you to configure best-practice CloudTrail infrastructure with predefined CloudFormation templates.
AWS CloudTrail is a service to audit all activity within your AWS account.
Enabling CloudTrail is critical for understanding the history of account changes and detecting suspicious activity.
Let’s try to dig deeper with an example event:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "AWSService",
"invokedBy": "ec2.amazonaws.com"
},
"eventTime": "2019-12-10T01:44:26Z",
"eventSource": "sts.amazonaws.com",
"eventName": "AssumeRole",
"awsRegion": "us-east-1",
"sourceIPAddress": "ec2.amazonaws.com",
"userAgent": "ec2.amazonaws.com",
"requestParameters": {
"roleSessionName": "i-00888ddddd3333322",
"roleArn": "arn:aws:iam::123456789012:role/S3SecurityDataAssumeRole"
},
"responseElements": {
"credentials": {
"sessionToken": "IQoJb3JpZ2luX2VjEIL//==",
"accessKeyId": "AAAASSSRRRRRZZZZZZCC",
"expiration": "Dec 10, 2019 7:46:42 AM"
}
},
"requestID": "09a32e7f-f51d-4f84-8f53-38e1cd773e3f",
"eventID": "f12888b4-4208-4ead-aa6a-1a21ffb24c15",
"resources": [
{
"ARN": "arn:aws:iam::123456789012:role/S3SecurityDataAssumeRole",
"accountId": "123456789012",
"type": "AWS::IAM::Role"
}
],
"eventType": "AwsApiCall",
"recipientAccountId": "123456789012",
"sharedEventID": "0aa168dc-6402-4bbf-9bc6-4d2bf58f106c"
}Code language: JSON / JSON with Comments (json)Here is what we can extract from the message:
ec2 service made the AssumeRole call at 2019–12–10T01:44:26Zus-east-1arn:aws:iam::123456789012:role/S3SecurityDataAssumeRole with the session i-00888ddddd3333322, indicating the EC2 hostnameerrorMessage key, also indicating a successful operationCloudTrail can capture two main types of events:
mike created a S3 bucket called production-secrets).jill downloaded a S3 object froms3://production-secrets/app/secret.txt).Before enabling data plane logging, be mindful of the pricing implications. Data plane settings must also be explicitly set, whereas the default operation for CloudTrail is to log events from the control plane.
CloudTrail can be configured to capture events from either all regions or a single region.
If you use AWS Organizations, an organization-level trail can be configured to automatically capture all events from each member account. Use the following API call to set an existing CloudTrail as your organization-level trail in your “master” account:$ aws cloudtrail update-trail –name <Trail> –is-organization-trail
There are several ways CloudTrail delivers data for security monitoring, and we have highlighted the notable ones below:
Buckets are required to create a new CloudTrail, and can optionally be configured to send messages to a SNS topic when new data is delivered. This feature can be used to initiate data processing pipelines and has increased reliability over using S3 event notifications. Data will be delivered between 5 and 15 minutes from the activity occurring.
CloudWatch Events provides the capability of analyzing CloudTrail data in real-time. When a CloudTrail is created, data automatically begins sending to CloudWatch Events and can be processed by using Event Rules to forward information to Lambda functions, Kinesis Streams, and more. The downside to this approach is that each region/account must be instrumented to centralize the data, and only write-level events are captured.
CloudTrail can also be sent to a CloudWatch Log group, with the main advantage of processing multi-region data in real-time from a single place. The downside to this approach, however, is cost and flexibility, since only a single subscription filter can be associated with a log group.
AWS Macie is a service that uses machine learning on S3 data to identify anomalous activity, and similarly, AWS GuardDuty is a broader service that can identify attacker activity (such as reconnaissance) in an account.
Enabling these services can provide quick wins in analyzing CloudTrail data.
The data sent in CloudTrail is sensitive, so it’s recommended to protect it with KMS encryption.
When creating the CloudTrail, a KMS Key can be associated by passing in the key ID. Additionally, to allow users or roles to process the data, an appropriate policy must allow kms:Decrypt permissions on the key.
CloudTrail recently introduced a feature called Insights to help detect higher than normal API call volume on write-based events.
To enable this feature, use the Insights tab on the CloudTrail Console or the following command:
$ aws cloudtrail put-insight-selectors --trail-name <Trail> --insight-selectors '[{"InsightType": "ApiCallRateInsight"}]'
Here is an example Insights event:
{
"eventVersion": "1.07",
"eventTime": "2019-10-15T21:14:00Z",
"awsRegion": "us-east-1",
"eventID": "EXAMPLEc-9eac-4af6-8e07-26a5ae8786a5",
"eventType": "AwsCloudTrailInsight",
"recipientAccountId": "123456789012",
"sharedEventID": "EXAMPLE8-02b2-4e93-9aab-08ed47ea5fd3",
"insightDetails": {
"state": "End",
"eventSource": "autoscaling.amazonaws.com",
"eventName": "PutLifecycleHook",
"insightType": "ApiCallRateInsight",
"insightContext": {
"statistics": {
"baseline": {
"average": 0.0017857143
},
"insight": {
"average": 4
},
"insightDuration": 1
}
}
},
"eventCategory": "Insight"
}Code language: JSON / JSON with Comments (json)The insightDetails.insightContext.statistics field shows the calculated baseline and how this set of API calls went above the threshold.
To set up a CloudTrail, run the following command from the panther-labs/tutorials repository:
$ make deploy tutorial=aws-security-logging stack=cloudtrail region=us-east-1 \
parameters="--parameter-overrides BucketID=<MyBucketName> TrailName=<MyTrailName>"Code language: SQL (Structured Query Language) (sql)This will create the following:
This CloudTrail can also be used as the organization’s master trail by issuing the aws update-trail command with the --is-organization-trail flag.
There are multiple ways to explore CloudTrail events in the AWS Console.
In the CloudTrail Dashboard, users have the capability to search the last 90 days of events with basic filtering and time range constraints:
To perform more advanced searches of CloudTrail data with a SQL interface, Athena can be used. To set up Athena, follow the steps below:
Run Advanced queries in Amazon Athena linkIn the Athena console, you can run SQL queries such as:
In this article, we covered the fundamentals of AWS CloudTrail. This service is critical for understanding your cloud security posture and provides a wide variety of rich data.
Panther is a cloud-native threat detection platform that ships with pre-built integrations, to make it easy to quickly analyze your data, triage alerts, and remediate incidents.
Use Panther’s built-in policies for continuous monitoring of CloudTrail resources or write your own detections in Python to fit your internal business use cases. Find documentation on how to use Panther to track real-time changes to your CloudTrail.
Get started with our Quick Start guide or contact us for a demo.
