Skip to content

AWS CloudTrail

Collect AWS CloudTrail log data

Logging

AWS CloudTrail Configuration

Event types

CloudTrail records management events, data events, network activity events, and Insights events. For example:

  • Management events: Also known as control panel operations, these are management operations performed on resources in an AWS account, such as creating, modifying, or deleting Amazon S3 buckets, starting or terminating Amazon EC2 instances, etc. They also include non-API events, such as console login events when users log into their accounts.
  • Data events: These are data plane operations, which are operations performed on or within resources, typically high-volume activities. For example, Amazon S3 object-level API operations (such as GetObject and PutObject), Amazon Lambda function invocation APIs, Amazon DynamoDB item-level API operations, etc., which can record the creation, reading, updating, or deletion of items in specific DynamoDB tables.
  • Network activity events: Used to record API operations executed from a private VPC to AWS services using VPC endpoints, including AWS API calls that successfully pass VPC endpoint policies and those that are denied access. VPC endpoint owners can use these logs to view operation logs denied by VPC endpoint policies or to determine if actors outside the data boundary are attempting to access data in S3 buckets.
  • Insights events: CloudTrail Insights continuously analyzes CloudTrail management events, and when it detects anomalous activity related to write API calls, it records Insights events and delivers them to the target S3 bucket, CloudWatch Events, or CloudWatch Logs group, allowing users to track and respond to anomalies.

Log collection methods

  • Write AWS CloudTrail logs to an S3 bucket
  • Trigger an Event to invoke a Lambda function via S3, and report data to the platform through DataKit or DataWay

Writing CloudTrail logs to an S3 bucket

To write AWS CloudTrail logs to an S3 bucket, follow these steps to configure and enable:

  1. Create an S3 bucket: In the AWS Management Console, go to the S3 service and create a dedicated bucket (recommended to use a unique name and enable version control), ensuring the bucket policy allows CloudTrail to write logs (the policy can be automatically generated when creating a trail).
  2. Create a CloudTrail trail: Go to the CloudTrail console, click "Create trail", set the trail name, and choose "Apply to all regions" or specify a region.
  3. Configure log storage: In the "Storage location", select "Existing S3 bucket" or create a new bucket, and specify the bucket name.
  4. Set advanced options (optional): Such as enabling multi-region trails, configuring log file validation, setting up CloudWatch Logs integration, etc.
  5. Complete creation: After confirming the configuration, click "Create trail". The system will automatically grant the necessary permissions to the specified S3 bucket, and subsequent API activity logs captured by CloudTrail will be written to this bucket (the path format is usually AWSLogs/<account ID>/CloudTrail/<region>/<year>/<month>/<day>/). After configuration, you can view the generated log files in the S3 bucket. These files are stored in JSON format and contain detailed API call records.

Lambda configuration

Reference: Lambda fetching S3 log data

Log parsing

TrueWatch has built-in CloudTrail log parsing. You can go to [Logs] - [Pipelines] - [Pipeline Library] - select CloudTrail, add it as a Central Pipeline, and save. This will take effect for newly reported data.