AWS S3¶

Use the script market "TrueWatch cloud sync" series of script packages to synchronize cloud monitoring and cloud asset data to TrueWatch.

Configuration¶

Install Func¶

It is recommended to activate TrueWatch integration - extension - hosted Func: all prerequisites will be automatically installed, please proceed with the script installation.

If you deploy Func yourself, refer to Self-deployed Func

Install Script¶

Note: Please prepare an Amazon AK that meets the requirements in advance (for simplicity, you can directly grant global read-only permissions ReadOnlyAccess)

To synchronize AWS S3 monitoring data, we install the corresponding collection script: "TrueWatch Integration (AWS-S3 Collection)" (ID: truewatch_aws_s3)

After clicking [Install], enter the corresponding parameters: Amazon AK, Amazon account name.

Click [Deploy Startup Script], the system will automatically create a Startup script set and automatically configure the corresponding startup script.

In addition, you can see the corresponding automatic trigger configuration under "Management / Automatic Trigger Configuration". Click [Execute] to immediately execute it once without waiting for the scheduled time. After a while, you can view the execution task records and corresponding logs.

We default collect some configurations, for details see metrics column Customize Cloud Object Metrics

Verification¶

1. In "Management / Automatic Trigger Configuration", confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task records and logs to check for any abnormalities.
2. On the TrueWatch platform, in "Infrastructure / Custom", check if there is any asset information.
3. On the TrueWatch platform, in "Metrics", check if there are any corresponding monitoring data.

Metrics¶

Configure Amazon - CloudWatch properly, the default metric sets are as follows. You can collect more metrics through configuration Amazon CloudWatch Metrics Details

Daily Storage Metrics for Amazon S3 Buckets in CloudWatch¶

The AWS/S3 namespace contains the following daily storage metrics for buckets.

| NumberOfObjects | Total number of objects stored in all storage classes in the bucket. This value is calculated by counting the total number of all objects (including current and non-current objects), delete markers, and all parts of all multipart uploads to the bucket that have not been completed. Valid storage type filter condition: AllStorageTypes (see StorageType dimension). Unit: Count Valid statistics: Average |

Amazon S3 CloudWatch Request Metrics in CloudWatch¶

The AWS/S3 namespace contains the following request metrics. These metrics include non-billable requests (GET requests from COPY and Replication).

S3 Replication Metrics in CloudWatch¶

You can use S3 replication metrics to monitor the progress of replication by tracking pending bytes, pending operations, and replication latency. For more details, see Monitoring Progress Using Replication Metrics.

Note:

You can enable alarms for replication metrics in Amazon CloudWatch. When setting alarms for replication metrics, set the Missing data treatment field to Treat missing data as ignore (maintain the alarm state).

S3 Storage Lens Usage and Activity Metrics in CloudWatch¶

You can publish S3 Storage Lens usage and activity metrics to Amazon CloudWatch to create a unified operational view in CloudWatch dashboards. S3 Storage Lens metrics are published to the AWS/S3/Storage-Lens namespace in CloudWatch. The CloudWatch publishing option is available for S3 Storage Lens dashboards upgraded to advanced metrics and recommendations.

For a list of S3 Storage Lens metrics published to CloudWatch, see Amazon S3 Storage Lens Metrics Glossary. For a complete list of dimensions, see Dimensions.

S3 Object Lambda Request Metrics in CloudWatch¶

S3 Object Lambda includes the following request metrics.

Amazon S3 on Outposts Metrics in CloudWatch¶

For a list of metrics used for S3 on Outposts in CloudWatch, see CloudWatch metrics (CloudWatch metrics).

Amazon S3 Dimensions in CloudWatch¶

The following dimensions are used to filter Amazon S3 metrics.

S3 Storage Lens Dimensions in CloudWatch¶

For a list of dimensions used to filter S3 Storage Lens metrics in CloudWatch, see Dimensions.

S3 Object Lambda Request Dimensions in CloudWatch¶

The following dimensions are used to filter data from object Lambda access points.

Objects¶

The collected AWS S3 object data structure can be viewed under "Infrastructure - Custom".

{
  "measurement": "aws_s3",
  "tags": {
    "name": "dataxxxx",
    "RegionId": "cn-northwest-1",
    "LocationConstraint": "cn-northwest-1",
    "Name": "dataxxxx"
  },
  "fields": {
    "CreationDate": "2022-03-09T06:13:31Z",
    "Grants": "{JSON Data}",
    "message": "{Instance JSON Data}"
  }
}

Note: Fields in tags, fields may change with subsequent updates.

Tip 1: The value of tags.name is the instance name, used for unique identification.

Tip 2: fields.message is a JSON serialized string.

Tip 3: fields.Grants is the bucket access control list.