Skip to content

Collector "AWS-RDS" Configuration Manual

Before reading this document, please read:

Tip

Before using this collector, you must install the "Integration Core Package" and its accompanying third-party dependencies.

1. Configuration Structure

The configuration structure of this collector is as follows:

Field Type Required Description
regions list Required List of regions to collect
regions[#] str Required Region ID. For example: 'cn-north-1'
See the appendix for the complete list
engines list No List of engine types (if not configured, collect all engine types of RDS instances)
engines[#] str No Engine type, currently supported: aurora-mysql, aurora-postgresql, mariadb, mysql, oracle-ee, oracle-ee-cdb, oracle-se2, oracle-se2-cdb, postgres, sqlserver-ee, sqlserver-se, sqlserver-ex, sqlserver-web

2. Configuration Examples

Specified Regions

Collect instance data from Ningxia and Beijing regions with engine types aurora-mysql, aurora-postgresql, and mysql

collector_configs = {
    'engines': ['aurora-mysql', 'aurora-postgresql', 'mysql'],
    'regions': [ 'cn-northwest-1', 'cn-north-1' ]
}

Configure Filters (Optional)

This collector script supports custom filters, allowing users to filter target resources by object attributes. The filter function returns True or False.

  • True: The target resource should be collected.

  • False: The target resource should not be collected.

Supported object attributes for filtering:

Attribute Description
name Instance ID
Engine Instance Engine
DBInstanceClass Instance Type
DBInstanceIdentifier Instance Identifier
AvailabilityZone Instance Availability Zone
SecondaryAvailabilityZone Secondary Availability Zone
Endpoint_Address Endpoint Address
DBInstanceStatus Instance Status 
# Example: Enable filter, filter by object's DBInstanceIdentifier and Engine attributes, configure as follows:
def filter_instance(instance):
    '''
    Collect instances with instance_id as xxx and engine as mysql
    '''
    # return True
    instance_id = instance['DBInstanceIdentifier']
    instance_engine = instance['Engine']
    if instance_id in ['xxx'] and instance_engine in ['mysql']:
        return True
    return False

@DFF.API('AWS-RDS Collection', timeout=3600, fixed_crontab='*/15 * * * *')
def run():
    Runner(main.DataCollector(account, collector_configs, filters=[filter_instance])).run()

3. Data Reporting Format

After data is successfully synchronized, you can view the data in the "Infrastructure - Resource Catalog" of TrueWatch.

The reported data example is as follows:

{
  "measurement": "aws_rds",
  "tags": {
    "name"                     : "xxxxx",
    "RegionId"                 : "cn-northwest-1",
    "Engine"                   : "mysql",
    "DBInstanceClass"          : "db.t3.medium",
    "DBInstanceIdentifier"     : "xxxxxx",
    "AvailabilityZone"         : "cn-northwest-1c",
    "SecondaryAvailabilityZone": "cn-northwest-1d"
  },
  "fields": {
    "InstanceCreateTime"  : "2018-03-28T19:54:07.871Z",
    "LatestRestorableTime": "2018-03-28T19:54:07.871Z",
    "Endpoint"            : "{Endpoint Address JSON Data}",
    "AllocatedStorage"    : 100,
    "message"             : "{Instance JSON Data}",
  }
}
Note

Fields in tags and fields may change with subsequent updates.

Tip

The value of tags.name is the instance ID, which serves as a unique identifier.

Tip

fields.message and fields.Endpoint are JSON serialized strings.

4. IAM Policy Permissions

Note

If users use the method of bringing IAM roles to collect resources, certain operation permissions need to be enabled.

This collector requires the following permissions:

rds:DescribeDBInstances

X. Appendix

Please refer to the AWS official documentation: