AWS Redshift¶
The core performance metrics of AWS Redshift include query performance, disk space utilization, CPU utilization, database connections, and disk I/O operations, which are key metrics for evaluating and optimizing data warehouse performance.
Configuration¶
Install Func¶
It is recommended to activate the TrueWatch Integration - Extensions - DataFlux Func (Automata): all prerequisites are automatically installed. Please proceed with the script installation.
For self-deployment of Func, refer to Self-deployment Func
Install Script¶
Note: Please prepare the required Amazon AK in advance (for simplicity, you can directly grant the global read-only permission
ReadOnlyAccess
)
Hosted Version Activation Script¶
- Log in to the TrueWatch console
- Click on the "Integration" menu, select "Cloud Account Management"
- Click "Add Cloud Account", select "AWS", and fill in the required information on the interface. If the cloud account information has been configured before, ignore this step
- Click "Test", and after a successful test, click "Save". If the test fails, please check if the relevant configuration information is correct and retest
- Click on the "Cloud Account Management" list to see the added cloud account, click on the corresponding cloud account to enter the details page
- Click the "Integration" button on the cloud account details page, find "AWS Redshift" under the
Not Installed
list, and click the "Install" button to pop up the installation interface for installation.
Manual Activation Script¶
-
Log in to the Func console, click on "Script Market", enter the TrueWatch script market, and search for:
integration_aws_redshift
-
Click "Install", then enter the corresponding parameters: AWS AK ID, AK Secret, and account name.
-
Click "Deploy Startup Script", the system will automatically create the
Startup
script set and automatically configure the corresponding startup script. -
After activation, you can see the corresponding automatic trigger configuration in "Management / Automatic Trigger Configuration". Click "Execute" to immediately execute once without waiting for the scheduled time. After a while, you can view the execution task records and corresponding logs.
Verification¶
- Confirm in "Management / Automatic Trigger Configuration" whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task records and logs to see if there are any exceptions
- In TrueWatch, check in "Infrastructure / Custom" to see if there is asset information
- In TrueWatch, check in "Metrics" to see if there is corresponding monitoring data
Metrics¶
After configuring Amazon CloudWatch, the default Measurement is as follows. More metrics can be collected through configuration Amazon CloudWatch Metrics Details
Redshift Metrics¶
Metric Name | Description | Unit | Dimensions |
---|---|---|---|
CPUUtilization | CPU usage percentage. For clusters, this metric represents the sum of CPU usage values for all nodes (leader node and compute nodes) | % | ClusterIdentifier |
HealthStatus | Health check of the cluster | 1: Normal or 0: Abnormal | ClusterIdentifier |
MaintenanceMode | Whether the cluster is in maintenance mode | 1: ON or 0: OFF | ClusterIdentifier |
PercentageDiskSpaceUsed | Percentage of disk space used | % | ClusterIdentifier |
DatabaseConnections | Number of database connections in the cluster | count | ClusterIdentifier |
CommitQueueLength | Number of transactions waiting to commit at a given point in time | count | ClusterIdentifier |
ConcurrencyScalingActiveClusters | Number of concurrency scaling clusters actively processing queries at any given time | count | ClusterIdentifier |
NetworkReceiveThroughput | Rate at which data is received by the node or cluster | byte/s | ClusterIdentifier |
NetworkTransmitThroughput | Rate at which data is written by the node or cluster | byte/s | ClusterIdentifier |
MaxConfiguredConcurrencyScalingClusters | Maximum number of concurrency scaling clusters configured from the parameter group | count | ClusterIdentifier |
NumExceededSchemaQuotas | Number of schemas that exceed quotas | count | ClusterIdentifier |
ReadIOPS | Average number of disk read operations per second | count/s | ClusterIdentifier |
ReadLatency | Average time required for disk read I/O operations | seconds | ClusterIdentifier |
ReadThroughput | Average number of bytes read from disk per second | byte | ClusterIdentifier |
TotalTableCount | Number of user tables open at a specific point in time | count | ClusterIdentifier |
WriteIOPS | Average number of disk write operations per second | count/s | ClusterIdentifier |
WriteLatency | Average time required for disk write I/O operations | seconds | ClusterIdentifier |
WriteThroughput | Average number of bytes written to disk per second | byte | ClusterIdentifier |
Object¶
The collected Amazon AWS Redshift object data structure can be seen in "Infrastructure - Custom"
{
"measurement": "aws_redshift",
"tags": {
"ClusterAvailabilityStatus" : "Modifying",
"ClusterIdentifier" : "hn-test",
"ClusterStatus" : "creating",
"ClusterSubnetGroupName" : "default",
"ClusterVersion" : "1.0",
"DBName" : "dev",
"MasterUsername" : "awsuser",
"NodeType" : "dc2.large",
"PreferredMaintenanceWindow" : "sat:19:30-sat:20:00",
"RegionId" : "cn-northwest-1",
"VpcId" : "vpc-b1ca3ff0fa4d",
"name" : "hn-test"
},
"fields": {
"AllowVersionUpgrade" : true,
"AutomatedSnapshotRetentionPeriod" : 1,
"ClusterNodes" : "[]",
"ClusterParameterGroups" : "[{\"ParameterApplyStatus\": \"in-sync\", \"ParameterGroupName\": \"default.redshift-1.0\"}]",
"ClusterSecurityGroups" : "[]",
"Encrypted" : false,
"Endpoint" : "{}",
"ManualSnapshotRetentionPeriod" : -1,
"NumberOfNodes" : 1,
"PendingModifiedValues" : "{\"MasterUserPassword\": \"****\"}",
"PubliclyAccessible" : false,
"VpcSecurityGroups" : "[{\"Status\": \"active\", \"VpcSecurityGroupId\": \"sg-467a\"}]",
"message" : "{instance JSON data}"
}
}
Note: The fields in
tags
,fields
may change with subsequent updatesTip 1: The value of
tags.name
is the instance ID, as a unique identifierTip 2:
fields.message
,fields.network_interfaces
,fields.blockdevicemappings
are JSON serialized strings ```