Skip to content

AWS Redshift

The core performance metrics of AWS Redshift include query performance, disk space utilization, CPU utilization, database connections, and disk I/O operations, which are key metrics for evaluating and optimizing data warehouse performance.

Configuration

Install Func

It is recommended to activate the TrueWatch Integration - Extensions - DataFlux Func (Automata): all prerequisites are automatically installed. Please proceed with the script installation.

For self-deployment of Func, refer to Self-deployment Func

Install Script

Note: Please prepare the required Amazon AK in advance (for simplicity, you can directly grant the global read-only permission ReadOnlyAccess)

Hosted Version Activation Script

  1. Log in to the TrueWatch console
  2. Click on the "Integration" menu, select "Cloud Account Management"
  3. Click "Add Cloud Account", select "AWS", and fill in the required information on the interface. If the cloud account information has been configured before, ignore this step
  4. Click "Test", and after a successful test, click "Save". If the test fails, please check if the relevant configuration information is correct and retest
  5. Click on the "Cloud Account Management" list to see the added cloud account, click on the corresponding cloud account to enter the details page
  6. Click the "Integration" button on the cloud account details page, find "AWS Redshift" under the Not Installed list, and click the "Install" button to pop up the installation interface for installation.

Manual Activation Script

  1. Log in to the Func console, click on "Script Market", enter the TrueWatch script market, and search for: integration_aws_redshift

  2. Click "Install", then enter the corresponding parameters: AWS AK ID, AK Secret, and account name.

  3. Click "Deploy Startup Script", the system will automatically create the Startup script set and automatically configure the corresponding startup script.

  4. After activation, you can see the corresponding automatic trigger configuration in "Management / Automatic Trigger Configuration". Click "Execute" to immediately execute once without waiting for the scheduled time. After a while, you can view the execution task records and corresponding logs.

Verification

  1. Confirm in "Management / Automatic Trigger Configuration" whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task records and logs to see if there are any exceptions
  2. In TrueWatch, check in "Infrastructure / Custom" to see if there is asset information
  3. In TrueWatch, check in "Metrics" to see if there is corresponding monitoring data

Metrics

After configuring Amazon CloudWatch, the default Measurement is as follows. More metrics can be collected through configuration Amazon CloudWatch Metrics Details

Redshift Metrics

Metric Name Description Unit Dimensions
CPUUtilization CPU usage percentage. For clusters, this metric represents the sum of CPU usage values for all nodes (leader node and compute nodes) % ClusterIdentifier
HealthStatus Health check of the cluster 1: Normal or 0: Abnormal ClusterIdentifier
MaintenanceMode Whether the cluster is in maintenance mode 1: ON or 0: OFF ClusterIdentifier
PercentageDiskSpaceUsed Percentage of disk space used % ClusterIdentifier
DatabaseConnections Number of database connections in the cluster count ClusterIdentifier
CommitQueueLength Number of transactions waiting to commit at a given point in time count ClusterIdentifier
ConcurrencyScalingActiveClusters Number of concurrency scaling clusters actively processing queries at any given time count ClusterIdentifier
NetworkReceiveThroughput Rate at which data is received by the node or cluster byte/s ClusterIdentifier
NetworkTransmitThroughput Rate at which data is written by the node or cluster byte/s ClusterIdentifier
MaxConfiguredConcurrencyScalingClusters Maximum number of concurrency scaling clusters configured from the parameter group count ClusterIdentifier
NumExceededSchemaQuotas Number of schemas that exceed quotas count ClusterIdentifier
ReadIOPS Average number of disk read operations per second count/s ClusterIdentifier
ReadLatency Average time required for disk read I/O operations seconds ClusterIdentifier
ReadThroughput Average number of bytes read from disk per second byte ClusterIdentifier
TotalTableCount Number of user tables open at a specific point in time count ClusterIdentifier
WriteIOPS Average number of disk write operations per second count/s ClusterIdentifier
WriteLatency Average time required for disk write I/O operations seconds ClusterIdentifier
WriteThroughput Average number of bytes written to disk per second byte ClusterIdentifier

Object

The collected Amazon AWS Redshift object data structure can be seen in "Infrastructure - Custom"

{
  "measurement": "aws_redshift",
  "tags": {
    "ClusterAvailabilityStatus"         : "Modifying",
    "ClusterIdentifier"                 : "hn-test",
    "ClusterStatus"                     : "creating",
    "ClusterSubnetGroupName"            : "default",
    "ClusterVersion"                    : "1.0",
    "DBName"                            : "dev",
    "MasterUsername"                    : "awsuser",
    "NodeType"                          : "dc2.large",
    "PreferredMaintenanceWindow"        : "sat:19:30-sat:20:00",
    "RegionId"                          : "cn-northwest-1",
    "VpcId"                             : "vpc-b1ca3ff0fa4d",
    "name"                              : "hn-test"
  },
  "fields": {
    "AllowVersionUpgrade"               : true,
    "AutomatedSnapshotRetentionPeriod"  : 1,
    "ClusterNodes"                      : "[]",
    "ClusterParameterGroups"            : "[{\"ParameterApplyStatus\": \"in-sync\", \"ParameterGroupName\": \"default.redshift-1.0\"}]",
    "ClusterSecurityGroups"             : "[]",
    "Encrypted"                         : false,
    "Endpoint"                          : "{}",
    "ManualSnapshotRetentionPeriod"     : -1,
    "NumberOfNodes"                     : 1,
    "PendingModifiedValues"             : "{\"MasterUserPassword\": \"****\"}",
    "PubliclyAccessible"                : false,
    "VpcSecurityGroups"                 : "[{\"Status\": \"active\", \"VpcSecurityGroupId\": \"sg-467a\"}]",
    "message"                           : "{instance JSON data}"
  }
}

Note: The fields in tags, fields may change with subsequent updates

Tip 1: The value of tags.name is the instance ID, as a unique identifier

Tip 2: fields.message, fields.network_interfaces, fields.blockdevicemappings are JSON serialized strings ```