Huawei Cloud RDS MYSQL
Collect Huawei Cloud RDS MYSQL data
Configuration¶
Install Func¶
It is recommended to activate the TrueWatch integration - extension - DataFlux Func (Automata): all preconditions are automatically installed, please proceed with the script installation.
If you deploy Func yourself, refer to Deploy Func Yourself
Install Script¶
Note: Please prepare the Huawei Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission
ReadOnlyAccess
)
To synchronize the monitoring data of Huawei Cloud RDS MYSQL, we install the corresponding collection script:
- integration_huaweicloud_rds: Collect monitoring Metrics data
- integration_huaweicloud_rds_slowlog_detail: Collect slow query detail log data
- integration_huaweicloud_rds_slowlog: Collect slow query statistics log data
After clicking [Install], enter the corresponding parameters: Huawei Cloud AK, Huawei Cloud account name.
Click [Deploy Startup Script], the system will automatically create the Startup
script set and automatically configure the corresponding startup script.
After the script is installed, find the script "TrueWatch Integration (Huawei Cloud-RDS Collection)" / "TrueWatch Integration (Huawei Cloud-RDS Slow Query Detail Log Collection)" / "TrueWatch Integration (Huawei Cloud-RDS Slow Query Statistics Log Collection)" in the "Development" section of Func, expand and modify this script, find collector_configs
and monitor_configs
respectively, edit the content in region_projects
, change the region and Project ID to the actual region and Project ID, and then click Save and Publish.
In addition, you can see the corresponding automatic trigger configuration in "Management / Automatic Trigger Configuration". Click [Execute] to execute it immediately without waiting for the scheduled time. Wait a moment, you can view the execution task record and corresponding log.
Verification¶
- In "Management / Automatic Trigger Configuration", confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task record and log to check for any exceptions.
- In TrueWatch, check whether asset information exists in "Infrastructure - Resource Catalog".
- In TrueWatch, check whether there is corresponding monitoring data in "Metrics".
- In TrueWatch, check whether there is corresponding slow log data in "Logs".
Metrics¶
Configure Huawei Cloud RDS MYSQL monitoring Metrics, you can collect more Metrics through configuration Huawei Cloud RDS MYSQL Metrics Details
Instance Monitoring Metrics¶
RDS for MySQL instance performance monitoring Metrics, as shown in the table below.
Metric ID | Metric Name | Metric Meaning | Value Range | Monitoring Period (Original Metric) |
---|---|---|---|---|
rds001_cpu_util |
CPU Utilization | This metric is used to count the CPU utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds002_mem_util |
Memory Utilization | This metric is used to count the memory utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds003_iops |
IOPS | This metric is used to count the current instance, the number of I/O requests processed by the system per unit time (average). | ≥ 0 counts/s | 1 minute |
rds004_bytes_in |
Network Input Throughput | This metric is used to count the average input traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds005_bytes_out |
Network Output Throughput | This metric is used to count the average output traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds006_conn_count |
Total Database Connections | This metric is used to count the total number of connections attempting to connect to the MySQL server, in counts. | ≥ 0 counts | 1 minute5 seconds1 second |
rds007_conn_active_count |
Current Active Connections | This metric is used to count the number of currently open connections, in counts. | ≥ 0 counts | 1 minute5 seconds1 second |
rds008_qps |
QPS | This metric is used to count the number of SQL statement queries, including stored procedures, in queries/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds009_tps |
TPS | This metric is used to count the number of transaction executions, including commits and rollbacks, in transactions/second. | ≥ 0 transactions/s | 1 minute5 seconds1 second |
rds010_innodb_buf_usage |
Buffer Pool Utilization | This metric is used to count the ratio of free pages to the total number of buffer pool pages in the InnoDB cache, in percentage. | 0-1 | 1 minute |
rds011_innodb_buf_hit |
Buffer Pool Hit Rate | This metric is used to count the ratio of read hits to read requests, in percentage. | 0-1 | 1 minute |
rds012_innodb_buf_dirty |
Buffer Pool Dirty Block Rate | This metric is used to count the ratio of dirty data in the InnoDB cache to the pages used in the InnoDB cache, in percentage. | 0-1 | 1 minute |
rds013_innodb_reads |
InnoDB Read Throughput | This metric is used to count the average number of bytes read per second by Innodb, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds014_innodb_writes |
InnoDB Write Throughput | This metric is used to count the average number of bytes written per second by Innodb, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds015_innodb_read_count |
InnoDB File Read Frequency | This metric is used to count the average number of reads from files per second by Innodb, in counts/second. | ≥ 0 counts/s | 1 minute |
rds016_innodb_write_count |
InnoDB File Write Frequency | This metric is used to count the average number of writes to files per second by Innodb, in counts/second. | ≥ 0 counts/s | 1 minute |
rds017_innodb_log_write_req_count |
InnoDB Log Write Request Frequency | This metric is used to count the average number of log write requests per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds018_innodb_log_write_count |
InnoDB Log Physical Write Frequency | This metric is used to count the average number of physical writes to log files per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds019_innodb_log_fsync_count |
InnoDB Log fsync() Write Frequency | This metric is used to count the average number of fsync() writes completed to log files per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds020_temp_tbl_rate |
Temporary Table Creation Rate | This metric is used to count the number of temporary tables created on the hard disk per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds021_myisam_buf_usage |
Key Buffer Utilization | This metric is used to count the utilization of MyISAM Key buffer, in percentage. | 0-1 | 1 minute |
rds022_myisam_buf_write_hit |
Key Buffer Write Hit Rate | This metric is used to count the write hit rate of MyISAM Key buffer, in percentage. | 0-1 | 1 minute |
rds023_myisam_buf_read_hit |
Key Buffer Read Hit Rate | This metric is used to count the read hit rate of MyISAM Key buffer, in percentage. | 0-1 | 1 minute |
rds024_myisam_disk_write_count |
MyISAM Disk Write Frequency | This metric is used to count the number of writes to disk indexes, in counts/second. | ≥ 0 counts/s | 1 minute |
rds025_myisam_disk_read_count |
MyISAM Disk Read Frequency | This metric is used to count the number of reads from disk indexes, in counts/second. | ≥ 0 counts/s | 1 minute |
rds026_myisam_buf_write_count |
MyISAM Buffer Pool Write Frequency | This metric is used to count the number of write requests to the buffer pool indexes, in counts/second. | ≥ 0 counts/s | 1 minute |
rds027_myisam_buf_read_count |
MyISAM Buffer Pool Read Frequency | This metric is used to count the number of read requests from the buffer pool indexes, in counts/second. | ≥ 0 counts/s | 1 minute |
rds028_comdml_del_count |
Delete Statement Execution Frequency | This metric is used to count the average number of Delete statement executions per second, in counts/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds029_comdml_ins_count |
Insert Statement Execution Frequency | This metric is used to count the average number of Insert statement executions per second, in counts/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds030_comdml_ins_sel_count |
Insert_Select Statement Execution Frequency | This metric is used to count the average number of Insert_Select statement executions per second, in counts/second. | ≥ 0 queries/s | 1 minute |
rds031_comdml_rep_count |
Replace Statement Execution Frequency | This metric is used to count the average number of Replace statement executions per second, in counts/second. | ≥ 0 queries/s | 1 minute |
rds032_comdml_rep_sel_count |
Replace_Selection Statement Execution Frequency | This metric is used to count the average number of Replace_Selection statement executions per second, in counts/second. | ≥ 0 queries/s | 1 minute |
rds033_comdml_sel_count |
Select Statement Execution Frequency | This metric is used to count the average number of Select statement executions per second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds034_comdml_upd_count |
Update Statement Execution Frequency | This metric is used to count the average number of Update statement executions per second, in counts/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds035_innodb_del_row_count |
Row Deletion Rate | This metric is used to count the average number of rows deleted from InnoDB tables per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds036_innodb_ins_row_count |
Row Insertion Rate | This metric is used to count the average number of rows inserted into InnoDB tables per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds037_innodb_read_row_count |
Row Read Rate | This metric is used to count the average number of rows read from InnoDB tables per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds038_innodb_upd_row_count |
Row Update Rate | This metric is used to count the average number of rows updated in InnoDB tables per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds039_disk_util |
Disk Utilization | This metric is used to count the disk utilization of the measurement object, in percentage. | 0-100% | 1 minute |
rds047_disk_total_size |
Total Disk Size | This metric is used to count the total disk size of the measurement object. | 40GB~4000GB | 1 minute |
rds048_disk_used_size |
Disk Usage | This metric is used to count the disk usage of the measurement object. | 0GB~4000GB | 1 minute |
rds049_disk_read_throughput |
Disk Read Throughput | This metric is used to count the number of bytes read from the disk per second. | ≥ 0 bytes/s | 1 minute |
rds050_disk_write_throughput |
Disk Write Throughput | This metric is used to count the number of bytes written to the disk per second. | ≥ 0 bytes/s | 1 minute |
rds072_conn_usage |
Connection Usage Rate | This metric is used to count the percentage of currently used MySQL connections to the total number of connections. | 0-100% | 1 minute |
rds173_replication_delay_avg |
Average Replication Delay | This metric is the average delay between the standby or read-only and the master, corresponding to seconds_behind_master. The average value of the 60-second period is taken. | ≥ 0s | 10 seconds |
rds073_replication_delay |
Real-time Replication Delay | This metric is the real-time delay between the standby or read-only and the master, corresponding to seconds_behind_master. This value is real-time. | ≥ 0s | 1 minute5 seconds |
rds074_slow_queries |
Slow Log Count Statistics | This metric is used to display the number of slow logs generated by MySQL per minute. | ≥ 0 | 1 minute |
rds075_avg_disk_ms_per_read |
Disk Read Time | This metric is used to count the average time spent reading the disk during a certain period. | ≥ 0ms | 1 minute |
rds076_avg_disk_ms_per_write |
Disk Write Time | This metric is used to count the average time spent writing to the disk during a certain period. | ≥ 0ms | 1 minute |
rds077_vma |
VMA Count | Monitor the virtual memory area size of the RDS process, in counts. | ≥ 0counts | 1 minute |
rds078_threads |
Thread Count in Process | Monitor the number of threads in the RDS process, in counts. | ≥ 0counts | 1 minute |
rds079_vm_hwm |
Peak Physical Memory Usage of Process | Monitor the peak physical memory usage of the RDS process, in KB. | ≥ 0 KB | 1 minute |
rds080_vm_peak |
Peak Virtual Memory Usage of Process | Monitor the peak virtual memory usage of the RDS process, in KB. | ≥ 0 KB | 1 minute |
rds081_vm_ioutils |
Disk I/O Utilization | Disk I/O utilization, in percentage. | 0-1 | 1 minute |
rds082_semi_sync_tx_avg_wait_time |
Average Transaction Wait Time | Monitor the average wait time in semi-synchronous replication mode, in microseconds. | ≥ 0 microseconds | 1 minute |
sys_swap_usage |
Swap Utilization | This metric is used to count the swap utilization of the measurement object, in percentage. | 0-100% | 1 minute |
rds_innodb_lock_waits |
Row Lock Wait Count | This metric is used to count the number of row lock waits in Innodb, in counts. Indicates the cumulative number of transactions waiting for row locks historically. Restarting will clear the lock waits. | ≥ 0 counts | 1 minute |
rds_bytes_recv_rate |
Database Bytes Received Per Second | This metric is used to count the number of bytes received by the database per second, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_bytes_sent_rate |
Database Bytes Sent Per Second | This metric is used to count the number of bytes sent by the database per second, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_innodb_pages_read_rate |
Innodb Average Data Read Per Second | This metric is used to count the average data read per second by Innodb, in pages/second. | ≥ 0 Pages/s | 1 minute |
rds_innodb_pages_written_rate |
Innodb Average Data Written Per Second | This metric is used to count the average data written per second by Innodb, in pages/second. | ≥ 0 Pages/s | 1 minute |
rds_innodb_os_log_written_rate |
Average Redo Log Written Per Second | This metric is used to count the average size of redo log written per second, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_innodb_buffer_pool_read_requests_rate |
Innodb_buffer_pool Read Requests Per Second | This metric is used to count the number of read requests per second by innodb_buffer_pool, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_innodb_buffer_pool_write_requests_rate |
Innodb_buffer_pool Write Requests Per Second | This metric is used to count the number of write requests per second by innodb_buffer_pool, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_innodb_buffer_pool_pages_flushed_rate |
Innodb_buffer_pool Page Flushes Per Second | This metric is used to count the number of page flushes per second by innodb_buffer_pool, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_innodb_log_waits_rate |
Log Buffer Insufficient Flush Wait Count | This metric is used to count the number of times waiting for flush to disk due to insufficient log buffer, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_created_tmp_tables_rate |
Temporary Tables Created Per Second | This metric is used to count the number of temporary tables created per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_wait_thread_count |
Waiting Thread Count | This metric is used to count the number of waiting threads, in counts. | ≥ 0 counts | 1 minute |
rds_innodb_row_lock_time_avg |
Historical Row Lock Average Wait Time | This metric is used to count the average wait time of historical row locks in innodb. | > 0ms | 1 minute |
rds_innodb_row_lock_current_waits |
Current Row Lock Wait Count | This metric is used to count the current number of row lock waits in innodb, in counts. Indicates the number of transactions currently waiting for row locks. | ≥ 0 counts | 1 minute |
rds_mdl_lock_count |
MDL Lock Count | This metric is used to count the number of MDL locks, in counts. | ≥ 0counts | 1 minute |
rds_buffer_pool_wait_free |
Dirty Pages Waiting to be Flushed | This metric counts the number of dirty pages waiting to be flushed, in counts. | ≥ 0counts | 1 minute |
rds_conn_active_usage |
Active Connection Usage Rate | This metric counts the ratio of active connections to the maximum number of connections, in percentage. | 0-100% | 1 minute |
rds_innodb_log_waits_count |
Log Wait Count | This metric is used to count the number of log waits, in counts. | ≥ 0counts | 1 minute |
rds_long_transaction |
Long Transaction Metric | This metric counts the time spent on long transactions, in seconds. Related operation commands must have BEGIN and COMMIT commands before and after to be considered a complete long transaction. | ≥ 0seconds | 1 minute |
RDS for MySQL database proxy monitoring Metrics, as shown in Table 2.
Metric ID | Metric Name | Metric Meaning | Value Range | Monitoring Period (Original Metric) |
---|---|---|---|---|
rds001_cpu_util | CPU Utilization | This metric is used to count the CPU utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds002_mem_util | Memory Utilization | This metric is used to count the memory utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds004_bytes_in | Network Input Throughput | This metric is used to count the average input traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds005_bytes_out | Network Output Throughput | This metric is used to count the average output traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_proxy_frontend_connections | Frontend Connections | The number of connections between the application and the Proxy. | ≥ 0 counts | 1 minute |
rds_proxy_backend_connections | Backend Connections | The number of connections between the Proxy and the RDS database. | ≥ 0 counts | 1 minute |
rds_proxy_average_response_time | Average Response Time | Average response time. | ≥ 0 ms | 1 minute |
rds_proxy_query_per_seconds | QPS | SQL statement query count. | ≥ 0 counts | 1 minute |
rds_proxy_read_query_proportions | Read Proportion | The proportion of read requests to total requests. | 0-100% | 1 minute |
rds_proxy_write_query_proportions | Write Proportion | The proportion of write requests to total requests. | 0-100% | 1 minute |
rds_proxy_frontend_connection_creation | Average Frontend Connections Created Per Second | Count the average number of frontend connections created per second by client applications for the database proxy service. | ≥ 0 counts/s | 1 minute |
rds_proxy_transaction_query | Average Queries in Transactions Per Second | Count the average number of select executions in transactions executed per second | ≥ 0 counts/s | 1 minute |
rds_proxy_multi_statement_query | Average Multi-Statements Executed Per Second | Count the average number of Multi-Statements executed per second | ≥ 0 counts/s | 1 minute |
Object¶
The collected Huawei Cloud RDS MYSQL object data structure can be seen in "Infrastructure - Resource Catalog"
{
"measurement": "huaweicloud_rds",
"tags": {
"RegionId" : "cn-north-4",
"project_id" : "c631f046252dxxxxxxxf253c62d48585",
"enterprise_project_id" : "d13ebb59-d4fe-xxxx-xxxx-zc22bcea6f987",
"instance_id" : "1d0c91561f4644dxxxxxxd68304b0520din01",
"instance_name" : "rds-df54-xxxx",
"status" : "ACTIVE",
"type" : "Single",
"engine" : "MySQL"
},
"fields": {
"engine_version" : "5.7",
"port" : "3306",
"security_group_id" : "d13ebb59-d4fe-xxxx-xxxx-c22bcea6f987",
"switch_strategy" : "xxx",
"enable_ssl" : "False",
"time_zone" : "UTC+08:00",
"created_time" : "2022-06-21T06:17:27+0000",
"updated_time" : "2022-06-21T06:20:03+0000",
"expiration_time" : "2022-06-23T06:20:03+0000",
"alias" : "xxx",
"private_ips" : "[\"192.xxx.x.144\"]",
"public_ips" : "[]",
"cpu" : "2",
"mem" : "4",
"volume" : "{volume information}",
"nodes" : "[{primary and standby instance information}]",
"related_instance" : "[]",
"backup_strategy" : "{database information}"
}
}
Note: The fields in
tags
andfields
may change with subsequent updates.Tip 1: The value of
tags.insatnce_id
is the instance ID, used as a unique identifier.Tip 2: The following fields are all JSON serialized strings.
fields.message
fields.private_ips
fields.public_ips
fields.volume
fields.nodes
fields.related_instance
fields.backup_strategy
Tip 3: The value of
type
is "Single", "Ha", "Replica", or "Enterprise", corresponding to single instance, primary-standby instance, read-only instance, and distributed instance (enterprise edition) respectively.