Huawei Cloud RDS MYSQL
Collect Huawei Cloud RDS MYSQL data
Configuration¶
Install Func¶
It is recommended to activate TrueWatch integration - extension - hosted version of Func: all preconditions are automatically installed, please continue with script installation.
If you deploy Func yourself, refer to Deploy Func Yourself
Install Script¶
Note: Please prepare the Huawei Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission
ReadOnlyAccess
)
To synchronize the monitoring data of Huawei Cloud RDS MYSQL, we install the corresponding collection scripts:
- integration_huaweicloud_rds: Collect monitoring Metrics data
- integration_huaweicloud_rds_slowlog_detail: Collect slow query detail log data
- integration_huaweicloud_rds_slowlog: Collect slow query statistical log data
After clicking [Install], enter the corresponding parameters: Huawei Cloud AK, Huawei Cloud account name.
Click [Deploy Startup Script], the system will automatically create a Startup
script set and automatically configure the corresponding startup scripts.
After the script is installed, find the script in Func "Development" 「TrueWatch Integration (Huawei Cloud-RDS Collection)」/「TrueWatch Integration (Huawei Cloud-RDS Slow Query Detail Log Collection)」/「TrueWatch Integration (Huawei Cloud-RDS Slow Query Statistical Log Collection)」, expand and modify this script, find collector_configs
and monitor_configs
and edit the content under region_projects
respectively, change the region and Project ID to the actual region and Project ID, then click save and publish.
In addition, you can see the corresponding automatic trigger configuration in "Management / Automatic Trigger Configuration". Click [Execute] to execute it immediately without waiting for the scheduled time. After a while, you can check the execution task record and corresponding logs.
Verification¶
- In "Management / Automatic Trigger Configuration", confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task record and logs to check for any exceptions
- In TrueWatch, "Infrastructure - Resource Catalog", check whether asset information exists
- In TrueWatch, "Metrics", check whether there is corresponding monitoring data
- In TrueWatch, "Logs", check whether there is corresponding slow log data
Metrics¶
Configure Huawei Cloud RDS MYSQL monitoring Metrics, you can collect more Metrics by configuration Huawei Cloud RDS MYSQL Metrics Details
Instance Monitoring Metrics¶
RDS for MySQL instance performance monitoring Metrics, as shown in the table below.
Metric ID | Metric Name | Metric Meaning | Value Range | Monitoring Period (Original Metric) |
---|---|---|---|---|
rds001_cpu_util |
CPU Utilization | This metric is used to statistically measure the CPU utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds002_mem_util |
Memory Utilization | This metric is used to statistically measure the memory utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds003_iops |
IOPS | This metric is used to statistically measure the number of I/O requests processed by the system per unit time (average). | ≥ 0 counts/s | 1 minute |
rds004_bytes_in |
Network Input Throughput | This metric is used to statistically measure the average input traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds005_bytes_out |
Network Output Throughput | This metric is used to statistically measure the average output traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds006_conn_count |
Total Database Connections | This metric is used to statistically measure the total number of connections attempting to connect to the MySQL server, in counts. | ≥ 0 counts | 1 minute5 seconds1 second |
rds007_conn_active_count |
Current Active Connections | This metric is used to statistically measure the number of currently open connections, in counts. | ≥ 0 counts | 1 minute5 seconds1 second |
rds008_qps |
QPS | This metric is used to statistically measure the number of SQL statement queries, including stored procedures, in queries/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds009_tps |
TPS | This metric is used to statistically measure the number of transactions executed, including committed and rolled back, in transactions/second. | ≥ 0 transactions/s | 1 minute5 seconds1 second |
rds010_innodb_buf_usage |
Buffer Pool Utilization | This metric is used to statistically measure the ratio of free pages to the total number of buffer pool pages in the InnoDB cache, in percentage. | 0-1 | 1 minute |
rds011_innodb_buf_hit |
Buffer Pool Hit Rate | This metric is used to statistically measure the ratio of read hits to read requests, in percentage. | 0-1 | 1 minute |
rds012_innodb_buf_dirty |
Buffer Pool Dirty Block Rate | This metric is used to statistically measure the ratio of dirty data in the InnoDB cache to the used pages in the InnoDB cache, in percentage. | 0-1 | 1 minute |
rds013_innodb_reads |
InnoDB Read Throughput | This metric is used to statistically measure the average number of bytes read per second by Innodb, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds014_innodb_writes |
InnoDB Write Throughput | This metric is used to statistically measure the average number of bytes written per second by Innodb, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds015_innodb_read_count |
InnoDB File Read Frequency | This metric is used to statistically measure the average number of reads from the file per second by Innodb, in counts/second. | ≥ 0 counts/s | 1 minute |
rds016_innodb_write_count |
InnoDB File Write Frequency | This metric is used to statistically measure the average number of writes to the file per second by Innodb, in counts/second. | ≥ 0 counts/s | 1 minute |
rds017_innodb_log_write_req_count |
InnoDB Log Write Request Frequency | This metric is used to statistically measure the average number of log write requests per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds018_innodb_log_write_count |
InnoDB Log Physical Write Frequency | This metric is used to statistically measure the average number of physical writes to the log file per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds019_innodb_log_fsync_count |
InnoDB Log fsync() Write Frequency | This metric is used to statistically measure the average number of fsync() writes completed to the log file per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds020_temp_tbl_rate |
Temporary Table Creation Rate | This metric is used to statistically measure the number of temporary tables created on the hard disk per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds021_myisam_buf_usage |
Key Buffer Utilization | This metric is used to statistically measure the utilization of MyISAM Key buffer, in percentage. | 0-1 | 1 minute |
rds022_myisam_buf_write_hit |
Key Buffer Write Hit Rate | This metric is used to statistically measure the write hit rate of MyISAM Key buffer, in percentage. | 0-1 | 1 minute |
rds023_myisam_buf_read_hit |
Key Buffer Read Hit Rate | This metric is used to statistically measure the read hit rate of MyISAM Key buffer, in percentage. | 0-1 | 1 minute |
rds024_myisam_disk_write_count |
MyISAM Disk Write Frequency | This metric is used to statistically measure the number of writes to the disk index, in counts/second. | ≥ 0 counts/s | 1 minute |
rds025_myisam_disk_read_count |
MyISAM Disk Read Frequency | This metric is used to statistically measure the number of reads from the disk index, in counts/second. | ≥ 0 counts/s | 1 minute |
rds026_myisam_buf_write_count |
MyISAM Buffer Pool Write Frequency | This metric is used to statistically measure the number of write requests to the buffer pool index, in counts/second. | ≥ 0 counts/s | 1 minute |
rds027_myisam_buf_read_count |
MyISAM Buffer Pool Read Frequency | This metric is used to statistically measure the number of read requests from the buffer pool index, in counts/second. | ≥ 0 counts/s | 1 minute |
rds028_comdml_del_count |
Delete Statement Execution Frequency | This metric is used to statistically measure the average number of Delete statement executions per second, in queries/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds029_comdml_ins_count |
Insert Statement Execution Frequency | This metric is used to statistically measure the average number of Insert statement executions per second, in queries/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds030_comdml_ins_sel_count |
Insert_Select Statement Execution Frequency | This metric is used to statistically measure the average number of Insert_Select statement executions per second, in queries/second. | ≥ 0 queries/s | 1 minute |
rds031_comdml_rep_count |
Replace Statement Execution Frequency | This metric is used to statistically measure the average number of Replace statement executions per second, in queries/second. | ≥ 0 queries/s | 1 minute |
rds032_comdml_rep_sel_count |
Replace_Selection Statement Execution Frequency | This metric is used to statistically measure the average number of Replace_Selection statement executions per second, in queries/second. | ≥ 0 queries/s | 1 minute |
rds033_comdml_sel_count |
Select Statement Execution Frequency | This metric is used to statistically measure the average number of Select statement executions per second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds034_comdml_upd_count |
Update Statement Execution Frequency | This metric is used to statistically measure the average number of Update statement executions per second, in queries/second. | ≥ 0 queries/s | 1 minute5 seconds1 second |
rds035_innodb_del_row_count |
Row Deletion Rate | This metric is used to statistically measure the average number of rows deleted from the InnoDB table per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds036_innodb_ins_row_count |
Row Insertion Rate | This metric is used to statistically measure the average number of rows inserted into the InnoDB table per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds037_innodb_read_row_count |
Row Read Rate | This metric is used to statistically measure the average number of rows read from the InnoDB table per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds038_innodb_upd_row_count |
Row Update Rate | This metric is used to statistically measure the average number of rows updated in the InnoDB table per second, in rows/second. | ≥ 0 rows/s | 1 minute |
rds039_disk_util |
Disk Utilization | This metric is used to statistically measure the disk utilization of the measurement object, in percentage. | 0-100% | 1 minute |
rds047_disk_total_size |
Total Disk Size | This metric is used to statistically measure the total disk size of the measurement object. | 40GB~4000GB | 1 minute |
rds048_disk_used_size |
Disk Usage | This metric is used to statistically measure the disk usage size of the measurement object. | 0GB~4000GB | 1 minute |
rds049_disk_read_throughput |
Disk Read Throughput | This metric is used to statistically measure the number of bytes read from the disk per second. | ≥ 0 bytes/s | 1 minute |
rds050_disk_write_throughput |
Disk Write Throughput | This metric is used to statistically measure the number of bytes written to the disk per second. | ≥ 0 bytes/s | 1 minute |
rds072_conn_usage |
Connection Usage Rate | This metric is used to statistically measure the percentage of currently used MySQL connections to the total number of connections. | 0-100% | 1 minute |
rds173_replication_delay_avg |
Average Replication Delay | This metric is the average delay between the standby or read-only and the master, corresponding to seconds_behind_master. The average value of the 60-second time period is taken. | ≥ 0s | 10 seconds |
rds073_replication_delay |
Real-time Replication Delay | This metric is the real-time delay between the standby or read-only and the master, corresponding to seconds_behind_master. This value is real-time. | ≥ 0s | 1 minute5 seconds |
rds074_slow_queries |
Slow Log Count Statistics | This metric is used to display the number of slow logs generated by MySQL per minute. | ≥ 0 | 1 minute |
rds075_avg_disk_ms_per_read |
Disk Read Time | This metric is used to statistically measure the average time taken to read the disk during a certain period. | ≥ 0ms | 1 minute |
rds076_avg_disk_ms_per_write |
Disk Write Time | This metric is used to statistically measure the average time taken to write to the disk during a certain period. | ≥ 0ms | 1 minute |
rds077_vma |
VMA Count | Monitor the virtual memory area size of the RDS process, in counts. | ≥ 0counts | 1 minute |
rds078_threads |
Number of Threads in Process | Monitor the number of threads in the RDS process, in counts. | ≥ 0counts | 1 minute |
rds079_vm_hwm |
Peak Physical Memory Usage of Process | Monitor the peak physical memory usage of the RDS process, in KB. | ≥ 0 KB | 1 minute |
rds080_vm_peak |
Peak Virtual Memory Usage of Process | Monitor the peak virtual memory usage of the RDS process, in KB. | ≥ 0 KB | 1 minute |
rds081_vm_ioutils |
Disk I/O Utilization | Disk I/O utilization, in percentage. | 0-1 | 1 minute |
rds082_semi_sync_tx_avg_wait_time |
Average Transaction Wait Time | Monitor the average wait time in semi-synchronous replication mode, in microseconds. | ≥ 0 microseconds | 1 minute |
sys_swap_usage |
Swap Utilization | This metric is used to statistically measure the swap utilization of the measurement object, in percentage. | 0-100% | 1 minute |
rds_innodb_lock_waits |
Row Lock Wait Count | This metric is used to statistically measure the number of row lock waits in Innodb, in counts. It represents the cumulative number of transactions waiting for row locks historically. Restarting will clear lock waits. | ≥ 0 counts | 1 minute |
rds_bytes_recv_rate |
Database Bytes Received Per Second | This metric is used to statistically measure the number of bytes received by the database per second, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_bytes_sent_rate |
Database Bytes Sent Per Second | This metric is used to statistically measure the number of bytes sent by the database per second, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_innodb_pages_read_rate |
Innodb Average Data Read Per Second | This metric is used to statistically measure the average amount of data read per second by Innodb, in pages/second. | ≥ 0 Pages/s | 1 minute |
rds_innodb_pages_written_rate |
Innodb Average Data Written Per Second | This metric is used to statistically measure the average amount of data written per second by Innodb, in pages/second. | ≥ 0 Pages/s | 1 minute |
rds_innodb_os_log_written_rate |
Average Redo Log Written Per Second | This metric is used to statistically measure the average amount of redo log written per second, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_innodb_buffer_pool_read_requests_rate |
innodb_buffer_pool Read Requests Per Second | This metric is used to statistically measure the number of read requests per second to innodb_buffer_pool, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_innodb_buffer_pool_write_requests_rate |
innodb_buffer_pool Write Requests Per Second | This metric is used to statistically measure the number of write requests per second to innodb_buffer_pool, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_innodb_buffer_pool_pages_flushed_rate |
innodb_buffer_pool Pages Flushed Per Second | This metric is used to statistically measure the number of pages flushed per second in innodb_buffer_pool, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_innodb_log_waits_rate |
Number of Waits Due to Log Buffer Insufficiency | This metric is used to statistically measure the number of waits per second due to insufficient log buffer causing flush to disk, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_created_tmp_tables_rate |
Temporary Tables Created Per Second | This metric is used to statistically measure the number of temporary tables created per second, in counts/second. | ≥ 0 counts/s | 1 minute |
rds_wait_thread_count |
Number of Waiting Threads | This metric is used to statistically measure the number of waiting threads, in counts. | ≥ 0 counts | 1 minute |
rds_innodb_row_lock_time_avg |
Historical Row Lock Average Wait Time | This metric is used to statistically measure the historical average wait time for row locks in innodb. | > 0ms | 1 minute |
rds_innodb_row_lock_current_waits |
Current Row Lock Wait Count | This metric is used to statistically measure the current number of row lock waits in innodb, in counts. It represents the number of transactions currently waiting for row locks. | ≥ 0 counts | 1 minute |
rds_mdl_lock_count |
MDL Lock Count | This metric is used to statistically measure the number of MDL locks, in counts. | ≥ 0counts | 1 minute |
rds_buffer_pool_wait_free |
Number of Dirty Pages Waiting to Be Flushed | This metric statistically measures the number of dirty pages waiting to be flushed, in counts. | ≥ 0counts | 1 minute |
rds_conn_active_usage |
Active Connection Usage Rate | This metric statistically measures the ratio of active connections to the maximum number of connections, in percentage. | 0-100% | 1 minute |
rds_innodb_log_waits_count |
Log Wait Count | This metric is used to statistically measure the number of log waits, in counts. | ≥ 0counts | 1 minute |
rds_long_transaction |
Long Transaction Metric | This metric statistically measures the time consumption data of long transactions, in seconds. Related operation commands are considered as a complete long transaction only if there are BEGIN and COMMIT commands before and after. | ≥ 0seconds | 1 minute |
RDS for MySQL database proxy monitoring Metrics, as shown in Table 2.
Metric ID | Metric Name | Metric Meaning | Value Range | Monitoring Period (Original Metric) |
---|---|---|---|---|
rds001_cpu_util | CPU Utilization | This metric is used to statistically measure the CPU utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds002_mem_util | Memory Utilization | This metric is used to statistically measure the memory utilization of the measurement object, in percentage. | 0-100% | 1 minute5 seconds1 second |
rds004_bytes_in | Network Input Throughput | This metric is used to statistically measure the average input traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds005_bytes_out | Network Output Throughput | This metric is used to statistically measure the average output traffic per second from all network adapters of the measurement object, in bytes/second. | ≥ 0 bytes/s | 1 minute |
rds_proxy_frontend_connections | Frontend Connections | The number of connections between the application and the Proxy. | ≥ 0 counts | 1 minute |
rds_proxy_backend_connections | Backend Connections | The number of connections between the Proxy and the RDS database. | ≥ 0 counts | 1 minute |
rds_proxy_average_response_time | Average Response Time | Average response time. | ≥ 0 ms | 1 minute |
rds_proxy_query_per_seconds | QPS | SQL statement query count. | ≥ 0 counts | 1 minute |
rds_proxy_read_query_proportions | Read Proportion | The proportion of read requests to total requests. | 0-100% | 1 minute |
rds_proxy_write_query_proportions | Write Proportion | The proportion of write requests to total requests. | 0-100% | 1 minute |
rds_proxy_frontend_connection_creation | Average Frontend Connections Created Per Second | Statistically measure the average number of frontend connections created per second by the client application for the database proxy service. | ≥ 0 counts/s | 1 minute |
rds_proxy_transaction_query | Average Queries in Transactions Per Second | Statistically measure the average number of select executions per second in transactions. | ≥ 0 counts/s | 1 minute |
rds_proxy_multi_statement_query | Average Multi-Statements Executed Per Second | Statistically measure the average number of Multi-Statements executed per second. | ≥ 0 counts/s | 1 minute |
Object¶
The collected Huawei Cloud RDS MYSQL object data structure can be seen in "Infrastructure - Resource Catalog"
{
"measurement": "huaweicloud_rds",
"tags": {
"RegionId" : "cn-north-4",
"project_id" : "c631f046252dxxxxxxxf253c62d48585",
"enterprise_project_id" : "d13ebb59-d4fe-xxxx-xxxx-zc22bcea6f987",
"instance_id" : "1d0c91561f4644dxxxxxxd68304b0520din01",
"instance_name" : "rds-df54-xxxx",
"status" : "ACTIVE",
"type" : "Single",
"engine" : "MySQL"
},
"fields": {
"engine_version" : "5.7",
"port" : "3306",
"security_group_id" : "d13ebb59-d4fe-xxxx-xxxx-c22bcea6f987",
"switch_strategy" : "xxx",
"enable_ssl" : "False",
"time_zone" : "UTC+08:00",
"created_time" : "2022-06-21T06:17:27+0000",
"updated_time" : "2022-06-21T06:20:03+0000",
"expiration_time" : "2022-06-23T06:20:03+0000",
"alias" : "xxx",
"private_ips" : "[\"192.xxx.x.144\"]",
"public_ips" : "[]",
"cpu" : "2",
"mem" : "4",
"volume" : "{volume information}",
"nodes" : "[{primary and standby instance information}]",
"related_instance" : "[]",
"backup_strategy" : "{database information}"
}
}
Note: The fields in
tags
andfields
may change with subsequent updatesNote 1: The value of
tags.insatnce_id
is the instance ID, used as a unique identifierNote 2: The following fields are all JSON serialized strings
fields.message
fields.private_ips
fields.public_ips
fields.volume
fields.nodes
fields.related_instance
fields.backup_strategy
Note 3: The value of
type
is "Single", "Ha", "Replica", or "Enterprise", corresponding to single instance, primary-standby instance, read-only instance, and distributed instance (enterprise edition) respectively.