Tencent Cloud MySQL¶
Use the "TrueWatch Cloud Sync" series script packages in the Script Market to synchronize cloud monitoring and cloud asset data to TrueWatch
Configuration¶
Install Func¶
It is recommended to activate the TrueWatch Integration - Extensions - DataFlux Func (Automata): all prerequisites are automatically installed. Please proceed with the script installation.
If you want to deploy Func yourself, refer to Self-deploying Func
Install Script¶
Note: Please prepare the required Tencent Cloud AK in advance (for simplicity, you can directly grant global read-only permission
ReadOnlyAccess)
To synchronize Tencent Cloud MySQL monitoring data, we install the corresponding collection script: (ID: integration_tencentcloud_mysql)
Click 【Install】, then enter the corresponding parameters: Tencent Cloud AK, Tencent Cloud account name.
Click 【Deploy Startup Script】, the system will automatically create the Startup script set and configure the corresponding startup scripts.
Additionally, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute it once without waiting for the scheduled time. Wait a moment, and you can view the execution task records and corresponding logs.
We have collected some configurations by default, see the Metrics section for details Configure Custom Cloud Object Metrics
Verification¶
- In 「Manage / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and check the corresponding task records and logs for any exceptions
- In TrueWatch, check if the asset information exists in 「Infrastructure / Custom」
- In TrueWatch, check if the corresponding monitoring data exists in 「Metrics」
Metrics¶
After configuring Tencent Cloud Cloud Monitoring, the default Measurement is as follows. More Metrics can be collected through configuration Tencent Cloud Cloud Monitoring Metrics Details
Resource Monitoring¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
BytesReceived |
Internal Inbound Traffic | Bytes received per second | Bytes/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
BytesSent |
Internal Outbound Traffic | Bytes sent per second | Bytes/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
Capacity |
Disk Usage Space | Includes MySQL data directory and binlog、relaylog、undolog、errorlog、slowlog log space |
MB | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
CpuUseRate |
CPU Utilization | Allows overuse during idle times, CPU utilization may exceed 100% | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
IOPS |
IOPS | Input/output operations per second (or read/write operations) | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
MemoryUse |
Memory Usage | Allows overuse during idle times, actual memory usage may exceed the purchased specification | MB | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
MemoryUseRate |
Memory Utilization | Allows overuse during idle times, memory utilization may exceed 100% | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
RealCapacity |
Data Usage Space | Only includes MySQL data directory, excluding binlog、relaylog、undolog、errorlog、slowlog log space |
MB | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
VolumeRate |
Disk Utilization | Disk usage space/instance purchased space | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
Engine Monitoring (General)- MyISAM¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
KeyCacheHitRate |
myisam Cache Hit Rate | myisam engine cache hit rate | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
KeyCacheUseRate |
myisam Cache Usage Rate | myisam engine cache usage rate | % | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (General)- InnoDB¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbCacheHitRate |
innodb Cache Hit Rate | Innodb engine cache hit rate | % | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbCacheUseRate |
innodb Cache Usage Rate | Innodb engine cache usage rate | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbNumOpenFiles |
Current InnoDB Open Tables Count | Innodb engine current open tables count | Count | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbOsFileReads |
innodb Disk Read Count | Innodb engine disk read file count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbOsFileWrites |
innodb Disk Write Count | Innodb engine disk write file count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbOsFsyncs |
innodbfsyn Count | Innodb engine fsync function call count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
Engine Monitoring (General)- Connections¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ConnectionUseRate |
Connection Utilization | Current open connections/maximum connections | % | InstanceId、InstanceType（Optional） | 5s、10s、60s、300s、3600s |
MaxConnections |
Maximum Connections | Maximum connections | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Qps |
Queries Per Second | Database SQL operations per second (including insert、select、update、delete、replace），QPS mainly reflects the actual processing capability of TencentDB instances | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ThreadsConnected |
Current Connections | Current open connections count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Tps |
Transactions Per Second | Database transactions processed per second | Transactions/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (General)- Access¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ComDelete |
Delete Count | Delete operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ComInsert |
Insert Count | Insert operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ComReplace |
Replace Count | Replace operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、6400s |
ComUpdate |
Update Count | Update operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Queries |
Total Access Volume | All executed SQL statements, including set, show, etc. | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
QueryRate |
Access Volume Ratio | Queries per second QPS/recommended queries per second | % | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SelectCount |
Select Count | Select operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SelectScan |
Full Table Scan Count | Full table search query count | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SlowQueries |
Slow Query Count | Query count exceeding long_query_time seconds | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (General)- Tables¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
CreatedTmpTables |
Memory Temporary Table Count | Temporary tables created per second | Count/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
TableLocksWaited |
Table Lock Wait Count | Table lock wait count | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
OpenedTables |
Opened Table Count | Engine opened table count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
TableLocksImmediate |
Immediate Table Lock Release Count | Engine immediate table lock release count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Tmp¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
CreatedTmpDiskTables |
Disk Temporary Table Count | Disk temporary tables created per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
CreatedTmpFiles |
Temporary File Count | Temporary files created per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Key¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
KeyBlocksUnused |
Unused Key Cache Blocks Count | myisam engine unused key cache blocks count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyBlocksUsed |
Used Key Cache Blocks Count | myisam engine used key cache blocks count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyReadRequests |
Key Cache Read Operations Count | myisam engine key cache read operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyReads |
Disk Read Operations Count | myisam engine disk read operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyWriteRequests |
Key Cache Write Operations Count | myisam engine key cache write operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyWrites |
Disk Write Operations Count | myisam engine disk write operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- InnoDB Row¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbRowLockTimeAvg |
InnoDB Average Row Lock Time (ms) | Innodb engine row lock average duration | ms | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowLockWaits |
InnoDB Row Lock Wait Count | Innodb engine row lock wait count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsDeleted |
InnoDB Row Delete Count | Innodb engine row delete count per second | Rows/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsInserted |
InnoDB Row Insert Count | Innodb engine row insert count per second | Rows/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsRead |
InnoDB Row Read Count | Innodb engine row read count per second | Rows/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsUpdated |
InnoDB Row Update Count | Innodb engine row update count per second | Rows/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、86400s |
Engine Monitoring (Extended)- InnoDB Data¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbDataRead |
InnoDB Read Volume | Innodb engine read data bytes per second | Bytes/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbDataReads |
InnoDB Total Read Volume | Innodb engine read data count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbDataWrites |
InnoDB Total Write Volume | Innodb engine write data count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbDataWritten |
InnoDB Write Volume | Innodb engine write data bytes per second | Bytes/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Handler¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
HandlerCommit |
Internal Commit Count | Internal commit operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
HandlerReadRndNext |
Read Next Row Request Count | Read next row request count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
HandlerRollback |
Internal Rollback Count | Internal rollback operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Buff¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbBufferPoolPagesFree |
InnoDB Free Page Count | Innodb engine memory free page count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s |
InnodbBufferPoolPagesTotal |
InnoDB Total Page Count | Innodb engine memory total page count | Count | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbBufferPoolReadRequests |
Innodb Buffer Pool Pre-read Page Count | Innodb engine logical read request count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbBufferPoolReads |
Innodb Disk Read Page Count | Innodb engine physical read request count per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
Engine Monitoring (Extended)- Others¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
LogCapacity |
Log Usage | Engine log usage count | MB | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s |
OpenFiles |
Open File Count | Engine open file count | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600、86400s |
Engine Monitoring (Extended)- Connections¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ThreadsCreated |
Created Thread Count | Threads created to handle connections | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ThreadsRunning |
Running Thread Count | Active (non-sleeping) thread count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Access¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ComCommit |
Commit Count | Commit operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ComRollback |
Rollback Count | Rollback operations per second | Operations/second | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Database Proxy (Proxy)¶
| Metric Name | Metric Chinese Name | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|
ProxyCpuUseRate |
CPU Utilization | % | InstanceId、InstanceType | 5s、60s、300s、3600s、86400s |
ProxyCurrentConnections |
Current Connection Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyMemoryUseRate |
Memory Utilization | % | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyQueries |
Request Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyRouteMaster |
Write Request Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyRouteSlave |
Read Request Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyMemoryUse |
Memory Usage | MB | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
Deployment Monitoring (Slave)¶
| Metric Name | Metric Chinese Name | Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
MasterSlaveSyncDistance |
Master-Slave Sync Distance | Master-slave binlog gap | MB | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SecondsBehindMaster |
Master-Slave Sync Delay Time | Master-slave sync delay time | Seconds | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SlaveIoRunning |
IO Thread Status | IO thread running status | Status value（0-Yes，1-No，2-Connecting） | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SlaveSqlRunning |
SQL Thread Status | SQL thread running status | Status value（0-Yes，1-No） | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Object¶
The collected Tencent Cloud MySQL object data structure can be seen in 「Infrastructure - Custom」
{
"measurement": "tencentcloud_cdb",
"tags": {
"name" : "cdb-xxxxxxx",
"RegionId" : "ap-shanghai",
"Region" : "ap-shanghai",
"InstanceId" : "cdb-imxxxx",
"InstanceName" : "smart_check_test",
"InstanceType" : "1",
"Zone" : "ap-shanghai-3",
"ZoneName" : "",
"DeviceType" : "UNIVERSAL",
"EngineVersion": "8.0",
"Vip" : "172.xx.x.9",
"Status" : "1",
"ProtectMode" : "0",
"ProjectId" : "0",
"PayType" : "1",
"WanStatus" : "0"
},
"fields": {
"WanPort" : 0,
"Memory" : 1000,
"Volume" : 25,
"DeadlineTime": "0000-00-00 00:00:00",
"CreateTime" : "2022-04-27 15:18:06",
"message" : "{Instance JSON Data}"
}
}
Logging¶
Slow Query Statistics¶
Prerequisites¶
Note 1: Before using this collector, you must install the "TrueWatch Integration Core Package" and its accompanying third-party dependencies
Note 2: The code execution of this script depends on the MySQL instance object collection. If the custom object collection for MySQL is not configured, the slow log script cannot collect slow log data
Install Script¶
On top of the previous steps, you need to install a corresponding MySQL Slow Query Statistics Log Collection Script
In 「Manage / Script Market」, click and install the corresponding script package:
- ID:
integration_tencentcloud_mysql_slowlog
After the data is synchronized normally, you can view the data in TrueWatch's 「Logging」.
The reported data example is as follows:
{
"measurement": "tencentcloud_cdb_slow_log",
"tags": {
"name" : "cdb-llxxxxx",
"Database" : "test",
"UserHost" : "9.xxx.xxx.122",
"UserName" : "root",
"InstanceId" : "cdb-lxxxxtk8",
"DeviceType" : "UNIVERSAL",
"EngineVersion" : "8.0",
"InstanceName" : "cdbxxxxx",
"InstanceType" : "1",
"PayType" : "1",
"ProjectId" : "0",
"ProtectMode" : "0",
"Region" : "ap-shanghai",
"RegionId" : "ap-shanghai",
"Status" : "1",
"Vip" : "172.xx.xxx.15",
"WanStatus" : "0",
"Zone" : "ap-shanghai-3",
"account_name" : "Script Development Tencent Account",
"cloud_provider": "tencentcloud"
},
"fields": {
"QueryTime" : 3.000195,
"SqlText" : "select sleep(3)",
"Timestamp" : 1652933796,
"LockTime" : 0,
"RowsExamined": 1,
"RowsSent" : 1,
"SqlTemplate" : "select sleep(?);",
"Md5" : "26A15F1AE530F28F",
"message" : "{Instance JSON Data}"
}
}
Some parameter descriptions are as follows
| Field | Type | Description |
|---|---|---|
QueryTime |
float | SQL execution duration (seconds) |
Timestamp |
integer | SQL execution time |
Md5 |
str | SQL statement MD5 |
LockTime |
float | Lock duration (seconds) |
RowsExamined |
integer | Scanned rows |
RowsSent |
integer | Result set rows |
Note: Fields in
tags、fieldsmay change with subsequent updatesNote 1:
tags.namevalue is the instance ID, used as a unique identifierNote 2:
fields.messageis the JSON serialized string ```