Skip to content

Huawei Cloud Data Migration CDM

Collect Huawei Cloud CDM monitoring metrics

Configuration

Install Func

It is recommended to activate TrueWatch Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation

For self-deployment of Func, refer to Self-deployment Func

Install Script

Note: Please prepare the Huawei Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission ReadOnlyAccess)

  1. Log in to the Func console, click 【Script Market】, enter the TrueWatch script market, and search for integration_huaweicloud_cdm

  2. Click 【Install】, then enter the corresponding parameters: Huawei Cloud AK, SK, and account name

  3. Click 【Deploy Startup Script】, the system will automatically create the Startup script set and configure the corresponding startup scripts

  4. After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to execute immediately without waiting for the scheduled time. Wait for a moment, you can check the execution task records and corresponding logs

Verification

  1. In 「Manage / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and check the corresponding task records and logs for any exceptions
  2. In TrueWatch, check if asset information exists in 「Infrastructure - Resource Catalog」
  3. In TrueWatch, check if there are corresponding monitoring data in 「Metrics」

Metrics

Collect Huawei Cloud Data Migration CDM metrics, more metrics can be collected through configuration Huawei Cloud CDM Metrics Details

Huawei Cloud CDM metrics are under the huaweicloud_SYS.CDM measurement, below are the CDM related metrics descriptions

Metric ID Metric Name Metric Meaning Value Range Unit Measurement Object (Dimension) Monitoring Period (Original Metric)
bytes_in Network Inbound Rate This metric is used to count the network traffic per second flowing into the measurement object. ≥ 0 bytes/s bytes/s CDM cluster instance 1 minute
bytes_out Network Outbound Rate This metric is used to count the network traffic per second flowing out of the measurement object. ≥ 0 bytes/s bytes/s CDM cluster instance 1 minute
cpu_usage CPU Usage This metric is used to count the CPU usage of the measurement object. 0%~100% % CDM cluster instance 1 minute
mem_usage Memory Usage This metric is used to count the memory usage of the measurement object. 0%~100% % CDM cluster instance 1 minute
pg_pending_job Queued Jobs This metric is used to count the number of jobs in PENDING state in this CDM instance. >=0 Count CDM cluster instance 1 minute
pending_threads Queued Extraction Concurrency This metric is used to count the number of extraction concurrency threads in Waiting state in this CDM instance. ≥ 0 Count CDM cluster instance 1 minute
disk_usage Disk Utilization This metric is the disk usage rate collected from the physical machine level, the data accuracy is lower than that collected from the elastic cloud server internal. 0.001%~90% % CDM cluster instance 1 minute
disk_io Disk IO This metric is the number of bytes read and written per second collected from the physical machine level, the data accuracy is lower than that collected from the elastic cloud server internal. ≥ 0 byte/s byte/s CDM cluster instance 1 minute
tomcat_heap_usage Heap Memory Usage This metric is the heap memory usage rate collected from the physical machine level, the data accuracy is lower than that collected from the elastic cloud server internal. 0.001%~90% % CDM cluster instance 1 minute
tomcat_connect Tomcat Concurrent Connections This metric is the number of concurrent connections of Tomcat collected from the physical machine level. 0~2147483647 Count CDM cluster instance 1 minute
tomcat_thread_count Tomcat Threads This metric is the number of threads occupied by Tomcat collected from the physical machine level. 0~2147483647 Count CDM cluster instance 1 minute
pg_connect Database Connections This metric is the number of connections to the postgres database collected from the physical machine level. 0~2147483647 Count CDM cluster instance 1 minute
pg_submission_row Historical Record Table Rows This metric is the number of rows in the submission table of the postgres database collected from the physical machine level. 0~2147483647 Count CDM cluster instance 1 minute
pg_failed_job_rate Failed Job Rate This metric is the failed job rate collected from the sqoop process at the physical machine level. 0.001%~100% % CDM cluster instance 1 minute
inodes_usage Inodes Utilization This metric is the disk inodes usage rate collected from the physical machine level, the data accuracy is lower than that collected from the elastic cloud server internal. 0.001%~100% % CDM cluster instance 1 minute

Object

After data synchronization is normal, you can view the data in TrueWatch's 「Infrastructure - Resource Catalog」.

{
  "measurement": "huaweicloud_cdm",
  "tags": {
    "RegionId"                : "cn-south-4",
    "project_id"              : "756ada1aa17e4049b2a16ea41912e52d",
    "enterprise_project_id"   : "0824ss-xxxx-xxxx-xxxx-12334fedffg",
    "cluster_id"              : "xxxxxx",
    "cluster_name"            : "test",
    "instance_name"           : "test-xxxx",
    "instance_id"             : "xxxxxxxxxx",
    "status"                  : "AVAILABLE"
  },
  "fields": {
    "azName"                        : "xszxxxxxx",
    "dbuser"                        : "root",
    "flavorName"                    : "xxxx",
    "clusterMode"                   : "xxxxx",
    "publicEndpoint"                : "cn-south-4a",
    "statusDetail"                  : "cdmk2-xxx-xxx",
    "create_time"                   : "2024-11-11T08:15:58Z",
    "update_time"                   : "2024-11-11T08:15:58Z",
    "recentEvent"                   : "xxxxx",
    "datastore"                     : "xxx",
    "isAutoOff"                     : "xxx",
    "task"                          : "xxx",
    "actionProgress"                : "xxx"
  }
}

Note: The fields in tags, fields may change with subsequent updates ```