Skip to content

Tencent Cloud Elasticsearch Service

Collect metrics data from Tencent Cloud Elasticsearch Service

Configuration

Install Func

It is recommended to activate TrueWatch Integration - Extensions - DataFlux Func (Automata): all prerequisites are automatically installed. Please proceed with the script installation.

For self-deployment of Func, refer to Self-deploy Func

Install Script

Note: Please prepare the Tencent Cloud AK with the required permissions in advance (for simplicity, you can grant the global read-only permission ReadOnlyAccess).

  1. Log in to the Func console, click on 【Script Market】, enter the TrueWatch Script Market, and search for: integration_tencentcloud_es

  2. Click on 【Install】, then enter the corresponding parameters: Tencent Cloud AK ID, AK Secret, and account name.

  3. Click on 【Deploy Startup Script】, the system will automatically create the Startup script set and configure the corresponding startup scripts.

  4. After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click on 【Execute】 to immediately execute it once without waiting for the scheduled time. After a short while, you can check the execution task records and corresponding logs.

Verification

  1. Confirm in 「Manage / Automatic Trigger Configuration」 whether the corresponding task has the automatic trigger configuration, and check the task records and logs for any anomalies.
  2. In TrueWatch, check if the asset information exists in 「Infrastructure - Resource Catalog」.
  3. In TrueWatch, check if there is corresponding monitoring data in 「Metrics」.

Metrics

After configuring Tencent Cloud Cloud Monitor, the default Measurement is as follows. You can collect more metrics through configuration. Tencent Cloud Cloud Monitor Metrics Details

Tencent Cloud ES metrics are under the tencentcloud_QCE/CES Measurement. Below are the related metric descriptions:

Metric Name Metric Chinese Name Metric Description Unit Statistics Granularity
IsReadOnly Cluster Read-Only Status Whether the cluster is read-only None 5s, 10s, 60s, 300s, 3600s, 86400s
NodeBulkSpeed Node Single Cycle Bulk Speed Node Single Cycle Bulk Speed Count/s 5s, 10s, 60s, 300s, 3600s, 86400s
NodeCpuLoad1min Node CPU 1 Minute Load Node CPU 1 Minute Load None 5s, 10s, 60s, 300s, 3600s, 86400s
NodeIndexLatency Node Single Cycle Write Average Latency Node Single Cycle Write Average Latency ms 5s, 10s, 60s, 300s, 3600s, 86400s
NodeIndexSpeed Node Write Speed Node Write Speed Count/s 5s, 10s, 60s, 300s, 3600s, 86400s
NodeJvmMemUsage Node JVM Memory Usage Node JVM Memory Usage % 5s, 10s, 60s, 300s, 3600s, 86400s
NodeJvmOldMemUsage JVM_Old Area Memory Usage JVM_Old Area Memory Usage % 5s, 10s, 60s, 300s, 3600s, 86400s
NodeMemUsage Node Memory Usage Node Memory Usage % 5s, 10s, 60s, 300s, 3600s, 86400s
NodeOldGcDif Node Single Cycle OldGC Count Node Single Cycle OldGC Count Count 5s, 10s, 60s, 300s, 3600s, 86400s
NodeSearchLatency Node Single Cycle Query Average Latency Node Single Cycle Query Average Latency ms 5s, 10s, 60s, 300s, 3600s, 86400s
NodeStatus Node Health Status Node Health Status None 5s, 10s, 60s, 300s, 3600s, 86400s
ShardNum Cluster Shard Count Cluster Shard Count Count 5s, 10s, 60s, 300s, 3600s, 86400s
Status Cluster Health Status Cluster Health Status: 0: Green 1: Yellow 2: Red % 5s, 10s, 60s, 300s, 3600s, 86400s
ClusterSlowSearchCount1min Cluster 1min Slow Query Count Cluster 1min Slow Query Count Count 5s, 10s, 60s, 300s, 3600s, 86400s
ForceMergeQueue ForceMerge Thread Queue Count ForceMerge Thread Queue Count Count 5s, 10s, 60s, 300s, 3600s, 86400s
NodeCpuLoad1minRate Node CPU 1 Minute Load Rate Node CPU 1 Minute Load Rate % 5s, 10s, 60s, 300s, 3600s, 86400s
QueryCacheHitRate Query Cache Hit Rate Query Cache Hit Rate % 5s, 10s, 60s, 300s, 3600s, 86400s
ShardNumLimitPercen Cluster Shard Usage Rate Cluster Shard Usage Rate % 5s, 10s, 60s, 300s, 3600s, 86400s
DiskAwait Operation Wait Time Operation Wait Time ms 5s, 10s, 60s, 300s, 3600s, 86400s
DiskIoutil Disk IO Util Disk IO Util % 5s, 10s, 60s, 300s, 3600s, 86400s
DiskIps Writes Per Second Writes Per Second None 5s, 10s, 60s, 300s, 3600s, 86400s
DiskOps Reads Per Second Reads Per Second None 5s, 10s, 60s, 300s, 3600s, 86400s
DiskReadTraffic Disk Read Traffic Disk Read Traffic KBytes/s 5s, 10s, 60s, 300s, 3600s, 86400s
DiskWriteTraffic Disk Write Traffic Disk Write Traffic KBytes/s 5s, 10s, 60s, 300s, 3600s, 86400s
MaxP90SearchTaskMillisMax Query Task Time 90th Percentile Query Task Time 90th Percentile ms 5s, 10s, 60s, 300s, 3600s, 86400s
MaxP95SearchTaskMillis Query Task Time 95th Percentile Query Task Time 95th Percentile ms 5s, 10s, 60s, 300s, 3600s, 86400s
MaxP99SearchTaskMillis Query Task Time 99th Percentile Query Task Time 99th Percentile ms 5s, 10s, 60s, 300s, 3600s, 86400s
NodeDiskPathMaxUsage Node Disk Max Usage Rate Node Disk Max Usage Rate % 5s, 10s, 60s, 300s, 3600s, 86400s
ClusterHttpCurrentOpen Cluster HTTP Connection Count Cluster HTTP Connection Count ms 5s, 10s, 60s, 300s, 3600s, 86400s

Objects

The collected Tencent Cloud ES object data structure can be seen in 「Infrastructure - Resource Catalog」.

{
  "measurement": "tencentcloud_es",
  "tags": {
    "RegionId"          : "ap-shanghai",
    "InstanceId"        : "ap-shanghai-1",
    "InstanceName"      : "xxxx",
    "InstanceType"      : "BIND",
    "AppId"             : "119.45.172.xxx",
    "EsVip"             : "xxxxx  ",
    "EsPort"            : "xxxx",
    "VpcUid"            : "xxxxxxxxxx"
  },
  "fields": {
    "EsPrivateUrl"               : "xxxxxxxxxxx",
    "EsPublicUrl"                : "xxxxxxxxxxx",
    "Status"                     : "xxxxxxxxxxx",
    "NodeInfoList"               : "mmxxxx",
    "CpuNum"                     : "8xxxxx",
    "MemSize"                    : "xxxx",
    "MasterNodeInfo"             : "xxxxxxxxxxx",
    "KibanaNodeInfo"             : false,
    "OptionalWebServiceInfos"    : false,
    "ChargeType"                 : 1,
    "CreateTime"                 : "2024-06-11T08:10:57Z",
    "TagList"                    : "{Instance JSON Data}"
  }
}

Note: The fields in tags and fields may change with subsequent updates.