Skip to content

Volcengine ESCloud Cloud Search Service

Collect Metrics data from Volcengine ESCloud Cloud Search Service

Configuration

Install Func

It is recommended to enable TrueWatch Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation.

If you want to deploy Func yourself, refer to Self-deploy Func

Install Script

Note: Please prepare the required Volcengine AK in advance (for simplicity, you can directly grant the global read-only permission ReadOnlyAccess)

To synchronize the monitoring data of ESCloud cloud resources, we install the corresponding collection script: 「TrueWatch Integration (Volcengine - Cloud Search Service Collection)」(ID: integration_volcengine_es_cloud)

Click 【Install】, then enter the corresponding parameters: Volcengine AK, Volcengine account name, and Regions.

Click 【Deploy Startup Script】, the system will automatically create the Startup script set and configure the corresponding startup script.

After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute it once without waiting for the scheduled time. Wait for a moment, and you can check the execution task records and corresponding logs.

Verification

  1. Confirm whether the corresponding task exists in the automatic trigger configuration in 「Manage / Automatic Trigger Configuration」, and check the task records and logs for any exceptions
  2. In TrueWatch, check if the asset information exists in 「Infrastructure - Resource Catalog」
  3. In TrueWatch, check if there is corresponding monitoring data in 「Metrics」

Metrics

Configure Volcengine ESCloud monitoring metrics to collect more metrics through configuration Volcengine Cloud Search Service Metrics Details

Volcengine ESCloud Cloud Search Service Monitoring Metrics

Volcengine ESCloud metrics are located under the volcengine_VCM_ESCloud Measurement. Here are the related metric descriptions

MetricName Subnamespace Metric Name MetricUnit Dimension
ActivePrimaryShardNumber cluster Primary Shard Count Count ResourceID
ActiveShardNumber cluster Active Shard Count Count ResourceID
AvgDiskUsagePercent total Average Disk Usage Rate Percent ResourceID
AvgJvmUsage total Average JVM Usage Rate Percent ResourceID
DeletedDocumentRate operation Delete Document Rate None ResourceID
DiskUsagePercent resource Node Disk Usage Rate Percent ResourceID,Node
FielddataMemorySize cluster Field data Memory Usage Bytes ResourceID,Node
FsIoTotalReadBytes resource Node Disk Read Bandwidth Bytes/Second ResourceID,Node
FsIoTotalWriteBytes resource Node Disk Write Bandwidth Bytes/Second ResourceID,Node
GreenIndexNumber cluster Green Status Index Count Count ResourceID
IndexOps operation Index Operations OPS None ResourceID
InitializingShardNumber cluster Initializing Shard Count Count ResourceID
InstanceCapacityUsage total Instance Used Capacity Percent ResourceID
InstanceDataNodeNumber total Instance Data Node Count Count ResourceID
InstanceHealthState total Cluster Status None ResourceID
InstanceIngestNodeNumber total Instance Ingest Role Node Count Count ResourceID
InstanceMasterNodeNumber total Instance Master Role Node Count Count ResourceID
InstanceMasterOnlyNodeNumber total Instance Dedicated Master Node Count Count ResourceID
InstanceStorageUsedBytes total Instance Storage Usage Size Bytes ResourceID
JvmUsage resource Node JVM Usage Rate Percent ResourceID,Node
MaxCpuUsage total Maximum CPU Usage Rate Percent ResourceID
MaxDiskUsagePercent total Maximum Disk Usage Rate Percent ResourceID
MaxJvmUsage total Maximum JVM Usage Rate Percent ResourceID
MeanCpuUsage total Average CPU Usage Rate Percent ResourceID
MergeOps operation Merge Operations OPS None ResourceID
NodeCpuUsage resource Node CPU Usage Rate Percent ResourceID,Node
NodeDiskReadIOPS resource Node Disk Read IOPS Count/Second ResourceID,Node
NodeFsIops resource Node Data Directory IOPS None ResourceID,Node
NodeHttpConnections resource Node HTTP Connections Count ResourceID,Node
NodeWriteIOPS resource Node Disk Write IOPS Count/Second ResourceID,Node
OldGcCountPerMin resource Old Generation GC Count Per Minute None ResourceID,Node
QueryOps operation Query Operations OPS None ResourceID
ReadOnlyIndexNumber cluster Read-Only Index Count Count ResourceID
RedIndexNumber total Red Status Index Count None ResourceID
RelocatingShardNumber cluster Relocating Shard Count None ResourceID
ReplicaShardNumber cluster Replica Shard Count Count ResourceID
SearchLatency operation Query Latency Second ResourceID
SearchSuccessRatio operation Search Success Rate Percent ResourceID
BulkSuccessRatio operation Bulk Success Rate Percent ResourceID
ThreadPoolFlushRejected cluster Rejected Flush Tasks Count Count ResourceID,Node
ThreadPoolForceMergeQueue cluster Force Merge Tasks in Queue Count Count ResourceID,Node
ThreadPoolForceMergeRejected cluster Rejected Force Merge Tasks Count Count ResourceID,Node
TotalDocumentNumber total Total Document Count Count ResourceID
TotalIndexNumber total Total Index Count Count ResourceID
TotalPendingTasks cluster Total Pending Tasks Count Count ResourceID
TotalShardNumber total Total Shard Count Count ResourceID
UnsignedShardNumber cluster Unassigned Shard Count Count ResourceID
YellowIndexNumber cluster Yellow Status Index Count Count ResourceID
YoungGcCountPerMin resource Young Generation GC Count Per Minute None ResourceID,Node

Objects

The collected Volcengine ESCloud object data structure can be seen in 「Infrastructure - Resource Catalog」

  {
    "measurement": "volcengine_es_cloud",
    "tags": {
    "InstanceId"      : "o-0089apbhxxx",
    "RegionId"        : "cn-guangzhou",
    "Status"          : "Running",
    "ESEip"           : "118.145.xxx.xxx",
    "UserId"          : "2102598885",
    "ESEipId"         : "eip-11vuua67wc3r440yrhczl0xxx",
    "KibanaEipId"     : "'\'",
    "KibanaPublicDomain"                  : "'\'",
    "CerebroPublicDomain"                 : "'\'",
    "KibanaPrivateDomain"                 : "https://visual-o-xxxxxapbhpvmd.escloud.ivolces.com:5601",
    "CerebroPrivateDomain"                : "'\'",
    "CerebroEnabled"                      : "True",
    "EnableESPublicNetwork"               : "True",
    "EnableESPrivateNetwork"              : "vpc-13fpvlahzboxsxxxxxxxxxx",
    "EnableKibanaPublicNetwork"           : "True",
    "EnableKibanaPrivateNetwork"          : "True",
    "EnableESPrivateDomainPublic"         : "False",
    "EnableKibanaPrivateDomainPublic"     : "False",
    },
    "fileds": {
      "TotalNodes": "4",
      "ChargeEnabled": "True",
      "DeletionProtection": "True",
      "CreateTime": "2025-05-06T09:39:50+08:00",
      "ExpireDate": "'\'",
      "KibanaConfig": "'\'",
      "ResourceTags": "[]",
      "SubInstances": "[]",
      "TransferInfo": "{}",
      "MaintenanceDay": "[\"MONDAY\", \"TUESDAY\", \"WEDNESDAY\", \"THURSDAY\", \"FRIDAY\", \"SATURDAY\", \"SUNDAY\"",
      "ESPublicEndpoint": "https://elasticsearch-o-xxxxxxapbhpvmd.escloud.volces.com:9200",
      "ESPrivateEndpoint": "https://elasticsearch-o-xxxxxapbhpvmd.escloud.ivolces.com:9200",
      "ESPublicIpWhitelist": "127.0.0.1",
      "ESPrivateIpWhitelist": "0.0.0.0/0",
      "InstanceConfiguration": "{Json data}",
      "KibanaPublicIpWhitelist": "'\'",
      "KibanaPrivateIpWhitelist": "0.0.0.0/0"
    }
  }