Skip to content

Volcengine ESCloud Cloud Search Service

Collect Volcengine ESCloud Cloud Search Service Metrics

Configuration

Install Func

It is recommended to activate TrueWatch Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation

If deploying Func manually, refer to Manually Deploy Func

Install Script

Note: Please prepare the required Volcengine AK in advance (for simplicity, you can grant the global read-only permission ReadOnlyAccess)

To synchronize ESCloud cloud resource monitoring data, we install the corresponding collection script: 「TrueWatch Integration (Volcengine - Cloud Search Service Collection)」(ID: integration_volcengine_es_cloud)

After clicking 【Install】, input the corresponding parameters: Volcengine AK, Volcengine account name, and Regions.

Click 【Deploy Startup Script】, the system will automatically create the Startup script set and configure the corresponding startup scripts.

After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute once without waiting for the scheduled time. After a while, you can check the execution task records and corresponding logs.

Verification

  1. In 「Manage / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and check the task records and logs for any exceptions
  2. In TrueWatch, check if asset information exists in 「Infrastructure - Resource Catalog」
  3. In TrueWatch, check if there are corresponding monitoring data in 「Metrics」

Metrics

Configure Volcengine ESCloud monitoring metrics to collect more metrics through configuration Volcengine Cloud Search Service Metrics Details

Volcengine ESCloud Cloud Search Service Monitoring Metrics

Volcengine ESCloud metrics are located under the volcengine_VCM_ESCloud measurement. Here are the related metric descriptions

MetricName Subnamespace Metric Name MetricUnit Dimension
ActivePrimaryShardNumber cluster Primary Shard Count Count ResourceID
ActiveShardNumber cluster Active Shard Count Count ResourceID
AvgDiskUsagePercent total Cluster Disk Average Usage Percent ResourceID
AvgJvmUsage total Cluster JVM Average Usage Percent ResourceID
DeletedDocumentRate operation Delete Document Rate None ResourceID
DiskUsagePercent resource Node Disk Usage Percent ResourceID,Node
FielddataMemorySize cluster Fielddata Memory Usage Bytes ResourceID,Node
FsIoTotalReadBytes resource Node Disk Read Bandwidth Bytes/Second ResourceID,Node
FsIoTotalWriteBytes resource Node Disk Write Bandwidth Bytes/Second ResourceID,Node
GreenIndexNumber cluster Green Status Index Count Count ResourceID
IndexOps operation Index Operations OPS None ResourceID
InitializingShardNumber cluster Initializing Shard Count Count ResourceID
InstanceCapacityUsage total Instance Used Capacity Percent ResourceID
InstanceDataNodeNumber total Instance Data Node Count Count ResourceID
InstanceHealthState total Cluster Status None ResourceID
InstanceIngestNodeNumber total Instance Ingest Role Node Count Count ResourceID
InstanceMasterNodeNumber total Instance Master Role Node Count Count ResourceID
InstanceMasterOnlyNodeNumber total Instance Dedicated Master Node Count Count ResourceID
InstanceStorageUsedBytes total Instance Storage Usage Size Bytes ResourceID
JvmUsage resource Node JVM Usage Percent ResourceID,Node
MaxCpuUsage total CPU Maximum Usage Percent ResourceID
MaxDiskUsagePercent total Cluster Disk Maximum Usage Percent ResourceID
MaxJvmUsage total Cluster JVM Maximum Usage Percent ResourceID
MeanCpuUsage total CPU Average Usage Percent ResourceID
MergeOps operation Merge Operations OPS None ResourceID
NodeCpuUsage resource Node CPU Usage Percent ResourceID,Node
NodeDiskReadIOPS resource Node Disk Read IOPS Count/Second ResourceID,Node
NodeFsIops resource Node Data Directory IOPS None ResourceID,Node
NodeHttpConnections resource Node HTTP Connections Count ResourceID,Node
NodeWriteIOPS resource Node Disk Write IOPS Count/Second ResourceID,Node
OldGcCountPerMin resource Old Generation GC Count Per Minute None ResourceID,Node
QueryOps operation Query Operations OPS None ResourceID
ReadOnlyIndexNumber cluster Read-Only Index Count Count ResourceID
RedIndexNumber total Red Status Index Count None ResourceID
RelocatingShardNumber cluster Relocating Shard Count None ResourceID
ReplicaShardNumber cluster Replica Shard Count Count ResourceID
SearchLatency operation Query Latency Second ResourceID
SearchSuccessRatio operation Search Success Rate Percent ResourceID
BulkSuccessRatio operation Bulk Success Rate Percent ResourceID
ThreadPoolFlushRejected cluster Rejected Flush Task Count Count ResourceID,Node
ThreadPoolForceMergeQueue cluster ForceMerge Task Count in Queue Count ResourceID,Node
ThreadPoolForceMergeRejected cluster Rejected ForceMerge Task Count Count ResourceID,Node
TotalDocumentNumber total Total Document Count Count ResourceID
TotalIndexNumber total Total Index Count Count ResourceID
TotalPendingTasks cluster Total Pending Task Count Count ResourceID
TotalShardNumber total Total Shard Count Count ResourceID
UnsignedShardNumber cluster Unassigned Shard Count Count ResourceID
YellowIndexNumber cluster Yellow Status Index Count Count ResourceID
YoungGcCountPerMin resource Young Generation GC Count Per Minute None ResourceID,Node

Objects

The collected Volcengine ESCloud object data structure can be seen in 「Infrastructure - Resource Catalog」

  {
    "measurement": "volcengine_es_cloud",
    "tags": {
    "InstanceId"      : "o-0089apbhxxx",
    "RegionId"        : "cn-guangzhou",
    "Status"          : "Running",
    "ESEip"           : "118.145.xxx.xxx",
    "UserId"          : "2102598885",
    "ESEipId"         : "eip-11vuua67wc3r440yrhczl0xxx",
    "KibanaEipId"     : "'\'",
    "KibanaPublicDomain"                  : "'\'",
    "CerebroPublicDomain"                 : "'\'",
    "KibanaPrivateDomain"                 : "https://visual-o-xxxxxapbhpvmd.escloud.ivolces.com:5601",
    "CerebroPrivateDomain"                : "'\'",
    "CerebroEnabled"                      : "True",
    "EnableESPublicNetwork"               : "True",
    "EnableESPrivateNetwork"              : "vpc-13fpvlahzboxsxxxxxxxxxx",
    "EnableKibanaPublicNetwork"           : "True",
    "EnableKibanaPrivateNetwork"          : "True",
    "EnableESPrivateDomainPublic"         : "False",
    "EnableKibanaPrivateDomainPublic"     : "False",
    },
    "fileds": {
      "TotalNodes": "4",
      "ChargeEnabled": "True",
      "DeletionProtection": "True",
      "CreateTime": "2025-05-06T09:39:50+08:00",
      "ExpireDate": "'\'",
      "KibanaConfig": "'\'",
      "ResourceTags": "[]",
      "SubInstances": "[]",
      "TransferInfo": "{}",
      "MaintenanceDay": "[\"MONDAY\", \"TUESDAY\", \"WEDNESDAY\", \"THURSDAY\", \"FRIDAY\", \"SATURDAY\", \"SUNDAY\"",
      "ESPublicEndpoint": "https://elasticsearch-o-xxxxxxapbhpvmd.escloud.volces.com:9200",
      "ESPrivateEndpoint": "https://elasticsearch-o-xxxxxapbhpvmd.escloud.ivolces.com:9200",
      "ESPublicIpWhitelist": "127.0.0.1",
      "ESPrivateIpWhitelist": "0.0.0.0/0",
      "InstanceConfiguration": "{Json data}",
      "KibanaPublicIpWhitelist": "'\'",
      "KibanaPrivateIpWhitelist": "0.0.0.0/0"
    }
  }