Volcengine ESCloud Cloud Search Service
Collect Volcengine ESCloud Cloud Search Service Metrics
Configuration¶
Install Func¶
It is recommended to activate TrueWatch Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation
If deploying Func manually, refer to Manually Deploy Func
Install Script¶
Note: Please prepare the required Volcengine AK in advance (for simplicity, you can grant the global read-only permission
ReadOnlyAccess
)
To synchronize ESCloud cloud resource monitoring data, we install the corresponding collection script: 「TrueWatch Integration (Volcengine - Cloud Search Service Collection)」(ID: integration_volcengine_es_cloud
)
After clicking 【Install】, input the corresponding parameters: Volcengine AK, Volcengine account name, and Regions.
Click 【Deploy Startup Script】, the system will automatically create the Startup
script set and configure the corresponding startup scripts.
After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute once without waiting for the scheduled time. After a while, you can check the execution task records and corresponding logs.
Verification¶
- In 「Manage / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and check the task records and logs for any exceptions
- In TrueWatch, check if asset information exists in 「Infrastructure - Resource Catalog」
- In TrueWatch, check if there are corresponding monitoring data in 「Metrics」
Metrics¶
Configure Volcengine ESCloud monitoring metrics to collect more metrics through configuration Volcengine Cloud Search Service Metrics Details
Volcengine ESCloud Cloud Search Service Monitoring Metrics¶
Volcengine ESCloud metrics are located under the volcengine_VCM_ESCloud measurement. Here are the related metric descriptions
MetricName |
Subnamespace |
Metric Name | MetricUnit | Dimension |
---|---|---|---|---|
ActivePrimaryShardNumber |
cluster |
Primary Shard Count | Count | ResourceID |
ActiveShardNumber |
cluster |
Active Shard Count | Count | ResourceID |
AvgDiskUsagePercent |
total |
Cluster Disk Average Usage | Percent | ResourceID |
AvgJvmUsage |
total |
Cluster JVM Average Usage | Percent | ResourceID |
DeletedDocumentRate |
operation |
Delete Document Rate | None | ResourceID |
DiskUsagePercent |
resource |
Node Disk Usage | Percent | ResourceID,Node |
FielddataMemorySize |
cluster |
Fielddata Memory Usage | Bytes | ResourceID,Node |
FsIoTotalReadBytes |
resource |
Node Disk Read Bandwidth | Bytes/Second | ResourceID,Node |
FsIoTotalWriteBytes |
resource |
Node Disk Write Bandwidth | Bytes/Second | ResourceID,Node |
GreenIndexNumber |
cluster |
Green Status Index Count | Count | ResourceID |
IndexOps |
operation |
Index Operations OPS | None | ResourceID |
InitializingShardNumber |
cluster |
Initializing Shard Count | Count | ResourceID |
InstanceCapacityUsage |
total |
Instance Used Capacity | Percent | ResourceID |
InstanceDataNodeNumber |
total |
Instance Data Node Count | Count | ResourceID |
InstanceHealthState |
total |
Cluster Status | None | ResourceID |
InstanceIngestNodeNumber |
total |
Instance Ingest Role Node Count | Count | ResourceID |
InstanceMasterNodeNumber |
total |
Instance Master Role Node Count | Count | ResourceID |
InstanceMasterOnlyNodeNumber |
total |
Instance Dedicated Master Node Count | Count | ResourceID |
InstanceStorageUsedBytes |
total |
Instance Storage Usage Size | Bytes | ResourceID |
JvmUsage |
resource |
Node JVM Usage | Percent | ResourceID,Node |
MaxCpuUsage |
total |
CPU Maximum Usage | Percent | ResourceID |
MaxDiskUsagePercent |
total |
Cluster Disk Maximum Usage | Percent | ResourceID |
MaxJvmUsage |
total |
Cluster JVM Maximum Usage | Percent | ResourceID |
MeanCpuUsage |
total |
CPU Average Usage | Percent | ResourceID |
MergeOps |
operation |
Merge Operations OPS | None | ResourceID |
NodeCpuUsage |
resource |
Node CPU Usage | Percent | ResourceID,Node |
NodeDiskReadIOPS |
resource |
Node Disk Read IOPS | Count/Second | ResourceID,Node |
NodeFsIops |
resource |
Node Data Directory IOPS | None | ResourceID,Node |
NodeHttpConnections |
resource |
Node HTTP Connections | Count | ResourceID,Node |
NodeWriteIOPS |
resource |
Node Disk Write IOPS | Count/Second | ResourceID,Node |
OldGcCountPerMin |
resource |
Old Generation GC Count Per Minute | None | ResourceID,Node |
QueryOps |
operation |
Query Operations OPS | None | ResourceID |
ReadOnlyIndexNumber |
cluster |
Read-Only Index Count | Count | ResourceID |
RedIndexNumber |
total |
Red Status Index Count | None | ResourceID |
RelocatingShardNumber |
cluster |
Relocating Shard Count | None | ResourceID |
ReplicaShardNumber |
cluster |
Replica Shard Count | Count | ResourceID |
SearchLatency |
operation |
Query Latency | Second | ResourceID |
SearchSuccessRatio |
operation |
Search Success Rate | Percent | ResourceID |
BulkSuccessRatio |
operation |
Bulk Success Rate | Percent | ResourceID |
ThreadPoolFlushRejected |
cluster |
Rejected Flush Task Count | Count | ResourceID,Node |
ThreadPoolForceMergeQueue |
cluster |
ForceMerge Task Count in Queue | Count | ResourceID,Node |
ThreadPoolForceMergeRejected |
cluster |
Rejected ForceMerge Task Count | Count | ResourceID,Node |
TotalDocumentNumber |
total |
Total Document Count | Count | ResourceID |
TotalIndexNumber |
total |
Total Index Count | Count | ResourceID |
TotalPendingTasks |
cluster |
Total Pending Task Count | Count | ResourceID |
TotalShardNumber |
total |
Total Shard Count | Count | ResourceID |
UnsignedShardNumber |
cluster |
Unassigned Shard Count | Count | ResourceID |
YellowIndexNumber |
cluster |
Yellow Status Index Count | Count | ResourceID |
YoungGcCountPerMin |
resource |
Young Generation GC Count Per Minute | None | ResourceID,Node |
Objects¶
The collected Volcengine ESCloud object data structure can be seen in 「Infrastructure - Resource Catalog」
{
"measurement": "volcengine_es_cloud",
"tags": {
"InstanceId" : "o-0089apbhxxx",
"RegionId" : "cn-guangzhou",
"Status" : "Running",
"ESEip" : "118.145.xxx.xxx",
"UserId" : "2102598885",
"ESEipId" : "eip-11vuua67wc3r440yrhczl0xxx",
"KibanaEipId" : "'\'",
"KibanaPublicDomain" : "'\'",
"CerebroPublicDomain" : "'\'",
"KibanaPrivateDomain" : "https://visual-o-xxxxxapbhpvmd.escloud.ivolces.com:5601",
"CerebroPrivateDomain" : "'\'",
"CerebroEnabled" : "True",
"EnableESPublicNetwork" : "True",
"EnableESPrivateNetwork" : "vpc-13fpvlahzboxsxxxxxxxxxx",
"EnableKibanaPublicNetwork" : "True",
"EnableKibanaPrivateNetwork" : "True",
"EnableESPrivateDomainPublic" : "False",
"EnableKibanaPrivateDomainPublic" : "False",
},
"fileds": {
"TotalNodes": "4",
"ChargeEnabled": "True",
"DeletionProtection": "True",
"CreateTime": "2025-05-06T09:39:50+08:00",
"ExpireDate": "'\'",
"KibanaConfig": "'\'",
"ResourceTags": "[]",
"SubInstances": "[]",
"TransferInfo": "{}",
"MaintenanceDay": "[\"MONDAY\", \"TUESDAY\", \"WEDNESDAY\", \"THURSDAY\", \"FRIDAY\", \"SATURDAY\", \"SUNDAY\"",
"ESPublicEndpoint": "https://elasticsearch-o-xxxxxxapbhpvmd.escloud.volces.com:9200",
"ESPrivateEndpoint": "https://elasticsearch-o-xxxxxapbhpvmd.escloud.ivolces.com:9200",
"ESPublicIpWhitelist": "127.0.0.1",
"ESPrivateIpWhitelist": "0.0.0.0/0",
"InstanceConfiguration": "{Json data}",
"KibanaPublicIpWhitelist": "'\'",
"KibanaPrivateIpWhitelist": "0.0.0.0/0"
}
}