Collector 'Alibaba Cloud-elasticsearch Log' Configuration Manual¶

Before reading this document, please read:

Cloud Integration

Tip

Before using this collector, you must install the 'Integration Core Package' and its accompanying third-party dependencies.

Tip

The code execution of this script depends on the elasticsearch instance object collection. If the custom object collection of elasticsearch is not configured, the log script cannot collect log data.

1. Configuration Structure¶

The configuration structure of this collector is as follows:

Field	Type	Required	Description
`regions`	list	Required	List of regions to be collected
`regions[#]`	str	Required	Region ID. For example: `'cn-hangzhou'` See appendix for the complete list
`log_types`	list	Required	List of log types to be collected
`log_types[#]`	str	Required	Log type. For example: `'JVMLOG'` See table below for details

log_types (Log Type) Assignment Meanings:

Value	Description
`INSTANCELOG`	Main log
`SEARCHSLOW`	Searching slow log
`INDEXINGSLOW`	Indexing slow log
`JVMLOG`	GC log
`ES_SEARCH_ACCESS_LOG`	ES access log
`AUDIT`	Audit log

2. Configuration Example¶

Specified Region¶

Collect data from the Hangzhou region

collector_configs = {
    'regions': [ 'cn-hangzhou' ],
    'log_types': ['JVMLOG']
}

3. Data Reporting Format¶

After the data is successfully synchronized, you can view the data in the 'Infrastructure-Resource Catalog' of TrueWatch.

Example of reported data:

{
  "measurement": "aliyun_elasticsearch_log",
  "tags": {
    "RegionId"       : "cn-hangzhou",
    "esVersion"      : "7.10.0_with_X-Pack",
    "host"           : "10.14.xxx.xxx",
    "instanceId"     : "es-cn-xxxx",
    "name"           : "es-cn-xxxx",
    "paymentType"    : "prepaid",
    "resourceGroupId": "rg-aekzkcwe4dxxxx",
    "serviceVpc"     : "True",
    "status"         : "active"
  },
  "fields": {
    "timestamp"        : 1684304299000,
    "contentCollection": "[ {Log detail JSON data}, ...]",
    "message"          : "{Instance JSON data}"
  }
}

Note

The fields in tags and fields may change with subsequent updates.

Tip

The value of tags.name is the instance ID, which serves as a unique identifier, and fields.message is the JSON serialized string.

X. Appendix¶

Please refer to the official Alibaba Cloud documentation: