AWS EMR¶

Use the "TrueWatch Cloud Sync" series script packages in the script market to synchronize cloud monitoring and cloud asset data to TrueWatch

Configuration¶

Install Func¶

It is recommended to enable TrueWatch integration - extension - DataFlux Func (Automata): all prerequisites are automatically installed. Please continue with the script installation.

If you deploy Func yourself, refer to Self-deploy Func

Install Script¶

Note: Please prepare an Amazon AK that meets the requirements in advance (for simplicity, you can directly grant global read-only permission ReadOnlyAccess)

To synchronize EMR cloud resource monitoring data, we install the corresponding collection script: "TrueWatch Integration (AWS EMR Collection)" (ID: `truewatch_aws_emr)

Click [Install], then input the corresponding parameters: Amazon AK, Amazon account name.

Click [Deploy Startup Script], the system will automatically create a Startup script set and automatically configure the corresponding startup script.

In addition, you can see the corresponding automatic trigger configuration in "Manage / Automatic Trigger Configuration". Click [Execute] to immediately execute it once without waiting for the scheduled time. After a while, you can check the execution task records and corresponding logs.

We default to collecting some configurations, for details, see the metrics section Customize Cloud Object Metrics

Verification¶

1. In "Manage / Automatic Trigger Configuration", confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can check the corresponding task records and logs to check for any abnormalities.
2. On the TrueWatch platform, check under "Infrastructure / Custom" whether there is asset information.
3. On the TrueWatch platform, check under "Metrics" whether there is corresponding monitoring data.

Metrics¶

After configuring Amazon-CloudWatch, the default metric set is as follows. You can collect more metrics through configuration Amazon CloudWatch Metric Details

Objects¶

The collected AWS EMR object data structure can be seen in the "Infrastructure - Custom" section.

{
  "measurement": "aws_emr",
  "tags": {
    "Id": "xxxxx",
    "ClusterName": "xxxxx",
    "ClusterArn": "xxxx",
    "RegionId": "cn-north-1",
    "OutpostArn": "xxxx"
  },
  "fields": {
    "Status": "{Instance status JSON data}",
    "message": "{Instance JSON data}"
  }
}

Note: The fields in tags, fields may change with subsequent updates