AWS MWAA
Collect metrics data from AWS Managed Workflows for Apache Airflow
Configuration¶
Install Func¶
It is recommended to activate TrueWatch Integration - Extensions - DataFlux Func (Automata): all prerequisites are automatically installed, please proceed with script installation.
If you deploy Func manually, refer to Manual Func Deployment
Install Script¶
Note: Please prepare the required Amazon AK in advance (for simplicity, you can directly grant global read-only permissions
ReadOnlyAccess
)
-
Log in to the Func console, click on 【Script Market】, enter the TrueWatch Script Market, and search for:
integration_aws_mwaa
-
Click 【Install】, then input the corresponding parameters: AWS AK ID, AK Secret, and account name.
-
Click 【Deploy Startup Script】, the system will automatically create the
Startup
script set and configure the corresponding startup scripts. -
After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute once without waiting for the scheduled time. After a while, you can check the execution task records and corresponding logs.
Verification¶
- In 「Manage / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and check the task records and logs for any anomalies.
- In TrueWatch, check if asset information exists in 「Infrastructure - Resource Catalog」.
- In TrueWatch, check if there are corresponding monitoring data in 「Metrics」.
Metrics¶
Configure Amazon CloudWatch to collect more metrics Amazon CloudWatch Metrics Details
AWS MWAA metrics are under the aws_AWS/MWAA
Measurement, with the description column including the description, unit, and statistics of each metric.
Cluster Metrics¶
Metric | Description | Unit |
---|---|---|
CPUUtilization |
CPU Utilization | % |
MemoryUtilization |
Memory Utilization | % |
Database Metrics¶
Metric | Description | Unit |
---|---|---|
CPUUtilization |
CPU Utilization | % |
DatabaseConnections |
Database Connections | Count |
DiskQueueDepth |
Disk Queue Depth | Count |
FreeableMemory |
Freeable Memory | Byte |
VolumeWriteIOPS |
Volume Write IOPS | Count/s |
WriteIOPS |
Write IOPS | Count/s |
WriteLatency |
Write Latency | s |
WriteThroughput |
Write Throughput | Bytes/s |
Queue Metrics¶
Metric | Description | Unit |
---|---|---|
ApproximateAgeOfOldestTask |
Approximate Age of Oldest Task | s |
RunningTasks |
Running Tasks | Count |
QueuedTasks |
Queued Tasks | Count |
Object¶
The collected AWS MWAA object data structure can be viewed in 「Infrastructure - Resource Catalog」.
{
"measurement": "aws_mwaa",
"tags": {
"EnvironmentName" : "MyAirflowEnvironment",
"RegionId" : "ap-southeast-1"
},
"fields": {
"Arn" : "arn:aws:airflow:ap-southeast-1:876267738534:environment/MyAirflowEnvironment",
"WebserverUrl" : "cbf8bc48-413d-4f10-8771-80a6627fdfd1.c3.ap-southeast-1.airflow.amazonaws.com",
"DagS3Path" : "test/",
"PluginsS3Path" : "xxxxxx",
"AirflowVersion" : "2.10.3",
"Status" : "AVAILABLE",
"CreatedAt" : "2025-08-11T17:02:23+08:00",
"EnvironmentClass" : "mw1.micro",
"MaxWorkers" : "1",
"WebserverAccessMode" : "PUBLIC_ONLY"
}
}
Note: The fields in
tags
andfields
may change with subsequent updates.