--- title: Monitor ECS Fargate with OpenTelemetry source: https://docs.newrelic.com/docs/opentelemetry/integrations/ecs-monitoring/ecs-fargate --- Monitor Amazon ECS tasks running on AWS Fargate by deploying OpenTelemetry Collector Contrib as a sidecar container. This comprehensive guide walks you through creating task definitions, configuring the collector, and setting up monitoring for your serverless ECS workloads. ## Installation steps [#install] Follow these steps in order to set up monitoring for your ECS Fargate tasks. ### Before you begin [#requirements] Make sure your environment meets these requirements: **ECS requirements** - ECS cluster: Active ECS cluster (capacity providers not required for Fargate) - Task definitions: Existing task definitions you want to monitor - Launch type: Fargate launch type - Platform version: 1.4.0 or later for Task Metadata Endpoint v4 support - Network mode: Must use `awsvpc` network mode (required for Fargate) **Fargate requirements** - CPU and Memory: Fargate task sizes (0.25-16 vCPU, 0.5-120 GB RAM) - Network: VPC with subnets and security groups configured - Storage: Ephemeral storage limits (20-200 GB) **AWS permissions** - Task execution role: `AmazonECSTaskExecutionRolePolicy` attached - SSM permissions: Access to Systems Manager Parameter Store for configuration - CloudWatch permissions: For logging (optional but recommended) - VPC permissions: If using private subnets, ensure NAT Gateway or VPC endpoints for internet access **New Relic requirements** - Account: Active New Relic account - License key: [New Relic license key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key) for your account ### Store your New Relic license key [#store-license-key] Save your license key as a Systems Manager (SSM) parameter to securely store credentials for the OpenTelemetry Collector: ```bash aws ssm put-parameter \ --name "/newrelic-infra/ecs/license-key" \ --type SecureString \ --description 'New Relic license key for ECS monitoring' \ --value "YOUR_NEW_RELIC_LICENSE_KEY" ``` ### Create IAM policy and execution role [#create-role] Create an IAM policy so your ECS containers can securely retrieve the New Relic license key: ```bash aws iam create-policy \ --policy-name "NewRelicSSMLicenseKeyReadAccess" \ --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["ssm:GetParameters"],"Resource":["arn:aws:ssm:*:*:parameter/newrelic-infra/ecs/license-key"]}]}' \ --description "Provides read access to the New Relic SSM license key parameter" ``` Create an IAM role to be used as the task execution role: ```bash aws iam create-role \ --role-name "NewRelicECSTaskExecutionRole" \ --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ecs-tasks.amazonaws.com"},"Action":"sts:AssumeRole"}]}' \ --description "ECS task execution role for New Relic infrastructure" ``` Attach the required managed policies to the role: ```bash # Attach the standard ECS task execution policy aws iam attach-role-policy \ --role-name "NewRelicECSTaskExecutionRole" \ --policy-arn "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy" # Attach the New Relic SSM license key read access policy aws iam attach-role-policy \ --role-name "NewRelicECSTaskExecutionRole" \ --policy-arn "arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):policy/NewRelicSSMLicenseKeyReadAccess" ``` ### Store collector configuration [#store-config] Store the OpenTelemetry Collector configuration in AWS Systems Manager Parameter Store so you can manage and update settings without rebuilding container images: ```bash aws ssm put-parameter \ --name "/ecs/otel-collector/fargate-config" \ --type "String" \ --value "$(cat < processors: metricstransform/base: transforms: - include: container.cpu.utilized action: insert new_name: container.cpu.utilization - include: container.memory.usage action: insert new_name: container.memory.usage.total - include: container.storage.read_bytes action: insert new_name: container.blockio.io_service_bytes_recursive operations: - action: add_label new_label: operation new_value: read - include: container.storage.write_bytes action: insert new_name: container.blockio.io_service_bytes_recursive operations: - action: add_label new_label: operation new_value: write transform/promote_cluster_keys: metric_statements: - context: datapoint statements: - set(attributes["aws.ecs.cluster.name"], resource.attributes["aws.ecs.cluster.name"]) where resource.attributes["aws.ecs.cluster.name"] != nil - set(attributes["cloud.account.id"], resource.attributes["cloud.account.id"]) where resource.attributes["cloud.account.id"] != nil - set(attributes["cloud.region"], resource.attributes["cloud.region"]) where resource.attributes["cloud.region"] != nil - set(attributes["aws.ecs.task.arn"], resource.attributes["aws.ecs.task.arn"]) where resource.attributes["aws.ecs.task.arn"] != nil metricstransform/cluster_aggregation: transforms: - include: container.cpu.utilization action: insert new_name: ecs.cluster.cpu.utilization operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: mean - include: container.memory.usage.total action: insert new_name: ecs.cluster.memory.usage operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: sum - include: container.network.io.usage.rx_bytes action: insert new_name: ecs.cluster.network.rx_bytes operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: sum - include: container.network.io.usage.tx_bytes action: insert new_name: ecs.cluster.network.tx_bytes operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: sum - include: container.cpu.utilization action: insert new_name: ecs.cluster.running_task_count operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region, aws.ecs.task.arn] aggregation_type: mean - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: count - include: container.memory.usage.limit action: insert new_name: ecs.cluster.memory.limit operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: sum transform/running_task_count_one: metric_statements: - context: datapoint statements: - set(value_double, 1.0) where metric.name == "ecs.cluster.running_task_count" metricstransform/running_task_count_sum: transforms: - include: ecs.cluster.running_task_count action: update operations: - action: aggregate_labels label_set: [aws.ecs.cluster.name, cloud.account.id, cloud.region] aggregation_type: sum filter/cluster_metrics: metrics: include: match_type: regexp metric_names: - ^ecs\.cluster\..* metricstransform/task_metrics: transforms: - include: ecs.task.cpu.utilized action: insert new_name: ecs.task.cpu.utilization - include: container.memory.usage.total action: insert new_name: ecs.task.memory.usage filter/task_metrics: metrics: include: match_type: strict metric_names: - "ecs.task.cpu.utilization" - "ecs.task.cpu.utilized" - "ecs.task.memory.utilized" - "ecs.task.memory.usage" - "ecs.task.memory.usage.limit" - "ecs.task.network.io.usage.rx_bytes" - "ecs.task.network.io.usage.tx_bytes" filter/container_metrics: metrics: include: match_type: regexp metric_names: - ^container\..* transform/cluster_identifier: metric_statements: - context: resource statements: - set(attributes["aws.arn"], resource.attributes["aws.ecs.task.arn"]) where resource.attributes["aws.ecs.task.arn"] != nil and resource.attributes["aws.ecs.cluster.name"] != nil - replace_pattern(attributes["aws.arn"], ":task/.*", "") where attributes["aws.arn"] != nil - set(attributes["aws.arn"], Concat([attributes["aws.arn"], ":cluster/", resource.attributes["aws.ecs.cluster.name"]], "")) where attributes["aws.arn"] != nil and resource.attributes["aws.ecs.cluster.name"] != nil transform/task_identifier: metric_statements: - context: resource statements: - set(attributes["TaskArn"], resource.attributes["aws.ecs.task.arn"]) where resource.attributes["aws.ecs.task.arn"] != nil batch: send_batch_size: timeout: resource/cluster: attributes: - key: ClusterName from_attribute: aws.ecs.cluster.name action: insert - key: cloud.platform value: "aws_ecs" action: upsert resource/task: attributes: - key: ClusterName from_attribute: aws.ecs.cluster.name action: insert - key: ServiceName from_attribute: aws.ecs.service.name action: insert - key: TaskId from_attribute: aws.ecs.task.id action: insert - key: TaskArn from_attribute: aws.ecs.task.arn action: insert - key: TaskDefinitionFamily from_attribute: aws.ecs.task.family action: insert - key: LaunchType from_attribute: aws.ecs.launch_type action: insert - key: cloud.platform value: "aws_ecs" action: upsert - key: docker.host from_attribute: aws.ecs.task.id action: insert - key: docker.imageName from_attribute: container.image.name action: insert - key: docker.containerId from_attribute: container.id action: insert - key: docker.state from_attribute: aws.ecs.container.know_status action: insert exporters: otlphttp: endpoint: https://otlp.nr-data.net:443 headers: api-key: \${NEW_RELIC_LICENSE_KEY} debug: verbosity: basic service: pipelines: metrics/clusters: receivers: [awsecscontainermetrics] processors: [ metricstransform/base, transform/promote_cluster_keys, metricstransform/cluster_aggregation, transform/running_task_count_one, metricstransform/running_task_count_sum, filter/cluster_metrics, transform/cluster_identifier, resource/cluster, batch ] exporters: [otlphttp, debug] metrics/tasks: receivers: [awsecscontainermetrics] processors: [ metricstransform/base, metricstransform/task_metrics, filter/task_metrics, transform/task_identifier, resource/task, batch ] exporters: [otlphttp, debug] metrics/containers: receivers: [awsecscontainermetrics] processors: [ metricstransform/base, filter/container_metrics, resource/task, batch ] exporters: [otlphttp, debug] EOF )" ``` #### Configuration parameters The following parameters can be customized in the OpenTelemetry Collector configuration: | Parameter | Description | | ------------------------------ | --------------------------------------------------------------------------------------- | | `` | The interval in seconds for collecting metrics from the ECS container metrics endpoint. | | `` | Memory limit for the OpenTelemetry Collector in MiB | | `` | Number of metrics to batch before sending to New Relic | | `` | Maximum time to wait before sending a batch | | `` | Timeout for resource detection processors | ### Create task definition [#create-task-definition] Create a new ECS task definition for Fargate that includes the collector sidecar container. Choose between NRDOT Collector (New Relic's distribution) or OpenTelemetry Collector, then select the task definition for your container platform: #### NRDOT Collector > #### 💡 TIP > > **NRDOT Collector** is New Relic's distribution of the OpenTelemetry Collector with New Relic support for assistance. It bundles the `awsecscontainermetrics` receiver used by this configuration, so it runs with the same collector config you stored in the previous step. > #### ⚠️ IMPORTANT > > NRDOT Collector publishes Linux container images only. To monitor Windows containers, use the **OpenTelemetry Collector** tab. ```json { "family": "otel-ecs-fargate-metrics", "networkMode": "awsvpc", "requiresCompatibilities": ["FARGATE"], "cpu": "", "memory": "", "executionRoleArn": "arn:aws:iam::YOUR_ACCOUNT:role/NewRelicECSTaskExecutionRole", "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" }, "containerDefinitions": [ { "name": "busybox", "image": "busybox:latest", "cpu": , "memory": , "essential": true, "command": [ "sh", "-c", "while true; do echo 'Hello from BusyBox'; sleep 30; done" ], "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "", "awslogs-create-group": "true", "awslogs-region": "", "awslogs-stream-prefix": "" } } }, { "name": "otel-collector", "image": "newrelic/nrdot-collector:latest", "cpu": , "memoryReservation": , "essential": true, "command": ["--config=env:OTEL_CONFIG"], "environment": [ { "name": "FARGATE", "value": "true" } ], "secrets": [ { "name": "OTEL_CONFIG", "valueFrom": "arn:aws:ssm:us-east-1:YOUR_ACCOUNT:parameter/ecs/otel-collector/fargate-config" }, { "name": "NEW_RELIC_LICENSE_KEY", "valueFrom": "arn:aws:ssm:us-east-1:YOUR_ACCOUNT:parameter/newrelic-infra/ecs/license-key" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "", "awslogs-create-group": "true", "awslogs-region": "", "awslogs-stream-prefix": "" } } } ] } ``` #### OpenTelemetry Collector **Linux containers** ```json { "family": "otel-ecs-fargate-metrics", "networkMode": "awsvpc", "requiresCompatibilities": ["FARGATE"], "cpu": "", "memory": "", "executionRoleArn": "arn:aws:iam::YOUR_ACCOUNT:role/NewRelicECSTaskExecutionRole", "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" }, "containerDefinitions": [ { "name": "busybox", "image": "busybox:latest", "cpu": , "memory": , "essential": true, "command": [ "sh", "-c", "while true; do echo 'Hello from BusyBox'; sleep 30; done" ], "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "", "awslogs-create-group": "true", "awslogs-region": "", "awslogs-stream-prefix": "" } } }, { "name": "otel-collector", "image": "otel/opentelemetry-collector-contrib:latest", "cpu": , "memoryReservation": , "essential": true, "command": ["--config=env:OTEL_CONFIG"], "environment": [ { "name": "FARGATE", "value": "true" } ], "secrets": [ { "name": "OTEL_CONFIG", "valueFrom": "arn:aws:ssm:us-east-1:YOUR_ACCOUNT:parameter/ecs/otel-collector/fargate-config" }, { "name": "NEW_RELIC_LICENSE_KEY", "valueFrom": "arn:aws:ssm:us-east-1:YOUR_ACCOUNT:parameter/newrelic-infra/ecs/license-key" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "", "awslogs-create-group": "true", "awslogs-region": "", "awslogs-stream-prefix": "" } } } ] } ``` **Windows containers** ```json { "family": "otel-ecs-fargate-metrics-windows", "networkMode": "awsvpc", "requiresCompatibilities": ["FARGATE"], "cpu": "", "memory": "", "executionRoleArn": "arn:aws:iam::YOUR_ACCOUNT:role/NewRelicECSTaskExecutionRole", "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "WINDOWS_SERVER_2022_FULL" }, "containerDefinitions": [ { "name": "busybox", "image": "winamd64/hello-world", "cpu": , "memoryReservation": , "essential": true, "entryPoint": ["cmd"], "command": [ "/c", "for /L %i in () do @(echo Running... && timeout /t 30 /nobreak >nul)" ], "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "", "awslogs-region": "", "awslogs-stream-prefix": "" } } }, { "name": "otel-collector", "image": "otel/opentelemetry-collector-contrib:latest-windows-2022-amd64", "cpu": , "memoryReservation": , "essential": true, "entryPoint": ["C:\\otelcol-contrib.exe"], "command": ["--config", "env:OTEL_CONFIG"], "environment": [ { "name": "FARGATE", "value": "true" } ], "secrets": [ { "name": "OTEL_CONFIG", "valueFrom": "arn:aws:ssm:us-east-1:YOUR_ACCOUNT:parameter/ecs/otel-collector/fargate-config-windows" }, { "name": "NEW_RELIC_LICENSE_KEY", "valueFrom": "arn:aws:ssm:us-east-1:YOUR_ACCOUNT:parameter/newrelic-infra/ecs/license-key" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "", "awslogs-create-group": "true", "awslogs-region": "", "awslogs-stream-prefix": "" } } } ] } ``` #### Task definition parameters The following parameters can be customized in your ECS Fargate task definition: | Parameter | Description | | -------------------------------- | --------------------------------------------------------- | | `` | Total CPU units for the Fargate task | | `` | Total memory for the Fargate task in MiB | | `` | CPU units allocated to your application container | | `` | Memory allocated to your application container in MiB | | `` | CPU units allocated to the OpenTelemetry Collector | | `` | Memory reservation for the OpenTelemetry Collector in MiB | | `` | CloudWatch log group name for your application container | | `` | CloudWatch log group name for the OpenTelemetry Collector | | `` | AWS region for CloudWatch logs | | `` | Log stream prefix for your application container | | `` | Log stream prefix for the OpenTelemetry Collector | > #### 💡 TIP > > Key differences between Linux and Windows Fargate containers: > > - Runtime platform: Windows requires explicit `operatingSystemFamily: "WINDOWS_SERVER_2022_FULL"` > - Images: Windows uses specific Windows container images > - Entry point: Windows collector specifies `C:\\otelcol-contrib.exe` as entry point > - Memory allocation: Uses `memoryReservation` for flexible memory management > #### ⚠️ IMPORTANT > > Replace `YOUR_ACCOUNT` and region values with your actual AWS account ID and AWS region. ### Deploy and run the task [#deploy-task] Deploy your task definition to your ECS cluster: 1. Register the task definition: ```bash aws ecs register-task-definition --cli-input-json file://task-definition.json ``` 2. Create a service: ```bash aws ecs create-service \ --cluster your-cluster-name \ --service-name otel-monitoring-service \ --task-definition otel-ecs-fargate-metrics:1 \ --desired-count 1 \ --launch-type FARGATE \ --network-configuration "awsvpcConfiguration={subnets=[subnet-12345,subnet-67890],securityGroups=[sg-abcdef],assignPublicIp=ENABLED}" ``` ### Verify data collection [#verify-data] Check that your data is flowing to New Relic: - Check OpenTelemetry Collector status: Review container logs to confirm the collector is running without errors and successfully connecting to New Relic: ```bash aws logs get-log-events \ --log-group-name "/ecs/otel-collector-fargate" \ --log-stream-name "otel/otel-collector/TASK_ID" ``` - Verify data in New Relic UI: Navigate to **[one.newrelic.com](https://one.newrelic.com) > All Capabilities > Infrastructure** to confirm your ECS containers appear with metrics. For detailed guidance on exploring your data, see [Find and query your ECS monitoring data](https://docs.newrelic.com/docs/opentelemetry/integrations/ecs-monitoring/find-and-query-your-data). ## Configuration differences from EC2 [#fargate-differences] Key differences when running on Fargate: - **No host-level access**: Cannot access underlying host metrics like in EC2 - **Network mode**: Must use `awsvpc` network mode - **Resource constraints**: Limited by Fargate CPU/memory combinations - **Storage**: Ephemeral storage only, no persistent volumes - **Container insights**: Relies on Task Metadata Endpoint v4 for container stats ## Next steps [#next-steps] After setting up monitoring, you can: - [Create custom dashboards](https://docs.newrelic.com/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/) for your Fargate metrics - [Set up alerts](https://docs.newrelic.com/docs/alerts/create-alert/create-alert-condition/alert-conditions/) for container and task-level issues - [Correlate ECS metrics with application traces and logs](https://docs.newrelic.com/docs/opentelemetry/integrations/ecs-monitoring/overview#how-it-works)