---
title: Kubernetes APM auto-attach
source: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/installation/k8s-agent-operator
---

The Kubernetes APM auto-attach, formerly known as the Kubernetes agent operator, streamlines full-stack observability for Kubernetes environments by automating APM instrumentation alongside Kubernetes agent deployment. By enabling APM auto instrumentation, developers no longer need to manually manage [APM agents](https://docs.newrelic.com/docs/apm/new-relic-apm/getting-started/introduction-apm/). The Kubernetes APM auto-attach will automatically install, upgrade and remove APM agents.

It currently [supports](#k8s-supported-versions) Java, .NET, Node.js, Python, Ruby, and PHP.

## How it works [#how-it-works]

-   The `MutatingWebHook`, upon installation, becomes involved in intercepting API requests for deploying pods onto nodes.

-   Reflecting the configurations specified, it mutates the pod specification to add a NR init container and environment variables.

-   Following the establishment of the pod, the New Relic APM Agent is seamlessly integrated into the application housed within it.

![Diagram showing how APM agents are auto injected](https://docs.newrelic.com/images/K8s-APM-auto-attach-diagram.webp "Diagram showing how APM agents are auto injected")

## Before you begin [#before-begin]

Before installing the operator, check the following:

-   [Helm](https://helm.sh/): You must install it to use the charts. See the [Helm documentation](https://helm.sh/docs/) if you need help getting started.

-   [Kubectl](https://kubernetes.io/docs/tasks/tools/): You have to configure it to communicate with your cluster.

## Installation [#install-k8s-operator]

Depending on what you need, you can choose to install the Kubernetes APM auto-attach independently or together with our Kubernetes integrations.

We strongly recommend to install it together with the Kubernetes integration to take advantage of our entire [full stack observability](https://docs.newrelic.com/docs/apm/apm-ui-pages/monitoring/kubernetes-summary-page/) experience.

### Bundle installation in addition to the Kubernetes integration (recommended) [#bundle-installation]

The Kubernetes APM auto-attach chart is part of the [`nri-bundle`](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) chart, which manages the installation of all the components needed to enable a full Kubernetes observability.

Add the `k8s-agents-operator.enabled=true` parameter to your helm command or include it in the [`values.yaml`](https://github.com/newrelic/helm-charts/blob/master/charts/nri-bundle/values.yaml) file. See the [Install the Kubernetes integration](/install/kubernetes/?dropdown1=helm) page for more information about using Helm or check out the [`nri-bundle`](https://github.com/newrelic/helm-charts/tree/master/charts/nri-bundle) chart.

See this sample of Helm commands using parameters:

```shell
helm repo add newrelic https://helm-charts.newrelic.com

helm upgrade --install newrelic-bundle newrelic/nri-bundle \
    --set global.licenseKey=YOUR_NEW_RELIC_INGEST_LICENSE_KEY \
    --set global.cluster=CLUSTER_NAME \
    --namespace=newrelic \
    --set newrelic-infrastructure.privileged=true \
    --set global.lowDataMode=true \
    --set kube-state-metrics.enabled=true \
    --set kubeEvents.enabled=true \
    --set k8s-agents-operator.enabled=true \
    --create-namespace
```

### Standalone installation [#standalone-installation]

To install the Kubernetes APM auto-attach with the default configuration, run these commands:

```shell
helm repo add k8s-agents-operator https://newrelic.github.io/k8s-agents-operator
helm upgrade --install k8s-agents-operator k8s-agents-operator/k8s-agents-operator \
    --namespace newrelic \
    --create-namespace \
    --set licenseKey=YOUR_NEW_RELIC_INGEST_LICENSE_KEY
```

For a complete list of configuration options, see the [README](https://github.com/newrelic/k8s-agents-operator/tree/main/charts/k8s-agents-operator#values) chart.

## Configure auto-instrumentation [#configure-auto-instrumentation]

After APM auto-attach is all set up in your cluster, the next step is just to roll out the configs required to get it operational. That involves having at least one instrumentation Custom Resource (CR) active in the cluster.

Here's what the instrumentation CR lets you map out:

-   Name of the instrumentation CR
-   Where it will apply the instrumentation CR using [`selectors`](#selectors)
-   APM agent (one per CR)
-   APM agent version
-   APM config parameters (env vars)
-   License key (optional)

Run the following command to install the instrumentation CR:

```bash
kubectl apply -f ./instrumentation.yaml
```

Here you have a complete [instrumentation.yaml](https://github.com/newrelic/k8s-agents-operator/blob/main/local/instrumentation.yaml) for your reference.

## How to use selectors [#selectors]

Use selectors to control which resources the CR injects with APM agents. Three selectors are available. You can use them individually or in combination. When combined, the selectors are evaluated using a logical AND (`&&`) operation.

**PodLabelSelector**

Informs the APM Auto-attach which pods needs to be instrumented.

The following example uses `matchLabel` in the `PodLabelSelector` to select pods containing an specific tag and value:

```yaml
...
podLabelSelector:
  matchLabels:
    app.kubernetes.io/name: flask-hello-world
...
```

**NameSpaceLabelSelector**

Defines the auto-instrumented pods at the namespace level. Permitted when the CR and operator are in the same namespace. Otherwise,
this selector must be left empty.

The following example uses `matchExpressions` in the `NameSpaceLabelSelector` to select namespace containing an specific tag and value:

```yaml
...
namespaceLabelSelector:
  matchExpressions:
    - key: "kubernetes.io/metadata.name"
      operator: "In"
      values: ["backend"]
...
```

> #### 💡 TIP
>
> A selector that uses the `kubernetes.io/metadata.name` label will match namespaces based on their name.

**ContainerSelector**

Defines the auto-instrumented container at the container level.

> #### 💡 TIP
>
> If a `containerSelector` isn't defined, the integration automatically instruments the first non-init container listed in the pod's specification.

It provides four methods to select the appropriate container.

#### envSelector

Select containers based on the container environment variables.

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: select-by-match-envs
spec:
  containerSelector:
    envSelector:
      matchEnvs:
        DEBUG: 'true'
#...
```

For more information and examples, refer [envSelector](https://github.com/newrelic/k8s-agents-operator/blob/main/instrumentation.md#envSelector)

#### imageSelector

Select containers based on the container image URL.

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: select-by-any-container
spec:
  containerSelector:
    imageSelector:
      matchExpressions:
        - key: url
          operator: StartsWith
          values: ["docker.io/"]
#...
```

For more information and examples, refer [imageSelector](https://github.com/newrelic/k8s-agents-operator/blob/main/instrumentation.md#imageSelector)

#### nameSelector

Selects containers for instrumentation based on their name and type (`init` or `regular`).

The value should be a list of the container names you want to target. Containers not included in this list will be ignored.

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: select-by-match-names-and-any-container
spec:
  containerSelector:
    nameSelector:
      matchNames:
        anyContainer: app
#...
```

For more information and examples, refer [nameSelector](https://github.com/newrelic/k8s-agents-operator/blob/main/instrumentation.md#nameSelector)

#### namesFromPodAnnotation

Specifies the key of a pod annotation. The value of the specified annotation must be a comma-separated list of container names to select for instrumentation.

For example, if a pod has an annotation identifying which containers to instrument. In this case, the `use-these-for-newrelic` annotation specifies that the containers named `a` and `c` should be selected.

```yaml
#... pod spec
metadata:
  annotations:
    use-these-for-newrelic: a,c
spec:
  initContainers:
    - name: a
      #.. more container spec ..
  containers:
    - name: b
      # ...
    - name: c
#...
```

Next, to use that annotation, set the `namesFromPodAnnotation` field in your instrumentation resource to the annotation's key (`use-these-for-newrelic`):

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: select-by-names-from-pod-annotation
spec:
  containerSelector:
    namesFromPodAnnotation: use-these-for-newrelic
#...
```

This configuration instruments the `a` (`init`) and `c` (`regular`) containers, while ignoring the `b` container, because their names were specified in the pod's `use-these-for-newrelic` annotation.

For more information and examples, refer [namesFromPodAnnotation](https://github.com/newrelic/k8s-agents-operator/blob/main/instrumentation.md#namesFromPodAnnotation)

### MatchLabel and MatchExpressions

All the selectors support `matchLabel` and `matchExpressions`.

**How to use `matchLabel` and `matchExpressions`**

`matchExpressions` is a more expressive label selector in Kubernetes and supports set-based matching unlike the `matchLabels`, which you can only use for exact matching. You can use it with or without the `matchLabels` selector.

```yaml
...
selector:
  matchLabels:
    tier: frontend
  matchExpressions:
    - {key: name, operator: In, values: [payroll, web]}
    - {key: environment, operator: NotIn, values: [dev]}
...
```

You can add more expressions to the selector. As in the example, each expression must contain a key, an operator, and possibly (depending on the operator) a list of values. There are four valid operators:

-   `In`: Label's value must match one of the specified values.
-   `NotIn`: Label's value must not match any of the specified values.
-   `Exists`: Pod must include a label with the specified key (the value isn't important). When using this operator, you shouldn't specify the values field.
-   `DoesNotExist`: Pod must not include a label with the specified key. You must not specify the values property.

If you specify many expressions, all those expressions must evaluate to true for the selector to match a pod. If you specify both `matchLabels` and `matchExpressions`, all the labels must match and all the expressions must evaluate to true for the pod to match the selector.

### APM agent [#apm-agent]

You've got to specify the APM agent and its version within the instrumentation CR. We recommend using the latest version to take advantage of the newest features available.

| Language           | Image                                     | Available versions                                                                                                 |
| ------------------ | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| dotnet             | `newrelic-dotnet-init:latest`             | [.NET](https://hub.docker.com/repository/docker/newrelic/newrelic-dotnet-init/general)                             |
| dotnet-windows2022 | `newrelic-dotnet-windows2022-init:latest` | [.NET on Windows 2022](https://hub.docker.com/repository/docker/newrelic/newrelic-dotnet-windows2022-init/general) |
| dotnet-windows2025 | `newrelic-dotnet-windows2025-init:latest` | [.NET on Windows 2025](https://hub.docker.com/repository/docker/newrelic/newrelic-dotnet-windows2025-init/general) |
| java               | `newrelic-java-init:latest`               | [Java](https://hub.docker.com/repository/docker/newrelic/newrelic-java-init/general)                               |
| nodejs             | `newrelic-node-init:latest`               | [Node](https://hub.docker.com/repository/docker/newrelic/newrelic-node-init/general)                               |
| python             | `newrelic-python-init:latest`             | [Python](https://hub.docker.com/repository/docker/newrelic/newrelic-python-init/general)                           |
| ruby               | `newrelic-ruby-init:latest`               | [Ruby](https://hub.docker.com/repository/docker/newrelic/newrelic-ruby-init/general)                               |
| php                | `newrelic-php-init:latest`                | [PHP](https://hub.docker.com/repository/docker/newrelic/newrelic-php-init/general)                                 |

See this example:

```yaml
...
spec:
  agent:
    language: dotnet
    image: newrelic/newrelic-dotnet-init:latest
...
```

### APM configuration [#apm-config-parameters]

#### Using environment variables

The instrumentation CR provides the capability to inject environment variables in the pod to streamline the configuration of the APM agents. See this example:

```yaml
...
spec:
  agent:
    env:
      # Example overriding the appName configuration by using a label of the pod
      - name: NEW_RELIC_APP_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.labels['app.kubernetes.io/name']
...
```

In the above example, we show you how you can configure the agent settings globally using environment variables. See each agent's configuration documentation for available configuration options:

-   [Java](https://docs.newrelic.com/docs/apm/agents/java-agent/configuration/java-agent-configuration-config-file/)
-   [Node](https://docs.newrelic.com/docs/apm/agents/nodejs-agent/installation-configuration/nodejs-agent-configuration/)
-   [Python](https://docs.newrelic.com/docs/apm/agents/python-agent/configuration/python-agent-configuration/)
-   [.NET](https://docs.newrelic.com/docs/apm/agents/net-agent/configuration/net-agent-configuration/)
-   [Ruby](https://docs.newrelic.com/docs/apm/agents/ruby-agent/configuration/ruby-agent-configuration/)
-   [PHP](https://docs.newrelic.com/docs/apm/agents/php-agent/configuration/php-agent-configuration/)

> #### ⚠️ IMPORTANT
>
> You can inject these environment variables in the app deployment manifest.

#### Using a configmap (Java only)

When using the configmap, note the following requirements:

-   The Java configuration must be stored in a key named `newrelic.yaml`.
-   Setting the application name or license key in the `configmap` is not supported.
-   To set these properties, use environment variables or custom secrets instead.

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: newrelic-instrumentation-java
  namespace: newrelic
spec:
  agentConfigMap: "my-java-app-config" # This configmap must exist in every namespace that the pod in this `Instrumentation` is targeting
  agent: ...
```

### License keys (optional) [#license-keys]

When you install it, a license key is created and it's the license by default. Follow these steps, if you need to send the APM telemetry to a different account:

-   To create a secret containing a new license key, run this command:

    ```bash
    kubectl create secret generic newrelic-key-secret \
        --namespace my-monitored-namespace \
        --from-literal=new_relic_license_key=YOUR_NEW_RELIC_INGEST_LICENSE_KEY
    ```

-   To reference the secret from the instrumentation CR, run this command:

    ```yaml
    ...
    spec:
      licenseKeySecret: the-name-of-the-custom-secret
    ...
    ```

## Instrumentation CR examples [#cr-examples]

> #### 💡 TIP
>
> If your cluster uses a container runtime that requires an explicit registry prefix (such as CRI-O), prepend `docker.io/` to the image path. For example: `image: docker.io/newrelic/newrelic-java-init:latest`.

**Python agent: Pods with a specific label and value and overriding app name**

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: newrelic-instrumentation-python
  namespace: newrelic
spec:
  agent:
    language: python
    image: newrelic/newrelic-python-init:latest
    env:
      - name: NEW_RELIC_APP_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.labels['app']
  podLabelSelector:
    matchExpressions:
      - key: "app"
        operator: "In"
        values: ["flask-hello-world","flask-hello-world-v2"]
```

**Python agent: Pods with a specific label only monitoring init containers with a specific name**

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: newrelic-instrumentation-python
  namespace: newrelic
spec:
  agent:
    language: python
    image: newrelic/newrelic-python-init:latest
  podLabelSelector:
    matchExpressions:
      - key: "app"
        operator: "In"
        values: ["flask-hello-world","flask-hello-world-v2"]
  containerSelector:
    nameSelector:
      matchNames:
        anyContainer: flask-hello-app
```

**Java agent: Pods of a specific namespace**

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: newrelic-instrumentation-java
  namespace: newrelic
spec:
  agent:
    language: java
    image: newrelic/newrelic-java-init:latest
  namespaceLabelSelector:
    matchExpressions:
      - key: "kubernetes.io/metadata.name"
        operator: "In"
        values: ["java"]
```

**Java agent: Pods of a specific namespace only monitoring containers with a specific name**

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: newrelic-instrumentation-java
  namespace: newrelic
spec:
  agent:
    language: java
    image: newrelic/newrelic-java-init:latest
  namespaceLabelSelector:
    matchExpressions:
      - key: "kubernetes.io/metadata.name"
        operator: "In"
        values: ["java"]
  containerSelector:
    nameSelector:
      matchExpressions:
        - key: container
          operator: In
          values: ["java-app"]
```

**Ruby agent: Any namespace containing the Ruby label and sending data to a different account**

```yaml
apiVersion: newrelic.com/v1beta2
kind: Instrumentation
metadata:
  name: newrelic-instrumentation-ruby
  namespace: newrelic
spec:
  agent:
    language: ruby
    image: newrelic/newrelic-ruby-init:latest
  namespaceLabelSelector:
    matchExpressions:
      - key: "Ruby"
        operator: "Exists"
  licenseKeySecret: the-name-of-the-custom-secret
```

## Update APM instrumentation in applications [#upgrade-apm-instrumention]

By default, the Kubernetes APM auto-attach automatically installs the latest available version of the corresponding [APM agent](https://docs.newrelic.com/docs/apm/new-relic-apm/getting-started/introduction-apm/).

Once the monitoring of an application starts, it's not automatically updated to a newer version unless you choose to update. You can update the application by redeploying the pods or restarting your deployment.

## Remove APM instrumentation in applications [#remove-apm-instrumentation]

To remove the APM instrumentation from an application, you must change the matching label selector inside either the `podLabelSelector` or `namespaceLabelSelector` used or delete the instrumentation CR. Then, restart the deployment. The remove process takes just a few seconds.

## Update the Kubernetes APM auto-attach [#update-k8s-auto-attach]

### Bundle installation [#upgrading-bundle-installation]

Run an update of the `nri-bundle` chart with the following parameter:

```shell
k8s-agents-operator.enabled=true
```

### Standalone installation [#standalone-installation]

Run the `helm upgrade` command to update to a newer version of the Kubernetes APM auto-attach.

```shell
helm upgrade k8s-agents-operator newrelic/k8s-agents-operator -n newrelic
```

## Uninstalling the Kubernetes APM auto-attach [#uninstall-k8s-auto-attach]

### Bundle installation [#uninstall-bundle-installation]

Uninstall the `nri-bundle` chart or if you only want to remove Kubernetes APM auto-attach, run a helm upgrade with the following parameter:

```shell
k8s-agents-operator.enabled=false
```

### Standalone installation [#uninstall-standalone-installation]

To uninstall and delete the Kubernetes APM auto-attach, run this command:

```shell
helm uninstall k8s-agents-operator -n newrelic
```

## Find and use data [#find-data]

-   Get insights of your applications and resolve alert events using the [APM summary](https://docs.newrelic.com/docs/apm/agents/manage-apm-agents/agent-data/triage-run-diagnostics/) page.

-   Check out the [Kubernetes summary](https://docs.newrelic.com/docs/apm/apm-ui-pages/monitoring/kubernetes-summary-page/) page. It provides Kubernetes insights in the context of your monitored applications.

## Certificates [#certificates]

The Kubernetes APM auto-attach can support [`cert-manager`](https://github.com/cert-manager/cert-manager) if preferred.

-   Run this command to install the [`cert-manager`](https://github.com/cert-manager/cert-manager) Helm chart:

    ```shell
    helm install cert-manager jetstack/cert-manager \
        --namespace cert-manager \
        --create-namespace \
        --set crds.enabled=true
    ```

-   In your `values.yaml` file, set `admissionWebhooks.autoGenerateCert.enabled: false` and `admissionWebhooks.certManager.enabled: true`. Then, install the chart as normal.

## Available chart releases [#available-chart-releases]

Run this command to see the available charts:

```shell
helm search repo k8s-agents-operator
```

## Frequently asked questions [#faq]

**Can I route my applications telemetry to different accounts?**

Yes, you just need to add more secrets in the cluster. Check needed steps in the [license keys (optional)](#license-keys]) section.

> #### ⚠️ IMPORTANT
>
> The [K8s APM experience](https://docs.newrelic.com/docs/apm/apm-ui-pages/monitoring/kubernetes-summary-page/) is only available on the account where the data from the applications and the Kubernetes cluster is available.

**Can I install the Kubernetes APM auto-attach if my applications are already instrumented?**

Installing two APM agents in the same application can potentially lead to unexpected issues. Therefore, we strongly recommend removing any existing instrumentation before installing it.

**Can I use custom instrumentation with the Kubernetes APM auto-attach?**

Yes, custom instrumentation will work the same as without APM auto-attach. The main difference is that the agent is now injected by APM auto-attach instead of installed in the container with the rest of the application dependencies.

You can still import and call the agent API to add custom instrumentation into your application. You can also utilize a configuration file or environment variables to add custom instrumentation if the particular agent you're using supports it. Note that agents have order of precendence between configuration via environment variables and configuration via configuration files, so you will need to make sure your environment variable configuration via the operator is not clashing with your configuration via configuration file. See each agents custom instrumentation docs for details:

-   [Java](https://docs.newrelic.com/docs/apm/agents/java-agent/custom-instrumentation/java-custom-instrumentation/)
-   [Node](https://docs.newrelic.com/docs/apm/agents/nodejs-agent/extend-your-instrumentation/nodejs-custom-instrumentation/)
-   [Python](https://docs.newrelic.com/docs/apm/agents/python-agent/custom-instrumentation/python-custom-instrumentation/)
-   [.NET](https://docs.newrelic.com/docs/apm/agents/net-agent/custom-instrumentation/introduction-net-custom-instrumentation/)
-   [Ruby](https://docs.newrelic.com/docs/apm/agents/ruby-agent/api-guides/ruby-custom-instrumentation/)
-   [PHP](https://docs.newrelic.com/docs/apm/agents/php-agent/features/php-custom-instrumentation/)

**Can I install the Kubernetes APM auto-attach if my applications are running on a read-only file system?**

No, you can't. The APM agents are injected at application runtime, it needs access to write to the application container's file system.

**Can I setup Kubernetes APM auto-attach in Windows nodes?**

Currently, Kubernetes APM auto-attach is supported only for .NET applications running on Windows Server 2022 and Windows Server 2025 nodes.

**Can I setup Kubernetes APM auto-attach in Fargate nodes?**

EKS Fargate is supported

## Troubleshooting [#troubleshooting]

If your applications are not instrumented, you should check the following:

-   Please be sure to redeploy or deploy new applications after you've installed Kubernetes APM auto-attach. Notice that only auto-instruments new applications are deployed in the cluster.

-   Run this command to check that the secret is installed in the app's namespace:

    ```bash
    kubectl get secrets -n NAMESPACE
    ```

-   Check that the pod has the required labels that enable automatic instrumentation through CR when using `podLabelSelector`. Similarly, check that the namespace has the required labels when using `namespaceLabelSelector` inside the CR.

    ```bash
    kubectl get pod POD_NAME -n NAMESPACE -o jsonpath='{.metadata.annotations}'
    ```

-   Run this command to get logs from the APM auto-attach pod:

    ```bash
    kubectl logs AGENT_OPERATOR_POD -n newrelic
    ```

-   Run this command to ensure the `init` container has been injected and sucessfully executed inside the application's pod.

    ```bash
    kubectl describe pod POD_NAME -n NAMESPACE
    ```

## How to migrate from previous versions that required annotations [#migrate]

Starting with version 0.14, annotations within the application deployment manifest are no longer necessary for applications to be auto-instrumented.

It's advised to uninstall any versions preceding 0.14 and proceed with the installation of the latest release. Utilizing the label selectors within the instrumentation CR will enable the precise deployment of APM agents, thereby obviating the requirement for annotations.

## Support [#support]

Kubernetes APM auto-attach supports the following languages and their minimum supported versions according to our standard APM agent support policy:

-   **Java Agent:** 8.12
-   **.NET Agent:** 10.25
-   **Ruby Agent:** 9.10
-   **Node.js Agent:** 11.9
-   **Python:** 9.10
-   **PHP:** 11.12

For any issues:

-   Review the [issues section on GitHub](https://github.com/newrelic/k8s-agents-operator/issues) for any similar problems or consider opening a new issue.

-   You can reach out to the [New Relic support](https://support.newrelic.com/) team for assistance.