--- title: OpenTelemetry API support in New Relic agents source: https://docs.newrelic.com/docs/apm/agents/manage-apm-agents/opentelemetry-api-support --- New Relic APM agents now support OpenTelemetry (OTel) APIs, allowing you to instrument your applications using vendor-neutral OpenTelemetry APIs while maintaining full access to New Relic's powerful APM features. This capability gives you the flexibility to write portable instrumentation code without sacrificing the depth of insights and features that New Relic provides. ## Benefits [#benefits] Using OpenTelemetry API support with New Relic agents provides several advantages: - **Vendor-neutral instrumentation**: Write custom instrumentation using standard OpenTelemetry APIs that work across observability platforms - **No migration required**: Enable OTel API support in your existing New Relic agent without replacing your current instrumentation - **Gradual adoption**: Use OpenTelemetry APIs for new custom instrumentation while existing New Relic instrumentation continues to work - **Mixed-mode compatibility**: Seamlessly trace requests across services instrumented with New Relic agents, OpenTelemetry APIs, or pure OTel agents - **Community instrumentation**: Leverage native OpenTelemetry instrumentation for frameworks and libraries ## What this feature provides [#what-it-provides] When you enable OpenTelemetry API support in a New Relic agent, the agent can: - Capture spans, traces, metrics, and logs created using OpenTelemetry APIs - Integrate OTel-generated telemetry seamlessly into New Relic APM traces - Support native OpenTelemetry instrumentation that doesn't conflict with New Relic's built-in instrumentation - Maintain trace context across mixed New Relic and OpenTelemetry instrumented services > #### ⚠️ IMPORTANT > > This feature is **disabled by default**. You must explicitly enable it through agent configuration. ## Supported languages and features [#supported-languages] OpenTelemetry API support is available for the following New Relic APM agents: | Language | Minimum version | Traces API | Metrics API | Logs API | Span links | Events on spans | | -------- | --------------- | ---------- | ----------- | -------------- | ---------- | --------------- | | Java | [9.1.0] | ✓ | ✓ | ✓ | ✓ | ✓ | | Node.js | [13.13.0] | ✓ | ✓ | ✓ | ✓ | ✓ | | .NET | [10.49.0] | ✓ | ✓ | See note below | ✓ | ✓ | | Python | [11.5.0] | ✓ | ✗ | See note below | ✓ | ✗ | | Ruby | [10.6.0] | ✓ | ✗ | ✗ | ✓ | ✓ | > #### 💡 TIP > > **.NET and Python logs:** The .NET agent already uses the logging library that became the OpenTelemetry API for log data. The Python agent already supports the logging module. For backwards compatibility, enabling OpenTelemetry API support does not change how these agents instrument logs. ### Sampling in Mixed Environments All agents now support **trace ratio sampling**, aligning with OpenTelemetry standards and ensuring consistent sampling logic across mixed environments. **Key benefits**: - Consistent sampling decisions across New Relic and OTel-instrumented services - Preservation of upstream trace decisions from OTel SDKs For details on how adaptive sampling works, sampling targets per agent, and configuration options, see [Distributed tracing sampling](https://docs.newrelic.com/docs/distributed-tracing/concepts/how-new-relic-distributed-tracing-works/#trace-origin-sampling). ## Supported instrumentation types [#instrumentation-types] ### Native instrumentation (automatic) Native OpenTelemetry instrumentation is supported out of the box. These instrumentations are built directly into the library or framework by its maintainer and automatically work when you enable OpenTelemetry API support. The specific native instrumentations available vary by language agent. See the [configuration section](#configuration) for details on which instrumentations are supported for each language. > #### ⚠️ IMPORTANT > > **Known limitation**: Some native instrumentations may produce slightly different span names or metric names compared to New Relic's built-in instrumentation due to differences in OpenTelemetry semantic conventions (for example, Elasticsearch, RabbitMQ). ### Custom instrumentation with OpenTelemetry APIs You can write custom instrumentation using the standard OpenTelemetry APIs: - **Traces**: Create custom spans and add attributes using the OpenTelemetry Tracing API - **Metrics**: Record custom metrics using the OpenTelemetry Metrics API (where supported) - **Logs**: Write logs using the OpenTelemetry Logs API (where supported) See the [OpenTelemetry documentation](https://opentelemetry.io/docs/instrumentation/) for details on using these APIs in your language. ## How to enable OpenTelemetry API support [#enable] OpenTelemetry API support is disabled by default. Enable it through your agent's configuration: **Java agent configuration** ### Prerequisites Add the required OpenTelemetry dependencies to your application: ````groovy implementation(platform("io.opentelemetry:opentelemetry-bom:1.44.1")) implementation("io.opentelemetry:opentelemetry-sdk-extension-autoconfigure") implementation("io.opentelemetry:opentelemetry-exporter-otlp") ``` ### Enable OpenTelemetry API support **1. Enable OpenTelemetry SDK autoconfiguration** Use one of the following methods: System property: ```bash -Dotel.java.global-autoconfigure.enabled=true ``` Environment variable: ```bash export OTEL_JAVA_GLOBAL_AUTOCONFIGURE_ENABLED=true ``` **2. Enable New Relic OpenTelemetry support** System property: ```bash -Dnewrelic.config.opentelemetry.enabled=true ``` Environment variable: ```bash NEW_RELIC_OPENTELEMETRY_ENABLED=true ``` YAML configuration (`newrelic.yml`): ```yaml opentelemetry: enabled: true ``` The Java agent instruments the OpenTelemetry trace, metrics, and logs APIs to capture signals emitted by your application. It does **not** ship or automatically enable any OpenTelemetry instrumentations. OpenTelemetry auto-instrumentation is not supported. ### Supported instrumentation types #### Manual instrumentation Custom instrumentation using the OpenTelemetry trace, metrics, or logs APIs will be automatically captured by the agent. #### Library and native instrumentation OpenTelemetry standalone library instrumentations and native instrumentation built directly into libraries will work with the New Relic agent. However, OpenTelemetry bytecode-based auto-instrumentation will not be detected by the New Relic agent. ```` **Node.js agent configuration** ### Enable OpenTelemetry API support Enable OpenTelemetry API support in your `newrelic.js` configuration file: ````javascript exports.config = { // ... other config opentelemetry: { enabled: true, traces: { enabled: true } } // ... other config } ``` The Node.js agent does **not** automatically enable any OpenTelemetry instrumentations. It only instruments the OpenTelemetry APIs to capture telemetry emitted by your application. ### Supported OpenTelemetry APIs When OpenTelemetry support is enabled, the Node.js agent fully supports: #### Traces API Create custom spans and add attributes using the OpenTelemetry Tracing API. See the [OpenTelemetry Node.js tracing example](https://github.com/newrelic/newrelic-node-examples/tree/4a3332d55e4455dca0f8eeba34a87e4a513a679a/opentelemetry-example). #### Metrics API The **Metrics API is the most useful feature** for Node.js customers. It expands the metrics collection capability to include dimensional metrics, a frequently requested feature. Record custom metrics using the OpenTelemetry Metrics API. See the [OpenTelemetry Node.js metrics example](https://github.com/newrelic/newrelic-node-examples/tree/4a3332d55e4455dca0f8eeba34a87e4a513a679a/opentelemetry-example). #### Logs API The JavaScript OpenTelemetry Logs API is still in development, but the Node.js agent supports it. You can use the API to emit logs that will be shipped to New Relic. See the [OpenTelemetry Node.js logs example](https://github.com/newrelic/newrelic-node-examples/tree/4a3332d55e4455dca0f8eeba34a87e4a513a679a/opentelemetry-example). ### Supported instrumentation types #### Manual instrumentation (recommended) Customers who instrument their applications with the OpenTelemetry API will find that the agent processes and ships that data correctly. #### Native and library instrumentation (limited) The list of OpenTelemetry instrumentations has significant overlap with the Node.js agent's supported modules. Instrumentations that are not already covered by the New Relic agent may: - Conflict with agent internals - Not have sufficient customer demand We have not tested OpenTelemetry instrumentations outside of the official OpenTelemetry list and cannot recommend specific ones at this time. ### Known limitations - **No automatic instrumentations**: The agent does not automatically enable OpenTelemetry instrumentations. Customers must manually instrument their code using OpenTelemetry APIs. ```` **.NET agent configuration** ### Enable OpenTelemetry API support Enable OpenTelemetry API support using either environment variables or the `newrelic.config` XML configuration file. **Using environment variables:** ````bash NEW_RELIC_OPENTELEMETRY_ENABLED=true ``` **Using newrelic.config:** ```xml ``` **Traces** and **Metrics** are both enabled by default when OpenTelemetry support is enabled. ### Supported native instrumentations When you enable OpenTelemetry API support, the .NET agent can capture telemetry from OpenTelemetry-instrumented libraries. #### Traces instrumentations The agent supports libraries with native OpenTelemetry tracing instrumentation. Custom instrumentation written using the OpenTelemetry Tracing API is also supported. #### Metrics instrumentations When metrics are enabled, the agent supports OpenTelemetry metric instrumentations from libraries and frameworks. Custom instrumentation written using the OpenTelemetry Metrics API is also supported. #### Logs instrumentations The .NET agent already instruments the logging library that became the OpenTelemetry API for log data. **No changes are required** for logs when enabling OpenTelemetry API support. For backwards compatibility, log instrumentation continues to work as before. ```` **Python agent configuration** ### Enable OpenTelemetry API support Enable OpenTelemetry API support in your `newrelic.ini` configuration file or using environment variables: **Using newrelic.ini:** ````ini [newrelic] opentelemetry.enabled = true ``` **Using environment variable:** ```bash NEW_RELIC_OPENTELEMETRY_ENABLED=true ``` When OpenTelemetry support is enabled, the agent will automatically detect if `opentelemetry-api` is installed and switch to using OpenTelemetry instrumentation for any supported libraries, even if New Relic has an instrumentation hook for that library. ### How it works When OpenTelemetry support is enabled and the agent detects `opentelemetry-api` is installed: 1. **Native instrumentation**: For libraries with native OpenTelemetry instrumentation built in and are not supported by New Relic, the agent will use the native OpenTelemetry instrumentation. If the user wants to use the OpenTelemetry instrumentation instead of the built-in New Relic instrumentation hooks, that library can be added to the `opentelemetry.traces.include` setting. 2. **Library instrumentation**: If an OpenTelemetry instrumentation library is detected and New Relic does not already provide an instrumentation hook, the agent will use the OpenTelemetry instrumentation. Similar to native instrumentation, if the user wants to use the OpenTelemetry instrumentation instead of the built-in New Relic instrumentation hooks, that library can be added to the `opentelemetry.traces.include` setting. For the current list of available Python instrumentations, see the [official OpenTelemetry Python instrumentation list](https://opentelemetry.io/ecosystem/registry/?language=python&component=instrumentation). AWS and gRPC instrumentation libraries will continue to use New Relic's instrumentation. ML/AI libraries with native instrumentation are explicitly disabled for now and will be supported in a future release. ### Manual instrumentation Any custom instrumentation written using the OpenTelemetry Tracing API will be captured by the agent. **Example:** ```python from opentelemetry import trace tracer = trace.get_tracer(__name__) with tracer.start_as_current_span("my-custom-span") as span: span.set_attribute("custom.attribute", "value") # Your code here ``` ### Supported features #### Traces API ✅ **Fully supported**: Create custom spans and add attributes using the OpenTelemetry Tracing API #### Metrics API ❌ **Not yet supported**: OpenTelemetry Metrics API is not enabled in the Python agent #### Logs API ✅ **Already works**: The agent already supports the logging module. There is no OpenTelemetry API for logging in Python (only an SDK exists). What OpenTelemetry configures for logging is whether it's enabled and which endpoints to send data to. ### How it works When OpenTelemetry support is enabled: 1. **Agent searches** the system to determine if OpenTelemetry instrumentation libraries are installed 2. **Automatic switching**: If found and New Relic does not already support this library, the agent will use the OpenTelemetry library's instrumentation 3. **Seamless integration**: OpenTelemetry spans are integrated into New Relic traces ### Known limitations - **Metrics not supported**: OpenTelemetry Metrics API is not yet enabled - **Some AWS and gRPC libraries excluded**: Will continue to use New Relic instrumentation - **AI libraries disabled**: Native OpenTelemetry instrumentation for AI libraries is disabled for now ```` **Ruby agent configuration** ### Enable OpenTelemetry API support To enable OpenTelemetry API support in the Ruby agent, add the following to your `newrelic.yml` configuration file: ````yaml opentelemetry: enabled: true ``` Or set the environment variable: ```bash NEW_RELIC_OPENTELEMETRY_ENABLED=true ``` When OpenTelemetry support is enabled, the agent will automatically detect if `opentelemetry-api` is installed. This gem must be installed to use hybrid agent features. ### Manual instrumentation Any custom instrumentation written using the OpenTelemetry Tracing API will be captured by the agent. **Example:** ```ruby require 'opentelemetry' APP_TRACER = OpenTelemetry.tracer_provider.tracer('my-app', '1.0.0') APP_TRACER.in_span("my-custom-span") do |span| span.set_attribute("test.attribute", "value") # Your code here end ``` ### Supported features #### Traces API ✅ **Supported**: Create custom spans and add attributes using the OpenTelemetry Tracing API #### Metrics API ❌ **Not supported**: OpenTelemetry Metrics API #### Logs API ❌ **Not supported**: OpenTelemetry Logs API #### Span Links ✅ **Supported**: Span Links using the `OpenTelemetry::Trace::Span#add_link` API #### Events on Spans ✅ **Supported**: Events on Spans using the `OpenTelemetry::Trace::Span#add_event` API ### How it works When OpenTelemetry support is enabled, the New Relic Ruby agent: 1. **Listens** for calls made to OpenTelemetry tracing APIs 2. **Translates** those calls into New Relic data models 3. **Incorporates** OpenTelemetry spans into New Relic traces By default, OpenTelemetry instrumentation is excluded for libraries the New Relic Ruby agent also instruments. This includes natively instrumented libraries and libraries with instrumentation packages in [opentelemetry-ruby-contrib](https://github.com/open-telemetry/opentelemetry-ruby-contrib). ### Known limitations - **Limited API support**: Only Traces API is supported in this release - **Metrics and Logs**: Not available - **Span Links and Events**: Not currently supported ```` ## Verify it's working [#verify] After enabling OpenTelemetry API support, verify that it's working: 1. Deploy your application with the updated agent configuration. 2. If using custom instrumentation, ensure your code is creating spans/metrics using OpenTelemetry APIs. 3. Generate traffic to your application. 4. Check your New Relic APM UI: - Go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > APM & services > (select your app)** - View distributed traces to see spans created by OpenTelemetry APIs integrated into your traces ## Limitations and known issues [#limitations] ### Instrumentation limitations | Limitation | Description | | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Library instrumentation not supported | OpenTelemetry library instrumentations that require separate package installation are not supported in this release. Only native instrumentations that are built directly into the library by its maintainer work automatically. | | Java bytecode instrumentation not detected | Java OpenTelemetry auto-instrumentation that uses bytecode manipulation will not be detected by the New Relic Java agent. | ### Semantic convention differences Some OpenTelemetry instrumentations use different naming conventions for spans and metrics compared to New Relic's built-in instrumentation: - **Elasticsearch**: Span names and operation names may differ in casing and length - **RabbitMQ**: Distributed tracing works differently (span links vs. parent/child relationships) > #### 💡 TIP > > These differences may affect existing dashboards and alert conditions that rely on specific span or metric names. Review and update your queries if you switch from New Relic instrumentation to OpenTelemetry instrumentation for these services. ### Agent-specific limitations - **Python**: Only Traces API is supported; Metrics and Logs APIs are not available - **Ruby**: Only Traces API is supported; Metrics and Logs APIs are not available - **.NET**: Enabling OpenTelemetry metrics may increase dimensional metric ingest costs ## Troubleshooting [#troubleshooting] ### OpenTelemetry spans not appearing in traces If you've enabled OpenTelemetry API support but don't see OTel-generated spans in your traces: **Verify OpenTelemetry support is enabled** Check your agent configuration to confirm OpenTelemetry support is enabled: - Review environment variables or configuration files - Check agent logs for confirmation that OpenTelemetry support is active - Ensure you've restarted your application after changing configuration **Verify agent version** Confirm you're running the minimum required agent version that supports OpenTelemetry APIs. See the [supported languages table](#supported-languages) above. ### Need help? If you're experiencing issues with OpenTelemetry API support, please reach out to [New Relic Support](https://support.newrelic.com) for assistance. ## What's next? [#whats-next] - Learn about [distributed tracing in New Relic](https://docs.newrelic.com/docs/distributed-tracing/concepts/introduction-distributed-tracing/) - Explore [OpenTelemetry documentation](https://opentelemetry.io/docs/instrumentation/) for your language - Read about [OpenTelemetry data in New Relic](https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/introduction-opentelemetry-new-relic/)