---
title: Distributed tracing for your Java services
source: https://docs.newrelic.com/docs/apm/agents/java-agent/configuration/distributed-tracing-java-agent
---

Distributed tracing allows you to see the entire journey of your requests throughout a [distributed system](https://docs.newrelic.com/docs/distributed-tracing/concepts/introduction-distributed-tracing). For the Java agent, we offer two types of distributed tracing (for more details, see [How span sampling works](https://docs.newrelic.com/docs/understand-dependencies/distributed-tracing/get-started/how-new-relic-distributed-tracing-works#sampling)):

-   Standard (head-based sampling): Before any traces arrive, we determine a set percentage of traces to accept and analyze. This gives you a solid starting point to see how tracing can help you. It is turned on by default in Java agents 7.4.0 and higher. By default, the Java agent samples 120 traces per minute, with a max span limit of 2000 spans per minute.

-   Infinite Tracing (tail-based sampling): Our cloud-based service accepts all your traces and then sorts through them to find the most important. Infinite Tracing analyzes all of your traces and gives you configuration options to sample the traces that matter most to you.

Whether you just want to try out standard distributed tracing (head-based sampling) or also want to set up Infinite Tracing (tail-based sampling), you need to start by setting up standard tracing.

## Standard distributed tracing [#quick-start-apm]

This is the best approach to set up [standard distributed tracing](https://docs.newrelic.com/docs/distributed-tracing/concepts/how-new-relic-distributed-tracing-works/#head-based) if you haven't installed any APM agents for your services yet.

> #### 💡 TIP
>
> When you install the New Relic Java agent, standard distributed tracing is turned on by default. If you prefer to turn it off, see our [configuration guide](https://docs.newrelic.com/docs/apm/agents/java-agent/configuration/java-agent-configuration-config-file).

### Identify services [#identify]

Figure out which services touch your request so you can instrument each of them to send trace data to New Relic.

### Instrument each service with an APM agent [#update-agents]

For each service involved in your transactions, you'll perform separate installations of the agent. If some of your services use other languages, simply repeat the [installation steps](https://docs.newrelic.com/docs/distributed-tracing/enable-configure/quick-start) for those languages.

To start the installation routine, click the tile below. When you're finished installing each agent, return here to see tips for [viewing your traces](#view-traces).

1.  [Start installation](https://one.newrelic.com/launcher/nr1-core.settings?pane=eyJuZXJkbGV0SWQiOiJ0dWNzb24ucGxnLWluc3RydW1lbnQtZXZlcnl0aGluZyJ9&cards[0]=eyJuZXJkbGV0SWQiOiJzZXR1cC1uZXJkbGV0cy5zZXR1cC1qYXZhLWludGVncmF0aW9uIiwiYWNjb3VudElkIjoyNjQwNDA5fQ==&platform[accountId]=2498654&platform[timeRange][duration]=1800000&platform[$isFallbackTimeRange]=true)

### View traces [#view-traces]

After you instrument each of your services with APM agents, generate some traffic in your application so we can capture some traces. Here are some ways to view your traces in the UI:

**View traces that include a specific service**

Here's one way you can see traces for a particular service:

1.  Go to **[one.newrelic.com](https://one.newrelic.com/all-capabilities)> APM & services**.
2.  Click your entity (service).
3.  In the left pane's **Monitor** section, click **Distributed tracing**.
4.  For details, click on an individual trace. If **Group similar traces** is on in the top menu, click on a trace group, and then click on an individual trace.
5.  If you don't see the traces you want, you can filter by the `trace.id`.

**View traces across accounts**

To see traces that cross accounts:

1.  Go to **[one.newrelic.com > All capabilities](https://one.newrelic.com/all-capabilities) > Traces**.
2.  Select your entity (service) in the left pane.
3.  For details, click on an individual trace. If **Group similar traces** is on in the top menu, click on a trace group, and then click on an individual trace.
4.  If you don't see the traces you want, you can filter by the `trace.id`.

**Examine logs related to traces**

You can bring your logs and tracing details together to make troubleshooting easier and faster. With [logs in context](https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-apm-agents/), you can see log messages alongside traces in the New Relic UI.

After you've located an interesting trace using the steps in [View traces that include a specific service](#entity-explorer) or [View traces across accounts](#cross-acct-traces), do the following:

1.  If you've enabled logs in context, click on the **Logs** tab (next to **Trace details**).
2.  To view details related to an individual log, click directly on the message.

For more help finding your traces in the UI:

-   [Understand and use the distributed tracing UI](https://docs.newrelic.com/docs/distributed-tracing/ui-data/understand-use-distributed-tracing-ui)
-   [Query distributed trace data](https://docs.newrelic.com/docs/understand-dependencies/distributed-tracing/ui-data/query-distributed-trace-data)

## Infinite Tracing (optional) [#infinite-tracing]

With Infinite Tracing (tail-based sampling) we can analyze all of your data and determine which traces are the most relevant.

### Complete the setup for standard distributed tracing [#install-agents-first]

The Infinite Tracing setup builds on standard distributed tracing. So, make sure you've completed the [steps above](#quick-start-apm), and then continue with the trace observer setup.

### Set up the trace observer [#provision-trace-observer]

The trace observer is a New Relic AWS-based service that collects and analyzes all your traces. Follow the instructions in [Set up trace observer](https://docs.newrelic.com/docs/understand-dependencies/distributed-tracing/infinite-tracing/set-trace-observer). When you're done, return here with your trace observer information and continue with the next step to configure the agent.

### Configure the agent for Infinite Tracing [#configure-agent-inf]

Infinite Tracing configuration settings include the standard distributed tracing plus information about the trace observer. Note that server-side configuration is not available for Infinite Tracing.

**Java Infinite Tracing configuration**

Here's an overview of the settings, you'll need [`YOUR_TRACE_OBSERVER_HOST`](https://docs.newrelic.com/docs/understand-dependencies/distributed-tracing/infinite-tracing/set-trace-observer) for this configuration.

For more help with configuration, see [Java agent configuration: Config file](https://docs.newrelic.com/docs/agents/java-agent/configuration/java-agent-configuration-config-file).

| Type             | Required configuration                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Infinite Tracing | Configuration options: - Configuration file (`newrelic.yml`) (indented 2 spaces under the `common` section): ```yml distributed_tracing:   enabled: true infinite_tracing:   trace_observer:     host: "YOUR_TRACE_OBSERVER_HOST" ``` - Java system property: ```ini -Dnewrelic.config.distributed_tracing.enabled=true -Dnewrelic.config.infinite_tracing.trace_observer.host="YOUR_TRACE_OBSERVER_HOST" ``` - Environment variables: ```ini NEW_RELIC_DISTRIBUTED_TRACING_ENABLED=true NEW_RELIC_INFINITE_TRACING_TRACE_OBSERVER_HOST="YOUR_TRACE_OBSERVER_HOST" ``` |

> #### 💡 TIP
>
> If you need help with proxy configuration, see [Proxy support](https://docs.newrelic.com/docs/understand-dependencies/distributed-tracing/other-requirements/infinite-tracing-proxy-support).

### (Optional) Customize Infinite Tracing [#customize-inf]

After you add the agent configuration settings, you should start seeing data in the New Relic UI. After you spend some time analyzing your data, you may want to adjust some of the features of Infinite Tracing:

-   [Configure trace observer monitoring](https://docs.newrelic.com/docs/distributed-tracing/infinite-tracing/infinite-tracing-configure-trace-observer-monitoring)
-   [Configure span attribute trace filter](https://docs.newrelic.com/docs/distributed-tracing/infinite-tracing/infinite-tracing-configure-span-attribute-trace-filter)
-   [Configure random trace filter](https://docs.newrelic.com/docs/distributed-tracing/infinite-tracing/infinite-tracing-configure-random-trace-filter)

## Manual instrumentation (if automatic instrumentation doesn't work) [#manual-instrumentation]

Recommendation: Before performing any custom instrumentation, read:

-   [How distributed tracing works](https://docs.newrelic.com/docs/apm/distributed-tracing/getting-started/how-new-relic-distributed-tracing-works)
-   [Troubleshoot missing data](https://docs.newrelic.com/docs/apm/distributed-tracing/troubleshooting/troubleshooting-missing-trace-data)

If a service is not passing the trace header to other services, you can use the distributed tracing payload APIs to instrument the [calling service](#calling-service) and the [called service](#called-service). The calling service uses an API call to generate a payload, which is accepted by the called service.

**Instrument the calling service**

To instrument the calling service:

1.  Ensure the [version of the APM agent](#compatibility-requirements) that monitors the calling service supports distributed tracing.
2.  Invoke the agent API call for generating a distributed trace payload: [Java](https://docs.newrelic.com/docs/agents/java-agent/api-guides/guide-using-java-agent-api#trace-calls).

    > #### ⚠️ IMPORTANT
    >
    > To maintain proper ordering of spans in a trace, ensure you **generate the payload in the context of the span that sends it**.
3.  Add that payload to the call made to the destination service (for example, in a header).
4.  (Optional) Identify the call as an external call (see [GitHub](https://newrelic.github.io/java-agent-api/javadoc/index.html?com/newrelic/api/agent/TracedMethod.html)).

**Instrument the called service**

To instrument the called service:

1.  Ensure the [version of the APM agent](#compatibility-requirements) that monitors the called service supports distributed tracing.
2.  If the New Relic agent on the called service does not identify a New Relic transaction, use the agent API to declare a transaction. Here's one way to tell that a transaction is not in progress: when `Transaction.insertDistributedTraceHeaders(Headers)` is called, no headers are inserted (this API requires agent 6.4.0+). To create a transaction, see [Java agent transaction-related APIs](https://docs.newrelic.com/docs/agents/java-agent/api-guides/guide-using-java-agent-api#transactions).
3.  Extract the payload from the call that you received (for example, in a header).
4.  Invoke the call for accepting the payload: [Java](https://docs.newrelic.com/docs/agents/java-agent/api-guides/guide-using-java-agent-api#trace-calls).