---
title: APM logs in context
source: https://docs.newrelic.com/docs/logs/logs-context/get-started-logs-context
---

There are [several ways to report your logs to New Relic](https://docs.newrelic.com/docs/logs/get-started/get-started-log-management). Using our APM agents is one popular way, especially for smaller teams and DevOps teams that value the benefit of not having to use any other log management tools.

> #### 💡 TIP
>
> Got lots of logs? Check out our [tutorial on how to optimize and manage them](https://docs.newrelic.com/docs/tutorial-large-logs/get-started-managing-large-logs/).

## APM logs in context [#apm-logs-defined]

Our APM agents have a feature called **logs in context**. For more on the benefits of this feature, see [Logs in context](https://docs.newrelic.com/docs/logs/logs-context/logs-in-context).

For APM agents, our logs in context feature does a few things:

-   Decorates your app logs with important New Relic metadata (like `span.id`, `trace.id`, `hostname`, `entity.guid`, `entity.name`) that allows you to see your log data in various New Relic UI experiences.
-   Forwards your logs directly to New Relic, so you don't have need for any third-party tool.
-   Reports some [logs metrics](#log-metrics). These are displayed on the **Logs** chart on the APM **Summary** UI page.

Our current APM agent versions have these features enabled by default.

You have control over all aspects of this feature via APM agent configuration. For reasons you might want to disable one or more of these, see [Limitations](#limitations). For example, if you want to have an APM agent add New Relic metadata, you can use the log decoration feature, while at the same time disabling log forwarding and instead using another logging agent (for example, our infrastructure agent, or a third-party logging agent).

The implementation details and configuration instructions vary per agent (see [agent details](#agents)).

To learn more about the power of logs in context, see [this example use case](#response-time-example). The example describes how an engineering team used logs in context to troubleshoot their app's poor response time and rising error rates.

To see how logs in context can help you find the root cause of an issue in your apps and hosts, watch this short video (approx. 3:40 minutes):

[Video](https://www.youtube.com/embed/Vgae-GrlZWg)

## Get started [#get-started]

To set up APM logs in context:

1.  If you don't have one already, [create a New Relic account](https://newrelic.com/signup). It's free, forever.
2.  Install an APM agent, making sure you have the [latest APM agent version](#agents).
3.  The newest versions of our APM agents have logs in context (addition of metadata and forwarding) enabled by default. You may sometimes have to make some updates to the agent config file to get logs working correctly. For details on this, see [Enable logs for your agent](#agents).

That's it! Start troubleshooting your applications with APM logs in context by going to the APM UI and looking for associated log data.

![Screenshot of APM Summary page with logs in context options](https://docs.newrelic.com/images/apm_screenshot-full_logs-overview.webp "APM logs in context UI options")

Drill down into your logs, traces, and errors, all from the APM **Summary** page in New Relic.

## APM agent APIs [#agent-apis]

If your logging framework is not available with our existing logs in context solutions, you can [configure your logging libraries by using API calls](https://docs.newrelic.com/docs/logs/logs-context/annotate-logs-logs-context-using-apm-agent-apis) to annotate your logs.

## APM agent log configuration [#agents]

Our latest APM agents have logs in context (log decoration and log forwarding) enabled by default. Here's information about the APM agents that support logs in context and log forwarding:

-   [Go logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-go) for agent [v3.17.0 or higher](https://docs.newrelic.com/docs/release-notes/agent-release-notes/go-release-notes)
-   [Java logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/java-configure-logs-context-all) for agent [v7.6.0 or higher](https://docs.newrelic.com/docs/release-notes/agent-release-notes/java-release-notes)
-   [.NET logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/net-configure-logs-context-all) for agent [v9.8.0 or higher](https://docs.newrelic.com/docs/release-notes/agent-release-notes/net-release-notes)
-   [Node.js logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-nodejs) for agent [v8.11.0 or higher](https://docs.newrelic.com/docs/release-notes/agent-release-notes/nodejs-release-notes)
-   [PHP logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-php) for agent [v10.1.0 or higher](https://docs.newrelic.com/docs/release-notes/agent-release-notes/php-release-notes)
-   [Python logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-python) for agent [v7.12.0.176](https://docs.newrelic.com/docs/release-notes/agent-release-notes/python-release-notes)
-   [Ruby logs in context procedures](https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-ruby) for agent [v8.6.0 or higher](https://docs.newrelic.com/docs/release-notes/agent-release-notes/ruby-release-notes)

If you can't or don't want to use APM log forwarding, you can [forward your logs using another solution](https://docs.newrelic.com/docs/logs/get-started/get-started-log-management#integrate-logs).

## Limitations [#limitations]

APM logs in context features are enabled by default. This can have a negative impact on your security, compliance, billing, or system performance.

Here are some additional known limitations:

-   With the exception of the Node.js agent, APM log forwarding doesn't send the complete log. [Learn about log forwarding details.](#forwarding-details)
-   Startup logs are not available until after the agent is loaded.
-   If you're using Kubernetes, be aware that we decorate the logs via instrumentation, not from the Kubernetes API. This is separate and apart from the writing logs out to the filesystem. The logs never touch the host or exist in a place where the API can be called.
-   When an exception is thrown from your application, currently you will not see stack traces in the associated logs in context for Java or .NET agents. As a workaround, you can change your drop filter rules.
-   Fluentd can add the `processID` from the entity that generated the log, but APM logs can't see that. Also, in Kubernetes, the API is called to add metadata, but this data cannot be seen from within the application. If you need the entity metadata, we recommend that you use automatic logs in context, but do not ship the logs from the application. Instead, continue to use Fluentd, Fluent Bit, or another solution to forward the log files.

If you need to adjust the default settings, see [Disable automatic logging](https://docs.newrelic.com/docs/logs/logs-context/disable-automatic-logging).

## Ensure data privacy [#data-privacy]

> #### ⚠️ CAUTION
>
> You control what log data is sent to New Relic, so be sure to follow your organization's security guidelines to mask, obfuscate, or prevent sending personal identifiable information (PII), protected health information (PHI), or any other sensitive data.

Our log ingest pipeline automatically masks credit cards, Social Security numbers, national IDs, etc. For more information, see our [security documentation](https://docs.newrelic.com/docs/logs/get-started/new-relics-log-management-security-privacy) for log management.

You can also create custom rules to mask or hash sensitive data in your logs with our obfuscation feature. This is critical when it is impractical or impossible to restrict access to sensitive data, or when some data should never be stored by New Relic. Read our [obfuscation documentation](https://docs.newrelic.com/docs/logs/ui-data/obfuscation-ui/) to find out more.

## Details about log forwarding [#forwarding-details]

For all APM agents (except Node.js), the log forwarding option doesn't forward the entire log. For details about what gets sent with log forwarding, see the collapser below.

**Log forwarding details**

For example, let's say your log has attributes like this:

````json
{
    "v": "x",
    "level": "x",
    "name": "x",
    "hostname": "x",
    "pid": 01,
    "timestamp": "x",
    "message": "x",
    "bunches": "of custom stuff"
}
```

When an APM agent (with exception of Node.js) decorates the log with our metadata and forwards it, it might have just a few attributes in New Relic, like this:

```json
{
    "timestamp": "x",
    "message": "x",
    "span.id": "x",
    "trace.id": "x",
    "hostname": "x",
    "entity.guid": "x",
    "entity.name": "x",
}
```

````

If you require more of the log reported, options include:

-   Configure the APM agent to include specific log data.
-   Keep the [log decoration](#apm-logs-defined) in place but disable APM agent log forwarding and instead use your own log forwarding solution.

For more on these options, see the agent-specific logs-in-context docs.

## Log metrics [#log-metrics]

When you set up an APM agent, you automatically get a chart of logging metrics on the APM **Summary** page:

![Screenshot showing the logging metrics chart](https://docs.newrelic.com/images/logs_screenshot-crop_logging-metrics.webp "Screenshot showing the logging metrics chart")

This chart shows a count of logs. If you haven't disabled log forwarding, you can click on the chart links that will take you to the logs themselves. Even if you disable log forwarding, this chart still shows the potential logs you could inspect if you had log forwarding enabled.

Logging metrics are reported via the `apm.service.logging.lines` attribute, as shown in the following query:

```sql
SELECT count(apm.service.logging.lines) FROM Metric WHERE (entity.guid = 'AN_ENTITY_GUID') LIMIT MAX SINCE 60 seconds AGO TIMESERIES
```

If you don't want to see the logging metrics chart, you can turn it off—but not at the account level. To disable logging metrics, see the dedicated APM configuration docs for your agent (for example, [this `logging_metrics` config option for PHP](https://docs.newrelic.com/docs/apm/agents/php-agent/configuration/php-agent-configuration/#cfg-application_logging_metrics-enabled)).

> #### ⚠️ IMPORTANT
>
> If you disable logging metrics, that doesn't turn off the APM log forwarding feature. To stop forwarding APM logs, see [Manage or disable APM logs in context](https://docs.newrelic.com/docs/logs/logs-context/disable-automatic-logging/).

## Prevent duplicate logs [#duplicate-logs]

Using logs in context functionality will increase your data ingest. Depending on your account's pricing model, this may have an impact on your ingest limits and billing.

> #### ⚠️ CAUTION
>
> If you want to use the APM agent to send logs directly from your applications, you must disable or modify log forwarding solutions that are currently collecting logs from those applications. Otherwise you will be sending duplicate logs, which will result in double billing.

Check our [upgrade guide](https://docs.newrelic.com/docs/logs/logs-context/upgrade-to-automatic-logs-context/) to learn more about how you can avoid sending duplicate logs.

For more information, follow the procedures to disable your [specific log forwarder](https://docs.newrelic.com/docs/logs/forward-logs/enable-log-management-new-relic#log-forwarding).

## Manage your ingest limits [#ingest]

**Example:** Your engineering team is troubleshooting a problem with your app, so you temporarily increase the volume of logs collected by the APM agent to provide more granular logging. However, if you leave higher limits running for several days, this could lead to sending unnecessary data that will increase your bill.

To avoid any surprises, we recommend that you use [NRQL queries](https://docs.newrelic.com/docs/accounts/accounts-billing/new-relic-one-pricing-billing/usage-queries-alerts/#data-queries) to create [alert conditions](https://docs.newrelic.com/docs/accounts/accounts-billing/new-relic-one-pricing-billing/usage-queries-alerts/#alerts) to keep track of your ingest limits. For example:

**View data limits and usage in UI**

To [review your data limits in the UI](https://docs.newrelic.com/docs/data-apis/manage-data/view-system-limits): From the [user menu](https://docs.newrelic.com/docs/accounts/accounts-billing/general-account-settings/intro-account-settings), click **Manage your data > Limits**. Or, to view your usage, click **View your usage**.

![Screenshot of View data and usage page in UI](https://docs.newrelic.com/images/apm_screenshot-full_view-limits-usage.webp "View data and usage from UI")

**NRQL query: Estimated cost example**

See [cost-related queries](https://docs.newrelic.com/docs/accounts/accounts-billing/new-relic-one-pricing-billing/usage-queries-alerts#estimated-cost).

**NRQL alert: Usage threshold example**

To create an alert when your usage exceeds a fixed monthly threshold for gigabytes of data, add this [NRQL query](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-nrql-alert-conditions/) to your alert condition:

````sql
FROM NrMTDConsumption SELECT latest(GigabytesIngested) WHERE productLine = 'DataPlatform'
```

````

## Example: The power of logs in context [#response-time-example]

Here is a detailed use case of using APM logs in context to get to the root cause of a problem.

**Example scenario:** The on-call engineer receives a New Relic alert notification about poor response time and rising error rates for their app. They need to discover the root cause behind the increase in errors and latency, so they can decide whether to rotate a problematic host out of load balancing or to roll back the most recent release.

To start troubleshooting, they go to the New Relic UI.

**When and where did the errors begin?**

When the engineer looks at the app's **APM Summary** page, they see many more error logs leading up to the increase in response time. They want to see if the logs can expose the root cause of the problem.

![Screenshot of APM Summary errors in UI](https://docs.newrelic.com/images/apm_screenshot-crop_apmsummary-view-logs-.webp "APM Summary errors in UI")

**How can they quickly decide what's related to the problem?**

They have not set up log collection. But the Java agent for their app has been recently updated to include APM logs in context, so the app now automatically receives logs and linking metadata. The metrics that appear in the UI facet the count by log severity. (This is set by the Java agent's built-in log decorating capabilities.)

**What entities are related?**

On the same UI page, they click the **Logs** chart, so they can review the **Log summary** view. This view shows them when instrumented logs have been collected and when those logs had an `Error` severity level or worse. It also shows them a list of log patterns and what percentage of all logs fit each pattern.

**What patterns emerge in the logs?**

In the **Log patterns** UI, rare error messages have begun to occur frequently, and they now appear in the list. This helps the engineer to focus on problematic logs instead of all the noisy status updates.

![Screenshot of Log patterns UI](https://docs.newrelic.com/images/apm_screenshot-crop_logs-context-patterns-two.webp "Logs patterns UI")

**What patterns should they focus on?**

The engineer notices an interesting log pattern and wants to view only the logs that fit this pattern. They click on it to add the pattern's value as a filter to the logs being displayed. This narrows the focus.

**What can more log details reveal?**

The engineer wants to see all the values contained in the log record, so they click the **Log details** view. This helps them validate that the log itself is meaningful. It also lets them drill down further into either the Kubernetes environment where the app runs, or directly to any distributed traces or APM errors related to the log record.

![Screenshot of Log details in APM UI](https://docs.newrelic.com/images/apm_logs_ui_with_log_detail_side_panel.webp "Log details in APM UI")

The engineer decides to look at the APM error where they can see a full stack trace. They can also see logs from their **Errors inbox** UI.

![Screenshot of logs from Errors inbox UI](https://docs.newrelic.com/images/apm_screenshot-crop_logs-cropped-ui.webp "Logs from Errors inbox UI")

**What additional test data will help troubleshoot?**

The troubleshooting team has been running tests to isolate the cause, but not all logs have been collected for the test transactions. The default log collection in their Java agent's config file (approximately 10,000 logs per minute in `application_logging:forwarding:max_samples_stored:`) has been useful to validate that the services have an issue. But now they need to turn up the sampling to a higher number for more detailed troubleshooting.

**Problem solved. Now what?**

The troubleshooting team determines the problem stems from a recently introduced change, so they roll back that code. To save resources and ingest expenses, they turn down log collection for `application_logging:forwarding:max_samples_stored:` in their config file, but they keep log sampling running in case the issue repeats itself.

The troubleshooting team also updates their runbooks to point to the **Log patterns** page filtered to this app, and they [add the runbook to their alert condition](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/advanced-alerts/understand-technical-concepts/provide-runbook-instructions-alert-activity/). Next time they get an alert similar to this situation, they can use these lessons learned for faster troubleshooting.