Allow env tag or custom tags in Datadog log stream integration

Help
,
1

TLDR:
In the Log Streaming integration for Datadog, set the standard env (dd_env) tag and/or let users specify additional custom static value tags to be included in each log.

Details:
Auth0’s Log Streaming integration for Datadog sets the source and service tags. However, it doesn’t set the env (dd_env) tag. This can make managing logs in Datadog by environment more difficult.

In my case, I have two environments and one Datadog account. Across all my infrastructure shipping logs to Datadog, I set an env tag representing the environment. This makes it easy to manage Datadog logs (queries, alerts, etc.) across environments, since there’s a single field shared by all logs that indicates the environment.

Because Log Streaming doesn’t set, or allow me to specify, the env tag, I’m forced to resort to use the data.hostname (which distinguishes the tenant). That’s an exception to my general rule of identifying environments using the same tag name, making it more clumsy to query/filter the logs.

Addendum
It appears to be able to resolve this by setting up a Lookup function in a Datadog Pipeline that processes the logs from Auth0.

1 Like
2

For those encountering this issue, I was able to address it by configuring a log ingestion pipeline in Datadog for the auth0 source, using a Remapper to make the tenant tag into an attribute, followed by a Lookup Processor to map the tenant value (now available in an attribute) into another attribute, and finally followed by another Remapper which maps that attribute to an env tag.

1 Like