Loading

Central configuration for EDOT SDKs

Stack Preview 9.1.0 ECH Preview 9.1.0

Manage Elastic Distribution of OpenTelemetry (EDOT) SDKs through the APM Agent Central Configuration feature in the Applications UI. Changes are automatically propagated to the deployed EDOT SDKs. Refer to APM Agent Central Configuration for more information.

This feature implements the Open Agent Management Protocol (OpAMP). Refer to Open Agent Management Protocol for more information.

The central configuration architecture for the Elastic Distribution of OpenTelemetry (EDOT) provides a robust and scalable mechanism for managing fleets of EDOT SDKs remotely. The data flow, illustrated in this diagram, ensures that configuration changes are efficiently propagated from a central management point to each individual agent.

Diagram of Central config architecture

The process starts within Kibana, where administrators create and manage settings for the EDOT SDKs. Once defined, settings are written to and persisted in Elasticsearch, which acts as the single source of truth. The EDOT Collector, when configured in Gateway mode, includes the Elastic APM Config Extension, which reads the SDK settings from Elasticsearch, making them available for distribution.

Each EDOT SDK contains an embedded OpAMP Client. Following the Open Agent Management Protocol (OpAMP), these clients periodically poll the OpAMP server, bundled with the Collector's APM Config Extension, over HTTP. This polling action allows the SDKs to retrieve the latest configuration updates, enabling dynamic and centralized control over their behavior without requiring manual intervention or redeployment.

To use APM Agent Central Configuration for EDOT SDKs, you need:

  • An Elastic self-managed or Elastic Cloud deployment, version 9.1 or higher.
  • A standalone EDOT Collector, in either Agent or Collector mode.
  • EDOT SDKs instrumenting your application.

The following versions of EDOT and Elastic Stack support central configuration:

Component Minimum version
Kibana 9.1 or higher
EDOT Collector 8.19, 9.1 or higher
EDOT Android 1.2.0 or higher
EDOT iOS 1.4.0 or higher
EDOT Java 1.5.0 or higher
EDOT Node.js 1.2.0 or higher
EDOT PHP 1.1.1 or higher
EDOT Python 1.4.0 or higher
Note

Serverless deployments are not currently supported.

To activate APM Agent Central Configuration for EDOT SDKs, follow these steps.

  1. Retrieve your credentials

    You need a valid Elasticsearch API key to authenticate to the Elasticsearch endpoint.

    Retrieve your Elasticsearch URL and your API key:

    1. Retrieve the Elasticsearch URL for your Elastic Cloud deployment:

      1. Go to the Elastic Cloud console.
      2. Next to your deployment, select Manage.
      3. Under Applications next to Elasticsearch, select Copy endpoint.

    2. Create an API Key following these instructions.

    Make sure the API key has config_agent:read permissions and resources set to -.

    Example JSON payload 
    POST /_security/api_key
{
  "name": "apmconfig-opamp-test-sdk",
  "metadata": {
    "application": "apm"
  },
  "role_descriptors": {
    "apm": {
      "cluster": [],
      "indices": [],
      "applications": [
        {
          "application": "apm",
          "privileges": [
            "config_agent:read"
          ],
          "resources": [
            "*"
          ]
        }
      ],
      "run_as": [],
      "metadata": {}
    }
  }
}

  2. Edit the EDOT Collector configuration

    Edit the EDOT Collector configuration to activate the central configuration feature:

    extensions:
  bearertokenauth:
    scheme: "APIKey"
    token: "<ENCODED_ELASTICSEARCH_APIKEY>"

  apmconfig:
    opamp:
      protocols:
        http:
          # Default is localhost:4320
          # To specify a custom endpoint, uncomment the following line
          # and set the endpoint to the custom endpoint
          # endpoint: "<CUSTOM_OPAMP_ENDPOINT>"
    source:
      elasticsearch:
        endpoint: "<ELASTICSEARCH_ENDPOINT>"
        auth:
          authenticator: bearertokenauth

service:
  extensions: [bearertokenauth, apmconfig]
    Note

    For comprehensive authentication configuration options, refer to Authentication methods.

    Restart the Elastic Agent to also restart the Collector and apply the changes.

    Refer to Secure connection if you need to secure the connection between the EDOT Collector and Elastic using TLS or mutual TLS.

  3. Set the environment variables for the SDKs

    Activate the central configuration feature in the SDKs by setting the ELASTIC_OTEL_OPAMP_ENDPOINT environment variable to the URL endpoint of the apmconfig extension that you configured in the previous step. For example:

    export ELASTIC_OTEL_OPAMP_ENDPOINT="http://localhost:4320/v1/opamp"

    If the OpAMP server in the Collector requires authentication set the ELASTIC_OTEL_OPAMP_HEADERS environment variable.

    export ELASTIC_OTEL_OPAMP_HEADERS="Authorization=ApiKey an_api_key"

    Restart the instrumented application to apply the changes.

    Important

    Support for the ELASTIC_OTEL_OPAMP_HEADERS environment variable depends on each SDK. Refer to the configuration reference of each EDOT SDK for more information.

  4. Check that the EDOT SDK shows up

    Wait some time for the EDOT SDK to appear in Kibana under Agent Configuration.

    1. Go to Kibana → Observability → Applications and select a service.
    2. Select Settings and go to Agent Configuration.

    Your application must produce and send telemetry data for the EDOT SDK to appear in Agent Configuration. This is because central configuration requires an application name as the key, which can't be defined until the application name is associated with the EDOT SDK agent after receiveing telemetry.

    Note

    Central configuration uses the service.name and deployment.environment.name OpenTelemetry resource attributes to target specific instances with a configuration. If no environment is specified, the central configuration feature will match All as the environment.

For a list of settings that you can configure through APM Agent Central Configuration, refer to the configuration reference of each EDOT SDK:

Stack Preview 9.2.0

The Advanced Configuration feature allows you to define custom configuration options as key-value pairs. Settings are passed directly to your EDOT SDK.

Warning

Use this feature with caution. An incorrect or incompatible setting might affect the behavior of the EDOT SDK.

To deactivate central configuration, remove the ELASTIC_OTEL_OPAMP_ENDPOINT environment variable.

Restart the instrumented application to apply the changes.