Target Allocator

The OpenTelemetry Operator comes with an optional component, theTarget Allocator(TA). In a nutshell, the TA is a mechanism for decoupling the service discoveryand metric collection functions of Prometheus such that they can be scaledindependently. The Collector manages Prometheus metrics without needing toinstall Prometheus. The TA manages the configuration of the Collector’sPrometheus Receiver.

The TA serves two functions:

1. Even distribution of Prometheus targets among a pool of Collectors
2. Discovery of Prometheus Custom Resources

Getting Started

When creating an OpenTelemetryCollector Custom Resource (CR) and setting the TAas enabled, the Operator will create a new deployment and service to servespecifichttp_sd_config directives for each Collector pod as part of that CR.It will also change the Prometheus receiver configuration in the CR, so that ituses thehttp_sd_configfrom the TA. The following example shows how to get started with the TargetAllocator:

apiVersion:opentelemetry.io/v1alpha1kind:OpenTelemetryCollectormetadata:name:collector-with-taspec:mode:statefulsettargetAllocator:enabled:trueconfig:|    receivers:      prometheus:        config:          scrape_configs:          - job_name: 'otel-collector'            scrape_interval: 10s            static_configs:            - targets: [ '0.0.0.0:8888' ]            metric_relabel_configs:            - action: labeldrop              regex: (id|name)              replacement: $$1            - action: labelmap              regex: label_(.+)              replacement: $$1    exporters:      # NOTE: Prior to v0.86.0 use `logging` instead of `debug`.      debug:    service:      pipelines:        metrics:          receivers: [prometheus]          processors: []          exporters: [debug]

Behind the scenes, the OpenTelemetry Operator will convert the Collector’sconfiguration after the reconciliation into the following:

receivers:prometheus:config:scrape_configs:-job_name:otel-collectorscrape_interval:10shttp_sd_configs:-url:http://collector-with-ta-targetallocator:80/jobs/otel-collector/targets?collector_id=$POD_NAMEmetric_relabel_configs:-action:labeldropregex:(id|name)replacement:$$1-action:labelmapregex:label_(.+)replacement:$$1exporters:debug:service:pipelines:metrics:receivers:[prometheus]processors:[]exporters:[debug]

Note how the Operator removes any existing service discovery configurations(e.g.,static_configs,file_sd_configs, etc.) from thescrape_configssection and adds anhttp_sd_configs configuration pointing to a TargetAllocator instance it provisioned.

For more detailed information on the TargetAllocator, seeTargetAllocator.