Exposing the metrics

The OpenTelemetry Collector exposes metrics about the volume of data it has processed. These metrics are valuable for monitoring the Collector's operational status and diagnosing data collection issues.

Available metrics

The Collector exposes the following metrics related to data processing volume:

Spans metrics

  • otelcol_receiver_accepted_spans: Number of spans accepted by receivers
  • otelcol_receiver_refused_spans: Number of spans refused by receivers
  • otelcol_exporter_sent_spans: Number of spans sent by exporters
  • otelcol_exporter_enqueue_failed_spans: Number of spans that failed to enqueue in exporters

Logs metrics

  • otelcol_receiver_accepted_logs: Number of log records accepted by receivers
  • otelcol_receiver_refused_logs: Number of log records refused by receivers
  • otelcol_exporter_sent_logs: Number of log records sent by exporters
  • otelcol_exporter_enqueue_failed_logs: Number of log records that failed to enqueue in exporters

Metrics metrics

  • otelcol_receiver_accepted_metrics: Number of metric data points accepted by receivers
  • otelcol_receiver_refused_metrics: Number of metric data points refused by receivers
  • otelcol_exporter_sent_metrics: Number of metric data points sent by exporters
  • otelcol_exporter_enqueue_failed_metrics: Number of metric data points that failed to enqueue in exporters

Configuring metrics exposure

Configuring the Prometheus metrics endpoint

To expose the Collector's internal metrics, configure a Prometheus pull exporter in the OpenTelemetryCollector custom resource (CR). The following example configures the Collector to expose metrics in Prometheus format on port 8888:

apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
  name: <collector_name>
spec:
  config:
    service:
      telemetry:
        metrics:
          readers:
            - pull:
                exporter:
                  prometheus:
                    host: "0.0.0.0"
                    port: 8888
                    without_scope_info: true
                    without_type_suffix: true
                    without_units: true

Enabling metrics scraping

To enable Prometheus to automatically scrape the Collector metrics, set the spec.observability.metrics.enableMetrics field to true in the OpenTelemetryCollector CR. The Operator automatically creates a ServiceMonitor or PodMonitor resource based on the Collector's deployment mode:

apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
  name: <collector_name>
spec:
  observability:
    metrics:
      enableMetrics: true

Manual verification (optional)

You can verify the metrics endpoint directly by using kubectl port-forward to access the Collector's metrics port:

kubectl port-forward <collector_pod_name> 8888:8888

Then, open http://localhost:8888/metrics in your browser or use curl to verify that the metrics are exposed:

curl http://localhost:8888/metrics

Accessing the metrics

After enabling metrics scraping, you can verify and access the Collector metrics through the Prometheus web console in Alauda Container Platform:

  1. Access the Prometheus web console in your Alauda Container Platform cluster.

  2. Navigate to StatusTargets.

  3. Verify that the ServiceMonitors or PodMonitors in the <instance_name>-collector format have the Up status.

Once the targets are confirmed as Up, you can use the Prometheus query interface to explore the Collector metrics listed in the Available metrics section.