Upgrading the Auto-instrumentation Agents
Each Instrumentation resource names the agent image it injects in spec.<language>.image, and the agent is copied into the application pod by an init container that the Operator injects when the pod is admitted. Upgrading an agent therefore has two parts:
- Updating the image reference in the
Instrumentationresource - Restarting the instrumented workloads, because a running pod keeps the agent version it started with
The examples below upgrade the Java agent from 2.26.1 to 2.30.0. The same procedure applies to the other supported languages by substituting the corresponding field — spec.python.image, spec.nodejs.image, spec.dotnet.image, spec.go.image, or spec.apacheHttpd.image.
The Operator records the image it would inject by default in the instrumentation.opentelemetry.io/default-auto-instrumentation-<language>-image annotation, and on startup it rewrites spec.<language>.image for any language whose image still matches that recorded default. An image that you supply yourself never matches, so it is left untouched. That is always the case here, because the Operator provides the injection mechanism only and includes no pre-built instrumentation images: every spec.<language>.image holds an image that you built or sourced yourself.
Prerequisites
- The Alauda Build of OpenTelemetry v2 Operator is upgraded to 0.157.0, as described in Upgrading Alauda Build of OpenTelemetry v2.
- The agent image to upgrade to is built or sourced by you and is pullable from the cluster. The Operator does not include pre-built instrumentation libraries or container images; see Instrumentation Options.
- An active ACP CLI (
kubectl) session by a cluster administrator with thecluster-adminrole.
TOC
Updating the Instrumentation ResourceRestarting the Instrumented WorkloadsVerificationVersion ConsiderationsUpdating the Instrumentation Resource
-
List the
Instrumentationresources and the Java agent image each one injects:Example output
-
Set the environment variables used by the remaining commands:
-
Update the image reference:
NOTEThe
Instrumentationresource is read only when a pod is admitted, so this patch has no effect on pods that are already running, and none of them restart.The Operator fills in a default image for every language it supports, not only the one you configure, and warns when one of those defaults is too old to be upgraded automatically — for example
dotnet image ... 1.2.0 is at a version that cannot be automatically upgraded. The warning is unrelated to the language you are upgrading and does not prevent the patch from being applied.
Restarting the Instrumented Workloads
-
List the workloads that the updated
Instrumentationresource applies to:NOTEThe injection annotation can also be set on the namespace, in which case every workload in that namespace is instrumented and the command above returns nothing. Adjust the selection to match how injection is enabled in your deployment, and cover
DaemonSetandStatefulSetworkloads the same way. -
Restart each workload and wait for it to become ready:
WARNINGRestarting the workloads interrupts the traffic they serve. Roll them out during a maintenance window, or one at a time, if the applications do not tolerate a rolling restart.
Verification
-
Confirm that the injected init container runs the new agent image:
Example output
A pod listed without an image is not instrumented, either because it carries no injection annotation or because it was admitted before the annotation was added.
-
Confirm that the agent is loaded into the application JVM. The Operator mounts the agent under a directory named after the container it instruments:
Example output
-
Generate application traffic and confirm that traces still reach the Collector. If the Collector uses the
debugexporter, the span batches appear in its logs:Example output
Version Considerations
- The
InstrumentationAPI is unchanged in this release. Two optional fields are added:spec.initContainerSecurityContext, which sets the security context of the injected init containers, andspec.go.securityContext, which overrides the defaults of the Go sidecar. See Security Context Configuration. - The agents are released independently of the Operator and follow their own version streams. Review the release notes of the agent you are upgrading for behavior changes, such as new or renamed span attributes, before rolling it out to production.
- The exporter endpoint configured in
spec.exporter.endpointis unaffected by the upgrade. The Java agent still defaults to thehttp/protobufprotocol, so the endpoint must address the Collector OTLP HTTP receiver on port4318unlessOTEL_EXPORTER_OTLP_PROTOCOL=grpcis set explicitly.