#Tasks for Envoy Gateway

#Overview

Use these advanced Envoy Gateway tasks after completing the main configuration path: operator, gateway, route, and policy.

When applying configuration changes in the Gateway API, there are three primary approaches available:

1. Modify standard Gateway API resources: Directly edit Gateway, HTTPRoute, TCPRoute, UDPRoute, and other core resources.
2. Attach policies via PolicyAttachment: Use SecurityPolicy, ClientTrafficPolicy, BackendTrafficPolicy, and other policy resources to extend Gateway and Route behavior.
3. Configure global settings: Modify the EnvoyGatewayCtl to change envoy-gateway instance behavior or other global settings that affect all gateways.

Advanced tasks and special scenarios include cross-namespace routing, observability setup, deployment customization, and troubleshooting.

#Prerequisites

1. Configure EnvoyGatewayCtl
2. Configure Gateway
3. Configure Route
4. Configure GatewayAPI Policy

#Advanced Tasks

#OpenTelemetry (OTel)

Please follow instructions in OpenTelemetry Integration, but use EnvoyGatewayCtl to modify the envoy-gateway-config.

#How To Attach to Listener Created In Other Namespace

In the Gateway's listener configuration, you need to specify which namespaces are allowed to attach Routes to it.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example-gateway
spec:
  listeners:
    - name: http-80
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All # without limit
    - name: http-81
      protocol: HTTP
      port: 81
      allowedRoutes:
        namespaces:
          from: Same # only allow routes in the same namespace
    - name: http-82
      protocol: HTTP
      port: 82
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              team: frontend # only allow routes in the namespace with label team=frontend

Please refer to Cross-Namespace routing for more details.

#How To Use a Certificate Created In Another Namespace

To use a certificate created in another namespace, create a ReferenceGrant in the namespace where the certificate is stored. Please follow instructions in cross-namespace-certificate-references and referencegrant.

#How To Use SSL Passthrough

Please follow instructions in

• tls
• tls-passthrough

#How To Change the Minimum TLS Version

Please follow instructions in customize-gateway-tls-parameters

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: enforce-tls-13
  namespace: default
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  tls:
    minVersion: "1.3"
EOF

The .spec.tls field in ClientTrafficPolicy is clienttlssettings. If you need to customize cipher suites in addition to the minimum TLS version, refer to the same upstream task and API reference.

#How To Specify NodePort When Using NodePort Service

When using a NodePort service, Kubernetes assigns a NodePort value to each service port. When accessing the service through a node IP, use the assigned NodePort instead of the service port.

There are two approaches:

Manually retrieve the NodePort assignment by following get nodeport from svc port.

Manually specify the NodePort in the EnvoyProxy configuration instead of letting Kubernetes automatically assign it.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
spec:
  ipFamily: DualStack
  provider:
    kubernetes:
      envoyDeployment:
        container:
          imageRepository: registry.alauda.cn:60080/acp/envoyproxy/envoy
      envoyService:
        patch:
          type: StrategicMerge
          value:
            spec:
              ports:
                - nodeport: 31888
                  port: 80
        type: NodePort
    type: Kubernetes

#How To Specify a VIP When Using MetalLB

When using MetalLB as the LoadBalancer provider, you can specify a static VIP for the Gateway service through service annotations.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
  namespace: demo
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: LoadBalancer
        annotations:
          metallb.universe.tf/address-pool: production
          metallb.universe.tf/loadBalancerIPs: VIP_IP

Available Annotations:

#How To Add Pod Annotations In Envoy Gateway

Set annotations in .spec.provider.kubernetes.envoyDeployment.patch.value.spec.template.metadata.annotations of EnvoyProxy.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: demo
spec:
  infrastructure:
    parametersRef:
      group: gateway.envoyproxy.io
      kind: EnvoyProxy
      name: demo
  gatewayClassName: envoy-gateway-operator-cpaas-default
  listeners:
    - name: http
      port: 80
      protocol: HTTP
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
spec:
  provider:
    kubernetes:
      envoyDeployment:
        replicas: 1
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                metadata:
                  annotations:
                    your-annotation-key: 'your-annotation-value'
        container:
          imageRepository: registry.alauda.cn:60080/acp/envoyproxy/envoy
    type: Kubernetes

#How To Set NodeSelector And Tolerations For envoy-gateway-operator

Update the Subscription resource.

# example of nodeSelector and tolerations
kubectl patch subscription envoy-gateway-operator  -n envoy-gateway-operator   --type='merge' -p '
{
  "spec": {
    "config": {
      "nodeSelector": {
        "node-role.kubernetes.io/infra": ""
      },
      "tolerations": [
        {
          "effect": "NoSchedule",
          "key": "node-role.kubernetes.io/infra",
          "operator": "Equal",
          "value": "reserved"
        }
      ]
    }
  }
}'

#How To Set NodeSelector And Tolerations For envoy-gateway

Update the EnvoyGatewayCtl resource.

# in default $NAME=cpaas-default and $NS=envoy-gateway-operator
kubectl patch envoygatewayctl $NAME -n $NS --type='merge' -p '
{
  "spec": {
    "deployment": {
      "pod": {
        "nodeSelector": {
          "node-role.kubernetes.io/infra": ""
        },
        "tolerations": [
          {
            "effect": "NoSchedule",
            "key": "node-role.kubernetes.io/infra",
            "operator": "Equal",
            "value": "reserved"
          }
        ]
      }
    }
  }
}'

#How To Set NodeSelector And Tolerations For envoy-proxy

Update the EnvoyProxy resource.

kubectl patch envoyproxy $NAME -n $NS --type='merge' -p '
{
  "spec": {
    "provider": {
      "kubernetes": {
        "envoyDeployment": {
          "pod": {
            "nodeSelector": {
              "node-role.kubernetes.io/infra": ""
            },
            "tolerations": [
              {
                "effect": "NoSchedule",
                "key": "node-role.kubernetes.io/infra",
                "operator": "Equal",
                "value": "reserved"
              }
            ]
          }
        }
      }
    }
  }
}'

#How To Use hostNetwork In envoy-proxy

Using hostNetwork: true allows the Envoy proxy pods to use the host network namespace directly. This can be useful for:

• achieving better network performance
• accessing the gateway through the node IP directly

Considerations:

• Pods using hostNetwork will bind directly to the host's network interfaces
• Port conflicts may occur if multiple pods try to use the same port on the same node
• Security isolation is reduced as pods share the host's network namespace
• You should use nodeSelector or affinity rules to control pod placement and avoid port conflicts

There are two approaches to configure hostNetwork, depending on whether you want to use privileged ports (< 1024) directly:

#Approach 1: Using Port Offset (Default, Recommended)

This is the default and recommended approach. Envoy Gateway automatically adds an offset of 10000 to privileged ports to avoid requiring special permissions.

Pros:

• No special permissions or capabilities required
• More secure because it runs as a non-root user without additional privileges
• Simpler configuration
• Works out of the box

Cons:

• Clients must use offset ports (10080, 10443)

Configuration:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
  namespace: demo
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                spec:
                  hostNetwork: true                    # Enable host network mode
                  dnsPolicy: ClusterFirstWithHostNet   # Required for proper DNS resolution
        pod:
          nodeSelector:                                # Recommended: control pod placement to avoid conflict
            kubernetes.io/hostname: "demo"

Access:

• Port 80 → Access via http://<node-ip>:10080
• Port 443 → Access via https://<node-ip>:10443

#Approach 2: Using Privileged Ports Directly with useListenerPortAsContainerPort

This approach allows Envoy to bind to privileged ports (< 1024) directly, such as port 80 and 443.

Pros:

• Can use standard ports (80 and 443) directly
• Better compatibility with clients expecting standard ports

Cons:

• Requires NET_BIND_SERVICE capability
• Slightly reduced security compared to port offset approach
• More complex configuration

Configuration:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
  namespace: demo
spec:
  provider:
    type: Kubernetes
    kubernetes:
      useListenerPortAsContainerPort: true  # Disable port offset
      envoyDeployment:
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                spec:
                  hostNetwork: true
                  dnsPolicy: ClusterFirstWithHostNet
                  containers:
                  - name: envoy
                    command:
                      - /usr/local/bin/envoy-with-cap  # use envoy with filecap
                    securityContext:
                      capabilities:
                        add:
                        - NET_BIND_SERVICE  # Required for binding to privileged ports
        pod:
          nodeSelector:                                # Recommended: control pod placement to avoid conflict
            kubernetes.io/hostname: "demo"

Access:

• Port 80 → Access via http://<node-ip>:80
• Port 443 → Access via https://<node-ip>:443

#How To Access a LoadBalancer VIP from Within the Cluster

By default, Envoy Gateway creates LoadBalancer services with externalTrafficPolicy: Local. This policy preserves client source IP addresses but has an important limitation: requests from cluster nodes without Envoy Gateway pods will fail because traffic is not forwarded to other nodes.

Solution 1: Use Service ClusterIP (Recommended for in-cluster access)

For applications running inside the cluster, use the service ClusterIP instead of the LoadBalancer VIP. This avoids the routing limitation entirely.

Solution 2: Change to Cluster Traffic Policy

If you need to access the LoadBalancer VIP from any cluster node, change externalTrafficPolicy to Cluster:

kubectl patch envoyproxy $GATEWAY_NAME -n $GATEWAY_NS --type='json' -p='[
  {"op": "replace", "path": "/spec/provider/kubernetes/envoyService/externalTrafficPolicy", "value": "Cluster"}
]'

#How To Configure HSTS

HTTP Strict Transport Security (HSTS) tells browsers to interact with a hostname only over HTTPS. ACP configures HSTS on a Gateway with ClientTrafficPolicy .spec.headers.lateResponseHeaders. Late response headers are applied by the Gateway listener after every other header modification, so the Strict-Transport-Security header is present on every response of the Gateway, including responses generated by Envoy itself (for example the 404 returned when no route matches the request), which a route-level ResponseHeaderModifier filter cannot cover.

This feature is available in ACP 4.5 and later.

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: hsts-policy
  namespace: default
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  headers:
    lateResponseHeaders:
      set:
      - name: Strict-Transport-Security
        value: "max-age=31536000; includeSubDomains"
EOF

Verify the header on a normal response and on the Envoy-generated 404 of an unmatched host:

curl -sk -v --resolve www.example.com:443:<gateway-ip> https://www.example.com/ 2>&1 | grep -i strict-transport
curl -sk -v https://<gateway-ip>/ 2>&1 | grep -E "^< HTTP|strict-transport"

Notes:

• Use set instead of add so that backends cannot weaken the header value. addIfAbsent and remove are also available in lateResponseHeaders.
• HSTS does not redirect HTTP traffic to HTTPS; it only tells browsers that already received the header over HTTPS to stay on HTTPS. Configure an HTTP-to-HTTPS redirect on the route as well if needed.
• Browsers cache the decision for max-age seconds and it cannot be undone from the server during that period. Validate with a small max-age (for example 300) before rolling out one year.
• For the full API, see the official Envoy Gateway HTTP response headers documentation.

#How To Configure Custom Error Pages

By default, the Gateway serves plain Envoy-generated error bodies (for example no healthy upstream on 503) and passes backend error responses through unchanged. Two mechanisms return custom error pages; both are available in ACP 4.5 and later:

• BackendTrafficPolicy .spec.responseOverride[] rewrites responses that match a status code.
• HTTPRouteFilter .spec.directResponse returns a fixed response for a route rule without contacting any backend.

responseOverride customizes error pages by status code. The body can be an inline string or the response.body key of a ConfigMap:

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: error-page-policy
  namespace: default
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: backend
  responseOverride:
  - match:
      statusCodes:
      - type: Value
        value: 404
    response:
      contentType: text/html
      body:
        type: Inline
        inline: "<html><body><h1>Page not found</h1></body></html>"
  - match:
      statusCodes:
      - type: Value
        value: 500
    response:
      contentType: application/json
      body:
        type: ValueRef
        valueRef:
          group: ""
          kind: ConfigMap
          name: error-page-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: error-page-config
  namespace: default
data:
  response.body: '{"error": "Internal Server Error"}'
EOF

The optional source field of each rule controls which responses it rewrites: All (default) rewrites both backend and Envoy-generated responses, Backend only responses from backends, and Local only responses generated by Envoy itself, for example the 503 produced when a route matches but its backend has no ready endpoints. Use Local and Backend rules with the same status code to serve different pages for gateway errors and backend errors. A rule can also rewrite the status code (response.statusCode) or fetch the response from another URL (redirect).

directResponse answers a path without contacting any backend. Reference the filter from an HTTPRoute rule through an ExtensionRef; the HTTPRouteFilter must live in the same namespace as the HTTPRoute, and the body size cannot exceed 4096 bytes:

cat <<EOF | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
  namespace: default
spec:
  parentRefs:
  - name: eg
  hostnames:
  - www.example.com
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /maintenance
    filters:
    - type: ExtensionRef
      extensionRef:
        group: gateway.envoyproxy.io
        kind: HTTPRouteFilter
        name: maintenance-page
  - backendRefs:
    - name: backend
      port: 3000
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: HTTPRouteFilter
metadata:
  name: maintenance-page
  namespace: default
spec:
  directResponse:
    contentType: text/html
    statusCode: 503
    body:
      type: Inline
      inline: "<html><body><h1>Under maintenance</h1></body></html>"
EOF

Verify the rewritten page:

curl -s -w '\n%{http_code}\n' --resolve www.example.com:80:<gateway-ip> http://www.example.com/missing-page

Notes:

• responseOverride only applies to responses that pass through the targeted HTTPRoute. A request that matches no route at all receives the default Envoy 404 and no rule applies to it.
• A direct response is generated by Envoy itself, so a responseOverride rule on the same HTTPRoute with a matching status code (for example 503 with source: Local or the default source: All) rewrites the direct response. Keep direct responses on a separate HTTPRoute when both mechanisms are used together.
• For the full API, see the official Envoy Gateway documentation for response override and direct response.

#Related Documentation

• Configure EnvoyGatewayCtl
• Configure Gateway
• Configure Route
• Configure GatewayAPI Policy

#More Configuration

Please refer to EnvoyGateway Tasks

#Trademark Notice

Envoy® is a registered trademark of The Linux Foundation.

MetalLB™ is a trademark of The Linux Foundation.

Alauda is an independent vendor. This product is not affiliated with, endorsed by, or sponsored by The Linux Foundation. All trademarks are the property of their respective owners and are used here for identification purposes only.