Alauda Build of HAMi
  • English

    • Configure Dynamic MIG

    HAMi dynamic MIG uses nvidia-mig-parted to create NVIDIA MIG instances for HAMi workloads while keeping the HAMi resource API. Use this advanced path only when the installed Alauda Build of HAMi release supports dynamic MIG and the NVIDIA GPU base is prepared for MIG.

    Dynamic MIG is a HAMi allocation mode. NVIDIA GPU hardware support, driver requirements, and direct MIG behavior remain in the NVIDIA GPU MIG documentation.

    WARNING

    Changing MIG geometry can reset the GPU. Drain or stop workloads and any process that prevents a GPU reset before enabling or changing the mode. Coordinate NVIDIA metrics components through the NVIDIA GPU product documentation.

    TOC

    Before you beginEnable dynamic MIG on a nodeRequest a dynamic MIG deviceVerify the modeOperational limitsNext steps

    Before you begin

    • Install Alauda Build of HAMi with the NVIDIA backend enabled.
    • Confirm that the target NVIDIA GPU model and driver support MIG.
    • Confirm that no other device plugin or DRA driver exposes the same physical GPU.
    • Record the current HAMi node configuration. Direct ConfigMap changes can be overwritten by a later cluster-plugin update.

    Back up the current configuration before changing it:

    kubectl -n kube-system get configmap hami-device-plugin \
  -o yaml > hami-device-plugin-before-mig.yaml

    Enable dynamic MIG on a node

    Edit the config.json data in the hami-device-plugin ConfigMap and add an entry for each dynamic MIG node:

    kubectl -n kube-system edit configmap hami-device-plugin
    {
  "nodeconfig": [
    {
      "name": "<mig-node-name>",
      "operatingmode": "mig",
      "filterdevices": {
        "uuid": [],
        "index": []
      }
    }
  ]
}

    Preserve entries for other nodes. After updating the configuration, restart the HAMi scheduler and device plugin during the planned maintenance window:

    kubectl -n kube-system rollout restart deployment hami-scheduler
kubectl -n kube-system rollout restart daemonset hami-device-plugin

    If the deployment uses release-prefixed resource names, obtain the actual names with kubectl get deployment,daemonset -n kube-system | grep hami.

    Request a dynamic MIG device

    HAMi uses the same nvidia.com/gpualloc and nvidia.com/gpumem resources for HAMi-Core and dynamic MIG workloads. Use the mode annotation when the workload must run on a dynamic MIG node:

    apiVersion: v1
kind: Pod
metadata:
  name: hami-dynamic-mig
  annotations:
    nvidia.com/vgpu-mode: "mig"
spec:
  containers:
    - name: workload
      image: <your-gpu-image>
      resources:
        limits:
          nvidia.com/gpualloc: 1
          nvidia.com/gpumem: 8000

    HAMi selects the first supported MIG geometry that can satisfy the memory request. The node does not expose static resources such as nvidia.com/mig-1g.10gb for this HAMi path.

    Verify the mode

    Check the workload, device-plugin logs, and HAMi resource view:

    kubectl get pod hami-dynamic-mig -o wide
kubectl -n kube-system logs daemonset/hami-device-plugin --tail=200
kubectl get node <mig-node-name> -o jsonpath='{.status.allocatable}'

    The workload must become Running, the device-plugin logs must not contain an nvidia-mig-parted failed error, and the workload must continue to request the standard HAMi NVIDIA resource keys.

    Operational limits

    • nvidia.com/gpualloc cannot exceed the number of physical GPUs available to the workload.
    • Geometry changes can require another GPU reset.
    • Direct edits to chart-generated ConfigMaps can be overwritten during upgrade. Back up and revalidate the node configuration before every HAMi upgrade.
    • Use only geometries supported by the installed HAMi package and target GPU model. For the complete upstream geometry format, see Project-HAMi dynamic MIG support.

    Next steps