Verification

Use this page to confirm that Alauda Build of NPU Operator installed the NPU stack and that workloads can consume NPU resources.

Verify operator status

Check the NPUOperatorCtl instance:

kubectl -n npu-operator get npuoperatorctl

The Deployed condition should be True. A newly installed NPUOperatorCtl reports reason InstallSuccessful; after a successful Helm release upgrade, it reports UpgradeSuccessful.

Check component pods in their deployed namespaces:

kubectl -n npu-operator get pods
kubectl -n kube-system get pods -l name=ascend-device-plugin-ds
kubectl -n npu-exporter get pods -l app=npu-exporter
kubectl -n mindx-dl get pods
kubectl -n default get daemonset mindio-tft mindio-acp --ignore-not-found

With the default installation, the Operator, Driver, Runtime, and NPU Feature Discovery run in npu-operator; these resources follow the Operator installation namespace if you selected another one. The Ascend Device Plugin runs in kube-system, NPU Exporter runs in npu-exporter, optional MindCluster components run in mindx-dl, and optional MindIO DaemonSets run in default. Pods should be Running when their corresponding components are enabled; an empty optional component result is expected when those components are disabled.

Verify node resources

Check node allocatable resources:

kubectl get node ${nodeName} -o jsonpath='{.status.allocatable}'

Expected examples:

{
  "huawei.com/Ascend910": "8",
  "huawei.com/Ascend310P": "1"
}

The exact key and value depend on the chip model, device plugin version, and card count. A 910-family node, including a 910B node, may report huawei.com/Ascend910. Use the resource key that appears in the target node allocatable resources.

Verify driver readiness

Check that the driver DaemonSet is ready on NPU nodes:

kubectl -n npu-operator get pods -l app=npu-driver-daemonset -o wide

When runtime driver tree mode is enabled, the driver pod writes the ready gate on the host at /run/ascend/.ready/driver-ready and exposes the runtime driver tree from /run/ascend/driver.

NOTE

For Ascend 910B, driver initialization can take several minutes after a node reboot because the chip PHY startup is slower than 310P. Wait until the driver pod is Ready before checking workload injection.

Verify RuntimeClass integration

Check the RuntimeClass and runtime integration DaemonSet on the target workload cluster:

kubectl get runtimeclass ascend
kubectl -n npu-operator get ds ascend-runtime-containerd
kubectl -n npu-operator logs ds/ascend-runtime-containerd -c prepare-config --tail=100
kubectl -n npu-operator logs ds/ascend-runtime-containerd -c inject --tail=100

The RuntimeClass name and handler should both be ascend. Keep the delivered operator.runtimeClass value at ascend in v26.6.0 so that it matches the containerd handler registered by the runtime integration DaemonSet. The DaemonSet installs the ascend-docker-runtime payload and maintains the corresponding containerd handler configuration. In precompiled driver mode, the host-side runtime Driver tree is staged under /run/ascend/driver. In pre-installed Driver and ordinary OS managed install modes, the runtime uses the Host Driver under /usr/local/Ascend/driver.

Verify a workload

Run a small workload image on an NPU node. Use the resource key that the node actually advertises. Different Ascend fleets may report different huawei.com/Ascend* keys, so check node allocatable resources first instead of hard-coding one key for every cluster.

Replace the image with an internal image if your cluster is air-gapped or image whitelist policies are enforced.

NOTE

With the default admission webhook enabled, do not add runtimeClassName to this example. The webhook adds runtimeClassName: ascend after it detects the Ascend resource request. If the webhook is disabled, set runtimeClassName: ascend explicitly.

apiVersion: v1
kind: Pod
metadata:
  name: npu-smoke
spec:
  restartPolicy: Never
  containers:
  - name: probe
    image: <your-npu-image>
    command: ["/bin/bash", "-lc"]
    args:
    - |
      set -euo pipefail
      echo "=== devices ==="
      ls -la /dev/davinci* /dev/davinci_manager /dev/devmm_svm /dev/hisi_hdc 2>/dev/null
      echo "=== Ascend runtime paths ==="
      grep -E '/usr/local/(bin/npu-smi|Ascend/driver)' /proc/mounts || true
      echo "=== npu-smi ==="
      /usr/local/bin/npu-smi info
    resources:
      limits:
        <huawei.com/Ascend-resource-key>: 1

Apply the YAML, then check logs:

kubectl get pod npu-smoke -o jsonpath='{.spec.runtimeClassName}{"\n"}'
kubectl logs npu-smoke

The admitted Pod should use the ascend RuntimeClass. It should then reach Completed, list /dev/davinci*, and print NPU status. npu-smi is available in the workload at /usr/local/bin/npu-smi, and the Driver libraries are mounted under /usr/local/Ascend/.... In precompiled Driver mode, the host-side source path for the runtime Driver tree is /run/ascend/driver.

Verify monitoring

If NPU Exporter is enabled and the monitoring namespace exists, the operator creates its ServiceMonitor in that namespace:

kubectl -n monitoring get servicemonitor npu-exporter-servicemonitor

Use ACP monitoring pages to manage dashboards. For dashboard operations, see Manage dashboards.