#Install NPU Operator

#Prerequisites

• Cluster administrator access to the target workload cluster where Alauda Build of NPU Operator will be installed.
• ACP v4.0-v4.3.
• Ascend 910B or Ascend 310P worker nodes.
• Alauda Build of Node Feature Discovery installed on the target workload cluster.
• Driver mode selected before creating the NPUOperatorCtl instance.
• Access to the Alauda Build of NPU Operator package from Customer Portal.

See Versions and Components and Compatibility for supported combinations.

#Prepare NPU packages

Prepare the packages required for the Ascend NPU product line.

Alauda Build of Node Feature Discovery is a shared cluster plugin. It publishes node facts that NPU components use during scheduling, component placement, and node profile matching. Install it once on the target workload cluster before creating the NPUOperatorCtl instance.

Alauda Build of NPU Operator is an umbrella operator. After the operator is installed, the NPUOperatorCtl instance controls which NPU components are reconciled on the selected nodes.

MindIO TFT and MindIO ACP are not separate cluster plugin packages. When enabled, Alauda Build of NPU Operator deploys the MindIO DaemonSets with the delivered npu-node-provision image. In offline or controlled network environments, prepare the matching MindIO SDK zip on each target node before enabling these components.

#Choose a driver lifecycle mode

Choose the driver lifecycle mode before creating the NPUOperatorCtl instance.

In pre-installed driver mode, the operator does not create the driver DaemonSet. The host driver must already be installed and healthy on each selected NPU node before you enable device plugin, runtime integration, exporter, HCCN, or other components that need the driver.

#Installation flow

#1. Prepare the driver according to the selected mode

Skip driver image preparation if your NPU nodes already have a Huawei .run driver installed out of band and you plan to disable Driver management in the NPUOperatorCtl instance. In that mode, verify that /usr/local/Ascend/driver exists on every selected NPU node.

For operator-managed immutable OS / Alauda OS precompiled mode, import the ARM64 Driver image that matches each (HDK, chip, kernel, OS) node profile. NPU Operator derives the tag from the HDK version, detected chip, and complete kernel release after removing the architecture suffix:

<HDK>-<chip>-<kernel-release-without-architecture>

For published openEuler tags, the legacy tag contract rewrites the .oe separator in the kernel release to -oe. Other kernel releases remain unchanged. Do not append a separate OS stem that is not present in the detected kernel release.

The Driver image is package-external and is not imported with the NPU Operator package. The following published images are examples for the specified node profiles:

Pull the matching example from Docker Hub:

docker pull alaudadockerhub/ascend-driver:25.5.0-310p-6.6.0-0008.ctl4
docker pull alaudadockerhub/ascend-driver:25.5.0-910b-6.6.0-145.0.4.135-oe2403sp3

For other published Driver images, check the tags in the Alauda Ascend Driver repository on Docker Hub. A published tag is an image inventory entry, not by itself a product-support or validation statement. Before use, confirm that the chip, kernel, and operating system match the target node and that the combination is supported for your deployment.

Verify that the selected source image reports arm64, then mirror it to the target workload cluster registry. For example:

SOURCE_IMAGE=docker.io/alaudadockerhub/ascend-driver:25.5.0-310p-6.6.0-0008.ctl4
TARGET_IMAGE=<your-cluster-registry>/mlops/ascend-driver:25.5.0-310p-6.6.0-0008.ctl4

skopeo inspect --override-arch arm64 docker://$SOURCE_IMAGE
skopeo copy --override-arch arm64 \
  docker://$SOURCE_IMAGE \
  docker://$TARGET_IMAGE
skopeo inspect --override-arch arm64 docker://$TARGET_IMAGE

If no tag matches your node kernel, contact Alauda Customer Support with the output of uname -r and the chip model.

When the Driver tag is left empty, NPU Operator resolves it from the configured Driver version, the node chip label, and the complete kernel release label. Configure image mapping for the complete source reference, including its resolved tag. After the Driver Pod starts, verify both the admitted Pod image and the runtime imageID; a rewritten Pod image alone does not prove that the intended image was pulled.

For operator-managed ordinary OS install mode, use the delivered driver installer image and disable precompiled driver delivery. The installer flow installs the driver under /usr/local/Ascend/driver.

#2. Allow driver images in ImageWhiteList

If ACP enforces image whitelist policies and the operator manages the driver, add every required driver image tag to an ImageWhiteList in the target workload cluster where Alauda Build of NPU Operator will be installed.

Skip this step in pre-installed driver mode unless another enabled component image is blocked by the cluster image policy.

If you configure the whitelist from the ACP console, switch to the target workload cluster first. The example below is applied to the target workload cluster and uses the cpaas-system namespace in that cluster.

For the validated 310P profile, match the complete source reference rendered by NPU Operator and rewrite it to the image already imported into your cluster registry. The build-harbor.alauda.cn reference below is only the Operator-rendered match key; obtain the source image from Docker Hub as shown in the preceding step.

apiVersion: app.alauda.io/v1alpha1
kind: ImageWhiteList
metadata:
  name: npu-operator-precompiled-driver
  namespace: cpaas-system
spec:
  repoList:
  - build-harbor.alauda.cn/mlops/ascend-driver:25.5.0-310p-6.6.0-0008.ctl4
  rewriteRules:
  - regexp: ^build-harbor\.alauda\.cn/mlops/ascend-driver:25\.5\.0-310p-6\.6\.0-0008\.ctl4$
    replacement: <your-cluster-registry>/mlops/ascend-driver:25.5.0-310p-6.6.0-0008.ctl4

You may add the rule to a platform-generated ImageWhiteList or a self-managed object. If you update an existing object, preserve unrelated entries. Use one exact rule for each supported Driver tag; do not replace the tag with a broad wildcard. Apply the rule before creating or updating the NPUOperatorCtl instance, and recheck it after a product upgrade.

After the instance creates the managed policy, confirm that the rendered repository matches the rule:

kubectl get npuclusterpolicy cluster \
  -o jsonpath='{.spec.driver.imageSpec.repository}{"\n"}'

Then verify the admitted Pod reference and the image actually running:

kubectl -n npu-operator get pods \
  -l app.kubernetes.io/component=npu-driver \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{range .spec.containers[*]}  {.name}{" image="}{.image}{"\n"}{end}{range .status.containerStatuses[*]}  {.name}{" imageID="}{.imageID}{"\n"}{end}{end}'

The admitted image must point to <your-cluster-registry>/mlops/ascend-driver:25.5.0-310p-6.6.0-0008.ctl4. Its imageID must resolve to the target digest recorded during import.

#3. Upload packages

Download and upload these packages from Customer Portal:

• Alauda Build of NPU Operator operator package.
• Alauda Build of Node Feature Discovery cluster plugin package.
• Volcano cluster plugin package, only if you plan to enable ClusterD.

For the package upload workflow, see Upload Packages.

#4. Install Node Feature Discovery

Install Alauda Build of Node Feature Discovery from Administrator > Marketplace > Cluster Plugins.

NFD is the shared node discovery layer for the cluster. It provides OS, kernel, architecture, and PCI hardware labels that the NPU installation depends on. After you create the NPUOperatorCtl instance, NPU Feature Discovery uses these facts and the Kubernetes node role to maintain NPU-specific and component-placement labels.

#5. Install Alauda Build of NPU Operator

1. Go to Administrator > Marketplace > OperatorHub.
2. Switch to the target workload cluster and open Alauda Build of NPU Operator.
3. Click Install.
4. Keep the default namespace npu-operator unless your deployment plan requires a different namespace. Commands in this guide use the default; if you select another namespace, replace npu-operator with it. The Driver, Runtime, HCCN Bootstrap, Rebooter, and NPU Feature Discovery resources that use the Operator namespace follow that selection.
5. Select Manual upgrade strategy for production environments.
6. Wait until the operator subscription succeeds and the controller pods are running.

#6. Create an NPUOperatorCtl instance

1. Open Installed Operators > Alauda Build of NPU Operator.
2. Open the NPUOperatorCtl tab.
3. Click Create NPUOperatorCtl.
4. Configure the components that should be managed by the operator.

Common component switches:

For optional Ascend Operator, NodeD, ClusterD, Resilience Controller, MindIO TFT, and MindIO ACP components, enable them only when the workload or operations plan requires them. MindIO TFT and MindIO ACP use the delivered npu-node-provision image line; prepare node-side SDK zip files only for offline or controlled network environments.

#7. Verify node placement labels

Creating the NPUOperatorCtl instance with spec.npu-feature-discovery.enabled=true deploys the npu-feature-discovery DaemonSet. It derives node placement labels from Kubernetes node roles and NPU discovery. Some managed components require these labels, but you do not need to add them manually when discovery runs successfully on the selected nodes.

Check the discovery DaemonSet and the labels it maintains:

kubectl -n npu-operator get daemonset npu-feature-discovery
kubectl -n npu-operator get pods -l app=npu-feature-discovery -o wide
kubectl get nodes \
  -L node-role.kubernetes.io/control-plane,masterselector,workerselector,openfuyao.com/npu.present

Expected results:

• A node with node-role.kubernetes.io/control-plane has masterselector=dls-master-node after NPU Feature Discovery runs on it.
• A non-control-plane node has workerselector=dls-worker-node.
• A control-plane node that also hosts an NPU can have both placement labels after NPU discovery identifies the device.

If a required placement label is missing, first confirm that an npu-feature-discovery Pod can run on that node and that a control-plane node has the standard node-role.kubernetes.io/control-plane label. The delivered DaemonSet does not tolerate the standard control-plane NoSchedule taints, and it does not recognize only the legacy node-role.kubernetes.io/master role label as control-plane. Either condition can prevent automatic master labeling.

If the node is a control-plane node but has only the legacy role label, add the standard role label and let NPU Feature Discovery reconcile the placement labels:

kubectl label node <control-plane-node-id> node-role.kubernetes.io/control-plane= --overwrite

Only when automatic discovery cannot cover the selected node, add the matching placement label as a fallback:

kubectl label node <control-plane-node-id> masterselector=dls-master-node --overwrite
kubectl label node <npu-worker-node-id> workerselector=dls-worker-node --overwrite

Apply these labels only to nodes intended for the corresponding components. Where NPU Feature Discovery is running, it continuously reconciles the labels and can replace or remove values that do not match the detected node role. In particular, a manual masterselector is not durable on a legacy-master-only node while discovery is running; add the standard control-plane role label instead.

#Next step

After the instance is created, continue with Verification.