#Creating Clusters on Bare Metal

This document explains how to create Kubernetes clusters on physical servers using the bare-metal provider. The workflow is YAML-only — there is no Fleet Essentials UI for bare-metal clusters at this time.

#Prerequisites

Before creating clusters, ensure all of the following prerequisites are met.

#1. Required Plugin Installation

Install the following plugins on the global cluster:

• Alauda Container Platform Kubeadm Provider
• Alauda Container Platform Bare Metal Infrastructure Provider (umbrella chart that installs both the bare-metal manager and elemental-operator)

See the Installation Guide for details.

#2. OS Images Imported and Image Catalog Confirmed

Before creating any cluster resources, obtain the matching OS image pair from Alauda technical support and import both images into the target platform registry. Importing only one image is not sufficient:

Use the image references in the target platform registry after the import completes. Do not point cluster resources at a build registry, derive one image reference from the other, or substitute an independently built image. Verify in the registry or from the image-import result that both final references exist before continuing. Adding an image-catalog entry or a SeedImage reference does not import the corresponding image.

The bare-metal provider chart ships an elemental-image-catalog ConfigMap that maps Machine.spec.version to the elemental upgrade image used to (re)provision a node. You do not need to create this ConfigMap separately — confirm that the target Kubernetes version is present:

kubectl -n cpaas-system get configmap elemental-image-catalog -o yaml

Every value used as Machine.spec.version (for both the control plane and worker MachineDeployment resources) must appear as a key in this ConfigMap, with the leading v preserved. Confirm that the key resolves to the imported base-image in the target platform registry. The provider resolves the image at reprovision time by substituting the platform registry address for the registry portion of the entry. If a target version is missing, no reprovision plan is written and BaremetalMachine ends up in Failed / Reason=ImageCatalogMiss until the entry is added.

#3. Network Connectivity

• Every physical host must be able to reach global.platformUrl (elemental-system-agent registration, plan secret polling).
• Every physical host must be able to pull from the platform registry (global.registry.address) for both elemental install (during the first boot) and elemental upgrade (during every reprovision).
• For Internal Self-built VIP, the VIP must live in the same Layer-2 broadcast domain as the control-plane node IPs. The vrid must be unique in that domain, the network must allow VRRP and gratuitous ARP updates, and the node image must expose IPVS and allow Alive to set net.ipv4.conf.all.arp_accept=1 and net.ipv4.vs.conntrack=1.
• For External LoadBalancer, provision the listener and control-plane backends before cluster creation. Follow Plan the Control Plane Endpoint.

If the target VM or physical host does not receive an address from DHCP while booted into the live ISO, configure the network manually from the host console before waiting for registration. Check the NetworkManager connection name first, then apply the site-specific address, gateway, and DNS values:

nmcli connection show
nmcli connection modify "Wired connection 1" \
  ipv4.method manual \
  ipv4.addresses 192.168.254.73/24 \
  ipv4.gateway 192.168.254.1 \
  ipv4.dns 192.168.16.19 \
  ipv4.ignore-auto-dns yes
nmcli connection up "Wired connection 1"

#4. TPM Decision

Set MachineRegistration.spec.config.elemental.registration.emulate-tpm from whether the host exposes a real hardware TPM (/dev/tpm0), not from whether it is physical or virtual:

• emulate-tpm: false (or omit the field) — only when the host has a working hardware TPM, so elemental-register can use it for auth: tpm.
• emulate-tpm: true with emulated-tpm-seed: -1 — for any host without a hardware TPM. This includes both virtual machines and physical servers that ship without a TPM module (for example a Dell R620). On such a host, auth: tpm combined with emulate-tpm: false makes elemental-register fail TPM attestation and never send the registration request, so elemental-operator records zero registration POSTs.

#5. Public Registry Credential (Only When Installing Platform Components Later)

public-registry-credential is not required to create a bare-metal cluster. It only becomes necessary when later platform components on the new workload cluster need to pull from a credentialed public registry. If your test scope ends at cluster + node Ready, you can ignore this prerequisite.

#6. Host Disk and Boot Preparation

Classify every disk and virtual disk (VD) on each host before you boot the SeedImage ISO. Record which device is the OS install target and which devices, if any, must retain application data:

• Wipe the selected OS install disk and any other obsolete boot disks so that no bootable previous operating system remains. The host must boot the ISO, not an old on-disk OS. Set MachineRegistration.spec.config.elemental.install.device to the exact OS disk.
• Do not wipe a data disk that you intend to manage with the Adopt policy. Back it up independently, record its stable ID and filesystem UUID, and make sure it cannot be selected as the install device.
• A disk intended for InitializeIfBlank must be disposable and objectively blank. Formatting still requires a separate initialization approval after the host registers; the registration manifest does not authorize it.
• Remove residual Elemental partition labels — COS_STATE, COS_PERSISTENT, COS_OEM, COS_RECOVERY — from all disks, not only the intended install disk. Elemental resolves partitions by label (blkid -L COS_STATE); a stale label left on a second disk (for example from a previous install or a leftover multipath member) makes it resolve the wrong device and the reprovision snapshotter fails.
• On hosts where the same disk can appear through more than one path (multipath), make sure none of those paths exposes a disk that still carries a COS_* label; a residual label on any path can be resolved ahead of the intended install disk. Clean the disk rather than disabling multipath — the image keeps it enabled for network-attached storage boot.
• On a host with multiple disks, do not leave install.device empty and do not use /dev/sda as a persistent identity. Follow Select a Fixed System Disk on Multi-Disk Bare-Metal Hosts to reuse one ISO while selecting each host's system disk by WWN.
• Set the boot order so the host boots the virtual CD / ISO first.

Use only stable storage identities reported by the registered inventory. Linux paths such as /dev/sdX, /dev/vdX, /dev/nvmeXnY, and /dev/mapper/mpathX are runtime paths and must not be placed in MachineInventory.spec.storage.

#Cluster Creation Workflow

When using YAML, import the paired OS images first, then proceed through five required resource steps, with an optional storage-preparation step before pool membership. Every Kubernetes resource must be applied in the cpaas-system namespace.

#Resolving Placeholder Values

The example manifests below use <placeholder> syntax for environment-specific values:

#Step 1: Build the SeedImage and Register Hosts

Create a MachineRegistration that describes the registration URL and first-install cloud-config, and a SeedImage that points elemental-operator at the matching ISO base image.

Set SeedImage.spec.baseImage to the final target-platform registry reference of the imported base-image-iso. It must be paired with the imported base-image used by the image catalog for later reprovisioning; do not construct the ISO reference by changing the repository name or tag yourself.

apiVersion: elemental.cattle.io/v1beta1
kind: MachineRegistration
metadata:
  name: <cluster-name>-registration
  namespace: cpaas-system
  labels:
    app.kubernetes.io/part-of: cluster-api-provider-baremetal
spec:
  # ${...} placeholders are expanded by elemental-register from SMBIOS data.
  # System UUID is preferred over Serial Number because it is more reliable
  # across virtualised test environments.
  machineName: "<cluster-name>-${System Information/UUID}"
  machineInventoryLabels:
    app.kubernetes.io/part-of: cluster-api-provider-baremetal
    pool.baremetal.alauda.io/eligible: "true"
    elemental.cattle.io/serial-number: "${System Information/Serial Number}"
  machineInventoryAnnotations:
    elemental.cattle.io/machine-uuid: "${System Information/UUID}"
  config:
    elemental:
      install:
        device: <install-device>
        eject-cd: true
        reboot: true
        snapshotter:
          type: btrfs
          maxSnaps: 4
      registration:
        # false only when the host has a real hardware TPM (/dev/tpm0). Set true
        # (with emulated-tpm-seed: -1) on any host without one, including physical
        # servers that ship without a TPM module.
        emulate-tpm: false
---
apiVersion: elemental.cattle.io/v1beta1
kind: SeedImage
metadata:
  name: <cluster-name>-registration-iso
  namespace: cpaas-system
  labels:
    app.kubernetes.io/part-of: cluster-api-provider-baremetal
spec:
  type: iso
  baseImage: <base-image-iso>          # Use the imported target-platform registry reference.
  registrationRef:
    apiVersion: elemental.cattle.io/v1beta1
    kind: MachineRegistration
    name: <cluster-name>-registration
    namespace: cpaas-system
  targetPlatform: linux/amd64
  size: 20Gi
  cleanupAfterMinutes: 120

Do not add /etc/resolv.conf to SeedImage.spec.cloud-config. Keep the ISO generic, and put site-specific resolver configuration in MachineRegistration.spec.config.cloud-config only when the first-boot registration path needs it. During the tested Global deployment flow, the ISO did not carry resolver files; node DNS was configured later by the kubeadm bootstrap data.

install.device, install.eject-cd, and install.reboot are intentional. If the target disk is omitted, elemental install can select an unintended device. On multi-disk physical hosts, use the fixed system disk workflow instead of a changing /dev/sdX name. If eject-cd or reboot is false, a host can remain in the live environment after the first install and never become usable inventory for Cluster API.

Apply the manifest and wait for the SeedImage build to finish:

kubectl apply -f 01-machineregistration-seedimage.yaml
kubectl -n cpaas-system get seedimage <cluster-name>-registration-iso -w

When status.state reaches Completed, fetch the download URL and ISO checksum:

kubectl -n cpaas-system get seedimage <cluster-name>-registration-iso \
  -o jsonpath='{.status.downloadURL}{"\n"}'
kubectl -n cpaas-system get seedimage <cluster-name>-registration-iso \
  -o jsonpath='{.status.checksumURL}{"\n"}'

Boot every target host from this ISO. elemental-register runs first (creates the MachineInventory and uploads observedNetwork), then elemental install writes the on-disk OS. After install completes, the host stays available for plan execution. If the live ISO environment has no DHCP address, configure NetworkManager manually on the host console as described in Network Connectivity before waiting for the MachineInventory.

Confirm registration:

kubectl -n cpaas-system get machineinventories.elemental.cattle.io

Every inventory you intend to use must:

• Show Ready=True.
• Have a non-empty status.plan.secretRef.name.
• Have a spec.observedNetwork that matches the host's expected NIC (only required when you want the install-time IP to survive across reprovisions).
• When managed data disks are required, run an observer-capable OS image, report a fresh status.observedStorage, and reach status.storage.phase=Prepared before pool allocation.

Record the exact MachineInventory names — they are referenced by name in the next step.

#Optional: Prepare Managed Data Disks

Managed storage belongs to the long-lived MachineInventory, not to a CAPI Machine or BaremetalMachineTemplate. Configure it while the inventory is unallocated and before adding that inventory to a production pool.

Follow Manage Data Disks on Bare-Metal Hosts to:

1. Verify elemental-storage-observer.service and inspect status.observedStorage.
2. Select each device by a stable ID and choose Adopt or InitializeIfBlank.
3. Use the matching storagectl binary to validate and atomically patch the declaration and, when required, its initialization approval.
4. Wait for StoragePrepared=True/AllRequiredVolumesPrepared and status.storage.phase=Prepared.

If no disks should be managed, leave spec.storage absent or set volumes: []. The storage controller converges that inventory to Unmanaged with StoragePrepared=True/NoManagedVolumes and performs no disk operation.

#Step 2: Create MachineInventoryPool Resources

Create one pool per role. The pool reconciler validates that every member exists, computes capacity counters, and writes the baremetal.alauda.io/pool=<pool-name> annotation onto the inventory.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: MachineInventoryPool
metadata:
  name: <cluster-name>-control-plane-pool
  namespace: cpaas-system
spec:
  clusterName: <cluster-name>
  machineInventories:
    - name: <control-plane-inventory-1>
      hostname: <control-plane-host-1>    # Optional; falls back to the inventory name.
    - name: <control-plane-inventory-2>
      hostname: <control-plane-host-2>
    - name: <control-plane-inventory-3>
      hostname: <control-plane-host-3>
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: MachineInventoryPool
metadata:
  name: <cluster-name>-worker-pool
  namespace: cpaas-system
spec:
  clusterName: <cluster-name>
  machineInventories:
    - name: <worker-inventory-1>
      hostname: <worker-host-1>
    - name: <worker-inventory-2>
    - name: <worker-inventory-3>

Key parameters:

Apply and verify:

kubectl apply -f 02-machineinventorypool.yaml
kubectl -n cpaas-system get machineinventorypools.infrastructure.cluster.x-k8s.io

A healthy pool reports Ready=True, total = len(spec.machineInventories), and available = total - allocated - preparing - reprovisioning - unavailable. Inventories listed in spec.machineInventories that fail validation (missing, plan secret missing, Ready=False, or storage not prepared) raise the pool's unavailable counter and surface in the MembersValid condition. A non-empty storage declaration is an allocation gate: the inventory is not Available to CAPI until Prepare succeeds against a fresh observation.

Size the control-plane pool to at least KubeadmControlPlane.spec.replicas. Size the worker pool to at least MachineDeployment.spec.replicas. For rolling upgrades the pool must hold the entire replica count — the provider uses delete-then-add semantics from the same pool, never both at once.

#Step 3: Create the Control-Plane Cluster Resources

Create the BaremetalCluster (declares the selected control-plane endpoint mode), the control-plane BaremetalMachineTemplate (points at the control-plane pool), the KubeadmControlPlane (replicas + kubeadm config), and the CAPI Cluster.

The pre-GA Bare Metal API supports both endpoint modes in the same workflow:

• Internal: the provider deploys Alive and reconciles the VIP and control-plane backend membership.
• External: the provider skips Alive; the load balancer owner maintains the listener, health check, and backend membership.

There is no legacy provider-version tab because Bare Metal has not had an earlier GA release. Select the mode with <control-plane-load-balancer-type> in the manifest below.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: BaremetalCluster
metadata:
  name: <cluster-name>
  namespace: cpaas-system
spec:
  controlPlaneLoadBalancer:
    type: <control-plane-load-balancer-type>
    host: <control-plane-vip>
    port: <control-plane-port>
    # Required only for Internal. Remove this field for External.
    vrid: <vrid>
  networkType: kube-ovn      # Enables provider-managed Kube-OVN AppRelease reconciliation.
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: BaremetalMachineTemplate
metadata:
  name: <cluster-name>-control-plane-template
  namespace: cpaas-system
spec:
  template:
    spec:
      machineInventoryPoolRef:
        name: <cluster-name>-control-plane-pool
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: <cluster-name>-control-plane
  namespace: cpaas-system
spec:
  replicas: 3
  version: <kubernetes-version>
  rolloutStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0            # Bare-metal does not over-provision; replace one at a time.
  machineTemplate:
    nodeDrainTimeout: 5m
    nodeVolumeDetachTimeout: 5m
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: BaremetalMachineTemplate
      name: <cluster-name>-control-plane-template
  kubeadmConfigSpec:
    users:
      - name: boot
        sudo: ALL=(ALL) NOPASSWD:ALL
        shell: /bin/bash
        sshAuthorizedKeys:
          - "<ssh-authorized-keys>"
    clusterConfiguration:
      imageRepository: cloud.alauda.io/alauda
      dns:
        imageTag: <dns-image-tag>
      etcd:
        local:
          imageTag: <etcd-image-tag>
      apiServer:
        extraArgs:
          profiling: "false"
          tls-min-version: VersionTLS12
      controllerManager:
        extraArgs:
          profiling: "false"
          tls-min-version: VersionTLS12
      scheduler:
        extraArgs:
          profiling: "false"
          tls-min-version: VersionTLS12
    initConfiguration:
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
    joinConfiguration:
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <cluster-name>
  namespace: cpaas-system
  annotations:
    capi.cpaas.io/resource-group-version: infrastructure.cluster.x-k8s.io/v1beta1
    capi.cpaas.io/resource-kind: BaremetalCluster
    cpaas.io/kube-ovn-join-cidr: <kube-ovn-join-cidr>
    cpaas.io/kube-ovn-version: <kube-ovn-version>
    cpaas.io/sentry-deploy-type: Baremetal
    cpaas.io/alb-address-type: ClusterAddress
  labels:
    cluster-type: ProviderBaremetal
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
        - <pods-cidr>
    services:
      cidrBlocks:
        - <services-cidr>
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: <cluster-name>-control-plane
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: BaremetalCluster
    name: <cluster-name>

Cluster annotations. The bare-metal provider relies on a small set of Cluster annotations during reconcile. Authoritative ones the operator must set:

BaremetalCluster parameters:

Apply and watch:

kubectl apply -f 03-cluster.yaml
kubectl -n cpaas-system get clusters.cluster.x-k8s.io
kubectl -n cpaas-system get kubeadmcontrolplanes.controlplane.cluster.x-k8s.io
kubectl -n cpaas-system get baremetalmachines.infrastructure.cluster.x-k8s.io -w

Each new control-plane BaremetalMachine advances Pending → Allocated → Reprovisioning → Running. Watch:

• BaremetalMachine.status.machineInventoryRef.name — which inventory was picked.
• BaremetalMachine.status.planSecretRef.name — plan secret being driven. The secret carries baremetal.alauda.io/plan.type=reprovision.
• MachineInventory.status.plan.state — Applied once the host completes cloud-init clean, elemental upgrade, reboot, and kubeadm init/join.
• BaremetalCluster.status.conditions[EndpointReady] — true once the configured control-plane endpoint is reachable.

#Step 4: Deploy Worker Nodes

After the control plane is Ready, create the worker BaremetalMachineTemplate, the worker KubeadmConfigTemplate, and the MachineDeployment. The full worker YAML and parameter table are in Managing Nodes on Bare Metal → Worker Node Deployment.

#Cluster Verification

#Using kubectl

# Cluster status
kubectl -n cpaas-system get cluster <cluster-name>

# Control plane progress
kubectl -n cpaas-system get kubeadmcontrolplane <cluster-name>-control-plane

# Per-machine state
kubectl -n cpaas-system get machines.cluster.x-k8s.io
kubectl -n cpaas-system get baremetalmachines.infrastructure.cluster.x-k8s.io

# Pool counters
kubectl -n cpaas-system get machineinventorypools.infrastructure.cluster.x-k8s.io

# Storage lifecycle for inventories that declare managed volumes
kubectl -n cpaas-system get machineinventories.elemental.cattle.io \
  -o custom-columns='NAME:.metadata.name,PHASE:.status.storage.phase,PREPARED:.status.storage.conditions[?(@.type=="StoragePrepared")].status,ACTIVE:.status.storage.conditions[?(@.type=="StorageActive")].status'

# Workload cluster Nodes (use the workload kubeconfig)
kubectl get nodes -o wide

#Verify the Control Plane Endpoint

Confirm the configured mode and endpoint:

kubectl -n cpaas-system get baremetalcluster <cluster-name> \
  -o jsonpath='{.spec.controlPlaneLoadBalancer.type}{" "}{.spec.controlPlaneEndpoint.host}{":"}{.spec.controlPlaneEndpoint.port}{"\n"}'

For Internal, verify Alive and the node prerequisites:

kubectl --kubeconfig <workload-kubeconfig> -n kube-system get pods -l app=alive -o wide
kubectl --kubeconfig <workload-kubeconfig> get --raw=/version

Run the following command on each control-plane node, not on the management cluster:

sysctl net.ipv4.conf.all.arp_accept net.ipv4.vs.conntrack

For External, verify the load balancer frontend and each backend according to the External LoadBalancer contract. Confirm that the load balancer owner has registered every current control-plane node and that HTTPS /healthz returns HTTP 200 for each backend.

#Expected Results

A successfully created cluster shows:

• Cluster.status.conditions[Ready]=True.
• KubeadmControlPlane replicas all Ready.
• Every BaremetalMachine.status.phase=Running and status.ready=true.
• Every used MachineInventory.status.plan.state=Applied with baremetal.alauda.io/plan.type=reprovision on its plan secret.
• Every used inventory with non-empty spec.storage.volumes[] reports status.storage.phase=Active, StoragePrepared=True, and StorageActive=True; the corresponding BaremetalMachine reports StorageReady=True/AllVolumesReady.
• MachineInventoryPool.status satisfies available + allocated + preparing + reprovisioning + unavailable = total.
• Kubernetes Nodes Ready.
• For Internal, Alive Pods are Ready and the API is reachable through the Self-built VIP.
• For External, the load balancer frontend is reachable and its backend list matches the current control-plane nodes.

#Common Failure Modes

For the full operator-side state machine reference (every condition reason and recovery action), see Provider Overview → clean / reprovision plans.

#Next Steps

After creating a cluster:

• Manage Nodes
• Manage Data Disks
• Upgrade Cluster

#Appendix

#Complete KubeadmControlPlane Configuration

The hardened configuration recommended for production bare-metal clusters — admission control, audit policy, kubelet patches, encryption provider, and IPv6 bind addresses. Substitute the placeholders from the table in Resolving Placeholder Values.

apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: <cluster-name>-control-plane
  namespace: cpaas-system
spec:
  replicas: 3
  version: <kubernetes-version>
  rolloutStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0
  machineTemplate:
    nodeDrainTimeout: 5m
    nodeVolumeDetachTimeout: 5m
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: BaremetalMachineTemplate
      name: <cluster-name>-control-plane-template
  kubeadmConfigSpec:
    users:
      - name: boot
        sudo: ALL=(ALL) NOPASSWD:ALL
        shell: /bin/bash
        sshAuthorizedKeys:
          - "<ssh-authorized-keys>"
    files:
      - path: /etc/kubernetes/admission/psa-config.yaml
        owner: "root:root"
        permissions: "0644"
        content: |
          apiVersion: apiserver.config.k8s.io/v1
          kind: AdmissionConfiguration
          plugins:
          - name: PodSecurity
            configuration:
              apiVersion: pod-security.admission.config.k8s.io/v1
              kind: PodSecurityConfiguration
              defaults:
                enforce: "privileged"
                enforce-version: "latest"
                audit: "baseline"
                audit-version: "latest"
                warn: "baseline"
                warn-version: "latest"
              exemptions:
                usernames: []
                runtimeClasses: []
                namespaces:
                - kube-system
                - cpaas-system
      - path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
        owner: "root:root"
        permissions: "0644"
        content: |
          {
            "apiVersion": "kubelet.config.k8s.io/v1beta1",
            "kind": "KubeletConfiguration",
            "imagePullCredentialsVerificationPolicy": "NeverVerify",
            "protectKernelDefaults": true,
            "tlsCertFile": "/etc/kubernetes/pki/kubelet.crt",
            "tlsPrivateKeyFile": "/etc/kubernetes/pki/kubelet.key",
            "streamingConnectionIdleTimeout": "5m",
            "clientCAFile": "/etc/kubernetes/pki/ca.crt"
          }
      - path: /etc/kubernetes/audit/policy.yaml
        owner: "root:root"
        permissions: "0644"
        content: |
          apiVersion: audit.k8s.io/v1
          kind: Policy
          omitStages:
          - "RequestReceived"
          rules:
          - level: None
            users:
            - system:kube-controller-manager
            - system:kube-scheduler
            - system:serviceaccount:kube-system:endpoint-controller
            verbs: ["get", "update"]
            namespaces: ["kube-system"]
            resources:
            - group: ""
              resources: ["endpoints"]
          - level: None
            nonResourceURLs:
            - /healthz*
            - /version
            - /swagger*
          - level: None
            resources:
            - group: ""
              resources: ["events"]
          - level: None
            verbs: ["get", "list", "watch"]
          - level: None
            resources:
            - group: "coordination.k8s.io"
              resources: ["leases"]
          - level: None
            resources:
            - group: "authorization.k8s.io"
              resources: ["subjectaccessreviews", "selfsubjectaccessreviews"]
            - group: "authentication.k8s.io"
              resources: ["tokenreviews"]
          - level: Metadata
            resources:
            - group: ""
              resources: ["secrets", "configmaps"]
          - level: RequestResponse
            resources:
            - group: ""
            - group: "apps"
            - group: "rbac.authorization.k8s.io"
            - group: "storage.k8s.io"
            - group: "networking.k8s.io"
          - level: Metadata
    postKubeadmCommands:
      - chmod 600 /var/lib/kubelet/config.yaml
    clusterConfiguration:
      imageRepository: cloud.alauda.io/alauda
      dns:
        imageTag: <dns-image-tag>
      etcd:
        local:
          imageTag: <etcd-image-tag>
      apiServer:
        extraArgs:
          audit-log-format: json
          audit-log-maxage: "30"
          audit-log-maxbackup: "10"
          audit-log-maxsize: "200"
          profiling: "false"
          audit-log-mode: batch
          audit-log-path: /etc/kubernetes/audit/audit.log
          audit-policy-file: /etc/kubernetes/audit/policy.yaml
          tls-cipher-suites: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
          admission-control-config-file: /etc/kubernetes/admission/psa-config.yaml
          tls-min-version: VersionTLS12
          kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
        extraVolumes:
        - name: vol-dir-0
          hostPath: /etc/kubernetes
          mountPath: /etc/kubernetes
          pathType: Directory
      controllerManager:
        extraArgs:
          bind-address: "::"
          profiling: "false"
          tls-min-version: VersionTLS12
      scheduler:
        extraArgs:
          bind-address: "::"
          tls-min-version: VersionTLS12
          profiling: "false"
    initConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"
    joinConfiguration:
      patches:
        directory: /etc/kubernetes/patches
      nodeRegistration:
        kubeletExtraArgs:
          node-labels: "kube-ovn/role=master"

Worker bootstrap is symmetric — see Managing Nodes on Bare Metal → Bootstrap Template for the worker KubeadmConfigTemplate.