#VMware vSphere Infrastructure Preparation

This document helps you prepare the infrastructure and collect the values required to create a VMware vSphere workload cluster. Complete this checklist before you apply the manifests in Creating Clusters on VMware vSphere. The provider behavior and fields on this page are validated against VMware vSphere Provider v1.0.16.

#Scenarios

Use this checklist in the following scenarios:

• You are preparing a new VMware vSphere cluster deployment.
• You want to validate external dependencies before you start the deployment.
• You plan to enable optional creation-time topology variants such as multiple datacenters, multiple NICs, or extra worker nodes.

#Prerequisites

Before you begin, ensure the following conditions are met:

• You can access the global cluster with kubectl.
• The workload cluster objects must be stored in the cpaas-system namespace.
• You have access to the target vCenter inventory, networks, datastores, and templates.

#How to Use This Checklist

Use this checklist in the following order:

1. Collect the deployment parameters listed in this document.
2. Replace every placeholder in the manifest templates with the actual values collected here.
3. Reuse the same value everywhere a placeholder appears in multiple manifests.
4. If an optional feature is not enabled, omit the corresponding YAML block exactly as described in Creation-Time Topology Variants.
5. If an optional field (such as deviceName) is not needed, remove the entire line from the YAML manifest.

#Parameter Sources and Safe Collection

Separate parameter collection into the following three sources. A value that can be discovered from vCenter is not necessarily a product-supported version, and a product-derived value is not a substitute for a network or capacity decision.

#Optional vCenter discovery with govc

If govc is already authenticated through your protected local environment, use read-only inventory commands such as:

govc find / -type d
govc find / -type c
govc find / -type p
govc find / -type n
govc find / -type s
govc find / -type m

Interpret the results as candidates, not automatic selections:

• d: datacenters
• c: compute clusters
• p: resource pools
• n: networks or distributed port groups
• s: datastores
• m: VMs and templates; confirm the selected object is a template

Use the openssl command in Thumbprint to retrieve the certificate fingerprint without printing vCenter credentials.

#Terminology

The following terms are used consistently throughout the VMware vSphere cluster-creation documents.

#machine config pool

A machine config pool is the VSphereMachineConfigPool custom resource. It predefines node slots. Each slot can include:

• A node hostname
• A target datacenter
• Static IP configuration for each NIC
• Persistent disk definitions

#Node slot

A node slot is an entry under VSphereMachineConfigPool.spec.configs[]. A single slot usually maps to one node, such as cp-01 or worker-01. The slot hostname drives the Kubernetes node name, the kubelet serving certificate DNS SAN, and (combined with the resolved primary NIC addresses) the kubelet node-ip; it must be a valid DNS-1123 subdomain.

#Slot network layout

Each slot declares its NIC layout under network.primary and network.additional:

• network.primary is required. Its networkName must be set and is used as the node-ip source for the kubelet.
• network.additional is an optional list of extra NICs merged after the primary NIC in the order listed.

The dns values in VSphereMachineConfigPool.spec.configs[].network describe the static network metadata for each slot. In affected VMware deployments, these values might not update the guest operating system's /etc/resolv.conf reliably. The deployment manifests therefore also write /etc/resolv.conf through both KubeadmControlPlane.spec.kubeadmConfigSpec.files and KubeadmConfigTemplate.spec.template.spec.files.

#deviceName

deviceName is an optional field in the VSphereMachineConfigPool network configuration. It is used to control the NIC name seen inside the guest operating system, such as eth0 or eth1.

Use the following distinctions when you fill the values:

• networkName is the vCenter network or port group name.
• deviceName is the NIC name inside the guest operating system.
• If deviceName is omitted, CAPV typically assigns names such as eth0, eth1, and eth2 by NIC order.

#vCenter resource pool

A vCenter resource pool is the native vCenter inventory object, for example:

/Datacenter1/host/cluster1/Resources

When the failure-domain creation variant is enabled, this path is used by VSphereDeploymentZone.spec.placementConstraint.resourcePool.

#Compute cluster

The compute cluster is the target vCenter compute-cluster name. In these documents, it is primarily used when a VSphereFailureDomain is mapped to a specific deployment target.

#Datastore

The datastore is the vSphere storage location that stores VM disks. Both system disks and data disks must be placed on concrete datastores.

#VM template

The VM template is the source template used to create node virtual machines. When you enable multiple datacenters, the same template must already exist in every target datacenter and must be resolvable by the same template name.

#Thumbprint

The thumbprint is the SHA-1 fingerprint of the vCenter server certificate. CAPV uses it to validate the target vCenter server.

Use the following command to retrieve it:

openssl s_client -connect <vsphere_server>:443 -servername <vsphere_server> </dev/null 2>/dev/null | openssl x509 -noout -fingerprint -sha1

#Management Cluster Prerequisites

Use the following table to record the required values and validation results.

#vCenter and Template Prerequisites

#vCenter connection information

Note: These documents assume the default vCenter HTTPS port 443.

#VM template requirements

The template should also meet the following requirements:

• It uses an operating system supported by your platform image policy.
• It includes cloud-init.
• It includes VMware Tools or open-vm-tools.
• It includes containerd.
• It includes the baseline components required by kubeadm bootstrap.
• It includes pre-exported container image tar files under /root/images/. These files are imported into containerd by capv-load-local-images.sh before kubeadm runs, so that node bootstrap does not depend on pulling images from a remote registry.
• The tar files include the kubeadm control-plane images supplied with the OS template, including etcd and kube-apiserver, under the exact image references generated from clusterConfiguration.imageRepository, the Kubernetes version, and the component image tags. Although etcd and kube-apiserver run as static Pods, they still require container images. A missing or mismatched local reference causes the runtime to try the configured Registry instead.
• The /root/images/*.tar files must include the sandbox (pause) image whose reference exactly matches the sandbox_image value (containerd v1) or sandbox value (containerd v2) configured in /etc/containerd/config.toml. For example, if containerd is configured with sandbox_image = "registry.example.com/tkestack/pause:3.10", one of the tar files must contain that exact image reference. A mismatch causes containerd to pull the sandbox image from the network, which defeats the purpose of local preloading and fails in air-gapped environments.

#Load Balancer Prerequisites

VMware vSphere Provider v1.0.16 uses an external LoadBalancer. The provider does not deploy a Self-built VIP component. Provision the endpoint before cluster creation and follow Plan the Control Plane Endpoint.

#Cluster Baseline Parameters

#Registry tag validation

First read the permanent Registry address from the global cluster, or confirm the equivalent address with the platform administrator:

kubectl -n cpaas-system get cluster global \
  -o jsonpath='{.metadata.annotations.cpaas\.io/registry-address}{"\n"}'
kubectl -n cpaas-system get secret public-registry-credential -o name

The Secret command verifies existence only and does not print credential data. Export the confirmed Cluster Registry address and the image Registry used by the CPI and kubeadm manifests. If both placeholders identify the same Registry, use the same value for both variables.

export CLUSTER_REGISTRY_ADDRESS="<registry_address>"
export IMAGE_REGISTRY_ADDRESS="<image_registry>"
curl -sk "https://${CLUSTER_REGISTRY_ADDRESS}/v2/acp/chart-cpaas-kube-ovn/tags/list"
curl -sk "https://${IMAGE_REGISTRY_ADDRESS}/v2/tkestack/coredns/tags/list"
curl -sk "https://${IMAGE_REGISTRY_ADDRESS}/v2/ait/cloud-provider-vsphere/tags/list"

The /v2/ segment belongs only to these Registry HTTP API URLs. Image references use <registry>/<repository>:<tag>, and the Cluster annotation uses <host>:<port>.

Confirm that the responses include the values you plan to use for:

• <kube_ovn_version>
• <dns_image_tag>
• <cpi_image_tag>

The Registry does not need to serve the etcd or kube-apiserver images when the VM template contains their exact image references and capv-load-local-images.sh imports them before kubeadm runs. Static-Pod deployment alone is not sufficient: verify the local images whenever the OS template, Kubernetes version, imageRepository, or component tags change.

#Validate the locally preloaded images

Create and boot a disposable VM from the exact candidate template. Run the following commands inside that VM to exercise the same import path used before kubeadm and list the image references that are actually available to containerd:

sudo bash -euo pipefail <<'EOF'
shopt -s nullglob
image_files=(/root/images/*.tar)
if (( ${#image_files[@]} == 0 )); then
  echo "ERROR: no tar files found in /root/images" >&2
  exit 1
fi

systemctl restart containerd
for image_file in "${image_files[@]}"; do
  ctr -n k8s.io images import "${image_file}"
done
ctr -n k8s.io images list -q | sort -u
EOF

sudo grep -nE '^[[:space:]]*(sandbox_image|sandbox)[[:space:]]*=' \
  /etc/containerd/config.toml

Compare the first command's output with the complete references required by the selected Kubernetes version and manifest values. Confirm that every required kubeadm image is present, including the exact etcd and kube-apiserver references generated from <image_registry>/tkestack, <k8s_version>, and the component tags. Compare the second command's sandbox reference with the imported pause-image reference; they must be identical.

Perform this validation again whenever the template, Kubernetes version, imageRepository, etcd tag, or containerd sandbox setting changes. Discard the validation VM afterward; do not convert it back into the source template.

Set cpaas.io/registry-address in the workload Cluster annotation to <registry_address>.

#Minimum Single-Datacenter Parameters

#Datacenter and resource placement

#Primary NIC parameters

#Control plane machine config pool

#Worker machine config pool

#Compute sizing

#Disk sizing model

VSphereMachineTemplate.spec.template.spec.diskGiB and VSphereMachineConfigPool.spec.configs[].persistentDisks[] define different disks.

Do not add the persistent disk sizes to diskGiB. For example, if the OS image has a 100 GiB system disk and the node also needs three 100 GiB persistent disks, use a system disk value that is at least 100 GiB, and define the three 100 GiB data disks under persistentDisks. Setting diskGiB to 400 GiB would create a 400 GiB system disk in addition to the three data disks.

#Standard data disks

Each node role requires a set of dedicated data disks. The disks listed below are the minimum required disks. You can append additional disks to the persistentDisks list if your workload requires them.

#Persistent disk fields

#Persistent disk lifecycle

Persistent disks are owned by a slot in one specific VSphereMachineConfigPool, not by the short-lived VM. A slot in another pool is a different allocation even when it declares the same hostname, IP address, or disk names. For vSphere Provider v1.0.16, the expected replacement lifecycle is:

1. While a Machine uses the slot, status.configStatuses[].state is InUse and the VMDKs are attached to its VM.
2. When the Machine and VM are deleted, the slot becomes Released. The VMDKs remain available for replacement and are not permanent orphaned disks.
3. A replacement Machine that references the same existing pool can reuse the Released slot immediately. The provider selects a released slot before an unused slot and reattaches the declared persistent VMDKs to the replacement VM. releaseDelayHours is not a wait time for reuse.
4. If the slot remains unused until releaseDelayHours expires, the controller can make it generally available and reclaim persistent VMDKs only after it confirms they are no longer attached. Asynchronous progress appears in status.configStatuses[].reclaimStatus.

wipeFilesystem controls data handling after a persistent disk is attached to a new VM; it does not control slot selection or datastore reclamation. false preserves the existing filesystem contents. true deletes the mounted contents during the new VM's first boot; it does not make VMDK reclamation faster and must not be treated as a datastore-capacity setting.

#Redeployment and teardown capacity

Creating a new VSphereMachineConfigPool with a different metadata.name creates a separate set of slots. Those slots do not reuse VMDKs owned by an earlier pool. Repeated teardown and redeployment with different pool names can therefore temporarily consume multiple complete sets of persistent disks until every earlier pool finishes provider-managed reclaim.

Use these rules when retrying or tearing down a deployment:

• For Machine replacement or rolling updates within the same cluster, keep each VSphereMachineTemplate.spec.template.spec.machineConfigPoolRef.name pointed at the same existing pool. Changing KubeadmControlPlane.spec.machineNamingStrategy does not transfer disks between pools and is not a datastore cleanup control.
• If a new deployment intentionally uses a new pool name, include the new pool's complete persistent-disk allocation in datastore capacity planning. Do not assume matching hostnames or disk names enable reuse across pools.
• Before starting another deployment against a capacity-constrained datastore, verify that earlier pool objects have completed deletion and that their VMDKs have been reclaimed. Pool deletion is asynchronous because the finalizer waits for Machines to be removed and for persistent-disk reclaim to finish.
• Do not remove the pool finalizer or manually delete backing VMDKs to accelerate teardown. If reclaim does not progress, collect the pool status, Events, provider-controller logs, and vCenter attachment state before treating the condition as a provider defect.

Use the following fields when you investigate a released slot or a datastore capacity alert:

Inspect the pool and related Events without reading credentials:

kubectl -n <namespace> get vspheremachineconfigpool <pool_name> -o yaml
kubectl -n <namespace> describe vspheremachineconfigpool <pool_name>
kubectl -n <namespace> get events \
  --field-selector involvedObject.kind=VSphereMachineConfigPool,involvedObject.name=<pool_name>

#Control plane nodes (3 disks per node)

#Worker nodes (2 disks per node)

#Size parameters

#vSphere CPI parameters

#Optional Creation-Time Topology Parameters

#Multiple datacenters and multiple failure domains

#Second NIC parameters

#Final Readiness Check

Before you start the deployment, confirm all of the following items:

1. The global cluster is reachable.
2. The two cluster plugins are installed: Alauda Container Platform Kubeadm Provider and Alauda Container Platform VMware vSphere Infrastructure Provider.
3. ClusterResourceSet=true is enabled.
4. The vCenter server, username, password, and thumbprint are collected.
5. The external LoadBalancer satisfies the Layer 4 listener, backend, health-check, ownership, and reachability contract.
6. The Pod CIDR, Service CIDR, and kube-ovn-join-cidr do not overlap with existing networks.
7. The VM template is available in every required datacenter.
8. The required datastores and vCenter resource pool paths are confirmed.
9. The permanent platform Registry contains the required Kube-OVN, CoreDNS, and vSphere CPI tags.
10. The VM template contains the exact locally preloaded etcd and kube-apiserver image references required by the selected Kubernetes and component versions.
11. The cpaas.io/registry-address value is planned as <registry_address> without a scheme or /v2/ path.
12. The control plane and worker bootstrap manifests both write /etc/resolv.conf through kubeadm files.
13. The machine config pool values for the minimum single-datacenter topology are complete.
14. The baseline system disk and data disk sizing is confirmed, and persistent disk sizes are not added into diskGiB.
15. releaseDelayHours, same-pool persistent-disk reuse, new-pool capacity allocation, and provider-managed teardown reclaim are understood before datastore capacity is allocated.
16. Every required parameter has a real value.

#Next Steps

After you complete this checklist, continue with Creating Clusters on VMware vSphere.