#使用 Velero
Velero 是 VMware 提供的开源 Kubernetes 集群备份和迁移工具。KubeVirt 提供了 Velero 插件以支持虚拟机的备份与恢复。
#前提条件
- Kubernetes 版本：1.20 或更高（Velero 要求）
- S3 存储：Velero BackupStorageLocation 使用平台提供的 Ceph 或 MinIO 对象存储
- 块存储：平台提供的 Ceph RBD
#操作步骤
#准备工作
-
通过 平台管理 → 集群管理 → 备份与恢复 → 备份仓库 在 ACP 平台部署 Velero。使用对象存储信息创建备份仓库。
-
启用 CSI 功能并添加 KubeVirt 插件：
kubectl edit deploy -n cpaas-system velero在 Velero 部署中添加以下内容：
containers: - args: - server - --uploader-type=restic - --namespace=cpaas-system - --features=EnableCSI command: - /velero在 initContainers 部分添加：
initContainers: - image: registry.example.org/3rdparty/kubevirt/kubevirt-velero-plugin:v0.7.0 imagePullPolicy: IfNotPresent name: velero-plugin-kubevirt resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /target name: plugins -
验证 KubeVirt 插件安装：
POD=$(kubectl -n cpaas-system get pod -l app.kubernetes.io/name=velero -o jsonpath='{.items[0].metadata.name}') kubectl -n cpaas-system exec -ti "$POD" -- /velero get plugins | grep kubevirt示例输出：
kubevirt-velero-plugin/backup-datavolume-action BackupItemAction kubevirt-velero-plugin/backup-datavolume-action BackupItemAction kubevirt-velero-plugin/backup-virtualmachine-action BackupItemAction kubevirt-velero-plugin/backup-virtualmachine-action BackupItemAction kubevirt-velero-plugin/backup-virtualmachineinstance-action BackupItemAction kubevirt-velero-plugin/backup-virtualmachineinstance-action BackupItemAction kubevirt-velero-plugin/restore-pod-action RestoreItemAction kubevirt-velero-plugin/restore-pod-action RestoreItemAction kubevirt-velero-plugin/restore-pvc-action RestoreItemAction kubevirt-velero-plugin/restore-pvc-action RestoreItemAction kubevirt-velero-plugin/restore-vm-action RestoreItemAction kubevirt-velero-plugin/restore-vm-action RestoreItemAction kubevirt-velero-plugin/restore-vmi-action RestoreItemAction kubevirt-velero-plugin/restore-vmi-action RestoreItemAction -
调整 node-agent 挂载宿主机 kubelet 目录并开启特权模式（数据通过
/var/lib/kubelet迁移时必需）：kubectl edit ds -n cpaas-system node-agent更新为：
securityContext: privileged: true volumeMounts: - mountPath: /host_pods mountPropagation: HostToContainer name: host-pods - mountPath: /var/lib/kubelet/plugins mountPropagation: HostToContainer name: host-plugins - mountPath: /scratch name: scratch securityContext: runAsUser: 0 volumes: - hostPath: path: /var/lib/kubelet/pods type: "" name: host-pods - hostPath: path: /var/lib/kubelet/plugins type: "" name: host-plugins - emptyDir: {} name: scratch
#备份
-
按需创建带有多个磁盘的虚拟机。
-
创建备份资源：
apiVersion: velero.io/v1 kind: Backup metadata: annotations: velero.io/resource-timeout: 10m0s velero.io/source-cluster-k8s-gitversion: v1.30.4 velero.io/source-cluster-k8s-major-version: "1" velero.io/source-cluster-k8s-minor-version: "30" labels: velero.io/storage-location: default name: example-backup namespace: cpaas-system spec: csiSnapshotTimeout: 10m0s defaultVolumesToFsBackup: false hooks: {} includedNamespaces: - example-namespace itemOperationTimeout: 4h0m0s metadata: {} snapshotMoveData: true storageLocation: default ttl: 720h0m0s -
等待备份完成并验证：
POD=$(kubectl -n cpaas-system get pod -l app.kubernetes.io/name=velero -o jsonpath='{.items[0].metadata.name}') kubectl -n cpaas-system exec -ti "$POD" -- /velero backup describe example-backup --details kubectl -n cpaas-system exec -ti "$POD" -- /velero backup get example-backup示例输出：
NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR example-backup WaitingForPluginOperations 0 0 2025-01-25 14:14:08 +0000 UTC 29d default <none> -
查看数据迁移状态：
kubectl get dataupload -n cpaas-system示例输出：
NAME STATUS STARTED BYTES DONE TOTAL BYTES STORAGE LOCATION AGE NODE example-backup-fhc6b Completed 11h 21474836480 21474836480 default 11h 192.168.254.66 example-backup-qwwzh Completed 11h 21474836480 21474836480 default 11h 192.168.254.15 -
验证备份完成：
POD=$(kubectl -n cpaas-system get pod -l app.kubernetes.io/name=velero -o jsonpath='{.items[0].metadata.name}') kubectl -n cpaas-system exec -ti "$POD" -- /velero backup get example-backup示例输出：
NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR example-backup Completed 0 0 2025-01-25 14:14:08 +0000 UTC 29d default <none> -
检查备份仓库（如 MinIO）中的目录：
mc ls <backup-repository>示例输出：
[2025-01-26 09:23:08 CST] 0B backups/ [2025-01-26 09:23:08 CST] 0B kopia/ [2025-01-26 09:23:08 CST] 0B restic/
#恢复
-
删除原虚拟机。
-
通过 平台管理 → 集群管理 → 备份与恢复 → 恢复管理 创建恢复资源，或手动创建：
apiVersion: velero.io/v1 kind: Restore metadata: annotations: cpaas.io/description: "" finalizers: - restores.velero.io/external-resources-finalizer name: example-restore namespace: cpaas-system spec: backupName: example-backup excludedResources: - nodes - events - events.events.k8s.io - backups.velero.io - restores.velero.io - resticrepositories.velero.io - csinodes.storage.k8s.io - volumeattachments.storage.k8s.io - backuprepositories.velero.io hooks: {} includedNamespaces: - example-namespace itemOperationTimeout: 4h0m0s namespaceMapping: {} -
验证恢复：
kubectl exec -ti -n cpaas-system velero-5df7bb7598-ljjrn -- /velero restore get查看 datadownload 状态：
kubectl get datadownload -n cpaas-system示例输出：
NAME STATUS STARTED BYTES DONE TOTAL BYTES STORAGE LOCATION AGE NODE example-restore-7lfcm Completed 11m 21474836480 21474836480 default 15m 192.168.254.66 example-restore-cjcd8 Completed 15m 21474836480 21474836480 default 15m 192.168.254.66恢复状态：
NAME BACKUP STATUS STARTED COMPLETED ERRORS WARNINGS CREATED SELECTOR aa example-backup Completed 2025-01-26 01:53:14 +0000 UTC 2025-01-26 02:01:24 +0000 UTC 0 2 2025-01-26 01:53:14 +0000 UTC <none> -
确认虚拟机已恢复且正常运行。
#跨集群恢复
- 在新集群部署 Velero 并完成准备工作。
- 配置与原集群相同的备份仓库（相同桶和目录），备份资源将自动出现。
- 按上述恢复步骤操作。
#恢复到不同命名空间
在恢复 spec 中添加 namespaceMapping 字段：
spec:
namespaceMapping:
example-namespace: test-namespace#恢复到不同存储类
- 在恢复前，在
cpaas-system命名空间创建 ConfigMap：apiVersion: v1 kind: ConfigMap metadata: name: change-storage-class-config namespace: cpaas-system labels: velero.io/plugin-config: "" velero.io/change-storage-class: RestoreItemAction data: vm-cephrbd: vm-topolvm-a - 确保存储能力（如跨节点访问、RWX）一致。
- 注意：当前不支持 RWX 模式的修改。