为 Pod 安全应用 SecurityContextConstraints
本指南适用于平台管理员和安全管理员。它将向你展示如何在现有 Kyverno 部署之上安装一个 SecurityContextConstraints(SCC)引擎,以及如何将 SCC profile 绑定到 ServiceAccount、User 和 Group,从而在 admission 时自动强制执行 Pod 安全边界。
简介
OpenShift 的 SecurityContextConstraints(SCC)模型允许集群管理员定义一组 Pod 安全 profile,然后授予 subject(ServiceAccount、User、Group)使用特定 profile 的权限。当 Pod 被准入时,平台会挑选该 subject 被允许使用的最合适的 SCC,填充缺失的默认值,并根据该 profile 验证 Pod。工作负载本身不需要声明每一个安全字段——SCC profile 会替它们完成。
原生 Kubernetes 没有内置的等效机制。本指南会安装一个基于 Kyverno 的引擎,在任何已经运行 Kyverno 的标准 Kubernetes 集群上重现 SCC 体验。它使用:
- 一个
SecurityContextConstraints CRD(security.alauda.io/v1alpha1)来存储 SCC profile。
- 标准 Kubernetes RBAC(
use 动词加上 resourceNames)将 subject 绑定到 profile,因此 operator 工作流与 OpenShift 保持一致(oc adm policy add-scc-to-user 模式可一一对应)。
- 一对 Kyverno admission policy——一个 mutating,一个 validating——用于选择正确的 SCC、填充默认值,并拒绝任何没有被授予的 SCC 能接受的 Pod。
- 五个
GlobalContextEntry 资源,用于在内存中缓存 SCC profile 和相关 RBAC 对象,从而使 admission 决策无需额外的 API 调用。
结果是:应用团队仍然可以编写简单直接的 Pod manifest,集群会自动将其约束到该 ServiceAccount 被允许使用的安全 profile,并且从 OpenShift 迁移时无需更改绑定模型。
SCC 授权属于安全控制变更。应用团队不应被授予直接创建或修改 SCC RBAC 绑定的权限,因为这样做会让他们绕过集群安全边界。应用团队应描述工作负载需求,例如 anyuid、hostNetwork 或 hostPath;平台或安全管理员审查请求,并将最小权限的 SCC 绑定到相应 subject。
谁负责什么
使用下表判断本指南的哪些部分适合你。
如果你是平台或安全管理员,请按照第 1 部分和第 2 部分操作。如果你是应用经理,请先使用步骤 2.1 准备 SCC 请求,然后仅在管理员批准并完成绑定后,再使用步骤 2.5 和步骤 2.6。不要自行应用步骤 2.2 到步骤 2.4 中的 RBAC manifest。
正常工作流如下:
- 应用经理确定工作负载需求和目标 ServiceAccount。
- 平台或安全管理员选择最小权限的 SCC,并创建 RBAC 绑定。
- 应用经理使用已批准的 ServiceAccount 部署工作负载,并且只有在管理员要求将某个特定 SCC 固定时,才添加
alauda.io/required-scc。
- 管理员使用
kubectl auth can-i 验证授权,工作负载所有者验证已准入的 Pod 具有预期的 alauda.io/scc 注解。
场景
当满足以下任一情况时,应用本指南:
- 你正在从 OpenShift 迁移工作负载,并希望保留现有的
oc adm policy add-scc-to-* 绑定模型,以便平台团队和审计工具可以继续按原样工作。
- 你已经在使用 Kyverno,并且需要一个集中管理的安全边界,而不是要求每个 Pod manifest 都声明完整的
securityContext。
- 你运营的是多租户集群,并希望不同 namespace 中的不同 ServiceAccount 接收不同的安全上限——例如,限制为
restricted-v2 的应用 SA、允许 hostmount-anyuid 的日志收集 SA,以及允许 NET_BIND_SERVICE 的 ingress controller SA。
- 你希望有一个全局统一的位置来表达和审计“谁可以运行 privileged Pod”,而不是将例外分散到每个 namespace 中。
前提条件
开始之前,请确保以下条件全部满足:
-
Kubernetes 集群运行版本 1.30 或更高(CEL admission 已稳定)。
-
Kyverno 已经安装并运行,版本为 v4.3.1 或更高,并且可用 MutatingPolicy、ValidatingPolicy 和 GlobalContextEntry CRD。你可以通过以下命令验证:
kubectl get crd validatingpolicies.policies.kyverno.io mutatingpolicies.policies.kyverno.io globalcontextentries.kyverno.io
-
kyverno namespace 中包含以下 ServiceAccount(默认 Kyverno 安装):
kyverno-admission-controller
kyverno-background-controller
kyverno-reports-controller
-
你拥有 cluster-admin(或等效)权限,因为安装引擎需要创建 CRD、ClusterRole、ClusterRoleBinding、GlobalContextEntry 和 admission policy。
-
你已经检查了计划允许非 restricted Pod 的每个 namespace 上的 Pod Security Admission(PSA)enforce 标签。PSA 会在 Kyverno 之前 运行;如果某个 namespace 标记为 pod-security.kubernetes.io/enforce: restricted,那么在 Kyverno 介入之前,该 namespace 就会拒绝任何与 anyuid 或 hostnetwork-v2 之类宽松 SCC 匹配的 Pod。请在适当的 namespace 中将标签调整为 baseline 或 privileged,或者限制你在这些 namespace 中提供的 SCC profile 集合。
Tip
引擎安装是一次性工作,通常由平台管理员执行。第 2 部分同样是管理员工作流:平台或安全管理员会在审查工作负载需求后绑定 SCC profile。应用团队通常只需提供这些需求,然后使用分配的 ServiceAccount。
步骤
工作分为两部分:
- 第 1 部分 在集群范围内安装 SCC 引擎。每个集群只需执行一次。
- 第 2 部分 通过将 SCC profile 绑定到 ServiceAccount、User 和 Group 来授权工作负载使用 SCC,并在需要时将特定工作负载固定到特定 SCC。
第 1 部分:安装 SCC 引擎
步骤 1.1 — 安装 SecurityContextConstraints CRD
将以下 manifest 保存为 scc-crd.yaml。它定义了一个 cluster-scoped 的 SecurityContextConstraints 资源(简称 scc),其字段与 OpenShift SCC 语义一致。
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: securitycontextconstraints.security.alauda.io
spec:
group: security.alauda.io
names:
plural: securitycontextconstraints
singular: securitycontextconstraints
kind: SecurityContextConstraints
listKind: SecurityContextConstraintsList
shortNames:
- scc
scope: Cluster
versions:
- name: v1alpha1
served: true
storage: true
schema:
openAPIV3Schema:
description: |
SecurityContextConstraints governs the ability to make requests that affect
container security context. This custom CRD mirrors OpenShift SCC semantics
while keeping fields under spec for Kyverno CEL consumption.
type: object
required:
- spec
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
type: object
required:
- runAsUser
properties:
allowHostPorts:
description: Determines if the profile allows host ports in containers.
type: boolean
priority:
description: Higher priority SCC is evaluated first.
type: integer
format: int32
nullable: true
restrictiveScore:
description: Secondary sort key. Lower score means less restrictive.
type: integer
format: int32
minimum: 0
requiredDropCapabilities:
description: Capabilities that must be dropped.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
allowPrivilegedContainer:
description: Determines if privileged containers are allowed.
type: boolean
runAsUser:
description: Strategy controlling runAsUser.
type: object
nullable: true
properties:
type:
description: Strategy type for runAsUser.
type: string
enum:
- RunAsAny
- MustRunAs
- MustRunAsRange
- MustRunAsNonRoot
- MustRunAsNonRootOrSystem
uid:
description: Required when type=MustRunAs.
type: integer
format: int64
minimum: 0
uidRangeMin:
description: Minimum uid for MustRunAsRange.
type: integer
format: int64
minimum: 0
uidRangeMax:
description: Maximum uid for MustRunAsRange.
type: integer
format: int64
minimum: 0
users:
description: Users who can use this SCC.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
groups:
description: Groups who can use this SCC.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
allowHostDirVolumePlugin:
description: Determines if hostPath-like volume plugin usage is allowed.
type: boolean
seccompProfiles:
description: Allowed seccomp profiles. '*' allows all.
type: array
nullable: true
items:
type: string
pattern: "^(\\*|runtime/default|unconfined|localhost/.+)$"
x-kubernetes-list-type: atomic
allowHostIPC:
description: Determines if host IPC is allowed.
type: boolean
forbiddenSysctls:
description: Explicitly forbidden sysctls.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
seLinuxContext:
description: Strategy controlling SELinux labels.
type: object
nullable: true
properties:
type:
description: Strategy type for SELinux context.
type: string
seLinuxOptions:
description: Fixed SELinux options required by MustRunAs.
type: object
properties:
user:
type: string
role:
type: string
type:
type: string
level:
type: string
readOnlyRootFilesystem:
description: Forces readOnlyRootFilesystem when set to true.
type: boolean
fsGroup:
description: Strategy controlling fsGroup.
type: object
nullable: true
properties:
type:
type: string
ranges:
type: array
items:
type: object
properties:
min:
type: integer
format: int64
max:
type: integer
format: int64
x-kubernetes-list-type: atomic
supplementalGroups:
description: Strategy controlling supplemental groups.
type: object
nullable: true
properties:
type:
type: string
ranges:
type: array
items:
type: object
properties:
min:
type: integer
format: int64
max:
type: integer
format: int64
x-kubernetes-list-type: atomic
userNamespaceLevel:
description: Controls host user namespace usage.
type: string
default: AllowHostLevel
enum:
- AllowHostLevel
- RequirePodLevel
defaultAddCapabilities:
description: Capabilities added by default unless explicitly dropped.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
allowedUnsafeSysctls:
description: Explicitly allowed unsafe sysctls.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
allowedFlexVolumes:
description: Allowed flex volume drivers.
type: array
nullable: true
items:
type: object
required:
- driver
properties:
driver:
type: string
x-kubernetes-list-type: atomic
volumes:
description: Allowed volume plugin types. '*' allows all.
type: array
nullable: true
items:
type: string
enum:
- '*'
- none
- hostPath
- emptyDir
- gcePersistentDisk
- awsElasticBlockStore
- gitRepo
- secret
- nfs
- iscsi
- glusterfs
- persistentVolumeClaim
- rbd
- flexVolume
- cinder
- cephfs
- flocker
- downwardAPI
- fc
- azureFile
- configMap
- vsphereVolume
- quobyte
- azureDisk
- photonPersistentDisk
- projected
- portworxVolume
- scaleIO
- storageos
- csi
- ephemeral
- image
x-kubernetes-list-type: atomic
allowHostPID:
description: Determines if host PID is allowed.
type: boolean
allowHostNetwork:
description: Determines if hostNetwork is allowed.
type: boolean
allowPrivilegeEscalation:
description: Determines if privilege escalation can be requested.
type: boolean
nullable: true
defaultAllowPrivilegeEscalation:
description: Default for allowPrivilegeEscalation when container omits it.
type: boolean
nullable: true
allowedCapabilities:
description: Capabilities that may be added.
type: array
nullable: true
items:
type: string
x-kubernetes-list-type: atomic
x-kubernetes-validations:
- rule: "!has(self.runAsUser) || self.runAsUser.type != 'MustRunAs' || has(self.runAsUser.uid)"
message: "runAsUser.uid is required when runAsUser.type is MustRunAs."
- rule: "!has(self.runAsUser) || self.runAsUser.type == 'MustRunAs' || !has(self.runAsUser.uid)"
message: "runAsUser.uid is only allowed when runAsUser.type is MustRunAs."
- rule: "!has(self.runAsUser) || self.runAsUser.type != 'MustRunAsRange' || (has(self.runAsUser.uidRangeMin) && has(self.runAsUser.uidRangeMax))"
message: "uidRangeMin and uidRangeMax are required when runAsUser.type is MustRunAsRange."
- rule: "!has(self.runAsUser) || self.runAsUser.type == 'MustRunAsRange' || (!has(self.runAsUser.uidRangeMin) && !has(self.runAsUser.uidRangeMax))"
message: "uidRangeMin and uidRangeMax are only allowed when runAsUser.type is MustRunAsRange."
- rule: "!has(self.runAsUser) || !has(self.runAsUser.uidRangeMin) || !has(self.runAsUser.uidRangeMax) || self.runAsUser.uidRangeMin <= self.runAsUser.uidRangeMax"
message: "uidRangeMin must be less than or equal to uidRangeMax."
additionalPrinterColumns:
- name: Priv
type: string
description: Determines if privileged containers are allowed
jsonPath: .spec.allowPrivilegedContainer
- name: Caps
type: string
description: Allowed capabilities
jsonPath: .spec.allowedCapabilities
- name: SELinux
type: string
description: SELinux strategy
jsonPath: .spec.seLinuxContext.type
- name: RunAsUser
type: string
description: RunAsUser strategy
jsonPath: .spec.runAsUser.type
- name: FSGroup
type: string
description: FSGroup strategy
jsonPath: .spec.fsGroup.type
- name: SupGroup
type: string
description: SupplementalGroups strategy
jsonPath: .spec.supplementalGroups.type
- name: Priority
type: string
description: SCC sort priority
jsonPath: .spec.priority
- name: Score
type: string
description: Secondary restrictive score
jsonPath: .spec.restrictiveScore
- name: ReadOnlyRootFS
type: string
description: Force read-only root filesystem
jsonPath: .spec.readOnlyRootFilesystem
- name: Volumes
type: string
description: Allowed volume plugins
jsonPath: .spec.volumes
conversion:
strategy: None
应用它,并等待 CRD 变为 Established 后再继续:
kubectl apply -f scc-crd.yaml
kubectl wait --for=condition=Established --timeout=120s \
crd/securitycontextconstraints.security.alauda.io
步骤 1.2 — 安装 13 个内置 SCC profile
将以下 manifest 保存为 scc-profiles.yaml。它定义了 13 个基于 OpenShift 内置集合建模的 SCC profile,按从最严格(restrictiveScore: 100)到最宽松(restrictiveScore: 0)排序。当同一个 subject 被授予多个 SCC 时,自动选择策略会优先选择更高的 restrictiveScore。
Tip
你不必安装所有 profile。可以根据平台提供的范围裁剪这个 manifest——但你必须至少保留一个 profile 可供每个 subject 使用,否则它们的 Pod 会在 admission 时被拒绝。
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: restricted-v2
spec:
priority: 0
restrictiveScore: 100
allowPrivilegedContainer: false
allowPrivilegeEscalation: false
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsRange
uidRangeMin: 1
uidRangeMax: 2147483647
seLinuxContext:
type: MustRunAs
fsGroup:
type: MustRunAs
supplementalGroups:
type: RunAsAny
allowedCapabilities:
- NET_BIND_SERVICE
requiredDropCapabilities:
- ALL
defaultAddCapabilities: []
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
seccompProfiles:
- runtime/default
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: restricted-v3
spec:
priority: 0
restrictiveScore: 100
allowPrivilegedContainer: false
allowPrivilegeEscalation: false
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsRange
uidRangeMin: 1000
uidRangeMax: 65534
seLinuxContext:
type: MustRunAs
fsGroup:
type: MustRunAs
ranges:
- min: 1000
max: 65534
supplementalGroups:
type: MustRunAs
ranges:
- min: 1000
max: 65534
userNamespaceLevel: RequirePodLevel
allowedCapabilities:
- NET_BIND_SERVICE
requiredDropCapabilities:
- ALL
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
seccompProfiles:
- runtime/default
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: restricted
spec:
priority: 0
restrictiveScore: 98
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsRange
uidRangeMin: 1
uidRangeMax: 2147483647
seLinuxContext:
type: MustRunAs
fsGroup:
type: MustRunAs
supplementalGroups:
type: RunAsAny
allowedCapabilities: []
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: nonroot-v2
spec:
priority: 0
restrictiveScore: 95
allowPrivilegedContainer: false
allowPrivilegeEscalation: false
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsNonRoot
seLinuxContext:
type: MustRunAs
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
allowedCapabilities:
- NET_BIND_SERVICE
requiredDropCapabilities:
- ALL
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
seccompProfiles:
- runtime/default
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: nonroot
spec:
priority: 0
restrictiveScore: 92
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsNonRoot
seLinuxContext:
type: MustRunAs
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
allowedCapabilities: []
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: hostnetwork-v2
spec:
priority: 0
restrictiveScore: 70
allowPrivilegedContainer: false
allowPrivilegeEscalation: false
allowHostNetwork: true
allowHostPID: false
allowHostIPC: false
allowHostPorts: true
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsRange
uidRangeMin: 1
uidRangeMax: 2147483647
seLinuxContext:
type: MustRunAs
fsGroup:
type: MustRunAs
supplementalGroups:
type: MustRunAs
allowedCapabilities:
- NET_BIND_SERVICE
requiredDropCapabilities:
- ALL
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
seccompProfiles:
- runtime/default
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: hostnetwork
spec:
priority: 0
restrictiveScore: 68
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: true
allowHostPID: false
allowHostIPC: false
allowHostPorts: true
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsRange
uidRangeMin: 1
uidRangeMax: 2147483647
seLinuxContext:
type: MustRunAs
fsGroup:
type: MustRunAs
supplementalGroups:
type: MustRunAs
allowedCapabilities: []
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: anyuid
spec:
priority: 10
restrictiveScore: 60
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: RunAsAny
seLinuxContext:
type: MustRunAs
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
allowedCapabilities: []
requiredDropCapabilities:
- MKNOD
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: nested-container
spec:
priority: 0
restrictiveScore: 58
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: false
runAsUser:
type: MustRunAsRange
uidRangeMin: 0
uidRangeMax: 65534
seLinuxContext:
type: MustRunAs
seLinuxOptions:
type: container_engine_t
fsGroup:
type: MustRunAs
ranges:
- min: 0
max: 65534
supplementalGroups:
type: MustRunAs
ranges:
- min: 0
max: 65534
userNamespaceLevel: RequirePodLevel
allowedCapabilities:
- SETUID
- SETGID
requiredDropCapabilities: []
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- image
- persistentVolumeClaim
- projected
- secret
seccompProfiles:
- '*'
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: hostmount-anyuid
spec:
priority: 0
restrictiveScore: 55
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: true
runAsUser:
type: RunAsAny
seLinuxContext:
type: MustRunAs
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
allowedCapabilities: []
requiredDropCapabilities:
- MKNOD
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- hostPath
- image
- nfs
- persistentVolumeClaim
- projected
- secret
readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: hostmount-anyuid-v2
spec:
priority: 0
restrictiveScore: 50
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: false
allowHostPID: false
allowHostIPC: false
allowHostPorts: false
allowHostDirVolumePlugin: true
runAsUser:
type: RunAsAny
seLinuxContext:
type: RunAsAny
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
allowedCapabilities: []
requiredDropCapabilities:
- MKNOD
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- hostPath
- image
- nfs
- persistentVolumeClaim
- projected
- secret
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: hostaccess
spec:
priority: 0
restrictiveScore: 40
allowPrivilegedContainer: false
allowPrivilegeEscalation: true
allowHostNetwork: true
allowHostPID: true
allowHostIPC: true
allowHostPorts: true
allowHostDirVolumePlugin: true
runAsUser:
type: MustRunAsRange
uidRangeMin: 1
uidRangeMax: 2147483647
seLinuxContext:
type: MustRunAs
fsGroup:
type: MustRunAs
supplementalGroups:
type: RunAsAny
allowedCapabilities: []
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
volumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- hostPath
- image
- persistentVolumeClaim
- projected
- secret
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
name: privileged
spec:
priority: 0
restrictiveScore: 0
allowPrivilegedContainer: true
allowPrivilegeEscalation: true
allowHostNetwork: true
allowHostPID: true
allowHostIPC: true
allowHostPorts: true
allowHostDirVolumePlugin: true
runAsUser:
type: RunAsAny
seLinuxContext:
type: RunAsAny
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
allowedCapabilities:
- '*'
requiredDropCapabilities: []
volumes:
- '*'
seccompProfiles:
- '*'
allowedUnsafeSysctls:
- '*'
应用这些 profile:
kubectl apply -f scc-profiles.yaml
kubectl get scc
你应该看到所有 13 个 profile 都被列出,并且其 Priority 和 Score 列已有值(同时还会显示其他 SCC 列,例如 Priv, RunAsUser 和 Volumes)。
步骤 1.3 — 安装 GlobalContextEntry、Kyverno reader RBAC 和 admission policy
此步骤一次安装三项内容:
- GlobalContextEntry(GCE)——五个内存缓存,Kyverno 在 admission 期间通过它们查找 SCC profile、ClusterRole、ClusterRoleBinding、RoleBinding 和 Role,而无需对每个请求发起 API 调用。
- Reader RBAC——一个 ClusterRole,授予 Kyverno 的三个 ServiceAccount 对 SCC CRD、上面四种 RBAC 资源,以及 policy 匹配的 Pod /
pods/ephemeralcontainers 资源的只读访问权限。
- 两个 admission policy——一个
MutatingPolicy 用于根据所选 SCC 填充默认值,一个 ValidatingPolicy 用于拒绝任何没有被授予的 SCC 能接受的 Pod。
Warning
这两个 policy 包含驱动 SCC 选择和验证的 CEL 逻辑。你不需要阅读或理解 CEL 就能使用该引擎——直接按原样应用 manifest 即可。这些表达式之所以很长,是因为它们逐字段复现了 OpenShift SCC 的 admission 算法。
将以下内容保存为 scc-gce.yaml 并应用:
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
name: scc-profiles
spec:
kubernetesResource:
group: security.alauda.io
version: v1alpha1
resource: securitycontextconstraints
projections:
- name: items
jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
name: scc-clusterroles
spec:
kubernetesResource:
group: rbac.authorization.k8s.io
version: v1
resource: clusterroles
projections:
- name: items
jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
name: scc-clusterrolebindings
spec:
kubernetesResource:
group: rbac.authorization.k8s.io
version: v1
resource: clusterrolebindings
projections:
- name: items
jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
name: scc-rolebindings
spec:
kubernetesResource:
group: rbac.authorization.k8s.io
version: v1
resource: rolebindings
projections:
- name: items
jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
name: scc-roles
spec:
kubernetesResource:
group: rbac.authorization.k8s.io
version: v1
resource: roles
projections:
- name: items
jmesPath: "@"
将以下内容保存为 scc-reader-rbac.yaml 并应用。需要 pods 和 pods/ephemeralcontainers 的读取权限,因为 Kyverno 在 policy readiness 门禁(RBACPermissionsGranted)期间会检查每个匹配资源的读权限;如果缺少这些权限,mutating policy 会一直处于 NotReady。
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kyverno-scc-reader
rules:
- apiGroups:
- security.alauda.io
resources:
- securitycontextconstraints
verbs:
- get
- list
- watch
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterroles
- clusterrolebindings
- rolebindings
- roles
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- pods
- pods/ephemeralcontainers
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kyverno-scc-reader
subjects:
- kind: ServiceAccount
name: kyverno-admission-controller
namespace: kyverno
- kind: ServiceAccount
name: kyverno-background-controller
namespace: kyverno
- kind: ServiceAccount
name: kyverno-reports-controller
namespace: kyverno
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kyverno-scc-reader
将以下内容保存为 scc-auto-pick.yaml。这是会拒绝任何没有被授予的 SCC 能接受的 Pod 的 ValidatingPolicy。
Warning
下面的示例配置为 validationActions: [Deny]。在现有集群上,第一次应用前请先将其改为 validationActions: [Warn],待你审查警告并创建所需的 SCC 绑定后,再改回 Deny。关于发布流程,请参见步骤 1.4。
apiVersion: policies.kyverno.io/v1alpha1
kind: ValidatingPolicy
metadata:
name: scc-auto-pick
labels:
reports.kyverno.io/disabled: "true"
annotations:
policies.kyverno.io/title: SCC Auto-Pick (CEL, CRD + RBAC)
pod-policies.kyverno.io/autogen-controllers: "none"
spec:
autogen:
podControllers:
controllers: []
validatingAdmissionPolicy:
enabled: false
evaluation:
admission:
enabled: true
background:
enabled: false
failurePolicy: Fail
validationActions:
- Deny
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["pods"]
matchConditions:
- name: skip-system-ns
expression: |
!(request.namespace.startsWith('kube-') ||
request.namespace.startsWith('cpaas-') ||
request.namespace.startsWith('alauda-') ||
request.namespace == 'kyverno' ||
request.namespace == 'cattle-system' ||
request.namespace == 'operators' ||
request.namespace == 'default')
variables:
- name: containers
expression: |
object.spec.containers + object.spec.?initContainers.orValue([]) +
object.spec.?ephemeralContainers.orValue([])
- name: required
expression: object.metadata.?annotations[?'alauda.io/required-scc'].orValue('')
- name: profiles
expression: |
cel.bind(items, globalContext.Get('scc-profiles', 'items'),
items == null ? [] : items)
- name: subjectMatches
expression: |
[
{'kind':'ServiceAccount',
'name': string(object.spec.?serviceAccountName.orValue('default')),
'namespace': string(request.namespace)},
{'kind':'Group', 'name':'system:serviceaccounts'},
{'kind':'Group', 'name':'system:serviceaccounts:'+request.namespace},
{'kind':'Group', 'name':'system:authenticated'},
{'kind':'User', 'name': request.userInfo.username}
]
+ request.userInfo.groups.map(g, {'kind':'Group','name': g})
- name: rolebindings
expression: |
cel.bind(rbs, globalContext.Get('scc-rolebindings','items'),
rbs == null ? [] : rbs)
- name: matchedClusterRoleRefsFromCRB
expression: |
cel.bind(crbs, globalContext.Get('scc-clusterrolebindings','items'),
crbs == null ? [] : crbs)
.filter(b, b.?roleRef.?kind.orValue('') == 'ClusterRole'
&& b.?subjects.orValue([]).exists(s,
variables.subjectMatches.exists(m,
s.kind == m.kind && s.name == m.name &&
(s.kind != 'ServiceAccount' ||
s.?namespace.orValue('') == m.?namespace.orValue('')))))
.map(b, b.roleRef.name)
- name: matchedClusterRoleRefsFromRB
expression: |
variables.rolebindings
.filter(b, b.?metadata.?namespace.orValue('') == request.namespace
&& b.?roleRef.?kind.orValue('') == 'ClusterRole'
&& b.?subjects.orValue([]).exists(s,
variables.subjectMatches.exists(m,
s.kind == m.kind && s.name == m.name &&
(s.kind != 'ServiceAccount' ||
s.?namespace.orValue('') == m.?namespace.orValue('')))))
.map(b, b.roleRef.name)
- name: matchedRoleRefsFromRB
expression: |
variables.rolebindings
.filter(b, b.?metadata.?namespace.orValue('') == request.namespace
&& b.?roleRef.?kind.orValue('') == 'Role'
&& b.?subjects.orValue([]).exists(s,
variables.subjectMatches.exists(m,
s.kind == m.kind && s.name == m.name &&
(s.kind != 'ServiceAccount' ||
s.?namespace.orValue('') == m.?namespace.orValue('')))))
.map(b, b.roleRef.name)
- name: matchedClusterRoleRefs
expression: |
variables.matchedClusterRoleRefsFromCRB + variables.matchedClusterRoleRefsFromRB
- name: allSccNames
expression: |
variables.profiles.map(p, p.metadata.name)
- name: assignedFromClusterRoles
expression: |
cel.bind(crs, globalContext.Get('scc-clusterroles','items'),
crs == null ? [] : crs)
.filter(r, variables.matchedClusterRoleRefs.exists(n, n == r.metadata.name))
.map(r, r.?rules.orValue([])
.filter(ru,
ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
.map(ru,
ru.?resourceNames.orValue([]).size() == 0
? variables.allSccNames
: ru.resourceNames)
)
.flatten()
.flatten()
- name: assignedFromRoles
expression: |
cel.bind(roles, globalContext.Get('scc-roles','items'),
roles == null ? [] : roles)
.filter(r,
r.?metadata.?namespace.orValue('') == request.namespace
&& variables.matchedRoleRefsFromRB.exists(n, n == r.metadata.name))
.map(r, r.?rules.orValue([])
.filter(ru,
ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
.map(ru,
ru.?resourceNames.orValue([]).size() == 0
? variables.allSccNames
: ru.resourceNames)
)
.flatten()
.flatten()
- name: assigned
expression: |
(variables.assignedFromClusterRoles + variables.assignedFromRoles)
.filter(n, variables.allSccNames.exists(s, s == n))
- name: safeSysctls
expression: |
['kernel.shm_rmid_forced',
'net.ipv4.ip_local_port_range',
'net.ipv4.ip_unprivileged_port_start',
'net.ipv4.tcp_syncookies',
'net.ipv4.ping_group_range']
- name: vtypes
expression: |
['hostPath','emptyDir','gcePersistentDisk','awsElasticBlockStore','gitRepo',
'secret','nfs','iscsi','glusterfs','persistentVolumeClaim','rbd','flexVolume',
'cinder','cephfs','flocker','downwardAPI','fc','azureFile','configMap',
'vsphereVolume','quobyte','azureDisk','photonPersistentDisk','projected',
'portworxVolume','scaleIO','storageos','csi','ephemeral','image']
- name: ordered
expression: |
variables.assigned.sortBy(n,
int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?priority.orValue(0)) * -100000 +
-int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?restrictiveScore.orValue(100))
)
- name: requiredExists
expression: variables.required == '' || variables.profiles.exists(pr, pr.metadata.name == variables.required)
- name: requiredBound
expression: variables.required == '' || variables.assigned.exists(n, n == variables.required)
- name: candidateNames
expression: |
variables.required != ''
? [variables.required]
: variables.ordered
- name: matched
expression: |
variables.candidateNames.exists(n,
cel.bind(p, variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.orValue({}),
(p.?allowPrivilegedContainer.orValue(false)
|| !variables.containers.exists(c, c.?securityContext.?privileged.orValue(false)))
&& (p.?allowPrivilegeEscalation.orValue(true)
|| !variables.containers.exists(c, c.?securityContext.?allowPrivilegeEscalation.orValue(true)))
&& (p.?allowHostNetwork.orValue(false) || !object.spec.?hostNetwork.orValue(false))
&& (p.?allowHostPID.orValue(false) || !object.spec.?hostPID.orValue(false))
&& (p.?allowHostIPC.orValue(false) || !object.spec.?hostIPC.orValue(false))
&& (p.?allowHostDirVolumePlugin.orValue(false)
|| !object.spec.?volumes.orValue([]).exists(v, has(v.hostPath)))
&& (
p.?runAsUser.?type.orValue('RunAsAny') == 'RunAsAny'
|| (
(p.?runAsUser.?type.orValue('RunAsAny') in ['MustRunAsNonRoot','MustRunAsNonRootOrSystem'])
&& !variables.containers.exists(c, c.?securityContext.?runAsUser.orValue(
object.spec.?securityContext.?runAsUser.orValue(1)) == 0)
)
|| (
p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAs'
&& variables.containers.all(c, c.?securityContext.?runAsUser.orValue(
object.spec.?securityContext.?runAsUser.orValue(1))
== p.?runAsUser.?uid.orValue(-1))
)
|| (
p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAsRange'
&& variables.containers.all(c,
c.?securityContext.?runAsUser.orValue(
object.spec.?securityContext.?runAsUser.orValue(1))
>= p.?runAsUser.?uidRangeMin.orValue(1)
&& c.?securityContext.?runAsUser.orValue(
object.spec.?securityContext.?runAsUser.orValue(1))
<= p.?runAsUser.?uidRangeMax.orValue(2147483647))
)
)
&& (p.?allowedCapabilities.orValue([]).exists(t, t == '*')
|| variables.containers.all(c,
c.?securityContext.?capabilities.?add.orValue([]).all(cap,
p.?allowedCapabilities.orValue([]).exists(a, a == cap))))
&& (p.?requiredDropCapabilities.orValue([]).size() == 0
|| variables.containers.all(c,
p.?requiredDropCapabilities.orValue([]).all(req,
c.?securityContext.?capabilities.?drop.orValue([]).exists(d, d == req || d == 'ALL'))))
&& (p.?volumes.orValue(['*']).exists(t, t == '*')
|| object.spec.?volumes.orValue([]).all(v,
variables.vtypes.filter(t, v[?t].hasValue()).all(t,
p.?volumes.orValue([]).exists(a, a == t))))
&& (p.?allowHostPorts.orValue(false)
|| variables.containers.all(c,
c.?ports.orValue([]).all(port, port.?hostPort.orValue(0) == 0)))
&& (p.?allowedUnsafeSysctls.orValue([]).exists(t, t == '*')
|| object.spec.?securityContext.?sysctls.orValue([]).all(s,
variables.safeSysctls.exists(safe, safe == s.name)
|| p.?allowedUnsafeSysctls.orValue([]).exists(a, a == s.name)))
&& (!p.?readOnlyRootFilesystem.orValue(false)
|| variables.containers.all(c, c.?securityContext.?readOnlyRootFilesystem.orValue(false) == true))
&& (p.?seccompProfiles.orValue([]).size() == 0
|| p.?seccompProfiles.orValue([]).exists(t, t == '*')
|| variables.containers.all(c,
p.?seccompProfiles.orValue([]).exists(a,
(c.?securityContext.?seccompProfile.?type.orValue(
object.spec.?securityContext.?seccompProfile.?type.orValue('')) == 'RuntimeDefault'
&& a == 'runtime/default')
|| (c.?securityContext.?seccompProfile.?type.orValue(
object.spec.?securityContext.?seccompProfile.?type.orValue('')) == 'Unconfined'
&& a == 'unconfined')
|| (c.?securityContext.?seccompProfile.?type.orValue(
object.spec.?securityContext.?seccompProfile.?type.orValue('')) == 'Localhost'
&& a == 'localhost/' + c.?securityContext.?seccompProfile.?localhostProfile.orValue(
object.spec.?securityContext.?seccompProfile.?localhostProfile.orValue(''))))))
&& (p.?allowedFlexVolumes.orValue([]).size() == 0
|| object.spec.?volumes.orValue([]).filter(v, v.?flexVolume.hasValue()).all(v,
p.?allowedFlexVolumes.orValue([]).exists(d, d.?driver.orValue('') == v.flexVolume.driver)))
)
)
validations:
- expression: variables.requiredExists
message: "required-scc does not exist"
messageExpression: |
"required SCC '" + variables.required + "' not found in scc-profiles"
- expression: variables.requiredBound
message: "required-scc is not bound to ServiceAccount"
messageExpression: |
"required SCC '" + variables.required +
"' is not bound to ServiceAccount '" +
object.spec.?serviceAccountName.orValue('default') +
"' in namespace '" + request.namespace + "'"
- expression: variables.matched
message: "Pod violates all SCCs assigned to its ServiceAccount"
messageExpression: |
variables.required != ''
? ("Pod " + object.metadata.name +
" does not satisfy required SCC '" + variables.required + "'")
: ("Pod " + object.metadata.name +
" does not satisfy any SCC profile assigned to ServiceAccount '" +
object.spec.?serviceAccountName.orValue('default') +
"' in namespace '" + request.namespace +
"' (candidates: " + variables.ordered.join(",") + ")")
将以下内容保存为 scc-fill-defaults.yaml 并应用。它是一个 MutatingPolicy,会在 Pod 上记录所选 SCC(alauda.io/scc 注解),并填充从该 SCC 继承而来的 runAsUser、seccompProfile 和 allowPrivilegeEscalation 默认值。
apiVersion: policies.kyverno.io/v1alpha1
kind: MutatingPolicy
metadata:
name: scc-fill-defaults
labels:
reports.kyverno.io/disabled: "true"
annotations:
policies.kyverno.io/title: SCC default value filler (CRD + RBAC, explicit-wins)
pod-policies.kyverno.io/autogen-controllers: "none"
spec:
autogen:
podControllers:
controllers: []
evaluation:
admission:
enabled: true
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE"]
resources: ["pods"]
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["UPDATE"]
resources: ["pods/ephemeralcontainers"]
matchConditions:
- name: skip-system-ns
expression: |
!(request.namespace.startsWith('kube-') ||
request.namespace.startsWith('cpaas-') ||
request.namespace.startsWith('alauda-') ||
request.namespace == 'kyverno' ||
request.namespace == 'cattle-system' ||
request.namespace == 'operators' ||
request.namespace == 'default')
variables:
- name: containers
expression: |
object.spec.containers + object.spec.?initContainers.orValue([]) +
object.spec.?ephemeralContainers.orValue([])
- name: required
expression: object.metadata.?annotations[?'alauda.io/required-scc'].orValue('')
- name: profiles
expression: |
cel.bind(items, globalContext.Get('scc-profiles', 'items'),
items == null ? [] : items)
- name: subjectMatches
expression: |
[
{'kind':'ServiceAccount',
'name': string(object.spec.?serviceAccountName.orValue('default')),
'namespace': string(object.metadata.namespace)},
{'kind':'Group', 'name':'system:serviceaccounts'},
{'kind':'Group', 'name':'system:serviceaccounts:'+object.metadata.namespace},
{'kind':'Group', 'name':'system:authenticated'},
{'kind':'User', 'name': request.userInfo.username}
]
+ request.userInfo.groups.map(g, {'kind':'Group','name': g})
- name: rolebindings
expression: |
cel.bind(rbs, globalContext.Get('scc-rolebindings','items'),
rbs == null ? [] : rbs)
- name: matchedClusterRoleRefsFromCRB
expression: |
cel.bind(crbs, globalContext.Get('scc-clusterrolebindings','items'),
crbs == null ? [] : crbs)
.filter(b, b.?roleRef.?kind.orValue('') == 'ClusterRole'
&& b.?subjects.orValue([]).exists(s,
variables.subjectMatches.exists(m,
s.kind == m.kind && s.name == m.name &&
(s.kind != 'ServiceAccount' ||
s.?namespace.orValue('') == m.?namespace.orValue('')))))
.map(b, b.roleRef.name)
- name: matchedClusterRoleRefsFromRB
expression: |
variables.rolebindings
.filter(b, b.?metadata.?namespace.orValue('') == object.metadata.namespace
&& b.?roleRef.?kind.orValue('') == 'ClusterRole'
&& b.?subjects.orValue([]).exists(s,
variables.subjectMatches.exists(m,
s.kind == m.kind && s.name == m.name &&
(s.kind != 'ServiceAccount' ||
s.?namespace.orValue('') == m.?namespace.orValue('')))))
.map(b, b.roleRef.name)
- name: matchedRoleRefsFromRB
expression: |
variables.rolebindings
.filter(b, b.?metadata.?namespace.orValue('') == object.metadata.namespace
&& b.?roleRef.?kind.orValue('') == 'Role'
&& b.?subjects.orValue([]).exists(s,
variables.subjectMatches.exists(m,
s.kind == m.kind && s.name == m.name &&
(s.kind != 'ServiceAccount' ||
s.?namespace.orValue('') == m.?namespace.orValue('')))))
.map(b, b.roleRef.name)
- name: matchedClusterRoleRefs
expression: |
variables.matchedClusterRoleRefsFromCRB + variables.matchedClusterRoleRefsFromRB
- name: allSccNames
expression: |
variables.profiles.map(p, p.metadata.name)
- name: assignedFromClusterRoles
expression: |
cel.bind(crs, globalContext.Get('scc-clusterroles','items'),
crs == null ? [] : crs)
.filter(r, variables.matchedClusterRoleRefs.exists(n, n == r.metadata.name))
.map(r, r.?rules.orValue([])
.filter(ru,
ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
.map(ru,
ru.?resourceNames.orValue([]).size() == 0
? variables.allSccNames
: ru.resourceNames)
)
.flatten()
.flatten()
- name: assignedFromRoles
expression: |
cel.bind(roles, globalContext.Get('scc-roles','items'),
roles == null ? [] : roles)
.filter(r,
r.?metadata.?namespace.orValue('') == object.metadata.namespace
&& variables.matchedRoleRefsFromRB.exists(n, n == r.metadata.name))
.map(r, r.?rules.orValue([])
.filter(ru,
ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
.map(ru,
ru.?resourceNames.orValue([]).size() == 0
? variables.allSccNames
: ru.resourceNames)
)
.flatten()
.flatten()
- name: assigned
expression: |
(variables.assignedFromClusterRoles + variables.assignedFromRoles)
.filter(n, variables.allSccNames.exists(s, s == n))
- name: safeSysctls
expression: |
['kernel.shm_rmid_forced',
'net.ipv4.ip_local_port_range',
'net.ipv4.ip_unprivileged_port_start',
'net.ipv4.tcp_syncookies',
'net.ipv4.ping_group_range']
- name: vtypes
expression: |
['hostPath','emptyDir','gcePersistentDisk','awsElasticBlockStore','gitRepo',
'secret','nfs','iscsi','glusterfs','persistentVolumeClaim','rbd','flexVolume',
'cinder','cephfs','flocker','downwardAPI','fc','azureFile','configMap',
'vsphereVolume','quobyte','azureDisk','photonPersistentDisk','projected',
'portworxVolume','scaleIO','storageos','csi','ephemeral','image']
- name: ordered
expression: |
variables.assigned.sortBy(n,
int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?priority.orValue(0)) * -100000 +
-int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?restrictiveScore.orValue(100))
)
- name: requiredExists
expression: variables.required == '' || variables.profiles.exists(pr, pr.metadata.name == variables.required)
- name: requiredBound
expression: variables.required == '' || variables.assigned.exists(n, n == variables.required)
- name: candidateNames
expression: |
variables.required != ''
? ((variables.requiredExists && variables.requiredBound) ? [variables.required] : [])
: variables.ordered
- name: isEphemeralSubresource
expression: request.operation == 'UPDATE'
- name: annotatedSelectedName
expression: object.metadata.?annotations[?'alauda.io/scc'].orValue('')
- name: matchedNames
expression: |
variables.candidateNames.filter(n,
cel.bind(p, variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.orValue({}),
cel.bind(defaultPE, p.?defaultAllowPrivilegeEscalation.orValue(
p.?allowPrivilegeEscalation.orValue(true)),
cel.bind(podRunAsUserForFill,
object.spec.?securityContext.?runAsUser.orValue(
(p.?runAsUser.?type.orValue('') == 'MustRunAs' && p.?runAsUser.?uid.hasValue())
? p.?runAsUser.?uid.orValue(1)
: 1),
cel.bind(seccompFirstForFill,
p.?seccompProfiles.orValue([]).filter(s, s != '' && s != '*')[?0].orValue(''),
cel.bind(fillSeccompType,
seccompFirstForFill == 'runtime/default' ? 'RuntimeDefault' :
seccompFirstForFill.startsWith('localhost/') ? 'Localhost' : '',
cel.bind(fillSeccompLocalhost,
fillSeccompType == 'Localhost'
? seccompFirstForFill.substring('localhost/'.size()) : '',
cel.bind(needPodSeccompFillForMatch,
!object.spec.?securityContext.?seccompProfile.hasValue() &&
object.spec.containers.all(c, !c.?securityContext.?seccompProfile.hasValue()) &&
object.spec.?initContainers.orValue([]).all(c, !c.?securityContext.?seccompProfile.hasValue()),
(p.?allowPrivilegedContainer.orValue(false)
|| !variables.containers.exists(c, c.?securityContext.?privileged.orValue(false)))
&& (p.?allowPrivilegeEscalation.orValue(true)
|| !variables.containers.exists(c, c.?securityContext.?allowPrivilegeEscalation.orValue(defaultPE)))
&& (p.?allowHostNetwork.orValue(false) || !object.spec.?hostNetwork.orValue(false))
&& (p.?allowHostPID.orValue(false) || !object.spec.?hostPID.orValue(false))
&& (p.?allowHostIPC.orValue(false) || !object.spec.?hostIPC.orValue(false))
&& (p.?allowHostDirVolumePlugin.orValue(false)
|| !object.spec.?volumes.orValue([]).exists(v, has(v.hostPath)))
&& (
p.?runAsUser.?type.orValue('RunAsAny') == 'RunAsAny'
|| (
(p.?runAsUser.?type.orValue('RunAsAny') in ['MustRunAsNonRoot','MustRunAsNonRootOrSystem'])
&& !variables.containers.exists(c, c.?securityContext.?runAsUser.orValue(
podRunAsUserForFill) == 0)
)
|| (
p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAs'
&& variables.containers.all(c, c.?securityContext.?runAsUser.orValue(
podRunAsUserForFill) == p.?runAsUser.?uid.orValue(-1))
)
|| (
p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAsRange'
&& variables.containers.all(c,
c.?securityContext.?runAsUser.orValue(podRunAsUserForFill)
>= p.?runAsUser.?uidRangeMin.orValue(1)
&& c.?securityContext.?runAsUser.orValue(podRunAsUserForFill)
<= p.?runAsUser.?uidRangeMax.orValue(2147483647))
)
)
&& (p.?allowedCapabilities.orValue([]).exists(t, t == '*')
|| variables.containers.all(c,
c.?securityContext.?capabilities.?add.orValue([]).all(cap,
p.?allowedCapabilities.orValue([]).exists(a, a == cap))))
&& (p.?requiredDropCapabilities.orValue([]).size() == 0
|| variables.containers.all(c,
p.?requiredDropCapabilities.orValue([]).all(req,
c.?securityContext.?capabilities.?drop.orValue([]).exists(d, d == req || d == 'ALL'))))
&& (p.?volumes.orValue(['*']).exists(t, t == '*')
|| object.spec.?volumes.orValue([]).all(v,
variables.vtypes.filter(t, v[?t].hasValue()).all(t,
p.?volumes.orValue([]).exists(a, a == t))))
&& (p.?allowHostPorts.orValue(false)
|| variables.containers.all(c,
c.?ports.orValue([]).all(port, port.?hostPort.orValue(0) == 0)))
&& (p.?allowedUnsafeSysctls.orValue([]).exists(t, t == '*')
|| object.spec.?securityContext.?sysctls.orValue([]).all(s,
variables.safeSysctls.exists(safe, safe == s.name)
|| p.?allowedUnsafeSysctls.orValue([]).exists(a, a == s.name)))
&& (!p.?readOnlyRootFilesystem.orValue(false)
|| variables.containers.all(c, c.?securityContext.?readOnlyRootFilesystem.orValue(false) == true))
&& (p.?seccompProfiles.orValue([]).size() == 0
|| p.?seccompProfiles.orValue([]).exists(t, t == '*')
|| variables.containers.all(c,
p.?seccompProfiles.orValue([]).exists(a,
(c.?securityContext.?seccompProfile.?type.orValue(
object.spec.?securityContext.?seccompProfile.?type.orValue(
(needPodSeccompFillForMatch && fillSeccompType != '') ? fillSeccompType : '')) == 'RuntimeDefault'
&& a == 'runtime/default')
|| (c.?securityContext.?seccompProfile.?type.orValue(
object.spec.?securityContext.?seccompProfile.?type.orValue(
(needPodSeccompFillForMatch && fillSeccompType != '') ? fillSeccompType : '')) == 'Unconfined'
&& a == 'unconfined')
|| (c.?securityContext.?seccompProfile.?type.orValue(
object.spec.?securityContext.?seccompProfile.?type.orValue(
(needPodSeccompFillForMatch && fillSeccompType != '') ? fillSeccompType : '')) == 'Localhost'
&& a == 'localhost/' + c.?securityContext.?seccompProfile.?localhostProfile.orValue(
object.spec.?securityContext.?seccompProfile.?localhostProfile.orValue(
(needPodSeccompFillForMatch && fillSeccompType == 'Localhost')
? fillSeccompLocalhost : ''))))))
&& (p.?allowedFlexVolumes.orValue([]).size() == 0
|| object.spec.?volumes.orValue([]).filter(v, v.?flexVolume.hasValue()).all(v,
p.?allowedFlexVolumes.orValue([]).exists(d, d.?driver.orValue('') == v.flexVolume.driver)))
)
)
)
)
)
)
)
)
- name: selectedName
expression: |
variables.isEphemeralSubresource
&& variables.annotatedSelectedName != ''
&& variables.candidateNames.exists(n, n == variables.annotatedSelectedName)
? variables.annotatedSelectedName
: variables.matchedNames[?0].orValue('')
- name: selectedSpec
expression: |
variables.profiles.filter(pr, pr.metadata.name == variables.selectedName)[?0].orValue({}).?spec.orValue({})
- name: defaultPE
expression: |
variables.selectedSpec.?defaultAllowPrivilegeEscalation.orValue(
variables.selectedSpec.?allowPrivilegeEscalation.orValue(true))
- name: seccompFirst
expression: |
variables.selectedSpec.?seccompProfiles.orValue([])
.filter(s, s != '' && s != '*')[?0].orValue('')
- name: defaultSeccompType
expression: |
variables.seccompFirst == 'runtime/default' ? 'RuntimeDefault' :
variables.seccompFirst.startsWith('localhost/') ? 'Localhost' : ''
- name: defaultSeccompLocalhostProfile
expression: |
variables.defaultSeccompType == 'Localhost'
? variables.seccompFirst.substring('localhost/'.size()) : ''
- name: needPodSeccomp
expression: |
variables.selectedName != '' && variables.defaultSeccompType != '' &&
!object.spec.?securityContext.?seccompProfile.hasValue() &&
object.spec.containers.all(c, !c.?securityContext.?seccompProfile.hasValue()) &&
object.spec.?initContainers.orValue([]).all(c, !c.?securityContext.?seccompProfile.hasValue())
- name: hasLiteralUid
expression: |
variables.selectedName != '' &&
variables.selectedSpec.?runAsUser.?type.orValue('') == 'MustRunAs' &&
variables.selectedSpec.?runAsUser.?uid.hasValue()
- name: literalUid
expression: |
variables.hasLiteralUid ? variables.selectedSpec.?runAsUser.?uid.orValue(-1) : -1
- name: needPodRunAsUser
expression: |
variables.hasLiteralUid &&
!object.spec.?securityContext.?runAsUser.hasValue()
mutations:
- patchType: ApplyConfiguration
applyConfiguration:
expression: |
(variables.isEphemeralSubresource || variables.selectedName == '') ? Object{} :
Object{
metadata: Object.metadata{
annotations: {
"alauda.io/scc": string(variables.selectedName)
}
}
}
- patchType: ApplyConfiguration
applyConfiguration:
expression: |
(variables.isEphemeralSubresource || !variables.needPodRunAsUser) ? Object{} :
Object{
spec: Object.spec{
securityContext: Object.spec.securityContext{
runAsUser: variables.literalUid
}
}
}
- patchType: ApplyConfiguration
applyConfiguration:
expression: |
(variables.isEphemeralSubresource || !variables.needPodSeccomp) ? Object{} :
(variables.defaultSeccompType == 'Localhost') ?
Object{
spec: Object.spec{
securityContext: Object.spec.securityContext{
seccompProfile: Object.spec.securityContext.seccompProfile{
type: 'Localhost',
localhostProfile: variables.defaultSeccompLocalhostProfile
}
}
}
} :
Object{
spec: Object.spec{
securityContext: Object.spec.securityContext{
seccompProfile: Object.spec.securityContext.seccompProfile{
type: variables.defaultSeccompType
}
}
}
}
- patchType: ApplyConfiguration
applyConfiguration:
expression: |
(variables.isEphemeralSubresource || variables.selectedName == '') ? Object{} :
Object{
spec: Object.spec{
containers: object.spec.containers.map(c, Object.spec.containers{
name: c.name,
securityContext: Object.spec.containers.securityContext{
allowPrivilegeEscalation:
c.?securityContext.?allowPrivilegeEscalation.hasValue()
? c.securityContext.allowPrivilegeEscalation
: variables.defaultPE
}
})
}
}
- patchType: ApplyConfiguration
applyConfiguration:
expression: |
(variables.isEphemeralSubresource || variables.selectedName == '' || !object.spec.?initContainers.hasValue()) ? Object{} :
Object{
spec: Object.spec{
initContainers: object.spec.initContainers.map(c, Object.spec.initContainers{
name: c.name,
securityContext: Object.spec.initContainers.securityContext{
allowPrivilegeEscalation:
c.?securityContext.?allowPrivilegeEscalation.hasValue()
? c.securityContext.allowPrivilegeEscalation
: variables.defaultPE
}
})
}
}
- patchType: ApplyConfiguration
applyConfiguration:
expression: |
(!variables.isEphemeralSubresource || variables.selectedName == '' || !object.spec.?ephemeralContainers.hasValue()) ? Object{} :
Object{
spec: Object.spec{
ephemeralContainers: object.spec.ephemeralContainers.map(c, Object.spec.ephemeralContainers{
name: c.name,
securityContext: Object.spec.ephemeralContainers.securityContext{
allowPrivilegeEscalation:
c.?securityContext.?allowPrivilegeEscalation.hasValue()
? c.securityContext.allowPrivilegeEscalation
: variables.defaultPE
}
})
}
}
请在两个 policy 上都保留 reports.kyverno.io/disabled: "true"。SCC 选择依赖于 admission 请求中的 User、Groups 和 ServiceAccount,而后台 report 扫描不具备等价的请求身份。mutating policy 也只会在 admission 期间填充默认值。将这些仅适用于 admission 的 policy 排除在后台报告之外,可以避免生成误导性的 PolicyReport,并避免对每个现有 Pod 进行不必要的重新编译。
这两个 policy 默认跳过以下 namespace:以 kube-、cpaas- 或 alauda- 开头的 namespace,以及 kyverno、cattle-system、operators 和 default。如果你的平台使用不同的系统 namespace,请相应调整两个 policy 中的 skip-system-ns 表达式。
步骤 1.4 — 通过 Warn → Deny 安全发布
该 validating policy 的交付配置为 failurePolicy: Fail 且 validationActions: [Deny],这意味着它会立即拒绝不符合要求的 Pod。在现有集群上,如果在未做好准备的情况下启用此配置,可能会破坏那些尚未绑定任何 SCC 的 ServiceAccount 所运行的工作负载。
请采用三阶段发布:
-
在第一次应用前先 Warn。在现有集群上应用 scc-auto-pick.yaml 之前,先将 validationActions 改为:
validationActions:
- Warn
然后应用该文件。此时该 policy 会在每个本来会被拒绝的 admission 响应中附加警告,但仍然允许 Pod 被准入。观察 Kyverno admission controller 日志以收集受影响的工作负载:
kubectl logs -n kyverno -l app.kubernetes.io/component=admission-controller \
--tail=500 | grep -i 'scc-auto-pick'
-
修复。对于每一个收到警告的工作负载,添加或修正 RBAC 绑定,使其 ServiceAccount 能够 use 一个合适的 SCC(参见第 2 部分)。使用以下命令确认:
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/<scc-name> \
--as="system:serviceaccount:<namespace>:<sa-name>" -n <namespace>
-
Deny。当合法工作负载不再产生警告后,把配置切回 Deny 并重新应用:
validationActions:
- Deny
Tip
如果你需要临时豁免整个 namespace,可以把它加入两个 policy 的 skip-system-ns 表达式中,或者创建一个 PolicyException 资源。关于 PolicyException 模式,请参见下面的 了解更多。
步骤 1.5 — 验证引擎已就绪
运行以下检查。所有资源都应存在,并且两个 policy 都应为 READY=true。
# 1. CRD is established and 13 profiles are loaded
kubectl get crd securitycontextconstraints.security.alauda.io
kubectl get scc
# 2. Five GCE caches exist
kubectl get globalcontextentry scc-profiles scc-clusterroles \
scc-clusterrolebindings scc-rolebindings scc-roles
# 3. Two admission policies are ready
kubectl get validatingpolicy scc-auto-pick
kubectl get mutatingpolicy scc-fill-defaults
# 4. Reader RBAC is in place
kubectl get clusterrole kyverno-scc-reader
kubectl get clusterrolebinding kyverno-scc-reader
如果 scc-fill-defaults 显示 READY=false,最常见原因是缺少对 pods/ephemeralcontainers 的读取权限——请确认第 1.3 步中的 kyverno-scc-reader ClusterRole 已完整应用。
第 2 部分:授权工作负载使用 SCC
安装引擎后,默认情况下不会授予任何 Pod 任何 SCC。直到管理员为某个 ServiceAccount(或 User、Group)创建 RBAC 绑定之前,在非系统 namespace 中以该 subject 运行的 Pod 都会被拒绝,并显示消息 Pod violates all SCCs assigned to its ServiceAccount。
请将每个 SCC 绑定视为一次安全授权决策。只有平台管理员或安全管理员才应被授予 SCC 绑定权限;普通应用用户和 namespace 所有者不应能够为自己授予更高的 Pod 权限。
步骤 2.1 — 选择合适的 SCC profile
将工作负载的安全需求与下表进行匹配。默认情况下,该引擎会先按 priority 再按 restrictiveScore 对已授予的 SCC 排序。选择该工作负载所需的最小权限 profile 集合,并在必须强制使用某个特定 profile 时使用 alauda.io/required-scc。
Warning
始终授予所需的最小权限。绑定到 privileged 的 ServiceAccount 可以运行任何 Pod,包括那些能够逃逸容器边界的 Pod。请仅为基础设施 DaemonSet 保留 privileged,并避免将其授予用户工作负载。
当应用经理请求 SCC 访问权限时,请提供以下信息:
- namespace 和 ServiceAccount,例如
databases/postgres-sa。
- 工作负载名称和控制器类型,例如
StatefulSet/postgres。
- 请求的 SCC 或所需能力,例如
anyuid,因为镜像以 UID 0 运行。
- 为什么更严格的 SCC(例如
restricted-v2)不够用。
- 该工作负载是否必须用
alauda.io/required-scc 固定到某个特定 SCC。
步骤 2.2 — 将 SCC 绑定到 ServiceAccount
最常见的管理员操作是将 SCC 绑定到工作负载 ServiceAccount。假设你有一个在 databases/postgres-sa 下运行的应用,并且镜像以 root(UID 0)运行。你希望这个 ServiceAccount 被允许使用 anyuid,同时仍保留 restricted-v2 给更严格的工作负载使用。在这个 root-UID 示例中,restricted-v2 不匹配(runAsUser.uidRangeMin: 1),因此 admission 会选择 anyuid。更一般地说,当一个 Pod 同时满足两个 profile 时,本指南中的默认 profile 集会优先选择 anyuid,因为除非你调整优先级或固定 alauda.io/required-scc,否则 anyuid 的 priority 高于 restricted-v2。
将以下内容保存为 bind-postgres-sa.yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: scc-use-anyuid-restricted
labels:
rbac.alauda.io/scc-use: "true"
rules:
- apiGroups: ["security.alauda.io"]
resources: ["securitycontextconstraints"]
resourceNames: ["anyuid", "restricted-v2"]
verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: postgres-sa-scc
namespace: databases
labels:
rbac.alauda.io/scc-use: "true"
subjects:
- kind: ServiceAccount
name: postgres-sa
namespace: databases
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: scc-use-anyuid-restricted
应用:
kubectl apply -f bind-postgres-sa.yaml
rbac.alauda.io/scc-use=true 标签是可选的。它不会影响 SCC 选择,但可以让你通过 kubectl get clusterrole,rolebinding -l rbac.alauda.io/scc-use=true -A 列出所有与 SCC 相关的 RBAC 对象。
Note
你也完全可以使用 ClusterRoleBinding 将这个 namespaced ServiceAccount 的 cluster-scoped use 权限授予它。通常,当你希望授权只在一个 namespace 内生效时,namespaced 的 RoleBinding 更清晰。
步骤 2.3 — 将 SCC 绑定到 User
当受信任的人类操作员(以 Kubernetes User 身份认证,例如通过 OIDC 或证书)需要直接启动 Pod 时——比如 SRE 运行 kubectl debug 或 kubectl run——你可以将该 SCC 授予这个 User principal。
保存为 bind-user-sre.yaml,将 [email protected] 替换为你的 User 名称:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: scc-use-hostaccess
labels:
rbac.alauda.io/scc-use: "true"
rules:
- apiGroups: ["security.alauda.io"]
resources: ["securitycontextconstraints"]
resourceNames: ["hostaccess"]
verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: sre-alice-hostaccess
labels:
rbac.alauda.io/scc-use: "true"
subjects:
- kind: User
name: [email protected]
apiGroup: rbac.authorization.k8s.io
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: scc-use-hostaccess
应用:
kubectl apply -f bind-user-sre.yaml
当 [email protected] 直接运行 kubectl run(而不是通过某个控制器的 ServiceAccount)时,他们创建的 Pod 会以其 User 身份被准入,并获得 hostaccess。
步骤 2.4 — 将 SCC 绑定到 Group
Group 绑定适用于由管理员管理的统一策略,例如“每个已认证用户都可以运行 restricted-v2 Pod”。有两个合成 Group 尤其相关:
system:authenticated —— 每个已认证的 principal。
system:serviceaccounts:<namespace> —— 某个特定 namespace 中的每个 ServiceAccount。
保存为 bind-group-authenticated.yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: scc-use-restricted-v2
labels:
rbac.alauda.io/scc-use: "true"
rules:
- apiGroups: ["security.alauda.io"]
resources: ["securitycontextconstraints"]
resourceNames: ["restricted-v2"]
verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: scc-use-restricted-v2-authenticated
labels:
rbac.alauda.io/scc-use: "true"
subjects:
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: scc-use-restricted-v2
Warning
system:authenticated Group 绑定是一个兜底机制,用于接住那些其 ServiceAccount 没有显式 SCC 绑定的工作负载。它可以作为步骤 1.4 中 Warn 阶段发布期间的迁移安全网。一旦每个工作负载都有了显式绑定,就应移除这个兜底。若长期保留,当未来新增某个默认较宽松的 SCC profile 时,它会扩大你的影响范围。
若要将绑定限制为单个 namespace 的 ServiceAccount,请将 subjects 改为:
subjects:
- kind: Group
name: system:serviceaccounts:my-namespace
apiGroup: rbac.authorization.k8s.io
步骤 2.5 — 使用 alauda.io/required-scc 固定到特定 SCC
默认情况下,引擎会选择 subject 被允许使用且 Pod 实际满足的、最严格的 SCC。如果某个工作负载必须始终以某个特定 profile 被准入——例如,一个对审计敏感的 deployment 必须使用 restricted-v3,即使其 ServiceAccount 也被允许 anyuid——请在 Pod 上设置 alauda.io/required-scc 注解:
apiVersion: v1
kind: Pod
metadata:
name: audited-app
namespace: payments
annotations:
alauda.io/required-scc: restricted-v3
spec:
serviceAccountName: payments-sa
securityContext:
runAsNonRoot: true
runAsUser: 1500
seccompProfile:
type: RuntimeDefault
containers:
- name: app
image: registry.example.com/payments/audited-app:1.2.3
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
alauda.io/required-scc 注解只会从该 subject 已经被授权使用的 SCC 中进行选择。它不会授予 SCC 访问权限。要使该注解生效,以下两个条件都必须满足:
- 集群中存在一个名为
restricted-v3 的 SecurityContextConstraints。
payments/payments-sa 已通过一个对该 resource name 授予 use 的 ClusterRole 或 Role 绑定到 restricted-v3.
如果任一条件不满足,Pod 会被拒绝。validating policy 会针对每种情况输出具体消息(参见 故障排查)。
当使用 PodTemplate 风格的控制器(Deployment、StatefulSet、Job)时,应把注解放在 Pod template 的 metadata 中,而不是放在 controller 上:
apiVersion: apps/v1
kind: Deployment
metadata:
name: audited-app
namespace: payments
spec:
selector:
matchLabels:
app: audited-app
template:
metadata:
labels:
app: audited-app
annotations:
alauda.io/required-scc: restricted-v3
spec:
serviceAccountName: payments-sa
# ...
步骤 2.6 — 验证绑定已生效
应用任何绑定后,请运行以下检查。
管理员验证——确认该 subject 可以 use 该 SCC:
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/anyuid \
--as="system:serviceaccount:databases:postgres-sa" -n databases
期望输出为 yes。如果结果是 no,请重新检查 ClusterRole 中的 apiGroups、resources、resourceNames 和 verbs。
应用所有者验证——在管理员确认绑定后,使用已批准的 ServiceAccount 创建或重新部署工作负载,然后检查已准入的 Pod 注解。快速探测示例如下:
kubectl -n databases run probe \
--image=registry.example.com/library/pause:3.10 \
--serviceaccount=postgres-sa \
--overrides='{"spec":{"securityContext":{"runAsUser":999}}}' \
--command -- /pause
kubectl -n databases get pod probe \
-o jsonpath='{.metadata.annotations.alauda\.io/scc}{"\n"}'
输出应该是引擎所选择的 SCC 名称(在此示例中为 anyuid)。如果应用所有者无法创建探测 Pod,管理员可以执行此检查,或者从真实工作负载中的 Pod 上进行检查。
Note
GlobalContextEntry 会基于 list/watch 刷新,并且通常会在几秒内将新的绑定传播到 admission 缓存,在高负载下有时最长可达一分钟。如果你在应用新绑定后立刻看到 Pod 被拒绝,请稍等片刻后重试,不要立即认为绑定配置有误。
结果
在完成第 1 部分以及至少一个第 2 部分绑定后,你应该能够验证以下所有内容:
kubectl get crd securitycontextconstraints.security.alauda.io 显示该 CRD 的状态为 Established=True。
kubectl get scc 列出了你安装的每个 SCC profile。
kubectl get globalcontextentry 返回全部五个 scc-* 条目。
kubectl get validatingpolicy scc-auto-pick 和 kubectl get mutatingpolicy scc-fill-defaults 都显示 READY=true。
- 在非系统 namespace 中,以已绑定 ServiceAccount 创建的 Pod 会收到一个
alauda.io/scc=<name> 注解,其中包含引擎选定的 SCC 名称。
- 在非系统 namespace 中,以未绑定 ServiceAccount 创建的 Pod 会在 admission 时被拒绝,并显示消息
Pod violates all SCCs assigned to its ServiceAccount。
故障排查
使用下表将症状映射到原因和解决步骤。
对于由 Deployment、StatefulSet、Job 和 DaemonSet 等控制器创建的 Pod,实际的工作负载身份通常是 Pod 的 ServiceAccount。对于由受信任的人类操作员直接创建的 Pod,例如 kubectl run 或 kubectl debug,User 和 Group SCC 绑定也可能匹配该 admission 请求。
了解更多
使用 PolicyException 临时绕过 policy
当你需要在短时间内允许某个单独的 ServiceAccount 超出其当前 SCC(例如紧急调试会话),而修改 RBAC 绑定并不合适时,可以使用 PolicyException 资源。这要求 Kyverno 的 admission controller 启动时带有 --enablePolicyException=true。
apiVersion: policies.kyverno.io/v1alpha1
kind: PolicyException
metadata:
name: postgres-debug-bypass
namespace: policy-exceptions
spec:
policyRefs:
- name: scc-auto-pick
kind: ValidatingPolicy
matchConditions:
- name: target-sa
expression: |
object.metadata.namespace == 'databases' &&
object.spec.?serviceAccountName.orValue('') == 'postgres-sa'
- name: must-be-debug-window
expression: |
object.metadata.?labels[?'debug-window'].orValue('') == 'open'
最佳实践:将 PolicyException 资源放在专用 namespace 中(例如 policy-exceptions),并限制写入权限;为每个例外添加 owner 和 expire-at 标签,并按周期进行审计。
引擎如何选择 SCC
当有多个 SCC 被授予某个 subject,且 Pod 满足其中多个 SCC 时,validating policy 会按以下顺序对候选项进行排序:
- 先看更高的
priority。
- 再看更高的
restrictiveScore。
Pod 完全满足的第一个候选项,就是被选中的 SCC。mutating policy 在决定填充哪个 SCC 的默认值时,也使用相同的排序。这与 OpenShift 的“最严格且可接受的 SCC 优先”意图一致,同时允许操作员通过每个 profile 的 priority 覆盖顺序。
OpenShift 命令映射
如果你来自 OpenShift,以下 oc 命令可直接映射为针对 SCC 引擎的 kubectl apply。这些操作会授予 SCC 的 use 权限,并且应仅由被允许更改集群 Pod 安全边界的管理员执行。
最佳实践:在现有 namespace 中授予一个 privileged 例外
当现有 namespace 通常遵循 PSA restricted 标准,但同一 namespace 中有一个工作负载必须运行 privileged 容器时,请使用此操作步骤。namespace 保持不变;相反,平台管理员从该 namespace 中移除 PSA restricted 强制执行,并让 Kyverno SCC 引擎成为执行安全边界的 admission 控制。
隔离模型如下:
- 该 namespace 中的每个 ServiceAccount 都可以使用
restricted-v2 作为默认安全上限。
- 这个例外工作负载使用一个专用 ServiceAccount。
- 只有该 ServiceAccount 可以使用内置的
privileged SCC。
- 该工作负载使用
alauda.io/required-scc 固定为 privileged。
- 其他 ServiceAccount 不会被授予
privileged,如果它们请求 privileged 设置,仍然会被拒绝。
Warning
PSA 和 Kyverno SCC 引擎是彼此独立的 admission 控制。SCC 授权不能覆盖 pod-security.kubernetes.io/enforce: restricted。在移除 PSA enforcement 之前,应先完成 restricted SCC 基线和例外工作负载绑定,以确保 namespace 始终不会存在无防护的过渡窗口。
步骤 1 - 审计现有 SCC 授权
在更改 PSA 之前,请检查普通 ServiceAccount 是否已经通过 User、Group、RoleBinding 或 ClusterRoleBinding 获得了宽松的 SCC。要特别关注 system:authenticated、system:serviceaccounts 和 system:serviceaccounts:payments 的绑定。
例如,默认 ServiceAccount 不应能够使用内置的 privileged SCC:
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/privileged \
--as=system:serviceaccount:payments:default \
-n payments
期望输出是 no。在继续之前,请移除任何非预期的宽泛授权。前面提到的可选标签 rbac.alauda.io/scc-use=true 可以帮助定位与 SCC 相关的 RBAC 对象,但不要仅依赖标签,因为没有标签的 Role 也可能授予 use 权限。
步骤 2 - 将 restricted 基线绑定到所有 namespace ServiceAccount
创建一个 namespaced RoleBinding,将合成的 system:serviceaccounts:payments Group 绑定到步骤 2.4 中的 scc-use-restricted-v2 ClusterRole。这样可以覆盖该 namespace 中现有和未来的所有 ServiceAccount。
保存为 payments-restricted-baseline.yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: all-serviceaccounts-restricted-v2
namespace: payments
labels:
rbac.alauda.io/scc-use: "true"
subjects:
- kind: Group
name: system:serviceaccounts:payments
apiGroup: rbac.authorization.k8s.io
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: scc-use-restricted-v2
应用该绑定,并验证普通 ServiceAccount 只能使用 restricted 基线:
kubectl apply -f payments-restricted-baseline.yaml
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/restricted-v2 \
--as=system:serviceaccount:payments:default \
-n payments
期望输出是 yes。
步骤 3 - 创建一个专用 ServiceAccount
SCC 授权是基于 subject 的;它不会将权限授予某个 Pod 名称。要建模单一工作负载例外,应使用一个不与无关工作负载共享的独占 ServiceAccount。
保存为 payment-agent-serviceaccount.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: payment-agent-privileged
namespace: payments
automountServiceAccountToken: false
应用该 ServiceAccount:
kubectl apply -f payment-agent-serviceaccount.yaml
Warning
内置的 privileged SCC 是有意设计为不受限制的。除了 privileged 容器之外,它还允许 host networking、host namespaces、host ports、host paths、所有 Linux capabilities、所有 volume 类型、所有 seccomp profile 和不安全的 sysctls。只有在工作负载确实需要完全权限时才授予它。如果工作负载只需要更小范围的例外,例如仅 anyuid、hostNetwork 或 hostPath,请改为绑定更窄的内置 profile。
除非工作负载必须调用 Kubernetes API,否则请保持 automountServiceAccountToken: false。启用 token 自动挂载会把 API 凭据注入到一个已经拥有更高运行时权限的容器中。
步骤 4 - 仅将 privileged 授予专用 ServiceAccount
创建一个 ClusterRole,其中的 resourceNames 只包含内置的 privileged SCC,然后通过 workload namespace 中的 RoleBinding 将其绑定到该专用 ServiceAccount。
保存为 payment-agent-scc-binding.yaml:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: scc-use-privileged
labels:
rbac.alauda.io/scc-use: "true"
rules:
- apiGroups: ["security.alauda.io"]
resources: ["securitycontextconstraints"]
resourceNames: ["privileged"]
verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: payment-agent-privileged-scc
namespace: payments
labels:
rbac.alauda.io/scc-use: "true"
subjects:
- kind: ServiceAccount
name: payment-agent-privileged
namespace: payments
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: scc-use-privileged
应用并验证绑定:
kubectl apply -f payment-agent-scc-binding.yaml
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/privileged \
--as=system:serviceaccount:payments:payment-agent-privileged \
-n payments
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/privileged \
--as=system:serviceaccount:payments:default \
-n payments
期望输出中,payment-agent-privileged 为 yes,default 为 no。该专用 ServiceAccount 还会从 namespace Group 绑定中继承 restricted-v2。
步骤 5 - 防止其他工作负载重用该 ServiceAccount
此操作步骤授予的是 ServiceAccount 的 privileged,而不是 Pod 或 controller 名称。Kubernetes RBAC 不提供对 spec.serviceAccountName 的字段级授权。如果一个用户可以在 payments 中创建任意 Pod、Deployment、StatefulSet、DaemonSet、Job 或 CronJob,那么他们就可能在另一个工作负载中引用 payment-agent-privileged,并获得相同的 SCC 访问权限。
在继续之前,请至少执行以下控制之一:
- 只允许专用的 GitOps 或部署身份创建或更新引用
payment-agent-privileged 的工作负载,并阻止普通 namespace 用户修改这些工作负载。
- 添加一个 admission guard:除非请求已获授权,否则拒绝对
payment-agent-privileged 的引用。将该 guard 应用于直接 Pod 以及在该 namespace 中启用的每一种 Pod-template controller。
不要仅依赖 workload label 或注解来授权使用该专用 ServiceAccount;通常能够创建工作负载的用户也可以复制这些值。如果无法限制部署所有权或 admission 时的 ServiceAccount 使用,那么这个模式只提供的是 ServiceAccount 级别的例外,不能将其表示为单个 Pod 的例外。
在强制实施上述某一种控制之前,不要继续进行 PSA 迁移。
步骤 6 - 将强制执行从 PSA 转移到 Kyverno SCC
只移除 PSA restricted enforcement 标签。保留 restricted audit 和 warning 标签,这样 API server 仍会报告那些会违反 restricted 标准的工作负载。
kubectl label namespace payments \
pod-security.kubernetes.io/enforce- \
pod-security.kubernetes.io/enforce-version- \
--overwrite
kubectl label namespace payments \
pod-security.kubernetes.io/audit=restricted \
pod-security.kubernetes.io/audit-version=latest \
pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/warn-version=latest \
--overwrite
移除 namespace 标签不会覆盖通过 API server admission 配置设置的集群级 PSA 默认值。如果集群默认仍然强制 restricted,请显式让该 namespace 退出 enforcement,同时保留审计和告警可见性:
kubectl label namespace payments \
pod-security.kubernetes.io/enforce=privileged \
pod-security.kubernetes.io/enforce-version=latest \
--overwrite
这里 enforce=privileged 表示 PSA 不施加额外限制。它并不授予 Pod privileged 访问权限;Kyverno SCC 绑定仍然是强制执行的授权边界。
步骤 7 - 在工作负载中固定已批准的 SCC
将 alauda.io/required-scc 放到 Pod template metadata 中,并使用专用 ServiceAccount。不要只把注解放在 Deployment metadata 上,因为 controller metadata 不会自动复制到其 Pod。
apiVersion: apps/v1
kind: Deployment
metadata:
name: payment-agent
namespace: payments
spec:
replicas: 1
selector:
matchLabels:
app: payment-agent
template:
metadata:
labels:
app: payment-agent
annotations:
alauda.io/required-scc: privileged
spec:
serviceAccountName: payment-agent-privileged
securityContext:
seccompProfile:
type: Unconfined
containers:
- name: agent
image: registry.example.com/payments/agent:1.0.0
securityContext:
privileged: true
allowPrivilegeEscalation: true
runAsUser: 0
该注解不会授予 SCC 权限。如果 SCC 缺失,或者 ServiceAccount 没有 use 权限,validating policy 会拒绝该 Pod。
步骤 8 - 验证该例外是隔离的
请以真实的应用部署身份运行 admission 探测,而不是使用 cluster-admin。对于直接创建的 Pod,SCC 选择会将请求的 User 和 Groups 纳入考虑,因此拥有单独宽泛 SCC 授权的管理员可能会得到与生产中所使用的工作负载身份不同的结果。
验证该例外工作负载被以预期的 SCC 准入:
kubectl -n payments get pod -l app=payment-agent \
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.annotations.alauda\.io/scc}{"\n"}{end}'
SCC 列应为 privileged。
接下来,确认普通 restricted Pod 仍然可以以 restricted-v2 准入:
cat <<'EOF' | kubectl create --dry-run=server -f - \
-o jsonpath='{.metadata.annotations.alauda\.io/scc}{"\n"}'
apiVersion: v1
kind: Pod
metadata:
name: restricted-probe
namespace: payments
spec:
serviceAccountName: default
securityContext:
runAsUser: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: probe
image: registry.example.com/library/pause:3.10
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
EOF
期望输出为 restricted-v2。
最后,确认普通 ServiceAccount 不能请求 privileged:
cat <<'EOF' | kubectl create --dry-run=server -f -
apiVersion: v1
kind: Pod
metadata:
name: unauthorized-privileged-probe
namespace: payments
annotations:
alauda.io/required-scc: privileged
spec:
serviceAccountName: default
containers:
- name: probe
image: registry.example.com/library/pause:3.10
securityContext:
privileged: true
EOF
当由正常的应用部署身份提交时,请求必须被拒绝,并带有一条说明所需 SCC 未绑定到 ServiceAccount default 的消息。如果该 Pod 被准入,请停止并在允许生产工作负载之前检查宽泛的 SCC RoleBinding、ClusterRoleBinding 以及 User 或 Group 授权。
回滚该例外
要安全地恢复 PSA restricted enforcement:
-
停止或替换该 privileged 工作负载,使其使用符合 restricted 的规范。
-
删除将 privileged 授予专用 ServiceAccount 的 RoleBinding:
kubectl -n payments delete rolebinding payment-agent-privileged-scc
-
确认该 ServiceAccount 不再能使用 privileged:
kubectl auth can-i use \
securitycontextconstraints.security.alauda.io/privileged \
--as=system:serviceaccount:payments:payment-agent-privileged \
-n payments
授权检查必须返回 no。不要删除内置的 privileged SCC。只有在没有其他 RoleBinding 或 ClusterRoleBinding 引用它时,才删除可复用的 scc-use-privileged ClusterRole。
-
对剩余工作负载执行 server-side dry-run 检查。
-
最后恢复 PSA restricted enforcement:
kubectl label namespace payments \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/enforce-version=latest \
--overwrite
后续步骤
- 在审查工作负载需求后,决定每个现有 namespace 和 ServiceAccount 应绑定到哪个 SCC,记录映射,并通过 GitOps 工作流应用这些绑定,以便保持可审计和可复现。
- 计划定期审查
PolicyException 资源——它们的用途是短期窗口,而不是永久例外。
- 如果你在大规模环境中运行,请监控 Kyverno admission controller 的
kyverno_admission_review_duration_seconds 指标,以便在 SCC profile 或 RBAC 绑定数量增长时检测 admission 延迟变化。