为 Pod 安全应用 SecurityContextConstraints

本指南适用于平台管理员和安全管理员。它将向你展示如何在现有 Kyverno 部署之上安装一个 SecurityContextConstraints(SCC)引擎,以及如何将 SCC profile 绑定到 ServiceAccount、User 和 Group,从而在 admission 时自动强制执行 Pod 安全边界。

简介

OpenShift 的 SecurityContextConstraints(SCC)模型允许集群管理员定义一组 Pod 安全 profile,然后授予 subject(ServiceAccount、User、Group)使用特定 profile 的权限。当 Pod 被准入时,平台会挑选该 subject 被允许使用的最合适的 SCC,填充缺失的默认值,并根据该 profile 验证 Pod。工作负载本身不需要声明每一个安全字段——SCC profile 会替它们完成。

原生 Kubernetes 没有内置的等效机制。本指南会安装一个基于 Kyverno 的引擎,在任何已经运行 Kyverno 的标准 Kubernetes 集群上重现 SCC 体验。它使用:

  • 一个 SecurityContextConstraints CRD(security.alauda.io/v1alpha1)来存储 SCC profile。
  • 标准 Kubernetes RBAC(use 动词加上 resourceNames)将 subject 绑定到 profile,因此 operator 工作流与 OpenShift 保持一致(oc adm policy add-scc-to-user 模式可一一对应)。
  • 一对 Kyverno admission policy——一个 mutating,一个 validating——用于选择正确的 SCC、填充默认值,并拒绝任何没有被授予的 SCC 能接受的 Pod。
  • 五个 GlobalContextEntry 资源,用于在内存中缓存 SCC profile 和相关 RBAC 对象,从而使 admission 决策无需额外的 API 调用。

结果是:应用团队仍然可以编写简单直接的 Pod manifest,集群会自动将其约束到该 ServiceAccount 被允许使用的安全 profile,并且从 OpenShift 迁移时无需更改绑定模型。

SCC 授权属于安全控制变更。应用团队不应被授予直接创建或修改 SCC RBAC 绑定的权限,因为这样做会让他们绕过集群安全边界。应用团队应描述工作负载需求,例如 anyuidhostNetworkhostPath;平台或安全管理员审查请求,并将最小权限的 SCC 绑定到相应 subject。

谁负责什么

使用下表判断本指南的哪些部分适合你。

角色你要做什么你不应做什么
平台管理员或安全管理员安装 SCC 引擎、批准 SCC 请求、创建 SCC RBAC 绑定、将 validating policy 从 Warn 切换到 Deny,以及审计例外。不要在没有工作负载级别理由和所有者的情况下授予过于宽泛的 SCC,例如 privilegedhostaccessanyuid
应用经理或应用所有者确定工作负载需要什么,例如 root UID、host networking、host ports、host paths、user namespaces 或固定 UID 范围。向平台或安全管理员提供 namespace、ServiceAccount、workload 名称和原因。经批准后,使用分配的 ServiceAccount 部署工作负载。不要创建 SCC RBAC 绑定,也不要给自己的 ServiceAccount 授予 SCC 权限。除非请求的 SCC 已经被批准并完成绑定,否则不要使用 alauda.io/required-scc

如果你是平台或安全管理员,请按照第 1 部分和第 2 部分操作。如果你是应用经理,请先使用步骤 2.1 准备 SCC 请求,然后仅在管理员批准并完成绑定后,再使用步骤 2.5 和步骤 2.6。不要自行应用步骤 2.2 到步骤 2.4 中的 RBAC manifest。

正常工作流如下:

  1. 应用经理确定工作负载需求和目标 ServiceAccount。
  2. 平台或安全管理员选择最小权限的 SCC,并创建 RBAC 绑定。
  3. 应用经理使用已批准的 ServiceAccount 部署工作负载,并且只有在管理员要求将某个特定 SCC 固定时,才添加 alauda.io/required-scc
  4. 管理员使用 kubectl auth can-i 验证授权,工作负载所有者验证已准入的 Pod 具有预期的 alauda.io/scc 注解。

场景

当满足以下任一情况时,应用本指南:

  • 你正在从 OpenShift 迁移工作负载,并希望保留现有的 oc adm policy add-scc-to-* 绑定模型,以便平台团队和审计工具可以继续按原样工作。
  • 你已经在使用 Kyverno,并且需要一个集中管理的安全边界,而不是要求每个 Pod manifest 都声明完整的 securityContext
  • 你运营的是多租户集群,并希望不同 namespace 中的不同 ServiceAccount 接收不同的安全上限——例如,限制为 restricted-v2 的应用 SA、允许 hostmount-anyuid 的日志收集 SA,以及允许 NET_BIND_SERVICE 的 ingress controller SA。
  • 你希望有一个全局统一的位置来表达和审计“谁可以运行 privileged Pod”,而不是将例外分散到每个 namespace 中。

前提条件

开始之前,请确保以下条件全部满足:

  1. Kubernetes 集群运行版本 1.30 或更高(CEL admission 已稳定)。

  2. Kyverno 已经安装并运行,版本为 v4.3.1 或更高,并且可用 MutatingPolicyValidatingPolicyGlobalContextEntry CRD。你可以通过以下命令验证:

    kubectl get crd validatingpolicies.policies.kyverno.io mutatingpolicies.policies.kyverno.io globalcontextentries.kyverno.io
  3. kyverno namespace 中包含以下 ServiceAccount(默认 Kyverno 安装):

    • kyverno-admission-controller
    • kyverno-background-controller
    • kyverno-reports-controller
  4. 你拥有 cluster-admin(或等效)权限,因为安装引擎需要创建 CRD、ClusterRole、ClusterRoleBinding、GlobalContextEntry 和 admission policy。

  5. 你已经检查了计划允许非 restricted Pod 的每个 namespace 上的 Pod Security Admission(PSA)enforce 标签。PSA 会在 Kyverno 之前 运行;如果某个 namespace 标记为 pod-security.kubernetes.io/enforce: restricted,那么在 Kyverno 介入之前,该 namespace 就会拒绝任何与 anyuidhostnetwork-v2 之类宽松 SCC 匹配的 Pod。请在适当的 namespace 中将标签调整为 baselineprivileged,或者限制你在这些 namespace 中提供的 SCC profile 集合。

Tip

引擎安装是一次性工作,通常由平台管理员执行。第 2 部分同样是管理员工作流:平台或安全管理员会在审查工作负载需求后绑定 SCC profile。应用团队通常只需提供这些需求,然后使用分配的 ServiceAccount。

步骤

工作分为两部分:

  • 第 1 部分 在集群范围内安装 SCC 引擎。每个集群只需执行一次。
  • 第 2 部分 通过将 SCC profile 绑定到 ServiceAccount、User 和 Group 来授权工作负载使用 SCC,并在需要时将特定工作负载固定到特定 SCC。

第 1 部分:安装 SCC 引擎

步骤 1.1 — 安装 SecurityContextConstraints CRD

将以下 manifest 保存为 scc-crd.yaml。它定义了一个 cluster-scoped 的 SecurityContextConstraints 资源(简称 scc),其字段与 OpenShift SCC 语义一致。

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: securitycontextconstraints.security.alauda.io
spec:
  group: security.alauda.io
  names:
    plural: securitycontextconstraints
    singular: securitycontextconstraints
    kind: SecurityContextConstraints
    listKind: SecurityContextConstraintsList
    shortNames:
      - scc
  scope: Cluster
  versions:
    - name: v1alpha1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          description: |
            SecurityContextConstraints governs the ability to make requests that affect
            container security context. This custom CRD mirrors OpenShift SCC semantics
            while keeping fields under spec for Kyverno CEL consumption.
          type: object
          required:
            - spec
          properties:
            apiVersion:
              type: string
            kind:
              type: string
            metadata:
              type: object
            spec:
              type: object
              required:
                - runAsUser
              properties:
                allowHostPorts:
                  description: Determines if the profile allows host ports in containers.
                  type: boolean
                priority:
                  description: Higher priority SCC is evaluated first.
                  type: integer
                  format: int32
                  nullable: true
                restrictiveScore:
                  description: Secondary sort key. Lower score means less restrictive.
                  type: integer
                  format: int32
                  minimum: 0
                requiredDropCapabilities:
                  description: Capabilities that must be dropped.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
                allowPrivilegedContainer:
                  description: Determines if privileged containers are allowed.
                  type: boolean
                runAsUser:
                  description: Strategy controlling runAsUser.
                  type: object
                  nullable: true
                  properties:
                    type:
                      description: Strategy type for runAsUser.
                      type: string
                      enum:
                        - RunAsAny
                        - MustRunAs
                        - MustRunAsRange
                        - MustRunAsNonRoot
                        - MustRunAsNonRootOrSystem
                    uid:
                      description: Required when type=MustRunAs.
                      type: integer
                      format: int64
                      minimum: 0
                    uidRangeMin:
                      description: Minimum uid for MustRunAsRange.
                      type: integer
                      format: int64
                      minimum: 0
                    uidRangeMax:
                      description: Maximum uid for MustRunAsRange.
                      type: integer
                      format: int64
                      minimum: 0
                users:
                  description: Users who can use this SCC.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
                groups:
                  description: Groups who can use this SCC.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
                allowHostDirVolumePlugin:
                  description: Determines if hostPath-like volume plugin usage is allowed.
                  type: boolean
                seccompProfiles:
                  description: Allowed seccomp profiles. '*' allows all.
                  type: array
                  nullable: true
                  items:
                    type: string
                    pattern: "^(\\*|runtime/default|unconfined|localhost/.+)$"
                  x-kubernetes-list-type: atomic
                allowHostIPC:
                  description: Determines if host IPC is allowed.
                  type: boolean
                forbiddenSysctls:
                  description: Explicitly forbidden sysctls.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
                seLinuxContext:
                  description: Strategy controlling SELinux labels.
                  type: object
                  nullable: true
                  properties:
                    type:
                      description: Strategy type for SELinux context.
                      type: string
                    seLinuxOptions:
                      description: Fixed SELinux options required by MustRunAs.
                      type: object
                      properties:
                        user:
                          type: string
                        role:
                          type: string
                        type:
                          type: string
                        level:
                          type: string
                readOnlyRootFilesystem:
                  description: Forces readOnlyRootFilesystem when set to true.
                  type: boolean
                fsGroup:
                  description: Strategy controlling fsGroup.
                  type: object
                  nullable: true
                  properties:
                    type:
                      type: string
                    ranges:
                      type: array
                      items:
                        type: object
                        properties:
                          min:
                            type: integer
                            format: int64
                          max:
                            type: integer
                            format: int64
                      x-kubernetes-list-type: atomic
                supplementalGroups:
                  description: Strategy controlling supplemental groups.
                  type: object
                  nullable: true
                  properties:
                    type:
                      type: string
                    ranges:
                      type: array
                      items:
                        type: object
                        properties:
                          min:
                            type: integer
                            format: int64
                          max:
                            type: integer
                            format: int64
                      x-kubernetes-list-type: atomic
                userNamespaceLevel:
                  description: Controls host user namespace usage.
                  type: string
                  default: AllowHostLevel
                  enum:
                    - AllowHostLevel
                    - RequirePodLevel
                defaultAddCapabilities:
                  description: Capabilities added by default unless explicitly dropped.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
                allowedUnsafeSysctls:
                  description: Explicitly allowed unsafe sysctls.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
                allowedFlexVolumes:
                  description: Allowed flex volume drivers.
                  type: array
                  nullable: true
                  items:
                    type: object
                    required:
                      - driver
                    properties:
                      driver:
                        type: string
                  x-kubernetes-list-type: atomic
                volumes:
                  description: Allowed volume plugin types. '*' allows all.
                  type: array
                  nullable: true
                  items:
                    type: string
                    enum:
                      - '*'
                      - none
                      - hostPath
                      - emptyDir
                      - gcePersistentDisk
                      - awsElasticBlockStore
                      - gitRepo
                      - secret
                      - nfs
                      - iscsi
                      - glusterfs
                      - persistentVolumeClaim
                      - rbd
                      - flexVolume
                      - cinder
                      - cephfs
                      - flocker
                      - downwardAPI
                      - fc
                      - azureFile
                      - configMap
                      - vsphereVolume
                      - quobyte
                      - azureDisk
                      - photonPersistentDisk
                      - projected
                      - portworxVolume
                      - scaleIO
                      - storageos
                      - csi
                      - ephemeral
                      - image
                  x-kubernetes-list-type: atomic
                allowHostPID:
                  description: Determines if host PID is allowed.
                  type: boolean
                allowHostNetwork:
                  description: Determines if hostNetwork is allowed.
                  type: boolean
                allowPrivilegeEscalation:
                  description: Determines if privilege escalation can be requested.
                  type: boolean
                  nullable: true
                defaultAllowPrivilegeEscalation:
                  description: Default for allowPrivilegeEscalation when container omits it.
                  type: boolean
                  nullable: true
                allowedCapabilities:
                  description: Capabilities that may be added.
                  type: array
                  nullable: true
                  items:
                    type: string
                  x-kubernetes-list-type: atomic
              x-kubernetes-validations:
                - rule: "!has(self.runAsUser) || self.runAsUser.type != 'MustRunAs' || has(self.runAsUser.uid)"
                  message: "runAsUser.uid is required when runAsUser.type is MustRunAs."
                - rule: "!has(self.runAsUser) || self.runAsUser.type == 'MustRunAs' || !has(self.runAsUser.uid)"
                  message: "runAsUser.uid is only allowed when runAsUser.type is MustRunAs."
                - rule: "!has(self.runAsUser) || self.runAsUser.type != 'MustRunAsRange' || (has(self.runAsUser.uidRangeMin) && has(self.runAsUser.uidRangeMax))"
                  message: "uidRangeMin and uidRangeMax are required when runAsUser.type is MustRunAsRange."
                - rule: "!has(self.runAsUser) || self.runAsUser.type == 'MustRunAsRange' || (!has(self.runAsUser.uidRangeMin) && !has(self.runAsUser.uidRangeMax))"
                  message: "uidRangeMin and uidRangeMax are only allowed when runAsUser.type is MustRunAsRange."
                - rule: "!has(self.runAsUser) || !has(self.runAsUser.uidRangeMin) || !has(self.runAsUser.uidRangeMax) || self.runAsUser.uidRangeMin <= self.runAsUser.uidRangeMax"
                  message: "uidRangeMin must be less than or equal to uidRangeMax."
      additionalPrinterColumns:
        - name: Priv
          type: string
          description: Determines if privileged containers are allowed
          jsonPath: .spec.allowPrivilegedContainer
        - name: Caps
          type: string
          description: Allowed capabilities
          jsonPath: .spec.allowedCapabilities
        - name: SELinux
          type: string
          description: SELinux strategy
          jsonPath: .spec.seLinuxContext.type
        - name: RunAsUser
          type: string
          description: RunAsUser strategy
          jsonPath: .spec.runAsUser.type
        - name: FSGroup
          type: string
          description: FSGroup strategy
          jsonPath: .spec.fsGroup.type
        - name: SupGroup
          type: string
          description: SupplementalGroups strategy
          jsonPath: .spec.supplementalGroups.type
        - name: Priority
          type: string
          description: SCC sort priority
          jsonPath: .spec.priority
        - name: Score
          type: string
          description: Secondary restrictive score
          jsonPath: .spec.restrictiveScore
        - name: ReadOnlyRootFS
          type: string
          description: Force read-only root filesystem
          jsonPath: .spec.readOnlyRootFilesystem
        - name: Volumes
          type: string
          description: Allowed volume plugins
          jsonPath: .spec.volumes
  conversion:
    strategy: None

应用它,并等待 CRD 变为 Established 后再继续:

kubectl apply -f scc-crd.yaml
kubectl wait --for=condition=Established --timeout=120s \
  crd/securitycontextconstraints.security.alauda.io

步骤 1.2 — 安装 13 个内置 SCC profile

将以下 manifest 保存为 scc-profiles.yaml。它定义了 13 个基于 OpenShift 内置集合建模的 SCC profile,按从最严格(restrictiveScore: 100)到最宽松(restrictiveScore: 0)排序。当同一个 subject 被授予多个 SCC 时,自动选择策略会优先选择更高的 restrictiveScore

Tip

你不必安装所有 profile。可以根据平台提供的范围裁剪这个 manifest——但你必须至少保留一个 profile 可供每个 subject 使用,否则它们的 Pod 会在 admission 时被拒绝。

apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: restricted-v2
spec:
  priority: 0
  restrictiveScore: 100
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: false
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 1
    uidRangeMax: 2147483647
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: MustRunAs
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities:
    - NET_BIND_SERVICE
  requiredDropCapabilities:
    - ALL
  defaultAddCapabilities: []
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  seccompProfiles:
    - runtime/default
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: restricted-v3
spec:
  priority: 0
  restrictiveScore: 100
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: false
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 1000
    uidRangeMax: 65534
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: MustRunAs
    ranges:
      - min: 1000
        max: 65534
  supplementalGroups:
    type: MustRunAs
    ranges:
      - min: 1000
        max: 65534
  userNamespaceLevel: RequirePodLevel
  allowedCapabilities:
    - NET_BIND_SERVICE
  requiredDropCapabilities:
    - ALL
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  seccompProfiles:
    - runtime/default
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: restricted
spec:
  priority: 0
  restrictiveScore: 98
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 1
    uidRangeMax: 2147483647
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: MustRunAs
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities: []
  requiredDropCapabilities:
    - KILL
    - MKNOD
    - SETUID
    - SETGID
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: nonroot-v2
spec:
  priority: 0
  restrictiveScore: 95
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: false
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsNonRoot
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: RunAsAny
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities:
    - NET_BIND_SERVICE
  requiredDropCapabilities:
    - ALL
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  seccompProfiles:
    - runtime/default
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: nonroot
spec:
  priority: 0
  restrictiveScore: 92
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsNonRoot
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: RunAsAny
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities: []
  requiredDropCapabilities:
    - KILL
    - MKNOD
    - SETUID
    - SETGID
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: hostnetwork-v2
spec:
  priority: 0
  restrictiveScore: 70
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: false
  allowHostNetwork: true
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: true
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 1
    uidRangeMax: 2147483647
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: MustRunAs
  supplementalGroups:
    type: MustRunAs
  allowedCapabilities:
    - NET_BIND_SERVICE
  requiredDropCapabilities:
    - ALL
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  seccompProfiles:
    - runtime/default
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: hostnetwork
spec:
  priority: 0
  restrictiveScore: 68
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: true
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: true
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 1
    uidRangeMax: 2147483647
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: MustRunAs
  supplementalGroups:
    type: MustRunAs
  allowedCapabilities: []
  requiredDropCapabilities:
    - KILL
    - MKNOD
    - SETUID
    - SETGID
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: anyuid
spec:
  priority: 10
  restrictiveScore: 60
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: RunAsAny
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: RunAsAny
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities: []
  requiredDropCapabilities:
    - MKNOD
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: nested-container
spec:
  priority: 0
  restrictiveScore: 58
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: false
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 0
    uidRangeMax: 65534
  seLinuxContext:
    type: MustRunAs
    seLinuxOptions:
      type: container_engine_t
  fsGroup:
    type: MustRunAs
    ranges:
      - min: 0
        max: 65534
  supplementalGroups:
    type: MustRunAs
    ranges:
      - min: 0
        max: 65534
  userNamespaceLevel: RequirePodLevel
  allowedCapabilities:
    - SETUID
    - SETGID
  requiredDropCapabilities: []
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - image
    - persistentVolumeClaim
    - projected
    - secret
  seccompProfiles:
    - '*'
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: hostmount-anyuid
spec:
  priority: 0
  restrictiveScore: 55
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: true
  runAsUser:
    type: RunAsAny
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: RunAsAny
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities: []
  requiredDropCapabilities:
    - MKNOD
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - hostPath
    - image
    - nfs
    - persistentVolumeClaim
    - projected
    - secret
  readOnlyRootFilesystem: false
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: hostmount-anyuid-v2
spec:
  priority: 0
  restrictiveScore: 50
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: false
  allowHostPID: false
  allowHostIPC: false
  allowHostPorts: false
  allowHostDirVolumePlugin: true
  runAsUser:
    type: RunAsAny
  seLinuxContext:
    type: RunAsAny
  fsGroup:
    type: RunAsAny
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities: []
  requiredDropCapabilities:
    - MKNOD
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - hostPath
    - image
    - nfs
    - persistentVolumeClaim
    - projected
    - secret
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: hostaccess
spec:
  priority: 0
  restrictiveScore: 40
  allowPrivilegedContainer: false
  allowPrivilegeEscalation: true
  allowHostNetwork: true
  allowHostPID: true
  allowHostIPC: true
  allowHostPorts: true
  allowHostDirVolumePlugin: true
  runAsUser:
    type: MustRunAsRange
    uidRangeMin: 1
    uidRangeMax: 2147483647
  seLinuxContext:
    type: MustRunAs
  fsGroup:
    type: MustRunAs
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities: []
  requiredDropCapabilities:
    - KILL
    - MKNOD
    - SETUID
    - SETGID
  volumes:
    - configMap
    - csi
    - downwardAPI
    - emptyDir
    - ephemeral
    - hostPath
    - image
    - persistentVolumeClaim
    - projected
    - secret
---
apiVersion: security.alauda.io/v1alpha1
kind: SecurityContextConstraints
metadata:
  name: privileged
spec:
  priority: 0
  restrictiveScore: 0
  allowPrivilegedContainer: true
  allowPrivilegeEscalation: true
  allowHostNetwork: true
  allowHostPID: true
  allowHostIPC: true
  allowHostPorts: true
  allowHostDirVolumePlugin: true
  runAsUser:
    type: RunAsAny
  seLinuxContext:
    type: RunAsAny
  fsGroup:
    type: RunAsAny
  supplementalGroups:
    type: RunAsAny
  allowedCapabilities:
    - '*'
  requiredDropCapabilities: []
  volumes:
    - '*'
  seccompProfiles:
    - '*'
  allowedUnsafeSysctls:
    - '*'

应用这些 profile:

kubectl apply -f scc-profiles.yaml
kubectl get scc

你应该看到所有 13 个 profile 都被列出,并且其 PriorityScore 列已有值(同时还会显示其他 SCC 列,例如 Priv, RunAsUserVolumes)。

步骤 1.3 — 安装 GlobalContextEntry、Kyverno reader RBAC 和 admission policy

此步骤一次安装三项内容:

  1. GlobalContextEntry(GCE)——五个内存缓存,Kyverno 在 admission 期间通过它们查找 SCC profile、ClusterRole、ClusterRoleBinding、RoleBinding 和 Role,而无需对每个请求发起 API 调用。
  2. Reader RBAC——一个 ClusterRole,授予 Kyverno 的三个 ServiceAccount 对 SCC CRD、上面四种 RBAC 资源,以及 policy 匹配的 Pod / pods/ephemeralcontainers 资源的只读访问权限。
  3. 两个 admission policy——一个 MutatingPolicy 用于根据所选 SCC 填充默认值,一个 ValidatingPolicy 用于拒绝任何没有被授予的 SCC 能接受的 Pod。
Warning

这两个 policy 包含驱动 SCC 选择和验证的 CEL 逻辑。你不需要阅读或理解 CEL 就能使用该引擎——直接按原样应用 manifest 即可。这些表达式之所以很长,是因为它们逐字段复现了 OpenShift SCC 的 admission 算法。

将以下内容保存为 scc-gce.yaml 并应用:

apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
  name: scc-profiles
spec:
  kubernetesResource:
    group: security.alauda.io
    version: v1alpha1
    resource: securitycontextconstraints
  projections:
    - name: items
      jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
  name: scc-clusterroles
spec:
  kubernetesResource:
    group: rbac.authorization.k8s.io
    version: v1
    resource: clusterroles
  projections:
    - name: items
      jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
  name: scc-clusterrolebindings
spec:
  kubernetesResource:
    group: rbac.authorization.k8s.io
    version: v1
    resource: clusterrolebindings
  projections:
    - name: items
      jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
  name: scc-rolebindings
spec:
  kubernetesResource:
    group: rbac.authorization.k8s.io
    version: v1
    resource: rolebindings
  projections:
    - name: items
      jmesPath: "@"
---
apiVersion: kyverno.io/v2alpha1
kind: GlobalContextEntry
metadata:
  name: scc-roles
spec:
  kubernetesResource:
    group: rbac.authorization.k8s.io
    version: v1
    resource: roles
  projections:
    - name: items
      jmesPath: "@"

将以下内容保存为 scc-reader-rbac.yaml 并应用。需要 podspods/ephemeralcontainers 的读取权限,因为 Kyverno 在 policy readiness 门禁(RBACPermissionsGranted)期间会检查每个匹配资源的读权限;如果缺少这些权限,mutating policy 会一直处于 NotReady

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kyverno-scc-reader
rules:
  - apiGroups:
      - security.alauda.io
    resources:
      - securitycontextconstraints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - rbac.authorization.k8s.io
    resources:
      - clusterroles
      - clusterrolebindings
      - rolebindings
      - roles
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - pods
      - pods/ephemeralcontainers
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kyverno-scc-reader
subjects:
  - kind: ServiceAccount
    name: kyverno-admission-controller
    namespace: kyverno
  - kind: ServiceAccount
    name: kyverno-background-controller
    namespace: kyverno
  - kind: ServiceAccount
    name: kyverno-reports-controller
    namespace: kyverno
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kyverno-scc-reader

将以下内容保存为 scc-auto-pick.yaml。这是会拒绝任何没有被授予的 SCC 能接受的 Pod 的 ValidatingPolicy

Warning

下面的示例配置为 validationActions: [Deny]。在现有集群上,第一次应用前请先将其改为 validationActions: [Warn],待你审查警告并创建所需的 SCC 绑定后,再改回 Deny。关于发布流程,请参见步骤 1.4。

apiVersion: policies.kyverno.io/v1alpha1
kind: ValidatingPolicy
metadata:
  name: scc-auto-pick
  labels:
    reports.kyverno.io/disabled: "true"
  annotations:
    policies.kyverno.io/title: SCC Auto-Pick (CEL, CRD + RBAC)
    pod-policies.kyverno.io/autogen-controllers: "none"
spec:
  autogen:
    podControllers:
      controllers: []
    validatingAdmissionPolicy:
      enabled: false
  evaluation:
    admission:
      enabled: true
    background:
      enabled: false
  failurePolicy: Fail
  validationActions:
    - Deny
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
  matchConditions:
    - name: skip-system-ns
      expression: |
        !(request.namespace.startsWith('kube-') ||
          request.namespace.startsWith('cpaas-') ||
          request.namespace.startsWith('alauda-') ||
          request.namespace == 'kyverno' ||
          request.namespace == 'cattle-system' ||
          request.namespace == 'operators' ||
          request.namespace == 'default')

  variables:
    - name: containers
      expression: |
        object.spec.containers + object.spec.?initContainers.orValue([]) +
        object.spec.?ephemeralContainers.orValue([])

    - name: required
      expression: object.metadata.?annotations[?'alauda.io/required-scc'].orValue('')

    - name: profiles
      expression: |
        cel.bind(items, globalContext.Get('scc-profiles', 'items'),
          items == null ? [] : items)

    - name: subjectMatches
      expression: |
        [
          {'kind':'ServiceAccount',
           'name': string(object.spec.?serviceAccountName.orValue('default')),
           'namespace': string(request.namespace)},
          {'kind':'Group', 'name':'system:serviceaccounts'},
          {'kind':'Group', 'name':'system:serviceaccounts:'+request.namespace},
          {'kind':'Group', 'name':'system:authenticated'},
          {'kind':'User',  'name': request.userInfo.username}
        ]
        + request.userInfo.groups.map(g, {'kind':'Group','name': g})

    - name: rolebindings
      expression: |
        cel.bind(rbs, globalContext.Get('scc-rolebindings','items'),
          rbs == null ? [] : rbs)

    - name: matchedClusterRoleRefsFromCRB
      expression: |
        cel.bind(crbs, globalContext.Get('scc-clusterrolebindings','items'),
          crbs == null ? [] : crbs)
        .filter(b, b.?roleRef.?kind.orValue('') == 'ClusterRole'
                && b.?subjects.orValue([]).exists(s,
            variables.subjectMatches.exists(m,
              s.kind == m.kind && s.name == m.name &&
              (s.kind != 'ServiceAccount' ||
               s.?namespace.orValue('') == m.?namespace.orValue('')))))
        .map(b, b.roleRef.name)

    - name: matchedClusterRoleRefsFromRB
      expression: |
        variables.rolebindings
          .filter(b, b.?metadata.?namespace.orValue('') == request.namespace
                  && b.?roleRef.?kind.orValue('') == 'ClusterRole'
                  && b.?subjects.orValue([]).exists(s,
              variables.subjectMatches.exists(m,
                s.kind == m.kind && s.name == m.name &&
                (s.kind != 'ServiceAccount' ||
                 s.?namespace.orValue('') == m.?namespace.orValue('')))))
          .map(b, b.roleRef.name)

    - name: matchedRoleRefsFromRB
      expression: |
        variables.rolebindings
          .filter(b, b.?metadata.?namespace.orValue('') == request.namespace
                  && b.?roleRef.?kind.orValue('') == 'Role'
                  && b.?subjects.orValue([]).exists(s,
              variables.subjectMatches.exists(m,
                s.kind == m.kind && s.name == m.name &&
                (s.kind != 'ServiceAccount' ||
                 s.?namespace.orValue('') == m.?namespace.orValue('')))))
          .map(b, b.roleRef.name)

    - name: matchedClusterRoleRefs
      expression: |
        variables.matchedClusterRoleRefsFromCRB + variables.matchedClusterRoleRefsFromRB

    - name: allSccNames
      expression: |
        variables.profiles.map(p, p.metadata.name)

    - name: assignedFromClusterRoles
      expression: |
        cel.bind(crs, globalContext.Get('scc-clusterroles','items'),
          crs == null ? [] : crs)
          .filter(r, variables.matchedClusterRoleRefs.exists(n, n == r.metadata.name))
          .map(r, r.?rules.orValue([])
            .filter(ru,
              ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
              ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
              ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
            .map(ru,
              ru.?resourceNames.orValue([]).size() == 0
                ? variables.allSccNames
                : ru.resourceNames)
          )
          .flatten()
          .flatten()

    - name: assignedFromRoles
      expression: |
        cel.bind(roles, globalContext.Get('scc-roles','items'),
          roles == null ? [] : roles)
          .filter(r,
            r.?metadata.?namespace.orValue('') == request.namespace
            && variables.matchedRoleRefsFromRB.exists(n, n == r.metadata.name))
          .map(r, r.?rules.orValue([])
            .filter(ru,
              ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
              ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
              ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
            .map(ru,
              ru.?resourceNames.orValue([]).size() == 0
                ? variables.allSccNames
                : ru.resourceNames)
          )
          .flatten()
          .flatten()

    - name: assigned
      expression: |
        (variables.assignedFromClusterRoles + variables.assignedFromRoles)
          .filter(n, variables.allSccNames.exists(s, s == n))

    - name: safeSysctls
      expression: |
        ['kernel.shm_rmid_forced',
         'net.ipv4.ip_local_port_range',
         'net.ipv4.ip_unprivileged_port_start',
         'net.ipv4.tcp_syncookies',
         'net.ipv4.ping_group_range']

    - name: vtypes
      expression: |
        ['hostPath','emptyDir','gcePersistentDisk','awsElasticBlockStore','gitRepo',
         'secret','nfs','iscsi','glusterfs','persistentVolumeClaim','rbd','flexVolume',
         'cinder','cephfs','flocker','downwardAPI','fc','azureFile','configMap',
         'vsphereVolume','quobyte','azureDisk','photonPersistentDisk','projected',
         'portworxVolume','scaleIO','storageos','csi','ephemeral','image']

    - name: ordered
      expression: |
        variables.assigned.sortBy(n,
          int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?priority.orValue(0)) * -100000 +
          -int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?restrictiveScore.orValue(100))
        )
    - name: requiredExists
      expression: variables.required == '' || variables.profiles.exists(pr, pr.metadata.name == variables.required)
    - name: requiredBound
      expression: variables.required == '' || variables.assigned.exists(n, n == variables.required)
    - name: candidateNames
      expression: |
        variables.required != ''
          ? [variables.required]
          : variables.ordered

    - name: matched
      expression: |
        variables.candidateNames.exists(n,
          cel.bind(p, variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.orValue({}),
             (p.?allowPrivilegedContainer.orValue(false)
               || !variables.containers.exists(c, c.?securityContext.?privileged.orValue(false)))
          && (p.?allowPrivilegeEscalation.orValue(true)
               || !variables.containers.exists(c, c.?securityContext.?allowPrivilegeEscalation.orValue(true)))
          && (p.?allowHostNetwork.orValue(false) || !object.spec.?hostNetwork.orValue(false))
          && (p.?allowHostPID.orValue(false)     || !object.spec.?hostPID.orValue(false))
          && (p.?allowHostIPC.orValue(false)     || !object.spec.?hostIPC.orValue(false))
          && (p.?allowHostDirVolumePlugin.orValue(false)
               || !object.spec.?volumes.orValue([]).exists(v, has(v.hostPath)))
          && (
               p.?runAsUser.?type.orValue('RunAsAny') == 'RunAsAny'
               || (
                    (p.?runAsUser.?type.orValue('RunAsAny') in ['MustRunAsNonRoot','MustRunAsNonRootOrSystem'])
                    && !variables.containers.exists(c, c.?securityContext.?runAsUser.orValue(
                         object.spec.?securityContext.?runAsUser.orValue(1)) == 0)
                  )
               || (
                    p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAs'
                    && variables.containers.all(c, c.?securityContext.?runAsUser.orValue(
                         object.spec.?securityContext.?runAsUser.orValue(1))
                         == p.?runAsUser.?uid.orValue(-1))
                  )
               || (
                    p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAsRange'
                    && variables.containers.all(c,
                         c.?securityContext.?runAsUser.orValue(
                           object.spec.?securityContext.?runAsUser.orValue(1))
                           >= p.?runAsUser.?uidRangeMin.orValue(1)
                         && c.?securityContext.?runAsUser.orValue(
                              object.spec.?securityContext.?runAsUser.orValue(1))
                           <= p.?runAsUser.?uidRangeMax.orValue(2147483647))
                  )
             )
          && (p.?allowedCapabilities.orValue([]).exists(t, t == '*')
               || variables.containers.all(c,
                    c.?securityContext.?capabilities.?add.orValue([]).all(cap,
                      p.?allowedCapabilities.orValue([]).exists(a, a == cap))))
          && (p.?requiredDropCapabilities.orValue([]).size() == 0
               || variables.containers.all(c,
                    p.?requiredDropCapabilities.orValue([]).all(req,
                      c.?securityContext.?capabilities.?drop.orValue([]).exists(d, d == req || d == 'ALL'))))
          && (p.?volumes.orValue(['*']).exists(t, t == '*')
               || object.spec.?volumes.orValue([]).all(v,
                    variables.vtypes.filter(t, v[?t].hasValue()).all(t,
                      p.?volumes.orValue([]).exists(a, a == t))))
          && (p.?allowHostPorts.orValue(false)
               || variables.containers.all(c,
                    c.?ports.orValue([]).all(port, port.?hostPort.orValue(0) == 0)))
          && (p.?allowedUnsafeSysctls.orValue([]).exists(t, t == '*')
               || object.spec.?securityContext.?sysctls.orValue([]).all(s,
                    variables.safeSysctls.exists(safe, safe == s.name)
                    || p.?allowedUnsafeSysctls.orValue([]).exists(a, a == s.name)))
          && (!p.?readOnlyRootFilesystem.orValue(false)
               || variables.containers.all(c, c.?securityContext.?readOnlyRootFilesystem.orValue(false) == true))
          && (p.?seccompProfiles.orValue([]).size() == 0
               || p.?seccompProfiles.orValue([]).exists(t, t == '*')
               || variables.containers.all(c,
                    p.?seccompProfiles.orValue([]).exists(a,
                      (c.?securityContext.?seccompProfile.?type.orValue(
                         object.spec.?securityContext.?seccompProfile.?type.orValue('')) == 'RuntimeDefault'
                         && a == 'runtime/default')
                      || (c.?securityContext.?seccompProfile.?type.orValue(
                         object.spec.?securityContext.?seccompProfile.?type.orValue('')) == 'Unconfined'
                         && a == 'unconfined')
                      || (c.?securityContext.?seccompProfile.?type.orValue(
                         object.spec.?securityContext.?seccompProfile.?type.orValue('')) == 'Localhost'
                         && a == 'localhost/' + c.?securityContext.?seccompProfile.?localhostProfile.orValue(
                              object.spec.?securityContext.?seccompProfile.?localhostProfile.orValue(''))))))
          && (p.?allowedFlexVolumes.orValue([]).size() == 0
               || object.spec.?volumes.orValue([]).filter(v, v.?flexVolume.hasValue()).all(v,
                    p.?allowedFlexVolumes.orValue([]).exists(d, d.?driver.orValue('') == v.flexVolume.driver)))
          )
        )

  validations:
    - expression: variables.requiredExists
      message: "required-scc does not exist"
      messageExpression: |
        "required SCC '" + variables.required + "' not found in scc-profiles"
    - expression: variables.requiredBound
      message: "required-scc is not bound to ServiceAccount"
      messageExpression: |
        "required SCC '" + variables.required +
        "' is not bound to ServiceAccount '" +
        object.spec.?serviceAccountName.orValue('default') +
        "' in namespace '" + request.namespace + "'"
    - expression: variables.matched
      message: "Pod violates all SCCs assigned to its ServiceAccount"
      messageExpression: |
        variables.required != ''
        ? ("Pod " + object.metadata.name +
           " does not satisfy required SCC '" + variables.required + "'")
        : ("Pod " + object.metadata.name +
           " does not satisfy any SCC profile assigned to ServiceAccount '" +
           object.spec.?serviceAccountName.orValue('default') +
           "' in namespace '" + request.namespace +
           "' (candidates: " + variables.ordered.join(",") + ")")

将以下内容保存为 scc-fill-defaults.yaml 并应用。它是一个 MutatingPolicy,会在 Pod 上记录所选 SCC(alauda.io/scc 注解),并填充从该 SCC 继承而来的 runAsUserseccompProfileallowPrivilegeEscalation 默认值。

apiVersion: policies.kyverno.io/v1alpha1
kind: MutatingPolicy
metadata:
  name: scc-fill-defaults
  labels:
    reports.kyverno.io/disabled: "true"
  annotations:
    policies.kyverno.io/title: SCC default value filler (CRD + RBAC, explicit-wins)
    pod-policies.kyverno.io/autogen-controllers: "none"
spec:
  autogen:
    podControllers:
      controllers: []
  evaluation:
    admission:
      enabled: true
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE"]
        resources: ["pods"]
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["UPDATE"]
        resources: ["pods/ephemeralcontainers"]
  matchConditions:
    - name: skip-system-ns
      expression: |
        !(request.namespace.startsWith('kube-') ||
          request.namespace.startsWith('cpaas-') ||
          request.namespace.startsWith('alauda-') ||
          request.namespace == 'kyverno' ||
          request.namespace == 'cattle-system' ||
          request.namespace == 'operators' ||
          request.namespace == 'default')

  variables:
    - name: containers
      expression: |
        object.spec.containers + object.spec.?initContainers.orValue([]) +
        object.spec.?ephemeralContainers.orValue([])
    - name: required
      expression: object.metadata.?annotations[?'alauda.io/required-scc'].orValue('')

    - name: profiles
      expression: |
        cel.bind(items, globalContext.Get('scc-profiles', 'items'),
          items == null ? [] : items)

    - name: subjectMatches
      expression: |
        [
          {'kind':'ServiceAccount',
           'name': string(object.spec.?serviceAccountName.orValue('default')),
           'namespace': string(object.metadata.namespace)},
          {'kind':'Group', 'name':'system:serviceaccounts'},
          {'kind':'Group', 'name':'system:serviceaccounts:'+object.metadata.namespace},
          {'kind':'Group', 'name':'system:authenticated'},
          {'kind':'User',  'name': request.userInfo.username}
        ]
        + request.userInfo.groups.map(g, {'kind':'Group','name': g})
    - name: rolebindings
      expression: |
        cel.bind(rbs, globalContext.Get('scc-rolebindings','items'),
          rbs == null ? [] : rbs)
    - name: matchedClusterRoleRefsFromCRB
      expression: |
        cel.bind(crbs, globalContext.Get('scc-clusterrolebindings','items'),
          crbs == null ? [] : crbs)
        .filter(b, b.?roleRef.?kind.orValue('') == 'ClusterRole'
                && b.?subjects.orValue([]).exists(s,
            variables.subjectMatches.exists(m,
              s.kind == m.kind && s.name == m.name &&
              (s.kind != 'ServiceAccount' ||
               s.?namespace.orValue('') == m.?namespace.orValue('')))))
        .map(b, b.roleRef.name)
    - name: matchedClusterRoleRefsFromRB
      expression: |
        variables.rolebindings
          .filter(b, b.?metadata.?namespace.orValue('') == object.metadata.namespace
                  && b.?roleRef.?kind.orValue('') == 'ClusterRole'
                  && b.?subjects.orValue([]).exists(s,
              variables.subjectMatches.exists(m,
                s.kind == m.kind && s.name == m.name &&
                (s.kind != 'ServiceAccount' ||
                 s.?namespace.orValue('') == m.?namespace.orValue('')))))
          .map(b, b.roleRef.name)
    - name: matchedRoleRefsFromRB
      expression: |
        variables.rolebindings
          .filter(b, b.?metadata.?namespace.orValue('') == object.metadata.namespace
                  && b.?roleRef.?kind.orValue('') == 'Role'
                  && b.?subjects.orValue([]).exists(s,
              variables.subjectMatches.exists(m,
                s.kind == m.kind && s.name == m.name &&
                (s.kind != 'ServiceAccount' ||
                 s.?namespace.orValue('') == m.?namespace.orValue('')))))
          .map(b, b.roleRef.name)
    - name: matchedClusterRoleRefs
      expression: |
        variables.matchedClusterRoleRefsFromCRB + variables.matchedClusterRoleRefsFromRB
    - name: allSccNames
      expression: |
        variables.profiles.map(p, p.metadata.name)
    - name: assignedFromClusterRoles
      expression: |
        cel.bind(crs, globalContext.Get('scc-clusterroles','items'),
          crs == null ? [] : crs)
          .filter(r, variables.matchedClusterRoleRefs.exists(n, n == r.metadata.name))
          .map(r, r.?rules.orValue([])
            .filter(ru,
              ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
              ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
              ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
            .map(ru,
              ru.?resourceNames.orValue([]).size() == 0
                ? variables.allSccNames
                : ru.resourceNames)
          )
          .flatten()
          .flatten()

    - name: assignedFromRoles
      expression: |
        cel.bind(roles, globalContext.Get('scc-roles','items'),
          roles == null ? [] : roles)
          .filter(r,
            r.?metadata.?namespace.orValue('') == object.metadata.namespace
            && variables.matchedRoleRefsFromRB.exists(n, n == r.metadata.name))
          .map(r, r.?rules.orValue([])
            .filter(ru,
              ru.?apiGroups.orValue([]).exists(g, g == 'security.alauda.io' || g == '*') &&
              ru.?resources.orValue([]).exists(x, x == 'securitycontextconstraints' || x == '*') &&
              ru.?verbs.orValue([]).exists(v, v == 'use' || v == '*'))
            .map(ru,
              ru.?resourceNames.orValue([]).size() == 0
                ? variables.allSccNames
                : ru.resourceNames)
          )
          .flatten()
          .flatten()

    - name: assigned
      expression: |
        (variables.assignedFromClusterRoles + variables.assignedFromRoles)
          .filter(n, variables.allSccNames.exists(s, s == n))

    - name: safeSysctls
      expression: |
        ['kernel.shm_rmid_forced',
         'net.ipv4.ip_local_port_range',
         'net.ipv4.ip_unprivileged_port_start',
         'net.ipv4.tcp_syncookies',
         'net.ipv4.ping_group_range']
    - name: vtypes
      expression: |
        ['hostPath','emptyDir','gcePersistentDisk','awsElasticBlockStore','gitRepo',
         'secret','nfs','iscsi','glusterfs','persistentVolumeClaim','rbd','flexVolume',
         'cinder','cephfs','flocker','downwardAPI','fc','azureFile','configMap',
         'vsphereVolume','quobyte','azureDisk','photonPersistentDisk','projected',
         'portworxVolume','scaleIO','storageos','csi','ephemeral','image']

    - name: ordered
      expression: |
        variables.assigned.sortBy(n,
          int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?priority.orValue(0)) * -100000 +
          -int(variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.?restrictiveScore.orValue(100))
        )
    - name: requiredExists
      expression: variables.required == '' || variables.profiles.exists(pr, pr.metadata.name == variables.required)
    - name: requiredBound
      expression: variables.required == '' || variables.assigned.exists(n, n == variables.required)
    - name: candidateNames
      expression: |
        variables.required != ''
          ? ((variables.requiredExists && variables.requiredBound) ? [variables.required] : [])
          : variables.ordered
    - name: isEphemeralSubresource
      expression: request.operation == 'UPDATE'
    - name: annotatedSelectedName
      expression: object.metadata.?annotations[?'alauda.io/scc'].orValue('')

    - name: matchedNames
      expression: |
        variables.candidateNames.filter(n,
          cel.bind(p, variables.profiles.filter(pr, pr.metadata.name == n)[?0].orValue({}).?spec.orValue({}),
            cel.bind(defaultPE, p.?defaultAllowPrivilegeEscalation.orValue(
                  p.?allowPrivilegeEscalation.orValue(true)),
              cel.bind(podRunAsUserForFill,
                    object.spec.?securityContext.?runAsUser.orValue(
                      (p.?runAsUser.?type.orValue('') == 'MustRunAs' && p.?runAsUser.?uid.hasValue())
                        ? p.?runAsUser.?uid.orValue(1)
                        : 1),
                cel.bind(seccompFirstForFill,
                      p.?seccompProfiles.orValue([]).filter(s, s != '' && s != '*')[?0].orValue(''),
                  cel.bind(fillSeccompType,
                        seccompFirstForFill == 'runtime/default' ? 'RuntimeDefault' :
                        seccompFirstForFill.startsWith('localhost/') ? 'Localhost' : '',
                    cel.bind(fillSeccompLocalhost,
                          fillSeccompType == 'Localhost'
                            ? seccompFirstForFill.substring('localhost/'.size()) : '',
                      cel.bind(needPodSeccompFillForMatch,
                            !object.spec.?securityContext.?seccompProfile.hasValue() &&
                            object.spec.containers.all(c, !c.?securityContext.?seccompProfile.hasValue()) &&
                            object.spec.?initContainers.orValue([]).all(c, !c.?securityContext.?seccompProfile.hasValue()),
                        (p.?allowPrivilegedContainer.orValue(false)
                          || !variables.containers.exists(c, c.?securityContext.?privileged.orValue(false)))
                        && (p.?allowPrivilegeEscalation.orValue(true)
                            || !variables.containers.exists(c, c.?securityContext.?allowPrivilegeEscalation.orValue(defaultPE)))
                        && (p.?allowHostNetwork.orValue(false) || !object.spec.?hostNetwork.orValue(false))
                        && (p.?allowHostPID.orValue(false)     || !object.spec.?hostPID.orValue(false))
                        && (p.?allowHostIPC.orValue(false)     || !object.spec.?hostIPC.orValue(false))
                        && (p.?allowHostDirVolumePlugin.orValue(false)
                            || !object.spec.?volumes.orValue([]).exists(v, has(v.hostPath)))
                        && (
                            p.?runAsUser.?type.orValue('RunAsAny') == 'RunAsAny'
                            || (
                                (p.?runAsUser.?type.orValue('RunAsAny') in ['MustRunAsNonRoot','MustRunAsNonRootOrSystem'])
                                && !variables.containers.exists(c, c.?securityContext.?runAsUser.orValue(
                                      podRunAsUserForFill) == 0)
                              )
                            || (
                                p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAs'
                                && variables.containers.all(c, c.?securityContext.?runAsUser.orValue(
                                      podRunAsUserForFill) == p.?runAsUser.?uid.orValue(-1))
                              )
                            || (
                                p.?runAsUser.?type.orValue('RunAsAny') == 'MustRunAsRange'
                                && variables.containers.all(c,
                                    c.?securityContext.?runAsUser.orValue(podRunAsUserForFill)
                                      >= p.?runAsUser.?uidRangeMin.orValue(1)
                                    && c.?securityContext.?runAsUser.orValue(podRunAsUserForFill)
                                      <= p.?runAsUser.?uidRangeMax.orValue(2147483647))
                              )
                          )
                        && (p.?allowedCapabilities.orValue([]).exists(t, t == '*')
                            || variables.containers.all(c,
                                c.?securityContext.?capabilities.?add.orValue([]).all(cap,
                                  p.?allowedCapabilities.orValue([]).exists(a, a == cap))))
                        && (p.?requiredDropCapabilities.orValue([]).size() == 0
                            || variables.containers.all(c,
                                p.?requiredDropCapabilities.orValue([]).all(req,
                                  c.?securityContext.?capabilities.?drop.orValue([]).exists(d, d == req || d == 'ALL'))))
                        && (p.?volumes.orValue(['*']).exists(t, t == '*')
                            || object.spec.?volumes.orValue([]).all(v,
                                variables.vtypes.filter(t, v[?t].hasValue()).all(t,
                                  p.?volumes.orValue([]).exists(a, a == t))))
                        && (p.?allowHostPorts.orValue(false)
                            || variables.containers.all(c,
                                c.?ports.orValue([]).all(port, port.?hostPort.orValue(0) == 0)))
                        && (p.?allowedUnsafeSysctls.orValue([]).exists(t, t == '*')
                            || object.spec.?securityContext.?sysctls.orValue([]).all(s,
                                variables.safeSysctls.exists(safe, safe == s.name)
                                || p.?allowedUnsafeSysctls.orValue([]).exists(a, a == s.name)))
                        && (!p.?readOnlyRootFilesystem.orValue(false)
                            || variables.containers.all(c, c.?securityContext.?readOnlyRootFilesystem.orValue(false) == true))
                        && (p.?seccompProfiles.orValue([]).size() == 0
                            || p.?seccompProfiles.orValue([]).exists(t, t == '*')
                            || variables.containers.all(c,
                                p.?seccompProfiles.orValue([]).exists(a,
                                  (c.?securityContext.?seccompProfile.?type.orValue(
                                      object.spec.?securityContext.?seccompProfile.?type.orValue(
                                        (needPodSeccompFillForMatch && fillSeccompType != '') ? fillSeccompType : '')) == 'RuntimeDefault'
                                    && a == 'runtime/default')
                                  || (c.?securityContext.?seccompProfile.?type.orValue(
                                      object.spec.?securityContext.?seccompProfile.?type.orValue(
                                        (needPodSeccompFillForMatch && fillSeccompType != '') ? fillSeccompType : '')) == 'Unconfined'
                                    && a == 'unconfined')
                                  || (c.?securityContext.?seccompProfile.?type.orValue(
                                      object.spec.?securityContext.?seccompProfile.?type.orValue(
                                        (needPodSeccompFillForMatch && fillSeccompType != '') ? fillSeccompType : '')) == 'Localhost'
                                    && a == 'localhost/' + c.?securityContext.?seccompProfile.?localhostProfile.orValue(
                                        object.spec.?securityContext.?seccompProfile.?localhostProfile.orValue(
                                          (needPodSeccompFillForMatch && fillSeccompType == 'Localhost')
                                            ? fillSeccompLocalhost : ''))))))
                        && (p.?allowedFlexVolumes.orValue([]).size() == 0
                            || object.spec.?volumes.orValue([]).filter(v, v.?flexVolume.hasValue()).all(v,
                                p.?allowedFlexVolumes.orValue([]).exists(d, d.?driver.orValue('') == v.flexVolume.driver)))
                      )
                    )
                  )
                )
              )
            )
          )
        )

    - name: selectedName
      expression: |
        variables.isEphemeralSubresource
          && variables.annotatedSelectedName != ''
          && variables.candidateNames.exists(n, n == variables.annotatedSelectedName)
          ? variables.annotatedSelectedName
          : variables.matchedNames[?0].orValue('')

    - name: selectedSpec
      expression: |
        variables.profiles.filter(pr, pr.metadata.name == variables.selectedName)[?0].orValue({}).?spec.orValue({})

    - name: defaultPE
      expression: |
        variables.selectedSpec.?defaultAllowPrivilegeEscalation.orValue(
          variables.selectedSpec.?allowPrivilegeEscalation.orValue(true))

    - name: seccompFirst
      expression: |
        variables.selectedSpec.?seccompProfiles.orValue([])
          .filter(s, s != '' && s != '*')[?0].orValue('')
    - name: defaultSeccompType
      expression: |
        variables.seccompFirst == 'runtime/default' ? 'RuntimeDefault' :
        variables.seccompFirst.startsWith('localhost/') ? 'Localhost' : ''
    - name: defaultSeccompLocalhostProfile
      expression: |
        variables.defaultSeccompType == 'Localhost'
          ? variables.seccompFirst.substring('localhost/'.size()) : ''
    - name: needPodSeccomp
      expression: |
        variables.selectedName != '' && variables.defaultSeccompType != '' &&
        !object.spec.?securityContext.?seccompProfile.hasValue() &&
        object.spec.containers.all(c, !c.?securityContext.?seccompProfile.hasValue()) &&
        object.spec.?initContainers.orValue([]).all(c, !c.?securityContext.?seccompProfile.hasValue())

    - name: hasLiteralUid
      expression: |
        variables.selectedName != '' &&
        variables.selectedSpec.?runAsUser.?type.orValue('') == 'MustRunAs' &&
        variables.selectedSpec.?runAsUser.?uid.hasValue()
    - name: literalUid
      expression: |
        variables.hasLiteralUid ? variables.selectedSpec.?runAsUser.?uid.orValue(-1) : -1
    - name: needPodRunAsUser
      expression: |
        variables.hasLiteralUid &&
        !object.spec.?securityContext.?runAsUser.hasValue()

  mutations:
    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: |
          (variables.isEphemeralSubresource || variables.selectedName == '') ? Object{} :
          Object{
            metadata: Object.metadata{
              annotations: {
                "alauda.io/scc": string(variables.selectedName)
              }
            }
          }

    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: |
          (variables.isEphemeralSubresource || !variables.needPodRunAsUser) ? Object{} :
          Object{
            spec: Object.spec{
              securityContext: Object.spec.securityContext{
                runAsUser: variables.literalUid
              }
            }
          }

    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: |
          (variables.isEphemeralSubresource || !variables.needPodSeccomp) ? Object{} :
          (variables.defaultSeccompType == 'Localhost') ?
          Object{
            spec: Object.spec{
              securityContext: Object.spec.securityContext{
                seccompProfile: Object.spec.securityContext.seccompProfile{
                  type: 'Localhost',
                  localhostProfile: variables.defaultSeccompLocalhostProfile
                }
              }
            }
          } :
          Object{
            spec: Object.spec{
              securityContext: Object.spec.securityContext{
                seccompProfile: Object.spec.securityContext.seccompProfile{
                  type: variables.defaultSeccompType
                }
              }
            }
          }

    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: |
          (variables.isEphemeralSubresource || variables.selectedName == '') ? Object{} :
          Object{
            spec: Object.spec{
              containers: object.spec.containers.map(c, Object.spec.containers{
                name: c.name,
                securityContext: Object.spec.containers.securityContext{
                  allowPrivilegeEscalation:
                    c.?securityContext.?allowPrivilegeEscalation.hasValue()
                      ? c.securityContext.allowPrivilegeEscalation
                      : variables.defaultPE
                }
              })
            }
          }

    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: |
          (variables.isEphemeralSubresource || variables.selectedName == '' || !object.spec.?initContainers.hasValue()) ? Object{} :
          Object{
            spec: Object.spec{
              initContainers: object.spec.initContainers.map(c, Object.spec.initContainers{
                name: c.name,
                securityContext: Object.spec.initContainers.securityContext{
                  allowPrivilegeEscalation:
                    c.?securityContext.?allowPrivilegeEscalation.hasValue()
                      ? c.securityContext.allowPrivilegeEscalation
                      : variables.defaultPE
                }
              })
            }
          }

    - patchType: ApplyConfiguration
      applyConfiguration:
        expression: |
          (!variables.isEphemeralSubresource || variables.selectedName == '' || !object.spec.?ephemeralContainers.hasValue()) ? Object{} :
          Object{
            spec: Object.spec{
              ephemeralContainers: object.spec.ephemeralContainers.map(c, Object.spec.ephemeralContainers{
                name: c.name,
                securityContext: Object.spec.ephemeralContainers.securityContext{
                  allowPrivilegeEscalation:
                    c.?securityContext.?allowPrivilegeEscalation.hasValue()
                      ? c.securityContext.allowPrivilegeEscalation
                      : variables.defaultPE
                }
              })
            }
          }

请在两个 policy 上都保留 reports.kyverno.io/disabled: "true"。SCC 选择依赖于 admission 请求中的 User、Groups 和 ServiceAccount,而后台 report 扫描不具备等价的请求身份。mutating policy 也只会在 admission 期间填充默认值。将这些仅适用于 admission 的 policy 排除在后台报告之外,可以避免生成误导性的 PolicyReport,并避免对每个现有 Pod 进行不必要的重新编译。

这两个 policy 默认跳过以下 namespace:以 kube-cpaas-alauda- 开头的 namespace,以及 kyvernocattle-systemoperatorsdefault。如果你的平台使用不同的系统 namespace,请相应调整两个 policy 中的 skip-system-ns 表达式。

步骤 1.4 — 通过 Warn → Deny 安全发布

该 validating policy 的交付配置为 failurePolicy: FailvalidationActions: [Deny],这意味着它会立即拒绝不符合要求的 Pod。在现有集群上,如果在未做好准备的情况下启用此配置,可能会破坏那些尚未绑定任何 SCC 的 ServiceAccount 所运行的工作负载。

请采用三阶段发布:

  1. 在第一次应用前先 Warn。在现有集群上应用 scc-auto-pick.yaml 之前,先将 validationActions 改为:

    validationActions:
      - Warn

    然后应用该文件。此时该 policy 会在每个本来会被拒绝的 admission 响应中附加警告,但仍然允许 Pod 被准入。观察 Kyverno admission controller 日志以收集受影响的工作负载:

    kubectl logs -n kyverno -l app.kubernetes.io/component=admission-controller \
      --tail=500 | grep -i 'scc-auto-pick'
  2. 修复。对于每一个收到警告的工作负载,添加或修正 RBAC 绑定,使其 ServiceAccount 能够 use 一个合适的 SCC(参见第 2 部分)。使用以下命令确认:

    kubectl auth can-i use \
      securitycontextconstraints.security.alauda.io/<scc-name> \
      --as="system:serviceaccount:<namespace>:<sa-name>" -n <namespace>
  3. Deny。当合法工作负载不再产生警告后,把配置切回 Deny 并重新应用:

    validationActions:
      - Deny
Tip

如果你需要临时豁免整个 namespace,可以把它加入两个 policy 的 skip-system-ns 表达式中,或者创建一个 PolicyException 资源。关于 PolicyException 模式,请参见下面的 了解更多

步骤 1.5 — 验证引擎已就绪

运行以下检查。所有资源都应存在,并且两个 policy 都应为 READY=true

# 1. CRD is established and 13 profiles are loaded
kubectl get crd securitycontextconstraints.security.alauda.io
kubectl get scc

# 2. Five GCE caches exist
kubectl get globalcontextentry scc-profiles scc-clusterroles \
  scc-clusterrolebindings scc-rolebindings scc-roles

# 3. Two admission policies are ready
kubectl get validatingpolicy scc-auto-pick
kubectl get mutatingpolicy scc-fill-defaults

# 4. Reader RBAC is in place
kubectl get clusterrole kyverno-scc-reader
kubectl get clusterrolebinding kyverno-scc-reader

如果 scc-fill-defaults 显示 READY=false,最常见原因是缺少对 pods/ephemeralcontainers 的读取权限——请确认第 1.3 步中的 kyverno-scc-reader ClusterRole 已完整应用。

第 2 部分:授权工作负载使用 SCC

安装引擎后,默认情况下不会授予任何 Pod 任何 SCC。直到管理员为某个 ServiceAccount(或 User、Group)创建 RBAC 绑定之前,在非系统 namespace 中以该 subject 运行的 Pod 都会被拒绝,并显示消息 Pod violates all SCCs assigned to its ServiceAccount

请将每个 SCC 绑定视为一次安全授权决策。只有平台管理员或安全管理员才应被授予 SCC 绑定权限;普通应用用户和 namespace 所有者不应能够为自己授予更高的 Pod 权限。

步骤 2.1 — 选择合适的 SCC profile

将工作负载的安全需求与下表进行匹配。默认情况下,该引擎会先按 priority 再按 restrictiveScore 对已授予的 SCC 排序。选择该工作负载所需的最小权限 profile 集合,并在必须强制使用某个特定 profile 时使用 alauda.io/required-scc

工作负载特征推荐 SCC
无状态服务、非 root、删除所有 capabilities、无 host 访问restricted-v2
与上面相同,但需要绑定低于 1024 的端口restricted-v2(该 profile 已允许 NET_BIND_SERVICE
与上面相同,但使用固定 UID 范围(如 1000–65534)和 user namespacesrestricted-v3
以非 root 用户运行,但不能删除所有 capabilities 的服务nonroot-v2(删除 ALL)或 nonroot(旧的删除集合)
需要以 root 运行(USER root)的镜像anyuid
需要 hostNetwork 和 host ports 的 ingress controller 或其他 Podhostnetwork-v2(删除 ALL)或 hostnetwork(旧的删除集合)
为收集日志/指标而挂载 hostPath、以非 root 运行的服务hostmount-anyuid
与上面相同,但不需要 SELinux relabelinghostmount-anyuid-v2
需要 hostNetworkhostPIDhostIPC 和 host paths 的诊断 Podhostaccess
使用 user namespaces 的 container-in-container 构建沙箱nested-container
完全 privileged 的工作负载(CNI、存储驱动、调试 Pod)privileged
Warning

始终授予所需的最小权限。绑定到 privileged 的 ServiceAccount 可以运行任何 Pod,包括那些能够逃逸容器边界的 Pod。请仅为基础设施 DaemonSet 保留 privileged,并避免将其授予用户工作负载。

当应用经理请求 SCC 访问权限时,请提供以下信息:

  • namespace 和 ServiceAccount,例如 databases/postgres-sa
  • 工作负载名称和控制器类型,例如 StatefulSet/postgres
  • 请求的 SCC 或所需能力,例如 anyuid,因为镜像以 UID 0 运行。
  • 为什么更严格的 SCC(例如 restricted-v2)不够用。
  • 该工作负载是否必须用 alauda.io/required-scc 固定到某个特定 SCC。

步骤 2.2 — 将 SCC 绑定到 ServiceAccount

最常见的管理员操作是将 SCC 绑定到工作负载 ServiceAccount。假设你有一个在 databases/postgres-sa 下运行的应用,并且镜像以 root(UID 0)运行。你希望这个 ServiceAccount 被允许使用 anyuid,同时仍保留 restricted-v2 给更严格的工作负载使用。在这个 root-UID 示例中,restricted-v2 不匹配(runAsUser.uidRangeMin: 1),因此 admission 会选择 anyuid。更一般地说,当一个 Pod 同时满足两个 profile 时,本指南中的默认 profile 集会优先选择 anyuid,因为除非你调整优先级或固定 alauda.io/required-scc,否则 anyuidpriority 高于 restricted-v2

将以下内容保存为 bind-postgres-sa.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: scc-use-anyuid-restricted
  labels:
    rbac.alauda.io/scc-use: "true"
rules:
  - apiGroups: ["security.alauda.io"]
    resources: ["securitycontextconstraints"]
    resourceNames: ["anyuid", "restricted-v2"]
    verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: postgres-sa-scc
  namespace: databases
  labels:
    rbac.alauda.io/scc-use: "true"
subjects:
  - kind: ServiceAccount
    name: postgres-sa
    namespace: databases
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-use-anyuid-restricted

应用:

kubectl apply -f bind-postgres-sa.yaml

rbac.alauda.io/scc-use=true 标签是可选的。它不会影响 SCC 选择,但可以让你通过 kubectl get clusterrole,rolebinding -l rbac.alauda.io/scc-use=true -A 列出所有与 SCC 相关的 RBAC 对象。

Note

你也完全可以使用 ClusterRoleBinding 将这个 namespaced ServiceAccount 的 cluster-scoped use 权限授予它。通常,当你希望授权只在一个 namespace 内生效时,namespaced 的 RoleBinding 更清晰。

步骤 2.3 — 将 SCC 绑定到 User

当受信任的人类操作员(以 Kubernetes User 身份认证,例如通过 OIDC 或证书)需要直接启动 Pod 时——比如 SRE 运行 kubectl debugkubectl run——你可以将该 SCC 授予这个 User principal。

保存为 bind-user-sre.yaml,将 [email protected] 替换为你的 User 名称:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: scc-use-hostaccess
  labels:
    rbac.alauda.io/scc-use: "true"
rules:
  - apiGroups: ["security.alauda.io"]
    resources: ["securitycontextconstraints"]
    resourceNames: ["hostaccess"]
    verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: sre-alice-hostaccess
  labels:
    rbac.alauda.io/scc-use: "true"
subjects:
  - kind: User
    name: [email protected]
    apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-use-hostaccess

应用:

kubectl apply -f bind-user-sre.yaml

[email protected] 直接运行 kubectl run(而不是通过某个控制器的 ServiceAccount)时,他们创建的 Pod 会以其 User 身份被准入,并获得 hostaccess

步骤 2.4 — 将 SCC 绑定到 Group

Group 绑定适用于由管理员管理的统一策略,例如“每个已认证用户都可以运行 restricted-v2 Pod”。有两个合成 Group 尤其相关:

  • system:authenticated —— 每个已认证的 principal。
  • system:serviceaccounts:<namespace> —— 某个特定 namespace 中的每个 ServiceAccount。

保存为 bind-group-authenticated.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: scc-use-restricted-v2
  labels:
    rbac.alauda.io/scc-use: "true"
rules:
  - apiGroups: ["security.alauda.io"]
    resources: ["securitycontextconstraints"]
    resourceNames: ["restricted-v2"]
    verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: scc-use-restricted-v2-authenticated
  labels:
    rbac.alauda.io/scc-use: "true"
subjects:
  - kind: Group
    name: system:authenticated
    apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-use-restricted-v2
Warning

system:authenticated Group 绑定是一个兜底机制,用于接住那些其 ServiceAccount 没有显式 SCC 绑定的工作负载。它可以作为步骤 1.4 中 Warn 阶段发布期间的迁移安全网。一旦每个工作负载都有了显式绑定,就应移除这个兜底。若长期保留,当未来新增某个默认较宽松的 SCC profile 时,它会扩大你的影响范围。

若要将绑定限制为单个 namespace 的 ServiceAccount,请将 subjects 改为:

subjects:
  - kind: Group
    name: system:serviceaccounts:my-namespace
    apiGroup: rbac.authorization.k8s.io

步骤 2.5 — 使用 alauda.io/required-scc 固定到特定 SCC

默认情况下,引擎会选择 subject 被允许使用且 Pod 实际满足的、最严格的 SCC。如果某个工作负载必须始终以某个特定 profile 被准入——例如,一个对审计敏感的 deployment 必须使用 restricted-v3,即使其 ServiceAccount 也被允许 anyuid——请在 Pod 上设置 alauda.io/required-scc 注解:

apiVersion: v1
kind: Pod
metadata:
  name: audited-app
  namespace: payments
  annotations:
    alauda.io/required-scc: restricted-v3
spec:
  serviceAccountName: payments-sa
  securityContext:
    runAsNonRoot: true
    runAsUser: 1500
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: registry.example.com/payments/audited-app:1.2.3
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]

alauda.io/required-scc 注解只会从该 subject 已经被授权使用的 SCC 中进行选择。它不会授予 SCC 访问权限。要使该注解生效,以下两个条件都必须满足

  • 集群中存在一个名为 restricted-v3 的 SecurityContextConstraints。
  • payments/payments-sa 已通过一个对该 resource name 授予 use 的 ClusterRole 或 Role 绑定到 restricted-v3.

如果任一条件不满足,Pod 会被拒绝。validating policy 会针对每种情况输出具体消息(参见 故障排查)。

当使用 PodTemplate 风格的控制器(Deployment、StatefulSet、Job)时,应把注解放在 Pod template 的 metadata 中,而不是放在 controller 上:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: audited-app
  namespace: payments
spec:
  selector:
    matchLabels:
      app: audited-app
  template:
    metadata:
      labels:
        app: audited-app
      annotations:
        alauda.io/required-scc: restricted-v3
    spec:
      serviceAccountName: payments-sa
      # ...

步骤 2.6 — 验证绑定已生效

应用任何绑定后,请运行以下检查。

管理员验证——确认该 subject 可以 use 该 SCC:

kubectl auth can-i use \
  securitycontextconstraints.security.alauda.io/anyuid \
  --as="system:serviceaccount:databases:postgres-sa" -n databases

期望输出为 yes。如果结果是 no,请重新检查 ClusterRole 中的 apiGroupsresourcesresourceNamesverbs

应用所有者验证——在管理员确认绑定后,使用已批准的 ServiceAccount 创建或重新部署工作负载,然后检查已准入的 Pod 注解。快速探测示例如下:

kubectl -n databases run probe \
  --image=registry.example.com/library/pause:3.10 \
  --serviceaccount=postgres-sa \
  --overrides='{"spec":{"securityContext":{"runAsUser":999}}}' \
  --command -- /pause

kubectl -n databases get pod probe \
  -o jsonpath='{.metadata.annotations.alauda\.io/scc}{"\n"}'

输出应该是引擎所选择的 SCC 名称(在此示例中为 anyuid)。如果应用所有者无法创建探测 Pod,管理员可以执行此检查,或者从真实工作负载中的 Pod 上进行检查。

Note

GlobalContextEntry 会基于 list/watch 刷新,并且通常会在几秒内将新的绑定传播到 admission 缓存,在高负载下有时最长可达一分钟。如果你在应用新绑定后立刻看到 Pod 被拒绝,请稍等片刻后重试,不要立即认为绑定配置有误。

结果

在完成第 1 部分以及至少一个第 2 部分绑定后,你应该能够验证以下所有内容:

  • kubectl get crd securitycontextconstraints.security.alauda.io 显示该 CRD 的状态为 Established=True
  • kubectl get scc 列出了你安装的每个 SCC profile。
  • kubectl get globalcontextentry 返回全部五个 scc-* 条目。
  • kubectl get validatingpolicy scc-auto-pickkubectl get mutatingpolicy scc-fill-defaults 都显示 READY=true
  • 在非系统 namespace 中,以已绑定 ServiceAccount 创建的 Pod 会收到一个 alauda.io/scc=<name> 注解,其中包含引擎选定的 SCC 名称。
  • 在非系统 namespace 中,以未绑定 ServiceAccount 创建的 Pod 会在 admission 时被拒绝,并显示消息 Pod violates all SCCs assigned to its ServiceAccount

故障排查

使用下表将症状映射到原因和解决步骤。

对于由 Deployment、StatefulSet、Job 和 DaemonSet 等控制器创建的 Pod,实际的工作负载身份通常是 Pod 的 ServiceAccount。对于由受信任的人类操作员直接创建的 Pod,例如 kubectl runkubectl debug,User 和 Group SCC 绑定也可能匹配该 admission 请求。

症状可能原因需要检查什么
Pod violates all SCCs assigned to its ServiceAccount (candidates: ...)Pod 的 ServiceAccount 至少绑定了一个 SCC,但 Pod 的 spec 违反了其中的每一个。消息末尾的候选列表列出了被考虑的 SCC。对每个候选 SCC,比较 Pod 与该 SCC 的字段。常见不匹配包括:容器 runAsUser 超出允许范围;当 requiredDropCapabilities: [ALL] 时缺少 drop: [ALL];当 SCC 要求 runtime/default 时缺少 seccompProfile.type
Pod violates all SCCs assigned to its ServiceAccount (candidates: )(空候选列表)没有 SCC 绑定到该 Pod 的 ServiceAccount。对从 kubectl get scc 得到的每个 SCC 名称,运行 kubectl auth can-i use securitycontextconstraints.security.alauda.io/<name> --as=system:serviceaccount:<ns>:<sa> -n <ns>。所有结果都应为 no。按步骤 2.2 添加绑定。
required SCC '<name>' not found in scc-profilesalauda.io/required-scc 注解引用了一个不存在的 SCC。运行 kubectl get scc <name>。修正该注解,或者安装缺失的 profile。
required SCC '<name>' is not bound to ServiceAccount '<sa>' in namespace '<ns>'该注解引用的 SCC,ServiceAccount 没有 use 权限。添加一个 RoleBinding,将 securitycontextconstraints/<name>use 权限授予该 SA,然后重试。
刚添加了绑定,但 Pod 仍然被拒绝Kyverno 的 GlobalContextEntry 会异步缓存 RBAC 对象;新绑定需要几秒钟才能传播。等待 10–30 秒后重试。检查 kubectl get globalcontextentry scc-rolebindings -o jsonpath='{.status.lastRefreshTime}{"\n"}',确认最近已刷新。
Pod 被准入,但 runAsUser 意外地被设置/未设置mutating policy 根据所选 SCC 填充了默认值,或者因为 Pod 已经声明了一个值所以没有填充。查看 Pod 上的 alauda.io/scc 注解,以了解选择了哪个 SCC,然后检查该 SCC 的 runAsUser.typerunAsUser.uid。Pod 如果自己声明了 runAsUser,则不会被覆盖。
scc-fill-defaultsscc-auto-pickREADY=falseKyverno 缺少对 policy 所匹配资源的读取权限(最常见的是 pods/ephemeralcontainers)。重新完整应用步骤 1.3 中的 kyverno-scc-reader ClusterRole。
位于 pod-security.kubernetes.io/enforce: restricted namespace 中的 Pod 在 Kyverno 看到之前就被拒绝Kubernetes Pod Security Admission 插件会在 Kyverno 之前运行,并独立执行 namespace 标签。根据该 namespace 中工作负载的需要,将 namespace 标签放宽为 baselineprivileged,或者限制你在该处提供的 SCC。

了解更多

使用 PolicyException 临时绕过 policy

当你需要在短时间内允许某个单独的 ServiceAccount 超出其当前 SCC(例如紧急调试会话),而修改 RBAC 绑定并不合适时,可以使用 PolicyException 资源。这要求 Kyverno 的 admission controller 启动时带有 --enablePolicyException=true

apiVersion: policies.kyverno.io/v1alpha1
kind: PolicyException
metadata:
  name: postgres-debug-bypass
  namespace: policy-exceptions
spec:
  policyRefs:
    - name: scc-auto-pick
      kind: ValidatingPolicy
  matchConditions:
    - name: target-sa
      expression: |
        object.metadata.namespace == 'databases' &&
        object.spec.?serviceAccountName.orValue('') == 'postgres-sa'
    - name: must-be-debug-window
      expression: |
        object.metadata.?labels[?'debug-window'].orValue('') == 'open'

最佳实践:将 PolicyException 资源放在专用 namespace 中(例如 policy-exceptions),并限制写入权限;为每个例外添加 ownerexpire-at 标签,并按周期进行审计。

引擎如何选择 SCC

当有多个 SCC 被授予某个 subject,且 Pod 满足其中多个 SCC 时,validating policy 会按以下顺序对候选项进行排序:

  1. 先看更高的 priority
  2. 再看更高的 restrictiveScore

Pod 完全满足的第一个候选项,就是被选中的 SCC。mutating policy 在决定填充哪个 SCC 的默认值时,也使用相同的排序。这与 OpenShift 的“最严格且可接受的 SCC 优先”意图一致,同时允许操作员通过每个 profile 的 priority 覆盖顺序。

OpenShift 命令映射

如果你来自 OpenShift,以下 oc 命令可直接映射为针对 SCC 引擎的 kubectl apply。这些操作会授予 SCC 的 use 权限,并且应仅由被允许更改集群 Pod 安全边界的管理员执行。

OpenShift 命令在该引擎中的等效操作
oc adm policy add-scc-to-user <scc> <user>创建一个对 securitycontextconstraints/<scc> 授予 use 的 ClusterRole,然后创建一个 subjects: [{kind: User, name: <user>}] 的 ClusterRoleBinding。
oc adm policy add-scc-to-user <scc> -z <sa> -n <ns>与上面相同的 ClusterRole,再加上 namespace <ns> 中的一个 RoleBinding,其 subjects: [{kind: ServiceAccount, name: <sa>, namespace: <ns>}]
oc adm policy add-scc-to-group <scc> <group>同样的 ClusterRole,再加上一个 subjects: [{kind: Group, name: <group>}] 的 ClusterRoleBinding。
oc get scckubectl get scc(该 CRD 的 shortNames: [scc] 使命令保持一致)。

最佳实践:在现有 namespace 中授予一个 privileged 例外

当现有 namespace 通常遵循 PSA restricted 标准,但同一 namespace 中有一个工作负载必须运行 privileged 容器时,请使用此操作步骤。namespace 保持不变;相反,平台管理员从该 namespace 中移除 PSA restricted 强制执行,并让 Kyverno SCC 引擎成为执行安全边界的 admission 控制。

隔离模型如下:

  • 该 namespace 中的每个 ServiceAccount 都可以使用 restricted-v2 作为默认安全上限。
  • 这个例外工作负载使用一个专用 ServiceAccount。
  • 只有该 ServiceAccount 可以使用内置的 privileged SCC。
  • 该工作负载使用 alauda.io/required-scc 固定为 privileged
  • 其他 ServiceAccount 不会被授予 privileged,如果它们请求 privileged 设置,仍然会被拒绝。
Warning

PSA 和 Kyverno SCC 引擎是彼此独立的 admission 控制。SCC 授权不能覆盖 pod-security.kubernetes.io/enforce: restricted。在移除 PSA enforcement 之前,应先完成 restricted SCC 基线和例外工作负载绑定,以确保 namespace 始终不会存在无防护的过渡窗口。

步骤 1 - 审计现有 SCC 授权

在更改 PSA 之前,请检查普通 ServiceAccount 是否已经通过 User、Group、RoleBinding 或 ClusterRoleBinding 获得了宽松的 SCC。要特别关注 system:authenticatedsystem:serviceaccountssystem:serviceaccounts:payments 的绑定。

例如,默认 ServiceAccount 不应能够使用内置的 privileged SCC:

kubectl auth can-i use \
  securitycontextconstraints.security.alauda.io/privileged \
  --as=system:serviceaccount:payments:default \
  -n payments

期望输出是 no。在继续之前,请移除任何非预期的宽泛授权。前面提到的可选标签 rbac.alauda.io/scc-use=true 可以帮助定位与 SCC 相关的 RBAC 对象,但不要仅依赖标签,因为没有标签的 Role 也可能授予 use 权限。

步骤 2 - 将 restricted 基线绑定到所有 namespace ServiceAccount

创建一个 namespaced RoleBinding,将合成的 system:serviceaccounts:payments Group 绑定到步骤 2.4 中的 scc-use-restricted-v2 ClusterRole。这样可以覆盖该 namespace 中现有和未来的所有 ServiceAccount。

保存为 payments-restricted-baseline.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: all-serviceaccounts-restricted-v2
  namespace: payments
  labels:
    rbac.alauda.io/scc-use: "true"
subjects:
  - kind: Group
    name: system:serviceaccounts:payments
    apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-use-restricted-v2

应用该绑定,并验证普通 ServiceAccount 只能使用 restricted 基线:

kubectl apply -f payments-restricted-baseline.yaml

kubectl auth can-i use \
  securitycontextconstraints.security.alauda.io/restricted-v2 \
  --as=system:serviceaccount:payments:default \
  -n payments

期望输出是 yes

步骤 3 - 创建一个专用 ServiceAccount

SCC 授权是基于 subject 的;它不会将权限授予某个 Pod 名称。要建模单一工作负载例外,应使用一个不与无关工作负载共享的独占 ServiceAccount。

保存为 payment-agent-serviceaccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: payment-agent-privileged
  namespace: payments
automountServiceAccountToken: false

应用该 ServiceAccount:

kubectl apply -f payment-agent-serviceaccount.yaml
Warning

内置的 privileged SCC 是有意设计为不受限制的。除了 privileged 容器之外,它还允许 host networking、host namespaces、host ports、host paths、所有 Linux capabilities、所有 volume 类型、所有 seccomp profile 和不安全的 sysctls。只有在工作负载确实需要完全权限时才授予它。如果工作负载只需要更小范围的例外,例如仅 anyuidhostNetworkhostPath,请改为绑定更窄的内置 profile。

除非工作负载必须调用 Kubernetes API,否则请保持 automountServiceAccountToken: false。启用 token 自动挂载会把 API 凭据注入到一个已经拥有更高运行时权限的容器中。

步骤 4 - 仅将 privileged 授予专用 ServiceAccount

创建一个 ClusterRole,其中的 resourceNames 只包含内置的 privileged SCC,然后通过 workload namespace 中的 RoleBinding 将其绑定到该专用 ServiceAccount。

保存为 payment-agent-scc-binding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: scc-use-privileged
  labels:
    rbac.alauda.io/scc-use: "true"
rules:
  - apiGroups: ["security.alauda.io"]
    resources: ["securitycontextconstraints"]
    resourceNames: ["privileged"]
    verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: payment-agent-privileged-scc
  namespace: payments
  labels:
    rbac.alauda.io/scc-use: "true"
subjects:
  - kind: ServiceAccount
    name: payment-agent-privileged
    namespace: payments
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-use-privileged

应用并验证绑定:

kubectl apply -f payment-agent-scc-binding.yaml

kubectl auth can-i use \
  securitycontextconstraints.security.alauda.io/privileged \
  --as=system:serviceaccount:payments:payment-agent-privileged \
  -n payments

kubectl auth can-i use \
  securitycontextconstraints.security.alauda.io/privileged \
  --as=system:serviceaccount:payments:default \
  -n payments

期望输出中,payment-agent-privilegedyesdefaultno。该专用 ServiceAccount 还会从 namespace Group 绑定中继承 restricted-v2

步骤 5 - 防止其他工作负载重用该 ServiceAccount

此操作步骤授予的是 ServiceAccount 的 privileged,而不是 Pod 或 controller 名称。Kubernetes RBAC 不提供对 spec.serviceAccountName 的字段级授权。如果一个用户可以在 payments 中创建任意 Pod、Deployment、StatefulSet、DaemonSet、Job 或 CronJob,那么他们就可能在另一个工作负载中引用 payment-agent-privileged,并获得相同的 SCC 访问权限。

在继续之前,请至少执行以下控制之一:

  • 只允许专用的 GitOps 或部署身份创建或更新引用 payment-agent-privileged 的工作负载,并阻止普通 namespace 用户修改这些工作负载。
  • 添加一个 admission guard:除非请求已获授权,否则拒绝对 payment-agent-privileged 的引用。将该 guard 应用于直接 Pod 以及在该 namespace 中启用的每一种 Pod-template controller。

不要仅依赖 workload label 或注解来授权使用该专用 ServiceAccount;通常能够创建工作负载的用户也可以复制这些值。如果无法限制部署所有权或 admission 时的 ServiceAccount 使用,那么这个模式只提供的是 ServiceAccount 级别的例外,不能将其表示为单个 Pod 的例外。

在强制实施上述某一种控制之前,不要继续进行 PSA 迁移。

步骤 6 - 将强制执行从 PSA 转移到 Kyverno SCC

只移除 PSA restricted enforcement 标签。保留 restricted audit 和 warning 标签,这样 API server 仍会报告那些会违反 restricted 标准的工作负载。

kubectl label namespace payments \
  pod-security.kubernetes.io/enforce- \
  pod-security.kubernetes.io/enforce-version- \
  --overwrite

kubectl label namespace payments \
  pod-security.kubernetes.io/audit=restricted \
  pod-security.kubernetes.io/audit-version=latest \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/warn-version=latest \
  --overwrite

移除 namespace 标签不会覆盖通过 API server admission 配置设置的集群级 PSA 默认值。如果集群默认仍然强制 restricted,请显式让该 namespace 退出 enforcement,同时保留审计和告警可见性:

kubectl label namespace payments \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/enforce-version=latest \
  --overwrite

这里 enforce=privileged 表示 PSA 不施加额外限制。它并不授予 Pod privileged 访问权限;Kyverno SCC 绑定仍然是强制执行的授权边界。

步骤 7 - 在工作负载中固定已批准的 SCC

alauda.io/required-scc 放到 Pod template metadata 中,并使用专用 ServiceAccount。不要只把注解放在 Deployment metadata 上,因为 controller metadata 不会自动复制到其 Pod。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-agent
  namespace: payments
spec:
  replicas: 1
  selector:
    matchLabels:
      app: payment-agent
  template:
    metadata:
      labels:
        app: payment-agent
      annotations:
        alauda.io/required-scc: privileged
    spec:
      serviceAccountName: payment-agent-privileged
      securityContext:
        seccompProfile:
          type: Unconfined
      containers:
        - name: agent
          image: registry.example.com/payments/agent:1.0.0
          securityContext:
            privileged: true
            allowPrivilegeEscalation: true
            runAsUser: 0

该注解不会授予 SCC 权限。如果 SCC 缺失,或者 ServiceAccount 没有 use 权限,validating policy 会拒绝该 Pod。

步骤 8 - 验证该例外是隔离的

请以真实的应用部署身份运行 admission 探测,而不是使用 cluster-admin。对于直接创建的 Pod,SCC 选择会将请求的 User 和 Groups 纳入考虑,因此拥有单独宽泛 SCC 授权的管理员可能会得到与生产中所使用的工作负载身份不同的结果。

验证该例外工作负载被以预期的 SCC 准入:

kubectl -n payments get pod -l app=payment-agent \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.annotations.alauda\.io/scc}{"\n"}{end}'

SCC 列应为 privileged

接下来,确认普通 restricted Pod 仍然可以以 restricted-v2 准入:

cat <<'EOF' | kubectl create --dry-run=server -f - \
  -o jsonpath='{.metadata.annotations.alauda\.io/scc}{"\n"}'
apiVersion: v1
kind: Pod
metadata:
  name: restricted-probe
  namespace: payments
spec:
  serviceAccountName: default
  securityContext:
    runAsUser: 1000
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: probe
      image: registry.example.com/library/pause:3.10
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
EOF

期望输出为 restricted-v2

最后,确认普通 ServiceAccount 不能请求 privileged

cat <<'EOF' | kubectl create --dry-run=server -f -
apiVersion: v1
kind: Pod
metadata:
  name: unauthorized-privileged-probe
  namespace: payments
  annotations:
    alauda.io/required-scc: privileged
spec:
  serviceAccountName: default
  containers:
    - name: probe
      image: registry.example.com/library/pause:3.10
      securityContext:
        privileged: true
EOF

当由正常的应用部署身份提交时,请求必须被拒绝,并带有一条说明所需 SCC 未绑定到 ServiceAccount default 的消息。如果该 Pod 被准入,请停止并在允许生产工作负载之前检查宽泛的 SCC RoleBinding、ClusterRoleBinding 以及 User 或 Group 授权。

回滚该例外

要安全地恢复 PSA restricted enforcement:

  1. 停止或替换该 privileged 工作负载,使其使用符合 restricted 的规范。

  2. 删除将 privileged 授予专用 ServiceAccount 的 RoleBinding:

    kubectl -n payments delete rolebinding payment-agent-privileged-scc
  3. 确认该 ServiceAccount 不再能使用 privileged

    kubectl auth can-i use \
      securitycontextconstraints.security.alauda.io/privileged \
      --as=system:serviceaccount:payments:payment-agent-privileged \
      -n payments

    授权检查必须返回 no。不要删除内置的 privileged SCC。只有在没有其他 RoleBinding 或 ClusterRoleBinding 引用它时,才删除可复用的 scc-use-privileged ClusterRole。

  4. 对剩余工作负载执行 server-side dry-run 检查。

  5. 最后恢复 PSA restricted enforcement:

    kubectl label namespace payments \
      pod-security.kubernetes.io/enforce=restricted \
      pod-security.kubernetes.io/enforce-version=latest \
      --overwrite

后续步骤

  • 在审查工作负载需求后,决定每个现有 namespace 和 ServiceAccount 应绑定到哪个 SCC,记录映射,并通过 GitOps 工作流应用这些绑定,以便保持可审计和可复现。
  • 计划定期审查 PolicyException 资源——它们的用途是短期窗口,而不是永久例外。
  • 如果你在大规模环境中运行,请监控 Kyverno admission controller 的 kyverno_admission_review_duration_seconds 指标,以便在 SCC profile 或 RBAC 绑定数量增长时检测 admission 延迟变化。