Skip to main content
Version: main 🚧

Central admission control

Enterprise
Available in these plansFreeDevProdScale
Centralized Admission Control
Supported Configurations
Running the control plane as a container with:
Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Centralized admission control is an advanced feature for cluster admins that have custom rules that they need to apply to the tenant cluster. Cluster admins can enforce Kubernetes admission webhooks that reference the control plane cluster or external policy services from within the tenant cluster. Examples of validating and mutating webhook based policy engines include OPA, Kyverno, and jsPolicy.

This feature requires the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers to be enabled. For more information, see the Kubernetes admission controllers reference.

warning
  • Central admission control is read-only after tenant cluster creation. Admins cannot bypass or alter hooks after vCluster deployment.
  • If you need to modify or delete webhook configurations, do not use central admission control. Instead, manually map the host policy service into the tenant cluster.

Central admission webhooks are different from the other policies:

  • LimitRange and ResourceQuota resources are created and enforced on the control plane cluster. They do not appear as resources in the tenant cluster.
  • A user could define LimitRange and ResourceQuota resources inside the tenant cluster, but they have full control over them and can delete them when needed.
  • Webhooks are different because they need to be configured inside the tenant cluster in order for them to be called by the vCluster's API server.
  • vCluster rewrites these definitions to point to a proxy for a control plane cluster service that handles webhook requests. The host might also have webhook configurations that use this service.
  • A user can still install a webhook service or webhook configuration into the tenant cluster outside of this config, but it would run inside the tenant cluster like any other workload.

Access tenant identifiers in webhooks​

vCluster rewrites the namespace of a namespace-scoped object to the host namespace where vCluster runs. This is not visible to the user, but the admission controller can use mechanisms like the namespace selector as usual. Some admission controllers such as Gatekeeper rely on the namespace always being present in the underlying cluster and wouldn't work otherwise. To access the tenant namespace in a webhook, read the request.options.vCluster.namespace.metadata.name field in the admission request object, which holds the full tenant namespace object.

In some cases a webhook running inside the tenant cluster needs the translated name of a pod. This is the name used when the pod is created on the control plane cluster. vCluster puts the translated pod name under the request.options.vCluster.name field in the admission request object. The final name is only available during the validation phase, so this only works with validatingWebhooks that have rules configured for pods.

Certificate configuration​

For webhooks using self-signed or private certificates, add the caBundle field:

# Get base64-encoded CA certificate from secret
kubectl get secret webhook-tls-secret -n webhook-namespace -o jsonpath='{.data.ca\.crt}'
warning

Missing caBundle with failurePolicy: Fail can make vCluster unrecoverable. Test with failurePolicy: Ignore first.

Kyverno example​

This example ensures that all new ConfigMap resources are sent to a mutating webhook on the control plane cluster.

Apply Enforce-mode policies after the tenant is ready

A Kyverno ClusterPolicy running on the control plane cluster in Enforce mode validates the resources that vCluster syncs from the tenant. This includes vCluster's own system pods, which don't carry an app.kubernetes.io/name label. A label-requiring policy applied before the tenant is provisioned rejects those pods, and the VirtualClusterInstance ends in Failed.

To avoid this, do one of the following:

  • Apply Enforce-mode policies only after the tenant cluster reaches Ready.
  • Start the policy in Audit mode.
  • Scope the policy to exclude vCluster system namespaces and service accounts.
centralAdmission:
  mutatingWebhooks:
    - apiVersion: admissionregistration.k8s.io/v1
      kind: MutatingWebhookConfiguration
      metadata:
        name: kyverno-webhook
      webhooks:
      - admissionReviewVersions:
        - v1
        clientConfig:
          # Add caBundle for self-signed certificates
          caBundle: LS0tLS1CRUdJTi... # Base64-encoded PEM certificate
          service:
            name: kyverno-svc
            namespace: kyverno
            path: /mutate/fail
            port: 443
        failurePolicy: Fail
        matchPolicy: Equivalent
        name: mutate.kyverno.svc-fail
        objectSelector: {}
        reinvocationPolicy: IfNeeded
        rules:
        - apiGroups:
          - ""
          apiVersions:
          - v1
          operations:
          - CREATE
          resources:
          - configmaps
          scope: '*'
        sideEffects: NoneOnDryRun
        timeoutSeconds: 10

Any time a ConfigMap is created, the vCluster API server forwards the admission request to a proxy, which in turn calls the actual admission hook running in the control plane cluster in the kyverno namespace.

The service providing the webhooks should not rely on the real names of the objects if they are synced on the control plane cluster (for example, pods). The requests that reach the host admission service hold the objects from the tenant cluster, so your policies need to be able to handle those objects (except for the namespace which is rewritten as outlined above).

Policies that key off name or namespace

The admission webhook fires against the translated name (<pod>-x-<namespace>-x-<tenant>) and the host namespace, not the tenant's view. Policies that key off metadata.name or the namespace produce wrong results:

  • A uniqueness or collision check silently passes resources that actually collide.
  • A naming or length check rejects tenant resources that are valid in the tenant's own view.
  • Namespace-scoped match and exclude rules apply to the wrong set of objects.

To read the original tenant identifiers, see Access tenant identifiers in webhooks.

The hook services do not require authentication as the proxy does not have access to the AdmissionConfiguration object or the files it references. Most policy engines (such as jsPolicy, Kyverno, or Gatekeeper) do not set authentication by default, so it should work with most installations.

If a user inside the tenant cluster (admin or not) tries to delete one of your admission hooks, that uses is prompted with the following error message:

kubectl delete kyverno-webhook
error: the resource kyverno-webhook is protected

Gatekeeper (OPA) example​

Ensure you install Gatekeeper to your Kubernetes cluster.

Create the following constraint template that denies pods without specified labels:

apiVersion: config.gatekeeper.sh/v1alpha1
kind: Config
metadata:
  name: config
  namespace: "gatekeeper-system"
spec:
  match:
    - excludedNamespaces: ["kube-*"]
      processes: ["*"]
  sync:
    syncOnly:
      - group: ""
        version: "v1"
        kind: "Namespace"
      - group: ""
        version: "v1"
        kind: "Pod"
---
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

Create the constraint:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: pod-required-labels
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    labelSelector:
      matchLabels:
        test: gatekeeper
  parameters:
    labels: ["gatekeeper"]

Then start the vCluster with the following configuration:

policies:
  centralAdmission:
    validatingWebhooks:
    - apiVersion: admissionregistration.k8s.io/v1
      kind: ValidatingWebhookConfiguration
      metadata:
        name: gatekeeper-webhook
      webhooks:
      - admissionReviewVersions:
        - v1
        clientConfig:
          # Add caBundle for self-signed certificates
          caBundle: LS0tLS1CRUdJTi... # Base64-encoded PEM certificate
          service:
            name: gatekeeper-webhook-service
            namespace: gatekeeper-system
            path: /v1/admit
            port: 443
        failurePolicy: Fail
        matchPolicy: Exact
        name: validation.gatekeeper.sh
        objectSelector: {}
        rules:
        - apiGroups:
          - '*'
          apiVersions:
          - '*'
          operations:
          - CREATE
          - UPDATE
          resources:
          - pods
          scope: Namespaced
        sideEffects: None
        timeoutSeconds: 10

Within the vCluster try to create the following pod which should get denied:

$ echo "apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    test: gatekeeper
spec:
  containers:
    - image: nginx
      name: nginx" | kubectl create -f -
admission webhook "validation.gatekeeper.sh" denied the request: you must provide labels: [gatekeeper]

Network policies​

If you are using network policies, admission webhooks traffic from the vCluster control plane must be allowed.

vcluster.yaml
policies:
  networkPolicy:
    controlPlane:
      egress:
      - to:
        # Allow vcluster control plane traffic to the gatekeeper webhook.
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: gatekeeper-system
          podSelector:
            matchLabels:
              gatekeeper.sh/operation: webhook

Config reference​

centralAdmission object ​

CentralAdmission defines what validating or mutating webhooks should be enforced within the virtual cluster.

validatingWebhooks object[] ​

ValidatingWebhooks are validating webhooks that should be enforced in the virtual cluster

kind string ​

Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to.

apiVersion string ​

APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values.

metadata object ​

Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.

name string ​

Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition.

labels object ​

Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services.

annotations object ​

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata.

webhooks object[] ​

Webhooks is a list of webhooks and the affected resources and operations.

name string ​

The name of the admission webhook. Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where "imagepolicy" is the name of the webhook, and kubernetes.io is the name of the organization.

clientConfig object ​

ClientConfig defines how to communicate with the hook.

url string ​

URL gives the location of the webhook, in standard URL form (scheme://host:port/path). Exactly one of url or service must be specified.

service object ​

Service is a reference to the service for this webhook. Either service or url must be specified.

If the webhook is running within the cluster, then you should use service.

namespace string ​

Namespace is the namespace of the service.

name string ​

Name is the name of the service.

path string ​

Path is an optional URL path which will be sent in any request to this service.

port integer ​

If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. port should be a valid port number (1-65535, inclusive).

caBundle string ​

CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate. If unspecified, system trust roots on the apiserver are used.

rules object[] ​

Rules describes what operations on what resources/subresources the webhook cares about. The webhook cares about an operation if it matches any Rule.

failurePolicy string ​

FailurePolicy defines how unrecognized errors from the admission endpoint are handled - allowed values are Ignore or Fail. Defaults to Fail.

matchPolicy string ​

matchPolicy defines how the "rules" list is used to match incoming requests. Allowed values are "Exact" or "Equivalent".

namespaceSelector object ​

NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook.

objectSelector object ​

ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector.

sideEffects string ​

SideEffects states whether this webhook has side effects.

timeoutSeconds integer ​

TimeoutSeconds specifies the timeout for this webhook.

admissionReviewVersions string[] ​

AdmissionReviewVersions is an ordered list of preferred AdmissionReview versions the Webhook expects.

matchConditions object[] ​

MatchConditions is a list of conditions that must be met for a request to be sent to this webhook. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.

mutatingWebhooks object[] ​

MutatingWebhooks are mutating webhooks that should be enforced in the virtual cluster

kind string ​

Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to.

apiVersion string ​

APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values.

metadata object ​

Standard object metadata; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata.

name string ​

Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition.

labels object ​

Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services.

annotations object ​

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata.

webhooks object[] ​

Webhooks is a list of webhooks and the affected resources and operations.

reinvocationPolicy string ​

reinvocationPolicy indicates whether this webhook should be called multiple times as part of a single admission evaluation. Allowed values are "Never" and "IfNeeded".

name string ​

The name of the admission webhook. Name should be fully qualified, e.g., imagepolicy.kubernetes.io, where "imagepolicy" is the name of the webhook, and kubernetes.io is the name of the organization.

clientConfig object ​

ClientConfig defines how to communicate with the hook.

url string ​

URL gives the location of the webhook, in standard URL form (scheme://host:port/path). Exactly one of url or service must be specified.

service object ​

Service is a reference to the service for this webhook. Either service or url must be specified.

If the webhook is running within the cluster, then you should use service.

namespace string ​

Namespace is the namespace of the service.

name string ​

Name is the name of the service.

path string ​

Path is an optional URL path which will be sent in any request to this service.

port integer ​

If specified, the port on the service that hosting webhook. Default to 443 for backward compatibility. port should be a valid port number (1-65535, inclusive).

caBundle string ​

CABundle is a PEM encoded CA bundle which will be used to validate the webhook's server certificate. If unspecified, system trust roots on the apiserver are used.

rules object[] ​

Rules describes what operations on what resources/subresources the webhook cares about. The webhook cares about an operation if it matches any Rule.

failurePolicy string ​

FailurePolicy defines how unrecognized errors from the admission endpoint are handled - allowed values are Ignore or Fail. Defaults to Fail.

matchPolicy string ​

matchPolicy defines how the "rules" list is used to match incoming requests. Allowed values are "Exact" or "Equivalent".

namespaceSelector object ​

NamespaceSelector decides whether to run the webhook on an object based on whether the namespace for that object matches the selector. If the object itself is a namespace, the matching is performed on object.metadata.labels. If the object is another cluster scoped resource, it never skips the webhook.

objectSelector object ​

ObjectSelector decides whether to run the webhook based on if the object has matching labels. objectSelector is evaluated against both the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector.

sideEffects string ​

SideEffects states whether this webhook has side effects.

timeoutSeconds integer ​

TimeoutSeconds specifies the timeout for this webhook.

admissionReviewVersions string[] ​

AdmissionReviewVersions is an ordered list of preferred AdmissionReview versions the Webhook expects.

matchConditions object[] ​

MatchConditions is a list of conditions that must be met for a request to be sent to this webhook. Match conditions filter requests that have already been matched by the rules, namespaceSelector, and objectSelector. An empty list of matchConditions matches all requests. There are a maximum of 64 match conditions allowed.