Admission Controllers In Kubernetes
We already know, Control Planes in Kubernetes exposes API of various Group, Version and Kinds [GVK]. An admission control plug-in works as a webhook in Kubernetes that intercept API requests before they pass to the API server and can prohibit or modify them which can be targeted to specific GVKs.
Introduction
Each admission control plug-in is run in sequence before a request is accepted into the cluster. If any of the plug-ins in the sequence reject the request, the entire request is rejected immediately and an error is returned to the end user. Each request must be authenticated and authorized before it can be accepted by the API Server and all communication must happen on port 443 or HTTPS.
Several admission controllers are enabled by default because most normal Kubernetes operations rely upon them. Most of these controllers comprise some of the Kubernetes source trees and are compiled as plugins.
Kubernetes supports two types of admission controllers,
- Mutating Admission Controller
- Validation Admission Controller
Mutating Admission Controller reads the incoming request into a valid schema and can perform ADD or Delete or Modify on the request.
The validation Admission Controller reads the incoming request into a valid schema and approves or rejects the request based on predefined checks.
Why do you need them?
Let’s start with a use case, Suppose you have a deployment which uses a Specific Version/Tag of an Image and you don’t want to use the default Latest image tag which could lead to backward compatibility issues. Here you can write a Validating admission controller that adds this check on your deployment that if the image tag is the Latest, the deployment would be rejected.
Many advanced features in Kubernetes require an admission control plug-in to be enabled to properly support the feature. As a result, a Kubernetes API server that is not properly configured with the right set of admission control plug-ins is an incomplete server and will not support all the features you expect.
How to build one?
Now that we have a basic understanding of the controllers, Let’s write a simple Validation Controller that checks if the deployment has Nodeselector/Node-affinity or not.
We would be using Client-Go for Kubernetes and the k8s.io/apiserver/pkg/server package that behaves like the API server itself. This server would be the one handling every incoming request and returning valid responses.
But before building a controller, we know the API server only accepts requests on port 443, we need to create tls certs for the same, run this command in your terminal,
openssl req -x509 -newkey rsa:2048 -keyout tls.key -out tls.crt --days 3650 --nodes --subj "/CN=resourceguard.kube-system.svc" --addext "subjectAltName = DNS:resourceguard.kube-system.svc"to use these certs, create a secret out of it,
kubectl create secret generic certs --from-file tls.crt --from-file tls.key --dry-run=client -oyaml> secrets.yamlTo create a basic server add following code,
import (
"net/http"
"os"
"time"
"github.com/spf13/pflag"
"k8s.io/apiserver/pkg/server"
"k8s.io/apiserver/pkg/server/options"
"k8s.io/component-base/cli/globalflag"
)
// Options Setting Up the HTTPS server For Request and Response
type Options struct {
SecureServingOptions options.SecureServingOptions
}
// AddFlagSet Adding Flag Support
func (o *Options) AddFlagSet(fs *pflag.FlagSet) {
o.SecureServingOptions.AddFlags(fs)
}
type Config struct {
SecureServingInfo *server.SecureServingInfo
}
const (
valkontroller = "val-kontroller"
)
func (o *Options) Config() *Config {
if err := o.SecureServingOptions.MaybeDefaultWithSelfSignedCerts("0.0.0.0", nil, nil); err != nil {
panic(err)
}
c := Config{}
if err := o.SecureServingOptions.ApplyTo(&c.SecureServingInfo); err != nil {
panic(err)
}
return &c
}
func DefaultServerOptions() *Options {
NewOption := &Options{
SecureServingOptions: *options.NewSecureServingOptions(),
}
NewOption.SecureServingOptions.BindPort = 8443
NewOption.SecureServingOptions.ServerCert.PairName = valkontroller
return NewOption
}
// Init Starting HTTPS Server
func Init() {
option := DefaultServerOptions()
fs := pflag.NewFlagSet(valkontroller, pflag.ExitOnError)
globalflag.AddGlobalFlags(fs, valkontroller)
option.AddFlagSet(fs)
sentrylog()
if err := fs.Parse(os.Args); err != nil {
panic(err)
}
c := option.Config()
mux := http.NewServeMux()
mux.Handle("/validate", http.HandlerFunc(ValidatePod))
// This Channel will Run Until Gets SIGTERM or SIGINT
stopCh := server.SetupSignalHandler()
ch, err := c.SecureServingInfo.Serve(mux, 60*time.Second, stopCh)
if err != nil {
sentry.CaptureException(err)
panic(err)
} else {
<-ch
}
}On Path /validate a validation function would run, where the logic is built,
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
admissionv1 "k8s.io/api/admission/v1beta1"
appsv1 "k8s.io/api/apps/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/serializer"
)
var (
scheme = runtime.NewScheme()
codecs = serializer.NewCodecFactory(scheme)
logger = log.New(os.Stdout, "info: ", log.LstdFlags)
)
func admissionReviewFromRequest(r *http.Request, deserializer runtime.Decoder) (*admissionv1.AdmissionReview, error) {
// Validate that the incoming content type is correct.
if r.Header.Get("Content-Type") != "application/json" {
return nil, fmt.Errorf("expected application/json content-type")
}
// Get the body data, which will be the AdmissionReview
// content for the request.
var body []byte
if r.Body != nil {
requestData, err := ioutil.ReadAll(r.Body)
if err != nil {
return nil, err
}
body = requestData
}
// Decode the request body into
admissionReviewRequest := &admissionv1.AdmissionReview{}
if _, _, err := deserializer.Decode(body, nil, admissionReviewRequest); err != nil {
return nil, err
}
return admissionReviewRequest, nil
}
func ValidatePod(w http.ResponseWriter, r *http.Request) {
logger.Printf("Validation controller was called")
deserializer := codecs.UniversalDeserializer()
// Parse the AdmissionReview from the http request.
admissionReviewRequest, err := admissionReviewFromRequest(r, deserializer)
if err != nil {
msg := fmt.Sprintf("error getting admission review from request: %v", err)
logger.Printf(msg)
w.WriteHeader(400)
w.Write([]byte(msg))
return
}
rawRequest := admissionReviewRequest.Request.Object.Raw
deployment := appsv1.Deployment{}
if _, _, err := deserializer.Decode(rawRequest, nil, &deployment); err != nil {
msg := fmt.Sprintf("error decoding raw pod: %v", err)
logger.Printf(msg)
w.WriteHeader(500)
w.Write([]byte(msg))
return
}
admissionResponse := &admissionv1.AdmissionResponse{}
admissionResponse.Allowed = true
podNodeselector := deployment.Spec.Template.Spec.NodeSelector
// note : Need to add extra check because Affinity is struct and *ref to podspec
podNodeselectorTerm := deployment.Spec.Template.Spec.Affinity
if len(podNodeselector) == 0 {
if podNodeselectorTerm == nil {
admissionResponse.Allowed = false
admissionResponse.Result = &metav1.Status{
Message: "Nodeselector is Missing in the Deployment Spec, NodeSelector is a required field",
}
logger.Printf("[Deployment Rejected] ResourceGroup = 'apps/v1 deployments' Namespace = %v Deployment = %v ", admissionReviewRequest.Request.Namespace, admissionReviewRequest.Request.Name)
}
}
var admissionReviewResponse admissionv1.AdmissionReview
admissionReviewResponse.Response = admissionResponse
admissionReviewResponse.SetGroupVersionKind(admissionReviewRequest.GroupVersionKind())
admissionReviewResponse.Response.UID = admissionReviewRequest.Request.UID
resp, err := json.Marshal(admissionReviewResponse)
if err != nil {
msg := fmt.Sprintf("error marshalling response json: %v", err)
logger.Printf(msg)
w.WriteHeader(500)
w.Write([]byte(msg))
return
}
w.Header().Set("Content-Type", "application/json")
w.Write(resp)
}Now build the Binary and create docker image,
FROM debian
COPY ./validatingwebhook /validatingwebhook
ENTRYPOINT ["/validatingwebhook","--tls-cert-file=/var/run/webhook/serving-cert/tls.crt", "--tls-private-key-file=/var/run/webhook/serving-cert/tls.key","--v=10"]Your webhook is now ready to be deployed
apiVersion: v1
kind: Service
metadata:
namespace: kube-system
labels:
app: validatingwebhook
name: validatingwebhook
spec:
ports:
- port: 443
protocol: TCP
targetPort: 8443
name: https
selector:
app: validatingwebhook
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: "validatingwebhook.kube-system.svc"
webhooks:
- name: "validatingwebhook.kube-system.svc"
rules:
- apiGroups: ["*"]
apiVersions: ["v1", "v1beta1"]
operations: ["CREATE"]
resources: ["deployments"]
clientConfig:
service:
namespace: "kube-system"
name: "validatingwebhook"
path: "/validate"
caBundle: "Cert that you created"
admissionReviewVersions: ["v1", "v1beta1"]
sideEffects: None
timeoutSeconds: 5
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: validatingwebhook
namespace: kube-system
automountServiceAccountToken: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: validatingwebhook
namespace: kube-system
labels:
app: validatingwebhook
spec:
replicas: 1
selector:
matchLabels:
app: validatingwebhook
template:
metadata:
labels:
app: validatingwebhook
spec:
serviceAccountName: validatingwebhook
securityContext:
fsGroup: 65534
hostNetwork: true
containers:
- name: validatingwebhook
image: validation-kontoller:latest
imagePullPolicy: Always
resources:
limits:
cpu: 500m
memory: 200Mi
requests:
cpu: 200m
memory: 200Mi
ports:
- containerPort: 8443
volumeMounts:
- name: serving-cert
mountPath: /var/run/webhook/serving-cert
volumes:
- name: serving-cert
secret:
secretName: certsSubscribe to Developer Stack
Get the latest posts delivered right to your inbox