IMPORTANT NOTE: This site is not official Red Hat documentation and is provided for informational purposes only. These guides may be experimental, proof of concept, or early adoption. Officially supported documentation is available at docs.openshift.com and access.redhat.com.
Azure Key Vault CSI on Azure Red Hat OpenShift
Author: Paul Czarkowski Modified: 08/16/2021
This document is adapted from the Azure Key Vault CSI Walkthrough specifically to run with Azure Red Hat OpenShift (ARO).
Prerequisites
- An ARO cluster
- The AZ CLI (logged in)
- Helm 3.x CLI
Environment Variables
-
Run this command to set some environment variables to use throughout
Note if you created the cluster from the instructions linked above these will re-use the same environment variables, or default them to
openshift
andeastus
.export KEYVAULT_RESOURCE_GROUP=${AZR_RESOURCE_GROUP:-"openshift"} export KEYVAULT_LOCATION=${AZR_RESOURCE_LOCATION:-"eastus"} export KEYVAULT_NAME=secret-store-$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 10 | head -n 1) export AZ_TENANT_ID=$(az account show -o tsv --query tenantId)
Installing the Kubernetes Secret Store CSI
-
Create an OpenShift Project to deploy the CSI into
oc new-project k8s-secrets-store-csi
-
Set SecurityContextConstraints to allow the CSI driver to run (otherwise the DaemonSet will not be able to create Pods)
oc adm policy add-scc-to-user privileged \ system:serviceaccount:k8s-secrets-store-csi:secrets-store-csi-driver
-
Add the Secrets Store CSI Driver to your Helm Repositories
helm repo add secrets-store-csi-driver \ https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
-
Update your Helm Repositories
helm repo update
-
Install the secrets store csi driver
helm install -n k8s-secrets-store-csi csi-secrets-store \ secrets-store-csi-driver/secrets-store-csi-driver \ --version v1.0.1 \ --set "linux.providersDir=/var/run/secrets-store-csi-providers"
-
Check that the Daemonsets is running
kubectl --namespace=k8s-secrets-store-csi get pods -l "app=secrets-store-csi-driver"
You should see the following
NAME READY STATUS RESTARTS AGE csi-secrets-store-secrets-store-csi-driver-cl7dv 3/3 Running 0 57s csi-secrets-store-secrets-store-csi-driver-gbz27 3/3 Running 0 57s
Deploy Azure Key Store CSI
-
Add the Azure Helm Repository
helm repo add csi-secrets-store-provider-azure \ https://raw.githubusercontent.com/Azure/secrets-store-csi-driver-provider-azure/master/charts
-
Update your local Helm Repositories
helm repo update
-
Install the Azure Key Vault CSI provider
helm install -n k8s-secrets-store-csi azure-csi-provider \ csi-secrets-store-provider-azure/csi-secrets-store-provider-azure \ --set linux.privileged=true --set secrets-store-csi-driver.install=false \ --set "linux.providersDir=/var/run/secrets-store-csi-providers" \ --version=v1.0.1
-
Set SecurityContextConstraints to allow the CSI driver to run
oc adm policy add-scc-to-user privileged \ system:serviceaccount:k8s-secrets-store-csi:csi-secrets-store-provider-azure
Create Keyvault and a Secret
-
Create a namespace for your application
oc new-project my-application
-
Create an Azure Keyvault in your Resource Group that contains ARO
az keyvault create -n ${KEYVAULT_NAME} \ -g ${KEYVAULT_RESOURCE_GROUP} \ --location ${KEYVAULT_LOCATION}
-
Create a secret in the Keyvault
az keyvault secret set \ --vault-name ${KEYVAULT_NAME} \ --name secret1 --value "Hello"
-
Create a Service Principal for the keyvault
Note: If this gives you an error, you may need upgrade your Azure CLI to the latest version.
```bash
export SERVICE_PRINCIPAL_CLIENT_SECRET="$(az ad sp create-for-rbac --skip-assignment --name http://$KEYVAULT_NAME --query 'password' -otsv)"
export SERVICE_PRINCIPAL_CLIENT_ID="$(az ad sp list --display-name http://$KEYVAULT_NAME --query '[0].appId' -otsv)"
```
-
Set an Access Policy for the Service Principal
az keyvault set-policy -n ${KEYVAULT_NAME} \ --secret-permissions get \ --spn ${SERVICE_PRINCIPAL_CLIENT_ID}
-
Create and label a secret for Kubernetes to use to access the Key Vault
kubectl create secret generic secrets-store-creds \ -n my-application \ --from-literal clientid=${SERVICE_PRINCIPAL_CLIENT_ID} \ --from-literal clientsecret=${SERVICE_PRINCIPAL_CLIENT_SECRET} kubectl -n my-application label secret \ secrets-store-creds secrets-store.csi.k8s.io/used=true
Deploy an Application that uses the CSI
-
Create a Secret Provider Class to give access to this secret
cat <<EOF | kubectl apply -f - apiVersion: secrets-store.csi.x-k8s.io/v1alpha1 kind: SecretProviderClass metadata: name: azure-kvname namespace: my-application spec: provider: azure parameters: usePodIdentity: "false" useVMManagedIdentity: "false" userAssignedIdentityID: "" keyvaultName: "${KEYVAULT_NAME}" objects: | array: - | objectName: secret1 objectType: secret objectVersion: "" tenantId: "${AZ_TENANT_ID}" EOF
-
Create a Pod that uses the above Secret Provider Class
cat <<EOF | kubectl apply -f - kind: Pod apiVersion: v1 metadata: name: busybox-secrets-store-inline namespace: my-application spec: containers: - name: busybox image: k8s.gcr.io/e2e-test-images/busybox:1.29 command: - "/bin/sleep" - "10000" volumeMounts: - name: secrets-store-inline mountPath: "/mnt/secrets-store" readOnly: true volumes: - name: secrets-store-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "azure-kvname" nodePublishSecretRef: name: secrets-store-creds EOF
-
Check the Secret is mounted
kubectl exec busybox-secrets-store-inline -- ls /mnt/secrets-store/
Output should match:
secret1
-
Print the Secret
kubectl exec busybox-secrets-store-inline \ -- cat /mnt/secrets-store/secret1
Output should match:
Hello
Cleanup
-
Uninstall Helm
helm uninstall -n k8s-secrets-store-csi azure-csi-provider
-
Delete the app
oc delete project my-application
-
Delete the Azure Key Vault
az keyvault delete -n ${KEYVAULT_NAME}
-
Delete the Service Principal
az ad sp delete --id ${SERVICE_PRINCIPAL_CLIENT_ID}
Uninstalling the Kubernetes Secret Store CSI
-
Delete the secrets store csi driver
helm delete -n k8s-secrets-store-csi csi-secrets-store
-
Delete the SecurityContextConstraints
oc adm policy remove-scc-from-user privileged \ system:serviceaccount:k8s-secrets-store-csi:secrets-store-csi-driver