Azure Key Vault CSI on Azure Red Hat OpenShift
Authors:
Paul Czarkowski,
Diana Sari
Last Editor:
Dustin Scott
Published Date:
16 August 2021
Modified Date: 25 May 2023
This document is adapted from the Azure Key Vault CSI Walkthrough specifically to run with Azure Red Hat OpenShift (ARO).
Prerequisites
- An ARO cluster
- The AZ CLI (logged in)
- The OC CLI (logged in)
- Helm 3.x CLI
Environment Variables
Run this command to set some environment variables to use throughout
Note if you created the cluster from the instructions linked above these will re-use the same environment variables, or default them to
openshift
andeastus
.export KEYVAULT_RESOURCE_GROUP=${AZR_RESOURCE_GROUP:-"openshift"} export KEYVAULT_LOCATION=${AZR_RESOURCE_LOCATION:-"eastus"} export KEYVAULT_NAME=secret-store-$(cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 10 | head -n 1) export AZ_TENANT_ID=$(az account show -o tsv --query tenantId)
Installing the Kubernetes Secret Store CSI
Create an OpenShift Project to deploy the CSI into
oc new-project k8s-secrets-store-csi
Set SecurityContextConstraints to allow the CSI driver to run (otherwise the DaemonSet will not be able to create Pods)
oc adm policy add-scc-to-user privileged \ system:serviceaccount:k8s-secrets-store-csi:secrets-store-csi-driver
Add the Secrets Store CSI Driver to your Helm Repositories
helm repo add secrets-store-csi-driver \ https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
Update your Helm Repositories
helm repo update
Install the secrets store csi driver
helm install -n k8s-secrets-store-csi csi-secrets-store \ secrets-store-csi-driver/secrets-store-csi-driver \ --version v1.3.2 \ --set "linux.providersDir=/var/run/secrets-store-csi-providers"
Check that the Daemonsets is running
oc -n k8s-secrets-store-csi get pods -l "app=secrets-store-csi-driver"
You should see the following
NAME READY STATUS RESTARTS AGE csi-secrets-store-secrets-store-csi-driver-cl7dv 3/3 Running 0 57s csi-secrets-store-secrets-store-csi-driver-gbz27 3/3 Running 0 57s
Deploy Azure Key Store CSI
Add the Azure Helm Repository
helm repo add csi-secrets-store-provider-azure \ https://azure.github.io/secrets-store-csi-driver-provider-azure/charts
Update your local Helm Repositories
helm repo update
Install the Azure Key Vault CSI provider
helm install -n k8s-secrets-store-csi azure-csi-provider \ csi-secrets-store-provider-azure/csi-secrets-store-provider-azure \ --set linux.privileged=true --set secrets-store-csi-driver.install=false \ --set "linux.providersDir=/var/run/secrets-store-csi-providers" \ --version=v1.4.1
Set SecurityContextConstraints to allow the CSI driver to run
oc adm policy add-scc-to-user privileged \ system:serviceaccount:k8s-secrets-store-csi:csi-secrets-store-provider-azure
Create Keyvault and a Secret
Create a namespace for your application
oc new-project my-application
Create an Azure Keyvault in your Resource Group that contains ARO
az keyvault create -n ${KEYVAULT_NAME} \ -g ${KEYVAULT_RESOURCE_GROUP} \ --location ${KEYVAULT_LOCATION}
Create a secret in the Keyvault
az keyvault secret set \ --vault-name ${KEYVAULT_NAME} \ --name secret1 --value "Hello"
Create a Service Principal for the keyvault
Note: If this gives you an error, you may need upgrade your Azure CLI to the latest version.
export SERVICE_PRINCIPAL_CLIENT_SECRET="$(az ad sp create-for-rbac \ --name http://$KEYVAULT_NAME --query 'password' -otsv)" export SERVICE_PRINCIPAL_CLIENT_ID="$(az ad sp list \ --display-name http://$KEYVAULT_NAME --query '[0].appId' -otsv)"
Set an Access Policy for the Service Principal
az keyvault set-policy -n ${KEYVAULT_NAME} \ --secret-permissions get \ --spn ${SERVICE_PRINCIPAL_CLIENT_ID}
Create and label a secret for Kubernetes to use to access the Key Vault
oc create secret generic secrets-store-creds \ -n my-application \ --from-literal clientid=${SERVICE_PRINCIPAL_CLIENT_ID} \ --from-literal clientsecret=${SERVICE_PRINCIPAL_CLIENT_SECRET} oc -n my-application label secret \ secrets-store-creds secrets-store.csi.k8s.io/used=true
Deploy an Application that uses the CSI
Create a Secret Provider Class to give access to this secret
cat <<EOF | oc apply -f - apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: azure-kvname namespace: my-application spec: provider: azure parameters: usePodIdentity: "false" useVMManagedIdentity: "false" userAssignedIdentityID: "" keyvaultName: "${KEYVAULT_NAME}" objects: | array: - | objectName: secret1 objectType: secret objectVersion: "" tenantId: "${AZ_TENANT_ID}" EOF
Create a Pod that uses the above Secret Provider Class
cat <<EOF | oc apply -f - kind: Pod apiVersion: v1 metadata: name: busybox-secrets-store-inline namespace: my-application spec: containers: - name: busybox image: k8s.gcr.io/e2e-test-images/busybox:1.29 command: - "/bin/sleep" - "10000" volumeMounts: - name: secrets-store-inline mountPath: "/mnt/secrets-store" readOnly: true volumes: - name: secrets-store-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "azure-kvname" nodePublishSecretRef: name: secrets-store-creds EOF
Check the Secret is mounted
oc exec busybox-secrets-store-inline -- ls /mnt/secrets-store/
Output should match:
secret1
Print the Secret
oc exec busybox-secrets-store-inline \ -- cat /mnt/secrets-store/secret1
Output should match:
Hello
Cleanup
Uninstall Helm
helm uninstall -n k8s-secrets-store-csi azure-csi-provider
Delete the app
oc delete project my-application
Delete the Azure Key Vault
az keyvault delete -n ${KEYVAULT_NAME}
Delete the Service Principal
az ad sp delete --id ${SERVICE_PRINCIPAL_CLIENT_ID}
Uninstalling the Kubernetes Secret Store CSI
Delete the secrets store csi driver
helm delete -n k8s-secrets-store-csi csi-secrets-store
Delete the SecurityContextConstraints
oc adm policy remove-scc-from-user privileged \ system:serviceaccount:k8s-secrets-store-csi:secrets-store-csi-driver