Cloud Experts Documentation

Azure Key Vault CSI on Azure Red Hat OpenShift

This content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

This document is adapted from the Azure Key Vault CSI Walkthroughexternal link (opens in new tab) specifically to run with Azure Red Hat OpenShift (ARO).

Prerequisites

  1. An ARO cluster
  2. The AZ CLI (logged in)
  3. The OC CLI (logged in)
  4. Helm 3.x CLI

Environment Variables

  1. Run this command to set some environment variables to use throughout

    Note if you created the cluster from the instructions linked above these will re-use the same environment variables, or default them to openshift and eastus.

    export KEYVAULT_RESOURCE_GROUP=${AZR_RESOURCE_GROUP:-"openshift"}
    export KEYVAULT_LOCATION=${AZR_RESOURCE_LOCATION:-"eastus"}
    export KEYVAULT_NAME=secret-store-$(cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 10 | head -n 1)
    export AZ_TENANT_ID=$(az account show -o tsv --query tenantId)
    export AZ_SUB_ID=$(az account show -o tsv --query id)
    
    Copy

Installing the Kubernetes Secret Store CSI

  1. Create an OpenShift Project to deploy the CSI into

    oc new-project k8s-secrets-store-csi
    
    Copy
  2. Set SecurityContextConstraints to allow the CSI driver to run (otherwise the DaemonSet will not be able to create Pods)

    oc adm policy add-scc-to-user privileged \
      system:serviceaccount:k8s-secrets-store-csi:secrets-store-csi-driver
    
    Copy
  3. Add the Secrets Store CSI Driver to your Helm Repositories

    helm repo add secrets-store-csi-driver \
      https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
    
    Copy
  4. Update your Helm Repositories

    helm repo update
    
    Copy
  5. Install the secrets store csi driver

    helm install -n k8s-secrets-store-csi csi-secrets-store \
      secrets-store-csi-driver/secrets-store-csi-driver \
      --version v1.3.2 \
      --set "linux.providersDir=/var/run/secrets-store-csi-providers"
    
    Copy
  6. Check that the Daemonsets is running

    oc -n k8s-secrets-store-csi get pods -l "app=secrets-store-csi-driver"
    
    Copy

    You should see the following

    NAME                                               READY   STATUS    RESTARTS   AGE
    csi-secrets-store-secrets-store-csi-driver-cl7dv   3/3     Running   0          57s
    csi-secrets-store-secrets-store-csi-driver-gbz27   3/3     Running   0          57s
    
    Copy
  7. Add pod security profile label for CSI Driver

    This is required starting in OpenShift v4.13
    oc label csidriver/secrets-store.csi.k8s.io security.openshift.io/csi-ephemeral-volume-profile=restricted
    
    Copy

Deploy Azure Key Store CSI

  1. Add the Azure Helm Repository

    helm repo add csi-secrets-store-provider-azure \
      https://azure.github.io/secrets-store-csi-driver-provider-azure/charts
    
    Copy
  2. Update your local Helm Repositories

    helm repo update
    
    Copy
  3. Install the Azure Key Vault CSI provider

    helm install -n k8s-secrets-store-csi azure-csi-provider \
      csi-secrets-store-provider-azure/csi-secrets-store-provider-azure \
      --set linux.privileged=true --set secrets-store-csi-driver.install=false \
      --set "linux.providersDir=/var/run/secrets-store-csi-providers" \
      --version=v1.4.1
    
    Copy
  4. Set SecurityContextConstraints to allow the CSI driver to run

    oc adm policy add-scc-to-user privileged \
      system:serviceaccount:k8s-secrets-store-csi:csi-secrets-store-provider-azure
    
    Copy

Create Keyvault and a Secret

  1. Create a namespace for your application

    oc new-project my-application
    
    Copy
  2. Create an Azure Keyvault in your Resource Group that contains ARO

    az keyvault create -n ${KEYVAULT_NAME} \
      -g ${KEYVAULT_RESOURCE_GROUP} \
      --location ${KEYVAULT_LOCATION}
    
    Copy
  3. Give your user account permissions to manage secrets in Key Vault

    az role assignment create --role "Key Vault Administrator" \
     --assignee "<your-email-address>" \
     --scope “/subscriptions/$SUB_ID/resourcegroups/$KEYVAULT_RESOURCE_GROUP/providers/microsoft.keyvault/vaults/$KEYVAULT_NAME"
    
    Copy

    Replace <your-email-address> with your actual value, which is your sign-in name.

  4. Create a secret in the Keyvault

    az keyvault secret set \
      --vault-name ${KEYVAULT_NAME} \
      --name secret1 --value "Hello"
    
    Copy
  5. Create a Service Principal for the key Vault

    Note: If this gives you an error, you may need upgrade your Azure CLI to the latest version.

    export SERVICE_PRINCIPAL_CLIENT_SECRET="$(az ad sp create-for-rbac \
      --name http://$KEYVAULT_NAME --query 'password' -otsv)"
    
    export SERVICE_PRINCIPAL_CLIENT_ID="$(az ad sp list \
      --display-name http://$KEYVAULT_NAME --query '[0].appId' -otsv)"
    
    Copy
  6. Give the Service Principal permissions to use secrets in Key Vault

    az role assignment create --role "Key Vault Secrets User" \
      --assignee ${SERVICE_PRINCIPAL_CLIENT_ID} \
      --scope “/subscriptions/$SUB_ID/resourcegroups/$KEYVAULT_RESOURCE_GROUP/providers/microsoft.keyvault/vaults/$KEYVAULT_NAME" 
    
    Copy
  7. Create and label a secret for Kubernetes to use to access the Key Vault

    oc create secret generic secrets-store-creds \
      -n my-application \
      --from-literal clientid=${SERVICE_PRINCIPAL_CLIENT_ID} \
      --from-literal clientsecret=${SERVICE_PRINCIPAL_CLIENT_SECRET}
    
    oc -n my-application label secret \
      secrets-store-creds secrets-store.csi.k8s.io/used=true
    
    Copy

Deploy an Application that uses the CSI

  1. Create a Secret Provider Class to give access to this secret

    cat <<EOF | oc apply -f -
    apiVersion: secrets-store.csi.x-k8s.io/v1
    kind: SecretProviderClass
    metadata:
      name: azure-kvname
      namespace: my-application
    spec:
      provider: azure
      parameters:
        usePodIdentity: "false"
        useVMManagedIdentity: "false"
        userAssignedIdentityID: ""
        keyvaultName: "${KEYVAULT_NAME}"
        objects: |
          array:
            - |
              objectName: secret1
              objectType: secret
              objectVersion: ""
        tenantId: "${AZ_TENANT_ID}"
    EOF
    
    Copy
  2. Create a Pod that uses the above Secret Provider Class

    cat <<EOF | oc apply -f -
    kind: Pod
    apiVersion: v1
    metadata:
      name: busybox-secrets-store-inline
      namespace: my-application
    spec:
      containers:
      - name: busybox
        image: k8s.gcr.io/e2e-test-images/busybox:1.29
        command:
          - "/bin/sleep"
          - "10000"
        volumeMounts:
        - name: secrets-store-inline
          mountPath: "/mnt/secrets-store"
          readOnly: true
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "azure-kvname"
            nodePublishSecretRef:
              name: secrets-store-creds
    EOF
    
    Copy
  3. Check the Secret is mounted

    oc exec busybox-secrets-store-inline -- ls /mnt/secrets-store/
    
    Copy

    Output should match:

    secret1
    
    Copy
  4. Print the Secret

    oc exec busybox-secrets-store-inline \
      -- cat /mnt/secrets-store/secret1
    
    Copy

    Output should match:

    Hello
    
    Copy

Cleanup

  1. Uninstall Helm

    helm uninstall -n k8s-secrets-store-csi azure-csi-provider
    
    Copy
  2. Delete the app

    oc delete project my-application
    
    Copy
  3. Delete the Azure Key Vault

    az keyvault delete -n ${KEYVAULT_NAME}
    
    Copy
  4. Delete the Service Principal

    az ad sp delete --id ${SERVICE_PRINCIPAL_CLIENT_ID}
    
    Copy

Uninstalling the Kubernetes Secret Store CSI

  1. Delete the secrets store csi driver

    helm delete -n k8s-secrets-store-csi csi-secrets-store
    
    Copy
  2. Delete the SecurityContextConstraints

    oc adm policy remove-scc-from-user privileged \
      system:serviceaccount:k8s-secrets-store-csi:secrets-store-csi-driver
    
    Copy

Interested in contributing to these docs?

Collaboration drives progress. Help improve our documentation The Red Hat Way.

Red Hat logo LinkedIn YouTube Facebook Twitter

Products

Tools

Try, buy & sell

Communicate

About Red Hat

We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Subscribe to our newsletter, Red Hat Shares

Sign up now
© 2023 Red Hat, Inc.