IMPORTANT NOTE: This site is not official Red Hat documentation and is provided for informational purposes only. These guides may be experimental, proof of concept, or early adoption. Officially supported documentation is available at docs.openshift.com and access.redhat.com.

Advanced Cluster Management Observability on ROSA


Authors: Connor Wooley
Last Editor: Dustin Scott
Published Date: 11 January 2022
Modified Date: 25 May 2023


This document will take you through deploying ACM Observability on a ROSA cluster. see here for the original documentation.

Prerequisites

  • An existing ROSA cluster
  • An Advanced Cluster Management (ACM) deployment

Set up environment

  1. Set environment variables

    export CLUSTER_NAME=my-cluster
    export S3_BUCKET=$CLUSTER_NAME-acm-observability
    export REGION=us-east-2
    export NAMESPACE=open-cluster-management-observability
    export SA=tbd
    export SCRATCH_DIR=/tmp/scratch
    export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
    export AWS_PAGER=""
    rm -rf $SCRATCH_DIR
    mkdir -p $SCRATCH_DIR
    

Prepare AWS Account

  1. Create an S3 bucket

    aws s3 mb s3://$S3_BUCKET
    
  2. Create a Policy for access to S3

    cat <<EOF > $SCRATCH_DIR/s3-policy.json
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "Statement",
                "Effect": "Allow",
                "Action": [
                    "s3:ListBucket",
                    "s3:GetObject",
                    "s3:DeleteObject",
                    "s3:PutObject",
                    "s3:PutObjectAcl",
                    "s3:CreateBucket",
                    "s3:DeleteBucket"
                ],
                "Resource": [
                    "arn:aws:s3:::$S3_BUCKET/*",
                    "arn:aws:s3:::$S3_BUCKET"
                ]
            }
        ]
    }
    EOF
    
  3. Apply the Policy

    S3_POLICY=$(aws iam create-policy --policy-name $CLUSTER_NAME-acm-obs \
      --policy-document file://$SCRATCH_DIR/s3-policy.json \
      --query 'Policy.Arn' --output text)
    echo $S3_POLICY
    
  4. Create service account

    aws iam create-user --user-name $CLUSTER_NAME-acm-obs  \
      --query User.Arn --output text
    
  5. Attach policy to user

    aws iam attach-user-policy --user-name $CLUSTER_NAME-acm-obs \
      --policy-arn ${S3_POLICY}
    
  6. Create Access Keys

    read -r ACCESS_KEY_ID ACCESS_KEY < <(aws iam create-access-key \
      --user-name $CLUSTER_NAME-acm-obs \
      --query 'AccessKey.[AccessKeyId,SecretAccessKey]' --output text)
    

ACM Hub

Log into the OpenShift cluster that is running your ACM Hub. We’ll set up Observability here

  1. Create a namespace for the observability

    oc new-project $NAMESPACE
    
  2. Generate a pull secret (this will check if the pull secret exists, if not, it will create it)

    DOCKER_CONFIG_JSON=`oc extract secret/multiclusterhub-operator-pull-secret -n open-cluster-management --to=-` || \
      DOCKER_CONFIG_JSON=`oc extract secret/pull-secret -n openshift-config --to=-` && \
      oc create secret generic multiclusterhub-operator-pull-secret \
      -n open-cluster-management-observability \
      --from-literal=.dockerconfigjson="$DOCKER_CONFIG_JSON" \
      --type=kubernetes.io/dockerconfigjson
    
  3. Create a Secret containing your S3 details

    cat << EOF | kubectl apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: thanos-object-storage
      namespace: open-cluster-management-observability
    type: Opaque
    stringData:
      thanos.yaml: |
        type: s3
        config:
          bucket: $S3_BUCKET
          endpoint: s3.$REGION.amazonaws.com
          signature_version2: false
          access_key: $ACCESS_KEY_ID
          secret_key: $ACCESS_KEY    
    EOF
    
  4. Create a CR for MulticlusterHub

    cat << EOF | kubectl apply -f -
    apiVersion: observability.open-cluster-management.io/v1beta2
    kind: MultiClusterObservability
    metadata:
      name: observability
    spec:
      observabilityAddonSpec: {}
      storageConfig:
        metricObjectStorage:
          name: thanos-object-storage
          key: thanos.yaml
    EOF
    

Access ACM Observability

  1. Log into Advanced Cluster management and access the new Grafana dashboard

ACM Grafana Dashboard