IMPORTANT NOTE: This site is not official Red Hat documentation and is provided for informational purposes only. These guides may be experimental, proof of concept, or early adoption. Officially supported documentation is available at docs.openshift.com and access.redhat.com.

Creating a ROSA cluster with Private Link enabled


Last Editor: Dustin Scott
Published Date: 11 June 2021
Modified Date: 25 May 2023


Prerequisites

Create VPC and Subnets

The following instructions use the AWS CLI to create the necessary networking to deploy a Private Link ROSA cluster into a Single AZ and are intended to be a guide. Ideally you would use an Automation tool like Ansible or Terraform to manage your VPCs.

When creating subnets, make sure that subnet(s) are created to an availability zone that has ROSA instances types available. If AZ is not “forced”, subnet is created to random AZ in the region. Force the AZ using --availability-zone argument in create-subnet command.

Use rosa list instance-types to list ROSA instance types and check available types availability in AZ with the following

aws ec2 describe-instance-type-offerings \
  --location-type availability-zone \
  --filters Name=location,Values=AZ_NAME_HERE \
  --region REGION_HERE --output text | egrep "YOU_PREFERRED_INSTANCE_TYPE"

As an example, you cannot install ROSA to us-east-1e AZ, but us-east-1b works fine.

Option 1 - VPC with a private subnet and AWS Site-to-Site VPN access.

Todo

Option 2 - VPC with public and private subnets and AWS Site-to-Site VPN access

Todo

Option 3 - VPC with public and private subnets (NAT)

This will create both a Private and Public subnet. All cluster resources will live in the private subnet, the public subnet only exists to NAT the egress traffic to the Internet.

architecture diagram showing privatelink with public subnet

As an alternative use the Terraform instructions provided here then skip down to the rosa create command.

  1. Set a Cluster name

    ROSA_CLUSTER_NAME=private-link
    
  2. Create a VPC to install a ROSA cluster into

    VPC_ID=`aws ec2 create-vpc --cidr-block 10.0.0.0/16 | jq -r .Vpc.VpcId`
    
    aws ec2 create-tags --resources $VPC_ID \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME | jq .
    
    aws ec2 modify-vpc-attribute --vpc-id $VPC_ID --enable-dns-hostnames | jq .
    
  3. Create a Public Subnet for the cluster to NAT egress traffic out of

    PUBLIC_SUBNET=`aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.128.0/17 | jq -r .Subnet.SubnetId`
    
    aws ec2 create-tags --resources $PUBLIC_SUBNET \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME-public | jq .
    
  4. Create a Private Subnet for the cluster machines to live in

    PRIVATE_SUBNET=`aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.0.0/17 | jq -r .Subnet.SubnetId`
    
    aws ec2 create-tags --resources $PRIVATE_SUBNET \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME-private | jq .
    
  5. Create an Internet Gateway for NAT egress traffic

    I_GW=`aws ec2 create-internet-gateway | jq -r .InternetGateway.InternetGatewayId`
    aws ec2 attach-internet-gateway --vpc-id $VPC_ID --internet-gateway-id $I_GW | jq .
    
    aws ec2 create-tags --resources $I_GW \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME | jq .
    
  6. Create a Route Table for NAT egress traffic

    R_TABLE=`aws ec2 create-route-table --vpc-id $VPC_ID | jq -r .RouteTable.RouteTableId`
    
    aws ec2 create-route --route-table-id $R_TABLE --destination-cidr-block 0.0.0.0/0 --gateway-id $I_GW | jq .
    
    aws ec2 describe-route-tables --route-table-id $R_TABLE | jq .
    
    aws ec2 associate-route-table --subnet-id $PUBLIC_SUBNET --route-table-id $R_TABLE | jq .
    
    aws ec2 create-tags --resources $R_TABLE \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME | jq .
    
  7. Create a NAT Gateway for the Private network

    EIP=`aws ec2 allocate-address --domain vpc | jq -r .AllocationId`
    NAT_GW=`aws ec2 create-nat-gateway --subnet-id $PUBLIC_SUBNET \
      --allocation-id $EIP | jq -r .NatGateway.NatGatewayId`
    
    aws ec2 create-tags --resources $EIP --resources $NAT_GW \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME | jq .
    
  8. Create a Route Table for the Private subnet to the NAT

    R_TABLE_NAT=`aws ec2 create-route-table --vpc-id $VPC_ID | jq -r .RouteTable.RouteTableId`
    
    while ! aws ec2 describe-route-tables --route-table-id $R_TABLE_NAT \
      | jq .; do sleep 1; done
    
    aws ec2 create-route --route-table-id $R_TABLE_NAT --destination-cidr-block 0.0.0.0/0 --gateway-id $NAT_GW | jq .
    
    aws ec2 associate-route-table --subnet-id $PRIVATE_SUBNET --route-table-id $R_TABLE_NAT | jq .
    
    aws ec2 create-tags --resources $R_TABLE_NAT $EIP \
      --tags Key=Name,Value=$ROSA_CLUSTER_NAME-private | jq .
    

Deploy ROSA

  1. Create ROSA cluster in the private subnet

    rosa create cluster --private-link \
      --cluster-name=$ROSA_CLUSTER_NAME \
      --machine-cidr=10.0.0.0/16 \
      --subnet-ids=$PRIVATE_SUBNET
    

Test Connectivity

  1. Create an Instance to use as a jump host

    TODO: CLI instructions

    Through the GUI:

    1. Navigate to the EC2 console and launch a new instance

    2. Select the AMI for your instance, if you don’t have a standard, the Amazon Linux 2 AMI works just fine

    3. Choose your instance type, the t2.micro/free tier is sufficient for our needs, and click Next: Configure Instance Details

    4. Change the Network settings to setup this host inside your private-link VPC select the private link vpc from dropdown

    5. Change the Subnet setting to use the private-link-public subnet select the subnet named private-link-public from the dropdown

    6. Change Auto-assign Public IP to Enable select Enable from dropdown

    7. Default settings for Storage and Tags are OK, if you do not need to change them for your own reasons, select 6. Configure Security Group from the top navigation or click through using the Next buttons

    8. If you already have a security group created to allow access from your computer to AWS, choose Select an existing security group and choose that group from the list and skip to Review and Launch. Otherwise, select Create a new security group and continue.

    9. To allow access only from your current public IP, change the Source heading to use My IP use the MY IP option from the Source dropdown

    10. Click Review and Launch, verify all settings are correct and follow the standard AWS instructions for finalizing the setup and selecting/creating the security keys.

    11. Once launched, open the instance summary for the jump host instance and note the public IP address.

  2. Create a ROSA admin user and save the login command for use later

    rosa create admin -c $ROSA_CLUSTER_NAME
    
  3. Note the DNS name of your private cluster, use the rosa describe command if needed

rosa describe cluster -c private-link
  1. update /etc/hosts to point the openshift domains to localhost. Use the DNS of your openshift cluster as described in the previous step in place of $YOUR_OPENSHIFT_DNS below

    127.0.0.1 api.$YOUR_OPENSHIFT_DNS
    127.0.0.1 console-openshift-console.apps.$YOUR_OPENSHIFT_DNS
    127.0.0.1 oauth-openshift.apps.$YOUR_OPENSHIFT_DNS
    
  2. SSH to that instance, tunneling traffic for the appropriate hostnames. Be sure to use your new/existing private key, the OpenShift DNS for $YOUR_OPENSHIFT_DNS and your jump host IP for $YOUR_EC2_IP

      sudo ssh -i PATH/TO/YOUR_KEY.pem \
      -L 6443:api.$YOUR_OPENSHIFT_DNS:6443 \
      -L 443:console-openshift-console.apps.$YOUR_OPENSHIFT_DNS:443 \
      -L 80:console-openshift-console.apps.$YOUR_OPENSHIFT_DNS:80 \
       ec2-user@$YOUR_EC2_IP
    
  3. Log into the cluster using oc login command from the create admin command above. ex.

    oc login https://api.private-test.3d1n.p1.openshiftapps.com:6443 --username cluster-admin --password GQSGJ-daqfN-8QNY3-tS9gU
    
  4. Check that you can access the Console by opening the console url in your browser.

Cleanup

  1. Delete ROSA

    rosa delete cluster -c $ROSA_CLUSTER_NAME -y
    
  2. Delete AWS resources

    aws ec2 delete-nat-gateway --nat-gateway-id $NAT_GW | jq .
    aws ec2 release-address --allocation-id=$EIP | jq .
    aws ec2 detach-internet-gateway --vpc-id $VPC_ID \
      --internet-gateway-id $I_GW | jq .
    aws ec2 delete-subnet --subnet-id=$PRIVATE_SUBNET | jq .
    aws ec2 delete-subnet --subnet-id=$PUBLIC_SUBNET | jq .
    aws ec2 delete-route-table --route-table-id=$R_TABLE | jq .
    aws ec2 delete-route-table --route-table-id=$R_TABLE_NAT | jq .
    aws ec2 delete-vpc --vpc-id=$VPC_ID | jq .