Home GitHub

IMPORTANT NOTE: This site is not official Red Hat documentation and is provided for informational purposes only. These guides may be experimental, proof of concept, or early adoption. Officially supported documentation is available at docs.openshift.com and access.redhat.com.

Using the Egressip Ipam Operator with a Private ARO Cluster

Prerequisites

Deploy the Egressip Ipam Operator

Via GUI

  1. Log into the ARO cluster’s Console

  2. Switch to the Administrator view

  3. Click on Operators -> Operator Hub

  4. Search for “Egressip Ipam Operator”

  5. Install it with the default settings

or

Via CLI

  1. Deploy the egress-ipam-operator

    cat << EOF | kubectl apply -f -
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: egressip-ipam-operator
    ---
    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: egressip-ipam-operator
      namespace: openshift-operators
      labels:
     operators.coreos.com/egressip-ipam-operator.egressip-ipam-operator: ''
    spec:
      channel: alpha
      installPlanApproval: Automatic
      name: egressip-ipam-operator
      source: community-operators
      sourceNamespace: openshift-marketplace
      startingCSV: egressip-ipam-operator.v1.2.2
    EOF
    

Configure EgressIP

  1. Create an EgressIPAM resource for your cluster. Update the CIDR to reflect the worker node subnet.

    cat << EOF | kubectl apply -f -
    apiVersion: redhatcop.redhat.io/v1alpha1
    kind: EgressIPAM
    metadata:
      name: egressipam-azure
      annotations:
       egressip-ipam-operator.redhat-cop.io/azure-egress-load-balancer: none
    spec:
      # Add fields here
      cidrAssignments:
     - labelValue: ""
       CIDR: 10.0.1.0/24
       reservedIPs: []
      topologyLabel: "node-role.kubernetes.io/worker"
      nodeSelector:
     matchLabels:
       node-role.kubernetes.io/worker: ""
    EOF
    
  2. Create test namespaces

    cat << EOF | kubectl apply -f -
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: egressipam-azure-test
      annotations:
     egressip-ipam-operator.redhat-cop.io/egressipam: egressipam-azure
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: egressipam-azure-test-1
      annotations:
     egressip-ipam-operator.redhat-cop.io/egressipam: egressipam-azure
    EOF
    
  3. Check the namespaces have IPs assigned

    kubectl get namespace egressipam-azure-test \
      egressipam-azure-test-1 -o yaml | grep egressips
    

    The output should look like:

     egressip-ipam-operator.redhat-cop.io/egressips: 10.0.1.8
     egressip-ipam-operator.redhat-cop.io/egressips: 10.0.1.7
    
  4. Check they’re actually set as Egress IPs

    oc get netnamespaces | egrep 'NAME|egress'
    

    The output should look like:

     NAME                                               NETID      EGRESS IPS
     egressip-ipam-operator                             6374875
     egressipam-azure-test                              6917470    ["10.0.1.8"]
     egressipam-azure-test-1                            16320378   ["10.0.1.7"]
    
  5. Finally check the Host Subnets for Egress IPS

    oc get hostsubnets
    

    The output should look like:

     NAME                                         HOST                                         HOST IP    SUBNET          EGRESS CIDRS   EGRESS IPS
     private-cluster-bj275-master-0               private-cluster-bj275-master-0               10.0.0.8   10.129.0.0/23
     private-cluster-bj275-master-1               private-cluster-bj275-master-1               10.0.0.7   10.128.0.0/23
     private-cluster-bj275-master-2               private-cluster-bj275-master-2               10.0.0.9   10.130.0.0/23
     private-cluster-bj275-worker-eastus1-zt59t   private-cluster-bj275-worker-eastus1-zt59t   10.0.1.4   10.128.2.0/23                  ["10.0.1.8"]
     private-cluster-bj275-worker-eastus2-bfrwt   private-cluster-bj275-worker-eastus2-bfrwt   10.0.1.5   10.129.2.0/23                  ["10.0.1.7"]
     private-cluster-bj275-worker-eastus3-fgjzk   private-cluster-bj275-worker-eastus3-fgjzk   10.0.1.6   10.131.0.0/23
    

Test Egress

  1. Log into your jumpbox and allow http into firewall

     sudo firewall-cmd --zone=public --add-service=http
    
  2. Install and start apache httpd

     sudo yum -y install httpd
     sudo systemctl start httpd
    
  3. Create a index.html

     echo HELLO | sudo tee /var/www/html/index.html
    
  4. tail apache logs

     sudo tail -f /var/log/httpd/access_log
    
  5. Start an interactive pod in one of your new namespaces

     kubectl run -n egressipam-azure-test -i \
       --tty --rm debug --image=alpine \
       --restart=Never -- wget -O - 10.0.3.4
    

    The output should look the following (the IP should match the egress IP of your namespace):

     10.0.1.7 - - [03/Feb/2022:19:33:54 +0000] "GET / HTTP/1.1" 200 6 "-" "Wget"
    
  6. Deploy a hello world pod

    oc new-project hello
    oc new-app --docker-image=docker.io/openshift/hello-openshift
    oc expose service/hello-openshift
    
  7. Create an interactive pod in one of the new namespaces

    kubectl run -n egressipam-azure-test -i --tty --rm debug --image=fedora --restart=Never
    
  8. Create

oc new-project hello oc new-app –docker-image=docker.io/openshift/hello-openshift oc create route edge –service=hello-openshift hello-openshift-tls
–hostname hello.apps.reinvent.aws.mobb.ninja