Cloud Experts Documentation

Using the Egressip Ipam Operator with a Private ARO Cluster

This content is authored by Red Hat experts, but has not yet been tested on every supported configuration.

This guide is only valid for ARO clusters created on version 4.10 or earlier.

Clusters created on version 4.11 and later use OVNKubernetes as their Container Network Interface, and egressip-ipam-operator does not support OVNKubernetes. Please see EgressIP as a possible alternative.

Prerequisites

Deploy the Egressip Ipam Operator

Via GUI

  1. Log into the ARO cluster’s Console

  2. Switch to the Administrator view

  3. Click on Operators -> Operator Hub

  4. Search for “Egressip Ipam Operator”

  5. Install it with the default settings

or

Via CLI

  1. Deploy the egress-ipam-operator

    cat << EOF | kubectl apply -f -
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: egressip-ipam-operator
    ---
    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      name: egressip-ipam-operator
      namespace: openshift-operators
      labels:
        operators.coreos.com/egressip-ipam-operator.egressip-ipam-operator: ''
    spec:
      channel: alpha
      installPlanApproval: Automatic
      name: egressip-ipam-operator
      source: community-operators
      sourceNamespace: openshift-marketplace
      startingCSV: egressip-ipam-operator.v1.2.2
    EOF
    

Configure EgressIP

  1. Create an EgressIPAM resource for your cluster. Update the CIDR to reflect the worker node subnet.

    cat << EOF | kubectl apply -f -
    apiVersion: redhatcop.redhat.io/v1alpha1
    kind: EgressIPAM
    metadata:
      name: egressipam-azure
      annotations:
        egressip-ipam-operator.redhat-cop.io/azure-egress-load-balancer: none
    spec:
      cidrAssignments:
        - labelValue: ""
          CIDR: 10.0.1.0/24
          reservedIPs: []
      topologyLabel: "node-role.kubernetes.io/worker"
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/worker: ""
    EOF
    
  2. Create test namespaces

    cat << EOF | kubectl apply -f -
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: egressipam-azure-test
      annotations:
        egressip-ipam-operator.redhat-cop.io/egressipam: egressipam-azure
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: egressipam-azure-test-1
      annotations:
        egressip-ipam-operator.redhat-cop.io/egressipam: egressipam-azure
    EOF
    
  3. Check the namespaces have IPs assigned

    kubectl get namespace egressipam-azure-test \
      egressipam-azure-test-1 -o yaml | grep egressips
    

    The output should look like:

    egressip-ipam-operator.redhat-cop.io/egressips: 10.0.1.8
    egressip-ipam-operator.redhat-cop.io/egressips: 10.0.1.7
    
  4. Check they’re actually set as Egress IPs

    oc get netnamespaces | egrep 'NAME|egress'
    

    The output should look like:

    NAME                                               NETID      EGRESS IPS
    egressip-ipam-operator                             6374875
    egressipam-azure-test                              6917470    ["10.0.1.8"]
    egressipam-azure-test-1                            16320378   ["10.0.1.7"]
    
  5. Finally check the Host Subnets for Egress IPS

    oc get hostsubnets
    

    The output should look like:

    NAME                                         HOST                                         HOST IP    SUBNET          EGRESS CIDRS   EGRESS IPS
    private-cluster-bj275-master-0               private-cluster-bj275-master-0               10.0.0.8   10.129.0.0/23
    private-cluster-bj275-master-1               private-cluster-bj275-master-1               10.0.0.7   10.128.0.0/23
    private-cluster-bj275-master-2               private-cluster-bj275-master-2               10.0.0.9   10.130.0.0/23
    private-cluster-bj275-worker-eastus1-zt59t   private-cluster-bj275-worker-eastus1-zt59t   10.0.1.4   10.128.2.0/23                  ["10.0.1.8"]
    private-cluster-bj275-worker-eastus2-bfrwt   private-cluster-bj275-worker-eastus2-bfrwt   10.0.1.5   10.129.2.0/23                  ["10.0.1.7"]
    private-cluster-bj275-worker-eastus3-fgjzk   private-cluster-bj275-worker-eastus3-fgjzk   10.0.1.6   10.131.0.0/23
    

Test Egress

  1. Log into your jumpbox and allow http into firewall

    sudo firewall-cmd --zone=public --add-service=http
    
  2. Install and start apache httpd

    sudo yum -y install httpd
    sudo systemctl start httpd
    
  3. Create a index.html

    echo HELLO | sudo tee /var/www/html/index.html
    
  4. tail apache logs

    sudo tail -f /var/log/httpd/access_log
    
  5. Start an interactive pod in one of your new namespaces

    kubectl run -n egressipam-azure-test -i \
    --tty --rm debug --image=alpine \
    --restart=Never -- wget -O - 10.0.3.4
    

    The output should look the following (the IP should match the egress IP of your namespace):

    10.0.1.7 - - [03/Feb/2022:19:33:54 +0000] "GET / HTTP/1.1" 200 6 "-" "Wget"
    

Interested in contributing to these docs?

Collaboration drives progress. Help improve our documentation The Red Hat Way.

Red Hat logo LinkedIn YouTube Facebook Twitter

Products

Tools

Try, buy & sell

Communicate

About Red Hat

We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Subscribe to our newsletter, Red Hat Shares

Sign up now
© 2023 Red Hat, Inc.