Using the Egressip Ipam Operator with a Private ARO Cluster
This content is authored by Red Hat experts, but has not yet been tested on every supported configuration.
This guide is only valid for ARO clusters created on version 4.10 or earlier.
Clusters created on version 4.11 and later use OVNKubernetes as their Container Network Interface, and egressip-ipam-operator does not support OVNKubernetes.
In addition, please refer
here
to create a private ARO cluster without using public IP address. This way, you will be using UserDefinedRouting
for
egress
.
Prerequisites
- A private ARO cluster that uses OpenShift SDN as its CNI
Deploy the Egressip Ipam Operator
Via GUI
Log into the ARO cluster’s Console
Switch to the Administrator view
Click on Operators -> Operator Hub
Search for “Egressip Ipam Operator”
Install it with the default settings
or
Via CLI
Deploy the
egress-ipam-operator
cat << EOF | kubectl apply -f - --- apiVersion: v1 kind: Namespace metadata: name: egressip-ipam-operator --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: egressip-ipam-operator namespace: openshift-operators labels: operators.coreos.com/egressip-ipam-operator.egressip-ipam-operator: '' spec: channel: alpha installPlanApproval: Automatic name: egressip-ipam-operator source: community-operators sourceNamespace: openshift-marketplace startingCSV: egressip-ipam-operator.v1.2.2 EOF
Copy
Configure EgressIP
Create an EgressIPAM resource for your cluster. Update the CIDR to reflect the worker node subnet.
cat << EOF | kubectl apply -f - apiVersion: redhatcop.redhat.io/v1alpha1 kind: EgressIPAM metadata: name: egressipam-azure annotations: egressip-ipam-operator.redhat-cop.io/azure-egress-load-balancer: none spec: cidrAssignments: - labelValue: "" CIDR: 10.0.1.0/24 reservedIPs: [] topologyLabel: "node-role.kubernetes.io/worker" nodeSelector: matchLabels: node-role.kubernetes.io/worker: "" EOF
CopyCreate test namespaces
cat << EOF | kubectl apply -f - --- apiVersion: v1 kind: Namespace metadata: name: egressipam-azure-test annotations: egressip-ipam-operator.redhat-cop.io/egressipam: egressipam-azure --- apiVersion: v1 kind: Namespace metadata: name: egressipam-azure-test-1 annotations: egressip-ipam-operator.redhat-cop.io/egressipam: egressipam-azure EOF
CopyCheck the namespaces have IPs assigned
kubectl get namespace egressipam-azure-test \ egressipam-azure-test-1 -o yaml | grep egressips
CopyThe output should look like:
egressip-ipam-operator.redhat-cop.io/egressips: 10.0.1.8 egressip-ipam-operator.redhat-cop.io/egressips: 10.0.1.7
CopyCheck they’re actually set as Egress IPs
oc get netnamespaces | egrep 'NAME|egress'
CopyThe output should look like:
NAME NETID EGRESS IPS egressip-ipam-operator 6374875 egressipam-azure-test 6917470 ["10.0.1.8"] egressipam-azure-test-1 16320378 ["10.0.1.7"]
CopyFinally check the Host Subnets for Egress IPS
oc get hostsubnets
CopyThe output should look like:
NAME HOST HOST IP SUBNET EGRESS CIDRS EGRESS IPS private-cluster-bj275-master-0 private-cluster-bj275-master-0 10.0.0.8 10.129.0.0/23 private-cluster-bj275-master-1 private-cluster-bj275-master-1 10.0.0.7 10.128.0.0/23 private-cluster-bj275-master-2 private-cluster-bj275-master-2 10.0.0.9 10.130.0.0/23 private-cluster-bj275-worker-eastus1-zt59t private-cluster-bj275-worker-eastus1-zt59t 10.0.1.4 10.128.2.0/23 ["10.0.1.8"] private-cluster-bj275-worker-eastus2-bfrwt private-cluster-bj275-worker-eastus2-bfrwt 10.0.1.5 10.129.2.0/23 ["10.0.1.7"] private-cluster-bj275-worker-eastus3-fgjzk private-cluster-bj275-worker-eastus3-fgjzk 10.0.1.6 10.131.0.0/23
Copy
Test Egress
Log into your jumpbox and allow http into firewall
sudo firewall-cmd --zone=public --add-service=http
CopyInstall and start apache httpd
sudo yum -y install httpd sudo systemctl start httpd
CopyCreate a index.html
echo HELLO | sudo tee /var/www/html/index.html
Copytail apache logs
sudo tail -f /var/log/httpd/access_log
CopyStart an interactive pod in one of your new namespaces
kubectl run -n egressipam-azure-test -i \ --tty --rm debug --image=alpine \ --restart=Never -- wget -O - 10.0.3.4
CopyThe output should look the following (the IP should match the egress IP of your namespace):
10.0.1.7 - - [03/Feb/2022:19:33:54 +0000] "GET / HTTP/1.1" 200 6 "-" "Wget"
Copy