Home GitHub

Using CloudFront + WAF

Problem Statement

  1. Operator requires WAF (Web Application Firewall) in front of their workloads running on OpenShift (ROSA)

  2. Operator does not want WAF running on OpenShift to ensure that OCP resources do not experience Denial of Service through handling the WAF

Proposed Solution

  1. Add a CustomDomain resource to the cluster using a wildcard DNS and TLS certificate.

  2. Set the Wildcard DNS CNAME’s to CloudFront and enable the CloudFront + WAF services to reverse proxy and inspect the traffic before sending it to the cluster.

Preparation

  1. Create a cluster

     rosa create cluster --cluster-name poc-waf --multi-az \
       --region us-east-2 --version 4.7.9 --compute-nodes 3 \
       --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 \
       --pod-cidr 10.128.0.0/14 --host-prefix 23
    
  2. When its ready create a admin user and follow the instructions to log in

     rosa create admin -c poc-waf
    
  3. Set some environment variables

     EMAIL=username.taken@gmail.com
     DOMAIN=waf.mobb.ninja
    

Certificate and DNS

  1. Use certbot to create a wildcard cert

     certbot certonly --manual \
       --preferred-challenges=dns \
       --email $EMAIL \
       --server https://acme-v02.api.letsencrypt.org/directory \
       --agree-tos \
       --manual-public-ip-logging-ok \
       -d "*.$DOMAIN"
    
  2. Follow Certbot’s instructions to create a DNS TXT record. certificate records will be saved on your system, in my case in /etc/letsencrypt/live/waf.mobb.ninja/. set that as an enviroment variable.

     CERTS=/etc/letsencrypt/live/waf.mobb.ninja
    

Custom OpenShift Domain

  1. Create a project and add the certs as a secret

     oc new-project my-custom-route
     oc create secret tls acme-tls --cert=$CERTS/fullchain1.pem --key=$CERTS/privkey1.pem
    
  2. Create a Custom Domain resource

     cat << EOF | oc apply -f -
     apiVersion: managed.openshift.io/v1alpha1
     kind: CustomDomain
     metadata:
       name: acme
     spec:
       domain: $DOMAIN
       certificate:
         name: acme-tls
         namespace: my-custom-route
     EOF
    
  3. Wait until your Custom Domain has an Endpoint

     watch  oc get customdomains
    

AWS WAF + CloudFront

  1. Create a WAF rule here https://console.aws.amazon.com/wafv2/homev2/web-acls/new?region=us-east-2 and use the Core and SQL Injection rules and set it as a CloudFront distribution resource type.

  2. View your WAF

     aws wafv2 list-web-acls --scope REGIONAL --region us-east-2 | jq .
    
  3. Add a certificate to ACM - https://us-east-2.console.aws.amazon.com/acm/home?region=us-east-1#/importwizard/, Paste in the cert, key, certchain from the files certbot game you.

    Make sure you create it in the US-EAST-1 region (otherwise cloud front can’t use it)

  4. Log into the AWS console and Create a Cloud Front distribution (make sure its the same region as your cluster).

    • Origin Domain Name:
    • Origin Protocol Policy: HTTPS only
    • Viewer Protocol Policy: Redirect HTTP to HTTPS
    • Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE
    • AWS WAF Web ACL: demo-waf-acl
    • Alternate Domain Names: *.
    • Custom SSL Certificate:
    • Origin Request Policy: create a new policy whitelist: Origin, user-agent, referer, host (IMPORTANT)
  5. Hit Create then wait until the Status is Ready.

DNS CNAME

  1. Create a CNAME in your DNS provider for *.<$DOMAIN> that points at the endpoint from the above status page. It should look something like d1vm7mfs9sc24l.cloudfront.net.

Deploy an Application

  1. Create a new application

     oc new-app --docker-image=docker.io/openshift/hello-openshift
    
  2. Create a route for the application

     oc create route edge --service=hello-openshift hello-openshift-tls \
         --hostname hello.waf.mobb.ninja
    

Test the WAF

  1. Make sure you can access your application with curl

     curl https://hello.waf.mobb.ninja
    

    You should get a simple hello response

     Hello OpenShift!
    
  2. Try do a XSS injection

     curl -X POST https://hello.waf.mobb.ninja \
       -F "user='<script><alert>Hello></alert></script>'"
    

    you should see this

     <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
     <HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
     <TITLE>ERROR: The request could not be satisfied</TITLE>
     </HEAD><BODY>
     <H1>403 ERROR</H1>