Repository files navigation

This guide is intended to demonstrate how to perform the OpenShift installation using the IPI method on AWS GovCloud. In addition, the guide will walk through performing this installation on a fresh GovCloud account. If you already have a VPC setup and subnets, please skip to Installing Openshift. Additionally this demostrates the install of Operator Catalog with addition to the Red Hat Gov Operator Catalog

For this demo, the AWS API communication is facilitated by a squid proxy. Without that access, we will not be able to install a cloud aware OpenShift cluster.

A Cloud Formation template that details the VPC with squid proxy used in this demo can be foundhere. This will be needed to install Openshift, bundle the images, and create the AMI. The following ports will need to be open to communicate with the OCP nodes and AWS API once the EC2 is running.

This guide will assume that the user has valid accounts and subscriptions to both Red Hat OpenShift and AWS GovCloud. In this demo we are using Cluster Admin credentials in AWS, but can use the VM Import/Export role + EC2 LB + Route 53 (if using for DNS).

You will need to install podman to pull the images, skopeo to copy the additional images, and jq for your pull secret.

sudo yum install podman skopeo jq -y

1. Use AWS Session Manager to connect to the EC2 squid proxy that seconds as the bootstramp. Download and compress the bundle on internet connected machine using the OpenShift4-mirror companion utility foundhere

   You will first need to retrieve an OpenShift pull secret. Once you have retrieved that, enter it into the literals of the value for--pull-secret in the command below. Pull secrets can be obtained fromhttps://cloud.redhat.com/openshift/install/aws/installer-provisioned

   OCP_VER=$(curl http://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable/release.txt2>&1| grep -oP"(?<=Version:\s\s).*")podman run -it --security-opt label=disable -v ./:/app/bundle quay.io/redhatgov/openshift4_mirror:latest \ ./openshift_mirror bundle \ --openshift-version${OCP_VER} \ --platform aws \ --skip-existing \ --skip-catalogs \ --pull-secret${PULL_SECRET}&& \git clone https://github.com/redhat-cop/ocp-disconnected-docs.git ./${OCP_VER}/ocp-disconnected&& \tar -zcvf openshift-${OCP_VER}.tar.gz${OCP_VER}

2. Transfer bundle from internet connected machine to disconnected vpc host.

3. Extract bundle on disconnected vpc host. From the directory containing the OCP bundle.

   OCP_VER=$(ls| grep -oP'(?<=openshift-)\d\.\d\.\d(?=.tar.gz)')    tar -xzvf openshift-${OCP_VER}.tar.gz

4. Create S3 Bucket and attach policies.

   export awsreg=$(aws configure get region)export s3name=$(date +%s"-rhcos")aws s3api create-bucket --bucket${s3name} --region${awsreg} --create-bucket-configuration LocationConstraint=${awsreg}aws iam create-role --role-name vmimport --assume-role-policy-document"file://${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniam/trust-policy.json"envsubst< ./${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniam/role-policy-templ.json> ./${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniam/role-policy.jsonaws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document"file://${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniam/role-policy.json"

5. Upload RHCOS Image to S3

   export RHCOS_VER=$(ls ./${OCP_VER}/rhcos/| grep -oP'.*(?=\.vmdk.gz)')gzip -d ./${OCP_VER}/rhcos/${RHCOS_VER}.vmdk.gzaws s3 mv ./${OCP_VER}/rhcos/${RHCOS_VER}.vmdk s3://${s3name}

6. Create AMI

   envsubst< ./${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniam/containers-templ.json> ./${OCP_VER}/ocp-disconnected/containers.jsontaskid=$(aws ec2 import-snapshot --region${awsreg} --description"rhcos-snapshot" --disk-container file://${OCP_VER}/ocp-disconnected/containers.json| jq -r'.ImportTaskId')until [[$resp=="completed" ]];do sleep 2;echo"Snapshot progress:"$(aws ec2 describe-import-snapshot-tasks --region${awsreg}| jq --arg task"$taskid" -r'.ImportSnapshotTasks[] | select(.ImportTaskId==$task) | .SnapshotTaskDetail.Progress')"%"; resp=$(aws ec2 describe-import-snapshot-tasks --region${awsreg}| jq --arg task"$taskid" -r'.ImportSnapshotTasks[] | select(.ImportTaskId==$task) | .SnapshotTaskDetail.Status');donesnapid=$(aws ec2 describe-import-snapshot-tasks --region${awsreg}| jq --arg task"$taskid"'.ImportSnapshotTasks[] | select(.ImportTaskId==$task) | .SnapshotTaskDetail.SnapshotId')aws ec2 register-image \  --region${awsreg} \  --architecture x86_64 \  --description"${RHCOS_VER}" \  --ena-support \  --name"${RHCOS_VER}" \  --virtualization-type hvm \  --root-device-name'/dev/xvda' \  --block-device-mappings'DeviceName=/dev/xvda,Ebs={DeleteOnTermination=true,SnapshotId='${snapid}'}'

7. Record the AMI ID from the output of the above command.

8. Create registry cert on disconnected vpc host

   export SUBJ="/C=US/ST=Virginia/O=Red Hat/CN=${HOSTNAME}"openssl req -newkey rsa:4096 -nodes -sha256 -keyout registry.key -x509 -days 365 -out registry.crt -subj"$SUBJ" -addext"subjectAltName = DNS:$HOSTNAME"

9. Make a copy of the install config

   mkdir ./${OCP_VER}/configcp ./${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniam/install-config-template.yaml ./${OCP_VER}/config/install-config.yaml

10. Edit install configFor this step, Open./${OCP_VER}/config/install-config.yaml and edit the following fields:

    baseDomain:i.e. example.comadditionalTrustBundle:copy and paste the content of ./registry.crt here.imageContentSources:mirrors:Only edit the registry hostname fields of this section. Make sure that you use the $HOSTNAME of the devices that you are currently using.metadata:name:i.e. test-clusternetworking:machineNetwork:  -cidr:i.e. 10.0.41.0/20. Shorten or lengthen this list as needed.platform:aws:region:the default region of your configured aws clizones:A list of availability zones that you are deploying into. Shorten or lengthen this list as needed.subnets:i.e. subnet-ef12d288. The length of this list must match the .networking.machineNetwork[].cidr length.amiID:the AMI ID recorded from step 9pullSecret:your pull secret enclosed in literalssshKey:i.e ssh-rsa AAAAB3... No quotes

    Don't forget to save and close the file!

11. Make a backup of the final config:

    cp -R ./${OCP_VER}/config/ ./${OCP_VER}/config.bak

12. Create manifests from install config.

    openshift-install create manifests --dir ./${OCP_VER}/config

13. create iam users and Policies

    cd ./${OCP_VER}/ocp-disconnected/aws-gov-ipi-dis-maniamchmod +x ./ocp-users.sh./ocp-users.sh prepPolicies./ocp-users.sh createUsers

14. Use the convenience script to create the aws credentials and kubernetes secrets:

    chmod +x ./secret-helper.sh./secret-helper.shcp secrets/* ../../config/openshift/cd -

15. start up the registry in the background

    oc image serve --dir=./${OCP_VER}/release/ --tls-crt=./registry.crt --tls-key=./registry.key&

16. Deploy the cluster

    openshift-install create cluster --dir ./${OCP_VER}/config

You can now access the cluster via CLI with oc or the web console with a web browser.

1. Locate the OpenShift access information provided by the final installer output.

   Example:

   INFO Waiting up to 10m0s for the openshift-console route to be created... INFO Install complete!                            INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/ec2-user/data/vid-pres/${OCP_VER}/config/auth/kubeconfig' INFO Access the OpenShift web-console here: https://console-openshift-console.apps.test-cluster.testocp1.net INFO Login to the console with user: "kubeadmin", and password: "z9yDP-2M6DS-oE9Im-Dcdzk" INFO Time elapsed: 48m34s

2. Set the default kube context used by oc and kubectl:

   Example:

   export KUBECONFIG=/home/ec2-user/data/vid-pres/4.7.0/config/auth/kubeconfig

   Config file optionaly availible at$OCP_VER/config/auth

3. Access the web console:

   URL Example:https://console-openshift-console.apps.test-cluster.testocp1.net

   Credentials Example:

   INFO Login to the console with user: "kubeadmin", and password: "z9yDP-2M6DS-oE9Im-Dcdzk

21. Create a RHEL 7.9 EC2 from the AWS console within your public subnet to be our local container registry

22. Please SSH into EC2 and register RHEL instance will subscription manager

    sudo subscription-manager register --auto-attach

23. Install Podman & Skopeo & JQ

    sudo yum install -y podman httpd-tools skopeo jq git

24. Create folders for the registry

    sudo mkdir -p /var/lib/registry

25. Deploy local podman registry

    sudo podman run --privileged -d --name registry -p 5000:5000 -v /var/lib/registry:/var/lib/registry --restart=always registry:2

26. Allow traffic from firewall

    sudo firewall-cmd --add-port=5000/tcp --zone=internal --permanentsudo firewall-cmd --add-port=5000/tcp --zone=public --permanentsudo firewall-cmd --reload

27. Add Red Hat pull secret to Podman creditials fileCopy pull secret from cloud.redhat.com and place it in the EC2 as pull.yaml

    sudo jq . pull.yaml >> /root/.docker/config.json

28. Clone Repo & Start the images transfer (this will take a while!)

    git clone https://github.com/afouladi7/disconnected_software_factory.gitsudo ./disconnected_images_transfer.sh

29. Update the machineconfig on Openshift to use the new local mirrorLogin to OC via the token, located in the top right of the OCP console.

    ./local_reg.sh./machine_config.sh

30. Wait till the master and workers have been updated. If you do see some pods still in a failing or image pull back state. Delete them and they should pull from the correct location.

31. Create aCatalogSource to import the RedHatGov operator catalog.

  oc apply -f - << EOF  apiVersion: operators.coreos.com/v1alpha1  kind: CatalogSource  metadata:    name: redhatgov-operators    namespace: openshift-marketplace  spec:    sourceType: grpc    image: quay.io/redhatgov/operator-catalog:latest    displayName: Red Hat NAPS Community Operators    publisher: RedHatGov  EOF

32. Create a project for your pipeline tooling to live.

oc new-project devsecops

33. Ploigos is hungry - delete anyLimitRange that might have been created from project templates:

oc delete limitrange --all -n devsecops

34. Install Ploigos Operator