Repository files navigation

This repository contains the scripts used for building the OS imagesused by the NeCTAR Research Cloud.

We use Ansible to provision images, which are build using Packer and theQEMU/KVM provider from ISO with kickstart/preseed.

More comprehensive notes can be found in the Nectar Wiki at:https://wiki.rc.nectar.org.au/wiki/ImageBuilding

For local building, you will need:

• Ansible v2.16
• Packer (latest)
• Qemu - qemu-system and qemu-user

Ensure your user account is able to use the kvm hypervisor - for example, by being in thekvm group.NOTE: This project Requires Ansible 2.16. Later versions are too old to buildRocky/AlmaLinux 8 with it's Python 3.6

The repository is intended for use with Jenkins, but it is possible to buildimages locally.

A Makefile is included which should handle most of what you might need to do.

First, run the init command to ensure all the dependencies are resolved:

make init target=<image name>

There are some Packer plugins like QEMU, Ansible, WinRM, etc that will beinstalled at this point. For Windows, this step will also download thelatest copy of the VirtIO Win ISO which is also required.

The build can then be started with:

make build target=<image name>

At the end of this process, you should have a QCOW2 image ready for uploadingto Glance.

A feature of this codebase is that we do include some testing functionality.

A simple testing framework written in Bash (or PowerShell for Windows) isdeployed as part of the image build process. The specific tests deployeddepend on the functionality included, and this is logic is handled withinAnsible.

We also include some scripts which are used to initiate the tests. Thesescripts are used for booting a server from a new image, waiting for it tobecome ready, SSH'ing in and then running the tests.

The scripts are:

• local-image-tests.sh: Intended for local usage and takes the QCOW2 imagefilename as an argument and will upload it to Glance
• run-image-tests.sh: Used as part of the Jenkins pipeline, and assumes theimage is already available in Glance.

As part of the Jenkins build pipline, after the tests have been executedsuccessfully, a promotion step happens.

This step set the new image topublic visibility and if there is a match toan older build of the same image, it willde-promote it. This means theold image will have its visibility set tocommunity and be moved into theNectar-Images-Archive project.

This process is handled by theaggrandise.py script.

The ISOs for the OS images do seem to change frequently, and we're now tryingto keep up with the minor releases, so a helper script has been included.

Runscripts/update-isos.sh to resolve the latest ISO URL and checksum andit will update thebase vars config for that OS. Other images using that samebase OS aren't updated automatically, so you'll need to handle those yourself.(e.g. VGPU images, Jenkins slaves, etc)

Developing and fixing image build issues can be a long slow process.Because of this, we include a Vagrant file is which is useful for testing theAnsible roles are working correctly before committing.

Runvagrant status in the top level directory for a list of availablevirtual machine profiles you can test.

To launch a test Vagrant instance, use the following command:

make vagrant-initmake vagrant-up target=<image name>

Vagrant is currently hard-coded to use the libvirt provider, but it is possibleto use virtualbox for example, but that functionality is not currently set up.

Once you're finished with your testing, you can destroy the running VM with:

make vagrant-down target=<image name>