- Notifications
You must be signed in to change notification settings - Fork15
a small, virtual setup to locally run the metal-stack
License
metal-stack/mini-lab
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a metal-stack partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.
ℹ This project can also be used as a template for writing your own metal-stack deployments.
- Linux machine with hardware virtualization support
- kvm as hypervisor for the VMs (you can check through the
kvm-okcommand) - docker >= 24.x.y (for using kind and our deployment base image)
- kind == v0.23.0 (for hosting the metal control plane)
- containerlab >= v0.56.0
- the lab creates a docker network on your host machine with the address block
203.0.113.0/24, designated as TEST-NET-3 for documentation and examples. - (recommended) haveged to have enough random entropy (only needed if the PXE process does not work)
Here is some code that should help you to set up most of the requirements:
# If UFW enabled.# Disable the firewall or allow traffic through Docker network IP range.sudo ufw statussudo ufw allow from 172.17.0.0/16# Install required tools / kvmsudo apt install -y git curl qemu qemu-kvm haveged jq# Install Dockercurl -fsSL https://get.docker.com| sh# if you want to be on the safe side, follow the original installation# instructions at https://docs.docker.com/engine/install/ubuntu/# Ensure that your user is member of the group "docker"# you need to login again in order to make this change take effectsudo usermod -G docker -a${USER}# Install containerlabbash -c"$(curl -sL https://get.containerlab.dev)"# Install kind (kubernetes in docker), for more details see https://kind.sigs.k8s.io/docs/user/quick-start/#installationsudo curl -Lo /usr/local/bin/kind"https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64"sudo chmod +x /usr/local/bin/kind
The following ports are used statically on your host machine:
| Port | Bind Address | Description |
|---|---|---|
| 6443 | 0.0.0.0 | kube-apiserver of the kind cluster |
| 4443 | 0.0.0.0 | HTTPS ingress |
| 4150 | 0.0.0.0 | nsqd |
| 8080 | 0.0.0.0 | HTTP ingress |
- to keep the demo small there is no EVPN
- machines have to be restarted manually with
make power-reset-<machine> - login to the machines is possible with virsh console, login to the firewall is possible with SSH from your local machine
git clone https://github.com/metal-stack/mini-lab.gitcd mini-labStart the mini-lab with a kind cluster, a metal-api instance as well as two containers wrapping leaf switches and another container that hosts two user-allocatable machines:
make# containerlab will ask you for root permissions (https://github.com/srl-labs/containerlab/issues/669)Before the upcoming steps, you need to bind some environment variables using the following command. This ensuresmetalctlkubectl are able to communicate with your mini-lab.
eval$(make dev-env)
After the deployment and waiting for a short amount of time, two machines in statusPXE booting become visible throughmetalctl machine ls:
docker compose run --rm metalctl machine lsID LAST EVENT WHEN AGE HOSTNAME PROJECT SIZE IMAGE PARTITION00000000-0000-0000-0000-000000000001 PXE Booting 3s00000000-0000-0000-0000-000000000002 PXE Booting 5s
Wait until the machines reach the waiting state:
docker compose run --rm metalctl machine lsID LAST EVENT WHEN AGE HOSTNAME PROJECT SIZE IMAGE PARTITION00000000-0000-0000-0000-000000000001 Waiting 8s v1-small-x86 mini-lab00000000-0000-0000-0000-000000000002 Waiting 8s v1-small-x86 mini-lab
Create a firewall and a machine with:
make firewallmake machine
Alternatively, you may want to issue themetalctl commands on your own:
docker compose run --rm metalctl network allocate \ --partition mini-lab \ --project 00000000-0000-0000-0000-000000000000 \ --name user-private-network# lookup the network ID and create a machinedocker compose run --rm metalctl machine create \ --descriptiontest \ --name machine \ --hostname machine \ --project 00000000-0000-0000-0000-000000000000 \ --partition mini-lab \ --image ubuntu-24.4 \ --size v1-small-x86 \ --networks<network-ID># create a firewall that is also connected to the virtual internet-mini-lab networkdocker compose run --rm metalctl firewall create \ --description fw \ --name fw \ --hostname fw \ --project 00000000-0000-0000-0000-000000000000 \ --partition mini-lab \ --image firewall-ubuntu-3.0 \ --size v1-small-x86 \ --networks internet-mini-lab,<network-ID>
See the installation process in action
make console-machine01# or console-machine02...Ubuntu 24.04 machine ttyS0machine login:Two machines are now installed and have status "Phoned Home"
docker compose run --rm metalctl machine lsID LAST EVENT WHEN AGE HOSTNAME PROJECT SIZE IMAGE PARTITION00000000-0000-0000-0000-000000000001 Phoned Home 2s 21s machine 00000000-0000-0000-0000-000000000000 v1-small-x86 Ubuntu 24.04 mini-lab00000000-0000-0000-0000-000000000002 Phoned Home 8s 18s fw 00000000-0000-0000-0000-000000000000 v1-small-x86 Firewall 3 Ubuntu mini-lab
Login with user name metal and the console password from
docker compose run --rm metalctl machine consolepassword 00000000-0000-0000-0000-000000000001
To remove the kind cluster, the switches and machines, run:
make cleanup
There are make targets to handle the power state of a machine:
make power-<on,reset,off>-<machine name>There are two versions, or flavors, of the mini-lab environment which differ in regards to the NOS running on the leaves:
sonic: runs 2 SONiC switchescapms: runs the SONiC flavor but with three instead of two machines (this is used forcluster-provider-metal-stack in order to have dedicated hosts for control plane / worker / firewall)gardener: installs theGardener in the mini-lab
In order to start specific flavor, you can define the flavor as follows:
export MINI_LAB_FLAVOR=sonicmakeAn Nginx is running inside of the www container to allow automatic testing of outgoing connections.
About
a small, virtual setup to locally run the metal-stack
Topics
Resources
License
Contributing
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.
