Repository files navigation

ALinux from Scratch (LFS) build usingpodman and theRPM packagemanager. These build steps follow LFS version 12.3-systemd.

Before you read any further, have you builtLinux fromScratch yourself? If not, I highlyrecommend doing it. The book is very well written and it is a fun exercise inwhich you learn the intimate details of what it takes to build an operatingsystem.

I built LFS about five years ago and I decided now was a good time to give itanother try. This time, though, I wanted to mix it up a bit. Instead of usingmy Linux system and a chroot to build LFS, I wanted to use podman instead. AndI wanted to use RPM infrastructure for building and installing the packagestoo. This repository holds the result of this work.

This build creates a bootable system, using a virtual machine, where you canget to a login prompt, login, and type inecho "hello world!". Not much elsehas been tested beyond that. Additional testing and hacking is left to me as afun exercise for a future rainy day activity. There is a chance that there aremajor errors with the system that have yet to be discovered. I am not an expertin building operating systems nor am I an expert in RPM packaging. Thisrepository should only be used as a guide of one way this could be done but itis nowhere near an optimal or proper way. Others may have done this already butthat was done for their fun, not mine.

Installpodman for use as the build environment:

sudo dnf install podman

To test the image in a virtual machine, install:

sudo dnf install qemu-kvm virt-manager

To continue onward after the initial LFS build, install createrepo:

sudo dnf install createrepo_c

Installpodman for use as the build environment and alsorpm andcurl:

sudo apt install podman rpm curl

To test the image in a virtual machine, install:

sudo apt install qemu-kvm virt-manager

Start libvirtd and add yourself to the libvirt group:

sudo systemctl start libvirtdsudo usermod -a -G libvirt $USERnewgrp libvirt

If you are lucky and cross your fingers the entire system can be built andbooted with the following three steps:

This downloads all the necessary source files and builds them using podman. Thiswill take some time. Here are the timing results from my personal desktop withan Intel i7-14700K CPU, SSD hard drive, 32 GiB of memory, default podmanconfiguration, andmake -j28:

real84m16.287s

Once done, the root filesystem can be found atbuild/containers/lfs-stage2.tar.gzand the kernel atbuild/boot/vmlinuz

If the build fails for some reason, you can continue the build using themanual procedure below. Running this command will start everything fromthe beginning which is usually what you don't want.

The tarball now has to be converted to a filesystem image. This requires rootprivileges as it is necessary to temporarily mount the image to copy in thefilesystem. Run this command and enter in your password if prompted. The rootfilesystem image will now be atbuild/lfs-12.3-root.img

The emulator is going to need access to two files: the root filesystem image,and the kernel. By default, it won't have access to files in your homedirectory. Running install will copy those files to/var/lib/libvirt/imagesas:

• lfs-12.3-root.img
• lfs-12.3-vmlinuz

Now create a virtual machine using the following steps:

• Start virt-manager
• Select File -> New Virtual Machine
• Select "Import existing disk image"
• For "Provide the existing storage path" and selectlfs-12.3-root.img
• For "Choose the operating system your are installing", select "Generic Linux 2022"
• Click on "Forward"
• Adjust memory and CPUs as needed and click on "Forward"
• For "Name", put in "LFS" or any name that you like
• Click on "Customize configuration before install"
• Click on "Finish"
• On the left sidebar, select "Boot Options" and then open "Direct kernel boot"
• Click on "Enable direct kernel boot"
• For "Kernel path" selectlfs-12.3-vmlinuz
• Leave "Initrd path" blank
• For "Kernel args" enter inroot=/dev/vda rw
• Click on "Apply"
• On the left sidebar, select "Video Virtio"
• Change Model from "Virtio" to "VGA"
• Click on "Apply"
• In the top left-hand corner, select "Begin Installation"

The operating system should now boot. Login as root and there is no password.Verify network connectivity withping 8.8.8.8.

If the boot hangs after a bunch of pci and pci_bus messages, change the videosettings under "Video Virtio" from "Virtio" to "VGA".

The build is split into four separate stages which correspond to chaptersin theLFS book. Those stages are:

• stage1a:Chapter 5, Compiling a Cross-Toolchain
• stage1b:Chapter 6, Cross Compiling Temporary Tools
• stage1c:Chapter 7, Entering Chroot and Building Additional Temporary Tools
• stage2:Chapter 8, Installing Basic System Software

The chapter breaks provide natural "save points" in the build process. Eachstage is built in a separate podman container and the results of the build areexported for use in the next stage.

The full procedure to build LFS withoutbuild-all is:

./lfs download  # optional./lfs 1a init./lfs 1a build./lfs 1a export./lfs 1b init./lfs 1b build./lfs 1b export./lfs 1c init./lfs 1c build./lfs 1c export./lfs 2 init./lfs 2 build./lfs 2 export./lfs mkimage./lfs install

Thelfs script is used to automate portions of the build process. Theavailable commands are as follows:

To create a podman image and container for use in building a stage, use thiscommand where<stage> is one of the stages listed above. The wordstage canbe omitted for brevity so that either "stage1b" or "1b" can be used. Forexample, to create a container for the first stage, use:

./lfs 1a init

Contexts are found in thecontainers directory. Each context has a<stage>.pkg.txt file that contains a list of source directories to buildrelative to the repository root.

This command always attempts to remove the existing podman container. If workhas already been done in a container, this work will be lost. It makes it easyto start over when needed but be aware that it will do so without anyconfirmation from you.

This command builds all packages, in order, found in<stage>.pkg.txt andthen uses RPM to install. In the event that the build fails, this command canbe reissued and it will skip packages that have already been installed. AllRPMs during the LFS build are installed using the--nodeps flag. Automaticdependency handling will be used after the base system is setup.

Packages are built with the-bb flag to only build RPMs while thepackages built after the initial OS install are built with-ba to create RPMsand SRPMs. These can be found in thebuild/{rpms,srpms} directories which arebind mounted in the containers at/rpmbuild/{RPMS,SRPMS}.

Once the build completes, issue this command to export the build results foruse in the next stage. A tarball will be created in the build directory underbuild/containers/lfs-<stage>.tar.gz and in the container context for thenext stage.

Once the containers have been created, they can individually be started orstopped with these commands. To get the list of containers that are currentlyrunning use:

podman ps

If you need to take a break during the build process, this command can beused to easily stop the containers and then to start them up at a later time.

Login to the container as the unprivileged user lfs (usingshell) or as theroot user (usingroot). In stage1a and stage1b you cansudo to root as thelfs user but in stage1c and stage2 that command is not available and you mustuse the./lfs <stage> root command. The repository root is bind mounted at/host.

Build a single RPM instage using the givensource_dir which contains anRPM spec file and any other files needed for the build. More then onesource directory can be provided if necessary. If this RPM has already beeninstalled, it will be rebuilt and reinstalled by replacing the older package.This command is useful during development when iterating on a specific packagewithout needing to runbuild.

By default,%check scriptlets are skipped when building RPMs. Prefix thiscommand withwith_check=1 to run any provided tests. This shouldn't bedone with the general build command as quite a few packages work fine witha few test failures.

Upstream files are downloaded during the build process if they have not alreadybeen downloaded. This command is useful if you want to download all thefiles upfront or if you need to download a specific package before building.If no source directories are provided on the command line, all upstream fileswill be downloaded.

A handful of environmental variables are injected into each container. Thiscommand shows you the current values of those variables. When debugging thelfs script, it can sometimes be useful to inject these variables into yourcurrent shell withsource ./lfs-env. These variables can be overridden byplacing changes in alocal-lfs.env file. Variables of note:

• lfs_host_image: The base podman image used in theFROM clause that isused in the containers for stage1a and stage1b.
• lfs_version: Defines which version of LFS is being built. This is used inURLs for downloading LFS specific patches and in the dist variable forrpmbuild.
• lfs_root_size: Size to use for the root partition filesystem image.

This resets everything to start a build from scratch again. It deleteseverything under thebuild directory except for upstream files that havebeen downloaded. All exports undercontainers are also removed along withany podman images or containers.

The same as./lfs clean but also removes the downloads directory.

In stage1a and stage1b, a Fedora image is used to build the initial RPMsusing therpm andrpmbuild commands provided by that distribution.Fedora image are used. The following macros are added or adjusted:

• %dist: Fedora uses a dist tag such asfc41 and Red Hat EnterpriseLinux uses dist tags such asel9. This build useslfs12_3.
• %_build_id_links: By default, Fedora wants to generate files in/usr/lib/.build-id for debugging purposes. This isn't necessary for an LFSbuild and just adds clutter to the build. This setting is set tonone todisable this feature.
• %debug_package: Fedora also wants to generate debuginfo packages whichwe don't need. Set to%{nil} to disable.
• %verify_sha256: A file is found in each source directory that containschecksums of the upstream files which are used by this macro to verify thatthe contents are indeed correct.

When using Fedora in stage1a and stage1b, there are two other macrosprovided in/usr/lib/rpm/redhat/macros that are changed. Sed scriptsin the Containerfile make these modifications:

• %source_date_epoch_from_changelog: The specfiles are not using changelogsand by setting this to zero, it removes a warning about not having a changelog.
• %_auto_set_build_flags: This sets a whole list of additionalCFLAGS thatFedora wants you to use when building packages but this causes a buildfailure somewhere. This is disabled to prevent this from happening.

Other macro files,/usr/lib/rpm/macros.d/macros.lfs-stage* are added withstage specific information. It has a%with_<stage> 1 macro used toidentify if a specific stage is being used and%with_stage1 to identifywhen in stage1. Quite a few packages need to be built twice, once as atemporary tool and once as the final package. The same specfile is usedbut differences in the builds are separated with conditionals using the%with_stage1 macro. Some packages are built three or four times and usethe additionalwith macros as necessary. An overall%with lfs macrois used to identify if a build is being performed in any non-pod stage.

A separate RPM database in/var/lib/lfs-rpm, is used during the Fedora stagesto track which packages have been built and installed for that stage, not forpackages which already exist on the system.

Logs can be found inlogs. Those files are:

• build.log: Timestamp and record of when each package build is startedand completed.
• last-started: Name of the last package that the build started
• last-success: Name of the last package that was built successfully
• <stage>/<package>.log: Standard output and standard error capture ofrpmbuild andrpm for that package.

To build packages beyond the initial set required to boot the operatingsystem, it is helpful to start using dependency tracking and resolution providedbyrpm anddnf. Stage 3 continues onward to build the packages necessary toget a minimaldnf version 5 executable:

./lfs 3 init./lfs 3 build

The stage 3 container now has a little over 100 packages installed at thispoint. Each step along the way has been adding more and more packages tosatisfy dependencies for later packages in the build. It is now time to createa podman image dedicated to building individual packages instead of an entireoperating system.

The image is going to be pruned down to basic tools needed for building. Tocreate this image, first run:

This creates a tarball to be used as the base filesystem for the generalbuild container. The list of packages included are found incontainers/lfs-pod/mkpod.pkg.txt. Going forward, rpm spec files must useBuildRequires for packages not found inmkpod.pkg.txt.

Thepod script is going to be used from now on instead of thelfs script.This creates a new podman image using the tarball created above but doesnot yet start a container.

Configuration for this environment can be found inpod-env. Right now itonly contains the build arch and the dist tag which is nowpod12_3. If youwould like to customize the build to use your own dist tag, create alocal-pod.env file and add to itpod_dist with the name of yourchoosing.

Creates or recreates a new container and builds the rpm for the given specfile. The results of the build are found inbuild/pod and this directory isalso a dnf repository available to the container. Subsequent builds can useBuildRequires to automatically install packages found here.

This calls./pod rpm for each spec file found inpackages.sources.txt. Torebuild all spec files up to stage 3 in the proper order, use:

./pod build ./rebuild.sources.txt

Export a filesystem for a bootable image using the list ofpackages found inimage.pkg.txt. The build container is used as the basefor this filesystem so any additional package required to boot must befound in the package list. An example of a working set can be found inpod-image.pkg.txt. Add additional packages to this list as desired.

After the image is exported, the image can be booted using./lfs mkimage build/pod/lfs-pod.tar.gz and./lfs install.

In the event of a build failure, the container will still be running andthis command can be used to login and investigate.

Create and start or destroy a build container.

Below is an example, and a working set, of commands to execute to boot animage with packages built with the podman build container:

./lfs 3 mkpod./pod init./pod build rebuild.spec.txt./pod export pod-image.pkg.txt./lfs mkimage-pod./lfs install

Now create the virtual machine as described above and login into the rootaccount which has no password.

I have subscribed to the LFS announcement mailing list and will try to keep thisup-to-date as new versions of LFS are published. No guarantee on when thathappens or even if I follow through with it.

The shell scripts have grown organically over time and are now in need ofsome refactoring. Another rainy day exercise may be to convert these to aPython script.

MIT

Contact me atlfs@blackchip.org