Repository files navigation

ALinux from Scratch (LFS) build usingpodman and theRPM packagemanager. These build steps follow LFS version 12.3-systemd.

Before you read any further, have you builtLinux fromScratch yourself? If not, I highlyrecommend doing it. The book is very well written and it is a fun exercise inwhich you learn the intimate details of what it takes to build an operatingsystem.

I built LFS about five years ago and I decided now was a good time to give itanother try. This time, though, I wanted to mix it up a bit. Instead of usingmy Linux system and a chroot to build LFS, I wanted to use podman instead. AndI wanted to use RPM infrastructure for building and installing the packagestoo. This repository holds the result of this work.

This build creates a bootable system, using a virutal machine, where you canget to a login prompt, login, and type inecho "hello world!". Not much elsehas been tested beyond that. Additional testing and hacking is left to me as afun exercise for a future rainy day activity. There is a chance that there aremajor errors with the system that have yet to be discovered. I am not an expertin building operating systems nor am I an expert in RPM packaging. Thisrepository should only be used as a guide of one way this could be done but itis nowhere near an optimal or proper way. Others may have done this already butthat was done for their fun, not mine.

Installpodman for use as the build environment:

sudo dnf install podman

To test the image in a virtual machine, install:

sudo dnf install qemu-kvm virt-manager

Installpodman for use as the build environment and alsorpm andcurl:

sudo apt install podman rpm curl

To test the image in a virtual machine, install:

sudo apt install qemu-kvm virt-manager

Start libvirtd and add yourself to the libvirt group:

sudo systemctl start libvirtdsudo usermod -a -G libvirt $USERnewgrp libvirt

If you are lucky and cross your fingers the entire system can be built andbooted with the following three steps:

This downloads all the necessary source files and builds them using podman. Thiswill take some time. Here are the timing results from my personal desktop withan Intel i7-7700 CPU, SSD hard drive, 16 GiB of memory, default podmanconfiguration, andmake -j8:

real131m27.156s

Once done, the root filesystem can be found atbuild/stage2/lfs-stage2.tar.gzand the kernel atbuild/boot/vmlinuz

If the build fails for some reason, you can continue the build using themanual procedure below. Running this command will start everything fromthe beginning which is usually what you don't want.

The tarball now has to be converted to a filesystem image. This requires rootprivileges as it is necessary to temporarily mount the image to copy in thefilesystem. Run this command and enter in your password if prompted. The rootfilesystem image will now be atbuild/lfs-12.3-root.img

The emulator is going to need access to two files: the root filesystem image,and the kernel. By default, it won't have access to files in your homedirectory. Running install will copy those files to/var/lib/libvirt/imagesas:

• lfs-12.3-root.img
• lfs-12.3-vmlinuz

Now create a virtual machine using the following steps:

• Start virt-manager
• Select File -> New Virtual Machine
• Select "Import existing disk image"
• For "Provide the existing storage path" and selectlfs-12.3-root.img
• For "Choose the operating system your are installing", select "Generic Linux 2022"
• Click on "Forward"
• Adjust memory and CPUs as needed and click on "Forward"
• For "Name", put in "LFS" or any name that you like
• Click on "Customize configuration before install"
• Click on "Finish"
• On the left sidebar, select "Boot Options" and then open "Direct kernel boot"
• Click on "Enable direct kernel boot"
• For "Kernel path" selectlfs-12.3-vmlinuz
• Leave "Initrd path" blank
• For "Kernel args" enter inroot=/dev/vda rw
• Click on "Apply"
• On the left sidebar, select "Video Virtio"
• Change Model from "Virtio" to "VGA"
• Click on "Apply"
• In the top left-hand corner, select "Begin Installation"

The operating system should now boot. Login as root and there is no password.Verify network connectivity withping 8.8.8.8.

If the boot hangs after a bunch of pci and pci_bus messages, change the videosettings under "Video Virtio" from "Virtio" to "VGA".

The build is split into four separate stages which correspond to chaptersin theLFS book. Those stages are:

• stage1a:Chapter 5, Compiling a Cross-Toolchain
• stage1b:Chapter 6, Cross Compiling Temporary Tools
• stage1c:Chapter 7, Entering Chroot and Building Additional Temporary Tools
• stage2:Chapter 8, Installing Basic System Software

The chapter breaks provide natural "save points" in the build process. Eachstage is built in a separate podman container and the results of the build areexported for use in the next stage.

The full procedure to build LFS withoutbuild-all is:

./lfs download  # optional./lfs 1a init./lfs 1a build./lfs 1a export./lfs 1b init./lfs 1b build./lfs 1b export./lfs 1c init./lfs 1c bootstrap./lfs 1c build./lfs 1c export./lfs 2 init./lfs 2 build./lfs 2 export./lfs mkimage./lfs install

Thelfs script is used to automate portions of the build process. Theavailable commands are as follows:

To create a podman image and container for use in building a stage, use thiscommand where<stage> is one of the stages listed above. The wordstage canbe omitted for brevity so that either "stage1b" or "1b" can be used. Forexample, to create a container for the first stage, use:

./lfs 1a init

Contexts are found in thecontainers directory. Each context has a<stage>.pkg.txt file that contains a list of spec files to build relativeto the repository root.

This command always attempts to remove the existing podman container. If workhas already been done in a container, this work will be lost. It makes it easyto start over when needed but be aware that it will do so without anyconfirmation from you.

This command builds all packages, in order, found in<stage>.pkg.txt andthen uses RPM to install. In the event that the build fails, this command canbe reissued and it will skip packages that have already been installed. AllRPMs during the LFS build are installed using the--nodeps flag. Automaticdependency handling will be used after the base system is setup.

Temporary packages are built with the-bb flag to only build RPMs while thepackages in stage2 are built with-ba to create RPMs and SRPMs. Thesecan be found in thebuild/<stage>/{rpms,srpms} directories which arebind mounted in the containers at/build/rpmbuild/{RPMS,SRPMS}. Allpackages are built with thex86_64 arch even if they qualify as anoarchpackage.

Once the build completes, issue this command to export the build results foruse in the next stage. A tarball will be created in the build directory underbuild/<stage>/lfs-<stage>.tar.gz and in the container context for thenext stage.

Once the containers have been created, they can individually be started orstopped with these commands. To get the list of containers that are currentlyrunning use:

podman ps

If you need to take a break during the build process, this command can beused to easily stop the containers and then to start them up at a later time.

Login to the container as the unprivileged user lfs (usingshell) or as theroot user (usingroot). In stage1a and stage1b you cansudo to root as thelfs user but in stage1c and stage2 that command is not available and you mustuse the./lfs <stage> root command. The repository root is bind mounted at/build/lfs-rpm and the sources directory at/build/rpmbuild/SOURCES.

Build a single RPM instage using the givenspecfile. More then onespecfile can be provided if necessary. If this RPM has already been installed,it will be rebuilt and reinstalled by replacing the older package. This commandis useful during development when iterating on a specific package withoutneeding to runbuild.

By default,%check scriptlets are skipped when building RPMs. Prefix thiscommand withwith_check=1 to run any provided tests. This shouldn't bedone with the general build command as quite a few packages work fine witha few test failures.

Source files are downloaded during the build process if they have not alreadybeen downloaded. This command is useful if you want to download all thepackages upfront or if you need to download a specific package before building.If no specfiles are provided on the command line, all source packages will bedownloaded.

A handful of environmental variables are injected into each container. Thiscommand shows you the current values of those variables. When debugging thelfs script, it can sometimes be useful to inject these variables into yourcurrent shell withsource ./lfs-env. These variables can be overridden byplacing changes in alocal-lfs.env file. Variables of note:

• lfs_version: Defines which version of LFS is being built. This is used inURLs for downloading LFS specific patches and in the dist variable forrpmbuild.
• lfs_host_image: The base podman image used in theFROM clause that isused in the containers for stage1a and stage1b.
• lfs_nproc: Number of parallel make processes to use when building. Thisis usually set to the number of processors on your machine.
• lfs_root_size: Size to use for the root partition filesystem image.

This resets everything to start a build from scratch again. It deleteseverything under thebuild directory except for the sources, all exportsundercontainers and removes any podman images or containers.

The same as./lfs clean but also removes the sources directory.

In stage1a and stage1b, therpm andrpmbuild commands provided by theFedora image are used. Some macros are installed under/usr/lib/rpm/macros.d/macros.lfs to assist with the build:

• %dist: Fedora uses a dist tag such asfc41 and Red Hat EnterpriseLinux uses dist tags such asel9. This build useslfs12_3.
• %_build_id_links: By default, Fedora wants to generate files in/usr/lib/.build-id for debugging purposes. This isn't necessary for an LFSbuild and just adds clutter to the build. This setting is set tonone todisable this feature.
• %debug_package: Fedora also wants to generate debuginfo packages whichwe don't need. Set to%{nil} to disable.
• %make: We use this macro in all the spec files when using the makecommand. This adds the-j flag which controls the number of parallelprocesses to use.
• %shlib: This sets the permissions to 755 for shared library files.
• %use_lfs_tools: This is used in stage1a and stage1b to add the LFS toolsdirectory to the front of the search path to ensure those tools are usedwhen available.
• %discard_docs: When building the temporary toolset, it isn't necessaryto keep any generated documentation. This macro removes those files andthat helps keep the build images a bit smaller.
• %remove_info_dir: When info documentation pages are generated, the/usr/share/info/dir file usually gets updated. This cannot be done atbuild or install time because multiple RPMs cannot "own" the same file.This macro deletes this file and then the following two macros are then used:
• %request_info_dir: This macro should be placed in the%post scriptletto place a request to regenerate the info dir file. Then use the followingmacro:
• %update_info_dir: This macro should be placed in the%posttrans scriptletto regenerate the info dir file. If multiple packages are installed that haveinfo files, the actual regeneration only occurs once.

When using Fedora in stage1a and stage1b, there are two other macrosprovided in/usr/lib/rpm/redhat/macros that are changed. Sed scriptsin the Containerfile make these modifications:

• %source_date_epoch_from_changelog: The specfiles are not using changelogsand by setting this to zero, it removes a warning about not having a changelog.
• %_auto_set_build_flags: This sets a whole list of additionalCFLAGS thatFedora wants you to use when building packages but this causes a buildfailure somewhere. This is disabled to prevent this from happening.

Another macro file,/usr/lib/rpm/macros.d/macros.lfs-stage is added withstage specific information. It has a%with_<stage> 1 macro used toidentify if a specific stage is being used and%with_stage1 to identifywhen in stage1. Quite a few packages need to be built twice, once as atemporary tool and once as the final package. The same specfile is usedbut differences in the builds are separated with conditionals using the%with_stage1 macro. Some packages are built three or four times and usethe additionalwith macros as necessary.

The RPM database is used during the build to track which packages have beenbuilt and installed for that stage, not for packages which already exist on thesystem. When the container for stage1a is created, the contents of the existingRPM database are deleted. Subsequent stages do not keep the database betweenexports.

At stage1c the host Fedora system is left behind and from that point forwardthe build relies only on the tools built so far. To continue, therpm andrpmbuild commands have to also be available. In stage1b, additional packagesare built for this purpose:

• lua
• pkg-config
• libgpg-error
• libgcrypt
• gettext
• zlib
• bzip2

The only packages after this that are necessary areelfutils,rpm to buildRPM itself andcmake which is needed to buildrpm. As you can tell, RPMuses cmake as its build system. I tried to cross-compile cmake in stage1b butI really just didn't have the patience to figure that out. cmake gives me aheadache.

I chose the other option which is to build those final packages in stage1cand install directly to the filesystem. This command does that finalbootstrapping for RPM and this needs to be executed after the stage1b exportand before the stage1c build.

Logs can be found inbuild/logs. Those files are:

• build.log: Timestamp and record of when each package build is startedand completed.
• last-started: Name of the last package that the build started
• last-success: Name of the last package that was built successfully
• <stage>/<package>.log: Standard output and standard error capture ofrpmbuild andrpm for that package.

To build packages beyond the initial set required to boot the operatingsystem, it is helpful to start using dependency tracking and resolution providedbyrpm anddnf. Stage 3 continues onward to build the packages necessary toget a minimaldnf version 5 executable:

./lfs 3 init./lfs 3 build

The stage 3 container now has a little over 100 packages installed at thispoint. Each step along the way has been adding more and more packages tosatisfy dependencies for later packages in the build. It is now time to createa podman image dedicated to building individual packages instead of an entireoperating system.

The image is going to be pruned down to basic tools needed for building. Tocreate this image, first run:

This creates a tarball to be used as the base filesystem for the generalbuild container. The list of packages included are found incontainers/lfs-pod/mkpod.pkg.txt. Going forward, rpm spec files must useBuildRequires for packages not found inmkpod.pkg.txt.

Thepod script is going to be used from now on instead of thelfs script.This creates a new podman image using the tarball created above but doesnot yet start a container.

Configuration for this environment can be found inpod-env. Right now itonly contains the build arch and the dist tag which is nowpod12_3. If youwould like to customize the build to use your own dist tag, create alocal-pod.env file and add to itpod_dist with the name of yourchoosing.

Creates or recreates a new container and builds the rpm for the given specfile. The results of the build are found inbuild/pod and this directory isalso a dnf repository available to the container. Subsequent builds can useBuildRequires to automatically install packages found here.

This calls./pod rpm for each spec file found inpackages.spec.txt. Torebuild all spec files up to stage 3 in the proper order, use:

./pod build rebuild.spec.txt

Export a filesystem for a bootable image using the list ofpackages found inimage.pkg.txt. The build container is used as the basefor this filesystem so any additional package required to boot must befound in the package list. An example of a working set can be found inpod-image.pkg.txt. Add additional packages to this list as desired.

After the image is exported, the image can be booted using./lfs mkimage build/pod/lfs-pod.tar.gz and./lfs install.

In the event of a build failure, the container will still be running andthis command can be used to login and investigate.

Create and start or destroy a build container.

Below is an example, and a working set, of commands to execute to boot animage with packages built with the podman build container:

./lfs 3 mkpod./pod init./pod build rebuild.spec.txt./pod export pod-image.pkg.txt./lfs mkimage-pod./lfs install

Now create the virtual machine as described above and login into the rootaccount which has no password.

I have subscribed to the LFS announcement mailing list and will try to keep thisup-to-date as new versions of LFS are published. No guarantee on when thathappens or even if I follow through with it.

MIT

Contact me atlfs@blackchip.org