The next step in my minimalist computing journey.

Enter Gentoo, my first source based GNU/Linux distro. Pre-packaged binaries, which is the approach most other (binary based) distros take, must often cater for the lowest common denominator to ensure packages can run on lots of differing setups out in the wild. On a source based distro, I can articulate my specific needs (USE flags on Gentoo) to finely tune the binaries to my system. For example, as I plan to steer clear of software like systemd, kde and gnome, I can ensure support for these packages is NOT built into other program binaries I build for my system.

This is a big win for performance and security.

Read the Gentoo AMD64 Handbook.

Below are my notes after walking through the handbook.

Preparation stage⌗

Make LiveUSB key⌗

Download the latest iso, and use the gentoo-usb.sh script provided.

Boot the system.

Wifi⌗

Now in the running LiveUSB environment, setup temporarily configured wifi using wpa_supplicant like so:

wpa_supplicant -B -i $INTERFACE -c <(wpa_passphrase $SSID $PSK)
dhcpcd $INTERFACE

Don’t waste time with iw, which doesnt support WPA/WPA2.

Partition disk⌗

I’m installing on my new system which is UEFI (not BIOS) based and has an NVME drive.

Aiming for 4 partitions:

  • /dev/nvme0n1p1: 2MiB for GRUB bootloader
  • /dev/nvme0n1p2: 128MiB for ESP (EFI) boot partition (FAT32)
  • /dev/nvme0n1p3: 18000MiB for swap
  • /dev/nvme0n1p4: Rest of the disk for /

Hence I’ll be making a small FAT32 based ESP (EFI System Partition) partition, done with set 2 boot on.

I decided on 18000MiB of swap given the system has 16GB of RAM (i.e. RAM size + 2GiB).

parted -a optimal /dev/nmve0n1
(parted) mklabel gpt
(parted) unit mib
(parted) mkpart primary 1 3
(parted) name 1 grub
(parted) set 1 bios_grub on
(parted) mkpart primary 3 131
(parted) name 2 boot
(parted) set 2 boot on
(parted) mkpart primary 131 18131
(parted) name 3 swap
(parted) mkpart primary 18131 -1
(parted) name 4 rootfs

Format partitions⌗

Nothing fancy here, fat32 for the ESP, and ext4 for /.

mkfs.vfat /dev/nvme0n1p2
mkfs.ext4 /dev/nvme0n1p4
mkswap /dev/nvme0n1p3
swapon /dev/nvme0n1p3
mkdir -p /mnt/gentoo
mount /dev/nvme0n1p4 /mnt/gentoo
mount /dev/nvme0n1p2 /mnt/gentoo/boot

Time⌗

To keep various underlying crypto (GPG, TLS) happy, time sync your clock before proceeding.

ntpd -q -g

Stage 3 tarball⌗

Stage tarball wtf?⌗

A stage tarball is just an archive containing a minimal environment.

A stage 3 tarball provides an almost-complete and almost-functional system (the most important parts still missing are a kernel and a bootloader).

There are a number of stages to choose from, some key upfront design decisions you’ll need to make:

  • init system: openrc or systemd
  • lib bitness: multilib (32 and 64) or pure 64 (no multilib)
  • hardened for a lean

Using the links ncurses based terminal web browser, head to https://gentoo.org/downloads/mirrors/

I used the aarnet mirror to download the latest openrc based multilib stage3 stage3-amd64-20210310T214503Z.tar.xz (about 200Mb), straight to /mnt/gentoo/ which should be mounted to the targeted primary / partition.

Download the stage3-amd64-20210310T214503Z.tar.xz.DIGESTS.xz too.

Validate the checksum, the following should get a match:

grep $(sha512sum stage3-amd64-20210310T214503Z.tar.xz) stage3-amd64-20210310T214503Z.tar.xz.DIGESTS.xz --color

If checksum good, unpack it (xattrs-include preserves extended file attributes, and numeric-owner preserves uid/gid):

tar xpvf stage3-amd64-20210310T214503Z.tar.xz --xattrs-include='*.*' --numeric-owner

Some interesting tar options being used here:

  • p preserves file permissions
  • xattrs-include preserve extended attributes
  • numeric-owner preserve uid and gid, regardless is they exist in the /etc/passwd of the host

Emerge compile options⌗

The fun bit, tuning Portage, the source based package manager Gentoo officially uses.

Some homework:

When MAKEOPTS="-jN" is used with EMERGE_DEFAULT_OPTS="--jobs K --load-average X.Y" the number of possible tasks created would be up to N*K. Therefore, both variables need to be set with each other in mind as they create up to K jobs each with up to N tasks.

The load average value is the same as displayed by top or uptime, and for an N-core system, a load average of N.0 would be a 100% load. Another rule of thumb here is to set X.Y=N*0.9 which will limit the load to 90%, thus maintaining responsiveness.

Given my zen 3 ryzen has 12 threads, are going with N of 6, K of 3, giving a total possible of tasks N*K of 18, and a load cap of 12*0.9 = 10.8 which I rounded down to 10

  • set N to 6 with a MAKEOPTS of -j4
  • set K to 3 with a EMERGE_DEFAULT_OPTS of --jobs=3
  • set X.Y to 8 with a EMERGE_DEFAULT_OPTS --load-average=10 and a MAKEOPTS of -l10
vi /mnt/gentoo/etc/portage/make.conf

Mirrors⌗ 

mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf

DNS⌗

Create /etc/resolv.conf and define a nameserver entry.

Bind mount pseudo-filesystems⌗ 

mount --types proc /proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev

chroot⌗ 

chroot /mnt/gentoo /bin/bash
source /etc/profile
export PS1="(chroot) ${PS1}"

Update the ebuild repository⌗

This snapshot contains a collection of files that informs Portage about available software titles (for installation), which profiles the system administrator can select, package or profile specific news items.

emerge-webrsync
emerge --sync

The profile⌗

A key decision point. This profile can affect defaults in USE, CFLAGS and other system level variables. It also locks the systems to a certain subset of package versions.

The default is multilib and openrc based, so I’m good to go. Use eselect profile set n to change the profile.

eselect profile list

Update the @world set⌗ 

emerge --ask --verbose --update --deep --newuse @world

Timezone⌗ 

echo "Australia/Canberra" > /etc/timezone
emerge --config sys-libs/timezone-data

Locale⌗ 

tee /etc/locale-gen <<EOF
en_US ISO-8859-1
en_US.UTF-8 UTF-8
EOF

locale-gen

Now to select system wide locale setting using eselect:

eselect locale list
eselect locale set 6

Reload with locale and timezone preferences:

env-update && source /etc/profile && export PS1="(chroot) ${PS1}"

Making a kernel⌗

Gentoo provides a number of possible kernel sources, with sys-kernel/gentoo-sources being a popular option.

Installing it will place the latest supported kernel sources in /usr/src/ and symlink it to /usr/src/linux:

emerge -q sys-kernel/gentoo-sources genkernel
emerge --ask sys-apps/pciutils

genkernel can automatically make a generic kernel, however on gentoo that kind of defeats the purpose. genkernel is still useful when crafting a custom kernel, in particular for making an initramfs.

Before customising a kernel, its important to get a solid understanding of the machine your targetting; two tools useful in this endevour are lsmod and lspci. To get the later need to install pciutils;

emerge -q sys-apps/pciutils
emerge -q app-arch/lzop app-arch/lz4
lspci > ~/lspci.txt
lsmod > ~/lsmod.txt

Time to get to work. The kernel team provides an ncurses TUI interface for going through the various options:

cd /usr/src/linux
make menuconfig

Read up on the Gentoo Kernel Configuration Guide.

Topics specific to my computer:

I worked through the various kernel sections in the above, including qualcomm PCIe wifi card, and integrated realtek 8169 ethernet adapter.

Finally to make sure my specific hardware was baked in, I worked through the lsmod and lspci dumps I made from the LiveCD environment.

AMDGPU update 2021-04-03: Somehow I missed the firmware section of the AMDGPU guide above, which hung my first boot.

After emerging sys-kernel/linux-firmware which populates /lib/firmware/ with a stack of firmware blobs, including a subdirectory amdgpu.

Due to the GPU crisis of 2021 (its basically impossible to purchase a modern GPU right now) am running my 2016 Radeon RX480.

This runs the Arctic Islands chipset, specifically POLARIS10. The following firmware blobs need to be registered into the custom kernel firmware loader, under generic options in make menuconfig:

amdgpu/polaris10_{ce,ce_2,k_smc,k2_smc,k_mc,mc,me,me_2,mec2,mec2_2,mec,mec_2,pfp,pfp_2,rlc,sdma1,sdma,smc,smc_sk,uvd,vce}.bin

Building the kernel⌗ 

emerge app-arch/lz4
make -j8 && make -j8 modules_install
make install

Building an initramfs⌗ 

emerge --ask sys-kernel/genkernel
genkernel --install --kernel-config=/usr/src/linux/.config initramfs

WWA: built kernel and initramfs, contine with kernel modules section in https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel#Kernel_modules

Install firmware⌗

Most firmware is housed in sys-kernel/linux-firmware.

emerge -a sys-kernel/linux-firmware

Setup ESP⌗

Ensure the ESP (EFI system partition) is mounted at /boot within the chroot environment:

mount /dev/nvme0n1p2 /mnt/gentoo/boot

Move kernel image to the correct place and set a .efi suffix:

mkdir -p /boot/EFI/Gentoo
cp /boot/vmlinuz-5.11.7-gentoo-r1 /boot/EFI/Gentoo/vmlinuz-5.11.7-gentoo-r1.efi

Configure the system⌗

fstab⌗

Having vim at this point would be incredibly useful. Dump the UUID of block devices with blkid in one buffer (or tab), and editing /etc/fstab in another.

emerge -a app-editors/neovim
nvim /etc/fstab

Once in neovim, create a new tab, dump the output of blkid into it, yanking the UUID of the block devices needed, grafting them into /etc/fstab:

:tabnew
:r !blkid
:tabp
:tabn

Networking⌗

Hostname⌗ 

nvim /etc/conf.d/hostname

Network setup⌗

Gentoo, by default, uses Netifrc as its default network manager. Under netifrc, config is defined in /etc/conf.d/net.

ip a to get device name of the network interface you plan on using.

emerge -a --no-replace net-misc/netifrc
nvim /etc/config.d/net

In neovim, I like to dump the output of ip a into the buffer with a :r !ip a, delete all text excluding the name of the network interface you want to configure, in my case wlp40s0.

To setup DHCP on the interface, and the wpa_supplicant layer to perform WPA/WPA2 authentication on top of the interface:

modules="wpa_supplicant"
config_wlp40s0="dhcp"

Auto start the network interface:

cd /etc/init.d
ln -s net.lo net.wlp40s0
rc-update add net.wlp40s0 default

Add dhcpcd to default runlevel, don’t add wpa_supplicant to any runlevel.

rc-update add dhcpcd default
rc-update del wpa_supplicant default

Create /etc/wpa_supplicant/wpa_supplicant.conf as follows:

ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=0
ap_scan=1
fast_reauth=1

network={
    ssid="hotspot"
    psk="weakpassword"
    scan_ssid=1
    key_mgmt=WPA-PSK
    priority=5
}

Tip: For a full blown configuration example, unpack the template with bzless /usr/share/doc/${P}/wpa_supplicant.conf.bz2 > /etc/wpa_supplicant/wpa_supplicant.conf

Finally edit /etc/conf.d/wpa_supplicant:

wpa_supplicant_args="-B -M -c/etc/wpa_supplicant/wpa_supplicant.conf"

With the interface setup:

rc-service net.wlp40s0 start
rc-service net.wlp40s0 stop

Troubleshooting tips⌗

Run wpa_supplicant interactive (not as a daemon). I idiotically set ap_scan=0 (great time waster). Also forget scan_ssid=1 to force scan for hidden ssid.

wpa_supplicant -i wlp40s0 -c /etc/wpa_supplicant/wpa_supplicant.conf -dd

Set root password⌗ 

passwd

Init (OpenRC) configuration⌗

Gentoo by default uses OpenRC as its init, which uses /etc/rc.conf as its config file.

Hardware clock⌗

Check your hardware clock using hwclock --verbose. Mine was in local time.

nvim /etc/conf.d/hwclock

Set clock="local"

Install system tools⌗

System logger⌗

Gentoo is not opinionated; sysklogd, rsyslog, syslog-ng and metalog are all supported.

emerge one, and add it to the default runlevel (so it auto boots).

emerge -a app-admin/syslog-ng
rc-update add syslog-ng default

syslog-ng does NOT include log rotate functionality.

emerge -a app-admin/logrotate

Cron daemon⌗

Like syslog, many crons to choose from; such as cronie, dcron, fcron and anacron.

emerge -a sys-process/cronie
rc-update add cronie default

File indexing⌗

File system indexer to provide fast lookups.

emerge -a sys-apps/mlocate

SSH⌗

If you want to enable remote access, tell OpenRC to bootstrap it as part of default runlevel. OpenSSH is already bundled in stage 3 tarball.

Personally, a NO for me on this particular box.

rc-update add sshd default

Filesystem tools⌗

Depending on the file system you’re running. I’m going conservative with this box and sticking with good old ext4.

emerge -a sys-fs/e2fsprogs

Networking tools⌗

DHCP client⌗ 

emerge -a net-misc/dhcpcd

Wireless tools⌗

Install iw for WEP networks, and general useful scanning abilities. Not for me, in this case.

Install wpa_supplicant for WPA or WPA2 networks.

emerge -a net-wireless/wpa_supplicant

Boot loader⌗

This hardware is a recent 2021 ryzen build, is UEFI based, aiming to use GRUB2 as the bootloader.

First make sure GRUB_PLATFORMS="efi-x64" is defined in /etc/portage/make.conf, before proceeding.

emerge -a sys-boot/grub:2

Next, make sure that the ESP vfat partition has been mounted to the chroot environment as /boot. If not:

mount /dev/nvme0n1p2 /mnt/gentoo/boot

Next install the GRUB boot loader:

grub-install --target=x86_64-efi --efi-directory=/boot

This failed for me, due to a missing perl module: Can’t locate Locale/gettext.pm

Recommendation was to run perlcleaner --all, which added missing modules.

Finally generate a GRUB config, which will probe the /boot for available kernel images, initramfs and UEFI support.

grub-mkconfig -o /boot/grub/grub.cfg

Next, the ultimate test, reboot to take your new kernel for a test drive.

Post boot⌗

Add regular user⌗ 

useradd -m -G users,wheel,audio,usb,video,portage -s /bin/bash ben
passwd ben

sudo⌗ 

emerge app-admin/sudo
chmod u+w /etc/sudoers
nvim /etc/sudoers

Uncomment the line that allows the wheel group to execute any command, after root password is provided.

chmod u-w /etc/sudoers

Cleanup stage 3 tarballs⌗ 

rm /stage3*tar.*

Install userspace progs⌗

Non root Xorg⌗

Its bad practice to run X as root - many eye opening CVE write-ups. Luckily there is a guide that walks through setting this up on gentoo, in a nutshell:

  • ensure the elogind global USE flag is specified
  • have elogind started in the boot runlevel so that pam_elogind can communicate with it
  • reboot

libvirt⌗

Install virt-manager.

USE flags needed:

  • gtk for app-emulation/virt-manager
  • spice for app-emulation/qemu
emerge -a app-emulation/virt-manager

This will install a dozens of infrastructual dependencies. Once the dust settles, setup libvirt for non-root usage:

sudo groupadd --system libvirt
sudo usermod -a -G libvirt $(whoami)
newgrp libvirt
sudo vim /etc/libvirt/libvirtd.conf

Uncomment the following lines:

unix_sock_group = "libvirt" # line 85
unix_sock_rw_perms = "0770" # line 102

Finally restart libvirtd:

sudo rc-service libvirtd restart

Useful⌗

Portage⌗

RTFM

  • emerge --sync updates the gentoo ebuilds repo in /var/db/repos/gentoo
  • emerge --update --deep --with-bdeps=y --newuse @world update the system, including dependencies, build dependencies and detect new USE flags (if changed)
  • emerge --search mupdf search through titles
  • emerge --searchdesc mupdf search through descriptions
  • emerge --ask net-print/cups-pdf install a named package
  • qlist -IRv list installed
  • emerge -ND @world to update packages with latest USE flags

Overlays⌗

An overlay is simply the place where people put third party ebuilds. These ebuilds undergo extensive testing before they can be put into the Portage tree.

layman has been superseded by built-in portage tools such as emaint and eselect.

Packages will be masked by an ~arch keyword. Before you can emerge them, you will need to add them to your /etc/portage/package.accept_keywords like so:

echo "media-tv/vdrseriestimer" >> /etc/portage/package.accept_keywords
echo "media-plugins/vdr-burn" >> /etc/portage/package.accept_keywords

https://overlays.gentoo.org/

  • emerge -a app-eselect/eselect-repository install the repository module for eselect
  • eselect repository list list official overlay repos
  • eselect repository enable <overlay> enable an oevrlay repo
  • emaint sync -a sync all ebuild repos
  • emaint sync -r <repo> sync a specific remote overlay repo
  • eix-sync synchronise everything
  • eix-remote update fetch caches for remote overlays
  • eix -R <name> search remote overlay repos for package name

Example:

  1. Search for overlay https://gpo.zugaina.org/app-editors/visual-studio-code
  2. Add the overlay layman -a dotnet
  3. Sync the overlay repo layman -s dotnet
  4. Install ebuild emerge app-editors/visual-studio-code

OpenRC cheatsheet⌗

Gentoo’s default init system.

  • rc-update output services/runlevel matrix
  • rc-service cronie status get service status
  • rc-update add net.wlp40s0 default register a service in the default runlevel

Bare git repo⌗

Things to include:

  • /etc/sudoers
  • /etc/conf.d/net
  • /etc/wpa_supplicant/wpa_supplicant.conf
  • /usr/src/linux/.config

USE flags⌗

CPU_FLAGS_X86⌗

For instruction set specific optimisations CPU_FLAGS_X86

emerge -a app-portage/cpuid2cpuflags
nvim /etc/portage/make.conf

In vim dump the flags output by cpuid2cpuflags into the buffer with :read, and into the variable CPU_FLAGS_X86:

:r !cpuid2cpuflags

Troubleshooting⌗

Firefox and libaom.so build woes⌗

emerge www-client/firefox fails with:

ERROR: media-libs/libaom-2.0.1::gentoo failed (compile phase):
    ninja -v -j8 -l10 failed

Scanning the ebuild environment with grep -i " error" /var/tmp/portage/media-libs/libaom-2.0.1/temp/environment and build logs revealed the root cause:

libaom.so.0: undefined reference to 'aom_sad4xh_sse2'

Luckily this was documented as a bug 671340 back in 2018.

In short you need to setup the CPU_FLAGS_X86 USE flag in /etc/portage/make.conf

libvirtd: Unable to create bridge virbr0: Package not installed⌗

Kernel is missing various networking options, needed to support a network bridge.

See kernel options as documented at QEMU.