bootstrap/gcc/notes/gentoo/gentoo_notes.txt

Based on portage-3.0.8 and gentoo-20201101 snapshot

The specifics of this document (software versions and whatnot) are subject to
change as gentoo evolves, but I hope the big lines of it won't change too
much.

This guide starts off with a system built from build_cross.sh, built for
x86_64 with i686 multilib.
It might be possible to adapt both that script, and this guide for other
architectures, but this guide won't go into detail.

You should be running a system with a x86_64 kernel (with i686 support).
Install the required sources (listed in gentoo.wget-list) into /sources.

Before entering the system, create the basic directory layout by unpacking
the baselayout tarball, and running:
make layout OS=Linux DESTDIR=<ROOT>
NOTE: In the future this might have to become `layout-usrmerge`

To enter the bootstrap system as a chroot, use the following commands to set
up the filesystems and entering:
mount -t proc proc proc
mount -t sysfs sysfs sys
mount -t devtmpfs devtmpfs dev
mount -t devpts devpts dev/pts
mount -t tmpfs tmpfs dev/shm
touch etc/resolv.conf
mount --bind /etc/resolv.conf etc/resolv.conf
./bootstrap/bin/chroot . /bootstrap/bin/env -i PATH=/bin:/sbin:/usr/bin:/usr/sbin:/bootstrap/bin:/bootstrap/sbin HOME=/root /bootstrap/bin/sh -l

To install all the prerequisite tools, place the sources (listed in
gentoo.wget-list) under /sources, then run the gentoo_tools.sh script.

Configure portage:
echo "portage:x:250:250:portage:/var/tmp/portage:/bin/false" > /etc/passwd
echo "portage::250:portage" > /etc/group
echo "C UTF-8" > /etc/locale.gen
cat > /etc/portage/make.conf << 'EOF'
FEATURES='-pid-sandbox -news'
ROOTPATH=/bootstrap/bin:/bootstrap/sbin
EOF
ln -sf ../../var/db/repos/gentoo/profiles/default/linux/amd64/17.1/ /etc/portage/make.profile

NOTE: You can copy the ebuild repository archive, gpgsig and md5sum to
/var/cache/distfiles to avoid downloading.

Install ebuild repository (pick appropriate date, see distfiles.gentoo.org/snapshots):
emerge-webrsync -k -v --revert=20201101

Install baselayout:
emerge -O1 sys-apps/baselayout
cp /usr/share/baselayout/{passwd,group} /etc/

Install some ebuild prerequisites:
emerge -O1 sys-apps/gentoo-functions  # Used by elt-patches and a ton of other things
emerge -O1 app-portage/elt-patches  # required by elibtoolize(libtool.eclass), used to install any library
emerge -O1 sys-devel/gnuconfig  # required by gnuconfig_update(gnuconfig.eclass), used by toolchain(-glibc).eclass, doesn't hurt to have
emerge -O1 sys-devel/binutils-config sys-devel/gcc-config

First, we will build an updated binutils+gcc combo that targets /usr instead
of /bootstrap. This makes it easier to bootstrap glibc and everything else. We
will use portage for this, even though it's a mess.

NOTE: Doing this is (probably) *BAAAAD*, since we're linking the compiler
support libraries with a potentially incompatible version of glibc in
/bootstrap, and upgrading it afterwards. Glibc should always be installed
first into a new prefix! However, due to the complex relationship between
glibc, gcc, libgcc and libstdc++, this is rather hard to do while wrangling
the change from /bootstrap to /usr, as well as the multilib libdir changes, so
for simplicity's sake, I won't bother. This just works™.

Install temporary libc files for target compiler:
ln -sf /bootstrap/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
ln -sf /bootstrap/lib32/ld-linux.so.2 /lib/ld-linux.so.2
ln -sf /bootstrap/lib/{crti,crtn,Scrt1}.o \
       /bootstrap/lib/{libc,libm,libpthread}.so /usr/lib64/
ln -sf /bootstrap/lib32/{crti,crtn,Scrt1}.o \
       /bootstrap/lib32/{libc,libm,libpthread}.so /usr/lib/
mkdir -p /usr/include
ln -sf /bootstrap/include/limits.h /usr/include/limits.h

Install binutils+GCC (linked against /bootstrap):
USE='-*' emerge -O1 sys-devel/binutils
USE='-* static-libs' EXTRA_ECONF=--disable-shared \
    CPPFLAGS='-I/usr/include' LDFLAGS='-L/usr/lib64 -L/usr/lib' \
    emerge -O1 dev-libs/gmp dev-libs/mpfr dev-libs/mpc
USE='-*' EXTRA_ECONF=--disable-bootstrap GCC_MAKE_TARGET=all \
    CPPFLAGS='-I/usr/include' LDFLAGS='-L/usr/lib64 -L/usr/lib' \
    CPATH=/bootstrap/include \
    emerge -O1 sys-devel/gcc
emerge --rage-clean dev-libs/gmp dev-libs/mpfr dev-libs/mpc
gcc-config 1  # Not automatic unlike binutils-config...

NOTE: The Glibc ebuild downloads a file called gcc-multilib-bootstrap, which
contains prebuilt binaries to bootstrap multilib gcc. This is _only_ used when
the command "echo 'int main(){}' | $CC -m32 -xc -" fails, and shouldn't be
being used in this bootstrap, but you can make sure of that by patching the
ebuild to `die` instead of running the `sed` command to enable its use.

NOTE: BOOTSTRAP_RAP=1 is only used in the Glibc ebuild, to avoid a pre-install
check that might pick up programs from /bootstrap to test against the
installed glibc, as well as picking up python from $PATH instead of
eselect-python. I'm not aware of how the support status is for this variable,
and as such it might disappear in the future, or be used in more packages.

Install initial glibc:
CC=/bootstrap/bin/gcc emerge -O1 sys-kernel/linux-headers
USE='-*' BOOTSTRAP_RAP=1 emerge -O1 sys-libs/glibc

NOTE: `sandbox` outputs a failure message to stderr for every program still
built against /bootstrap, which includes the compiler. Some autoconf scripts
check stderr to determine a failure, so this causes builds to fail. This is
why it's (partially) disabled for this stage.

Bootstrap the rest of the system:
USE=-seccomp emerge -O1 app-misc/pax-utils
cd /var/db/repos/gentoo/scripts
mkdir -p /var/run
ln -s /bootstrap/bin/python3 /usr/bin/python
rm -f /etc/profile.env
USE=-nls FEATURES=-usersandbox BOOTSTRAP_RAP=1 ./bootstrap.sh

Reinitialize the environment:
env-update
exec bash -l
export PATH=$PATH:/bootstrap/bin:/bootstrap/sbin

In the following stages, you'll get tons of LD_PRELOAD libsandbox.so errors
from programs still linked against /bootstrap. This is safe to ignore, and
will gradually disappear as programs are rebuilt.

It'd be nice to remove the bootstrap tools now, but among wget/openssl, gawk,
gzip, grep, and other tools, they're still required for some reason.

From this point onward, you're pretty much on your own to beat portage into
performing an `emerge -e @system` and fixing any unspecified dependencies you
encounter along the way. This can be done in a whole number of ways, and will
differ on any given day, since this method of bootstrapping is painfully
unsupported by gentoo devs. Below I detail my most recent procedure, which, I
can't stress enough, may or may not work at the time you're reading this.

Fix circular dependencies:
USE=-filecaps emerge -O1 sys-libs/pam
USE='-acl -xattr' emerge -O1 sys-apps/shadow  # required by acct-group/acct-user

Build everything:
emerge -be @system

Clean up bootstrap tools/configs:
rm -rf /bootstrap /sources
rm -f /etc/portage/make.conf

Reinitialize environment:
env-update
exec bash -l

NOTE: Re-entering chroot after this point:
chroot . /usr/bin/env -i /bin/su -l
NOTE: Cleaning up the chroot:
umount -R proc sys dev etc/resolv.conf

(optional) consider rebuilding everything again without /bootstrap, to ensure sanity:
emerge -be @system

To install everything into a clean root:
USE=build emerge --root /final sys-apps/baselayout
emerge --root /final -K -j$(nproc) @system
#emerge --root /final --sysroot /final -K --with-bdeps=y --root-deps -j$(nproc) @system
TODO: How to forcefully install _everything_

Now you're essentially done. You can move /final (as well as /var/db/repos,
/var/cache/distfiles and /var/cache/binpkgs) to a proper disk and start using
it, or copy the built binpkgs to an existing install and install them.  This
guide will further go over how to make a stage3 and install CD, in case you're
installing a new machine, or just want to know how that's done.

Prepare files for catalyst:
mkdir -p /var/tmp/catalyst/builds/default
cd /var/tmp/catalyst/builds/default
tar -C /final -cf stage3-amd64-latest.tar .
rm -rf /final

# Adjust to hash relevant for portage tree date, see https://gitweb.gentoo.org/proj/releng.git/log/
wget https://gitweb.gentoo.org/proj/releng.git/snapshot/releng-0430f1f0ced5a3137f284ec426b7431be7d53085.tar.bz2
tar xf releng-*.tar.bz2
cd releng-*/
sed -i -e 's:@TIMESTAMP@:now:g' -e "s:@REPO_DIR@:$PWD:g" releases/specs/amd64/*.spec
cd releases/specs/amd64

Build the stage3:
emerge dev-util/catalyst app-arch/pixz
catalyst -s now
catalyst -f stage1.spec && catalyst -f stage2.spec && catalyst -f stage3.spec
rm -f /var/tmp/catalyst/builds/default/stage3-amd64-latest.tar

Build the install CD (failed at time of writing, see https://bugs.gentoo.org/713946):
catalyst -f installcd-stage1.spec && catalyst -f installcd-stage2-minimal.spec