bootstrap/gcc/notes/gentoo/gentoo_notes.txt

Based on portage-3.0.8 and gentoo-20201031 snapshot

The specifics of this document (software versions and whatnot) are subject to
change as gentoo evolves, but I hope the big lines of it won't change too
much.

This guide starts off with a system built from build_cross.sh, built for
x86_64 with i686 multilib.
It might be possible to adapt both that script, and this guide for other
architectures, but this guide won't go into detail.

You should be running a system with a x86_64 kernel (with i686 support).
Install the required sources (listed in gentoo.wget-list) into /sources.

To enter the bootstrap system as a chroot, use the following commands to set
up the filesystems and entering:
mkdir -p proc sys dev tmp
mount -t proc proc proc
mount -t sysfs sysfs sys
mount -t devtmpfs devtmpfs dev
mount -t devpts devpts dev/pts
mount -t tmpfs tmpfs dev/shm
mkdir -p etc
touch etc/resolv.conf
mount --bind /etc/resolv.conf etc/resolv.conf
./bootstrap/bin/chroot . /bootstrap/bin/env -i PATH=/bin:/sbin:/usr/bin:/usr/sbin:/bootstrap/bin:/bootstrap/sbin HOME=/root /bootstrap/bin/sh -l

To install all the prerequisite tools, place the sources (listed in
gentoo.wget-list) under /sources, then run the gentoo_tools.sh script.

Configure portage:
echo "root:x:0:0:root:/root:/bin/bash" > /etc/passwd
echo "root:x:0:root" > /etc/group
echo "portage:x:250:250:portage:/var/tmp/portage:/bin/false" >> /etc/passwd
echo "portage::250:portage" >> /etc/group
echo "C UTF-8" > /etc/locale.gen
mkdir -p /tmp /root
chown root.root /tmp /root
chmod 1777 /tmp
chmod 0700 /root
cat > /etc/portage/make.conf << 'EOF'
FEATURES='-pid-sandbox -news'
ROOTPATH=/bootstrap/bin:/bootstrap/sbin
EOF
ln -sf ../../var/db/repos/gentoo/profiles/default/linux/amd64/17.1/ /etc/portage/make.profile

Install ebuild repository (pick appropriate date, see distfiles.gentoo.org/snapshots):
emerge-webrsync -v --revert=20201031

Install some ebuild prerequisites:
emerge -O1 sys-apps/gentoo-functions  # Used by elt-patches and a ton of other things
emerge -O1 app-portage/elt-patches  # required by elibtoolize(libtool.eclass), used to install any library
emerge -O1 sys-devel/gnuconfig  # required by gnuconfig_update(gnuconfig.eclass), used by toolchain(-glibc).eclass, doesn't hurt to have
emerge -O1 sys-devel/binutils-config sys-devel/gcc-config

First, we will build an updated binutils+gcc combo that targets /usr instead
of /bootstrap. This makes it easier to bootstrap glibc and everything else. We
will use portage for this, even though it's a mess.

NOTE: Doing this is (probably) *BAAAAD*, since we're linking the compiler
support libraries with a potentially incompatible version of glibc in
/bootstrap, and upgrading it afterwards. Glibc should always be installed
first into a new prefix! However, due to the complex relationship between
glibc, gcc, libgcc and libstdc++, this is rather hard to do while wrangling
the change from /bootstrap to /usr, as well as the multilib libdir changes, so
for simplicity's sake, I won't bother. This just works™.

Install temporary libc files for target compiler:
mkdir -p /lib64 /lib
ln -sf /bootstrap/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
ln -sf /bootstrap/lib32/ld-linux.so.2 /lib/ld-linux.so.2
mkdir -p /usr/include
ln -sf /bootstrap/include/limits.h /usr/include/limits.h
mkdir -p /usr/lib64 /usr/lib
ln -sf /bootstrap/lib/{crti,crtn,Scrt1}.o \
       /bootstrap/lib/{libc,libm,libpthread}.so /usr/lib64/
ln -sf /bootstrap/lib32/{crti,crtn,Scrt1}.o \
       /bootstrap/lib32/{libc,libm,libpthread}.so /usr/lib/

Install binutils+GCC (linked against /bootstrap):
USE='-*' emerge -O1 sys-devel/binutils
USE='-* static-libs' EXTRA_ECONF=--disable-shared \
    CPPFLAGS='-isystem/usr/include' LDFLAGS='-L/usr/lib64 -L/usr/lib' \
    emerge -O1 dev-libs/gmp dev-libs/mpfr dev-libs/mpc
USE='-*' EXTRA_ECONF=--disable-bootstrap GCC_MAKE_TARGET=all \
    CPPFLAGS='-isystem/usr/include' LDFLAGS='-L/usr/lib64 -L/usr/lib' \
    CPATH=/bootstrap/include \
    emerge -O1 sys-devel/gcc
emerge --rage-clean dev-libs/gmp dev-libs/mpfr dev-libs/mpc
gcc-config 1  # Not automatic unlike binutils-config...

NOTE: The Glibc ebuild downloads a file called gcc-multilib-bootstrap, which
contains prebuilt binaries to bootstrap multilib gcc. This is _only_ used when
the command "echo 'int main(){}' | $CC -m32 -xc -" fails, and shouldn't be
being used in this bootstrap, but you can make sure of that by patching the
ebuild to `die` instead of running the `sed` command to enable its use.

NOTE: BOOTSTRAP_RAP=1 is only used in the Glibc ebuild, to avoid a pre-install
check that might pick up programs from /bootstrap to test against the
installed glibc, as well as picking up python from $PATH instead of
eselect-python. I'm not aware of how the support status is for this variable,
and as such it might disappear in the future, or be used in more packages.

Install initial glibc:
CC=/bootstrap/bin/gcc emerge -O1 sys-kernel/linux-headers
USE='-*' BOOTSTRAP_RAP=1 emerge -O1 sys-libs/glibc

NOTE: `sandbox` outputs a failure message to stderr for every program still
built against /bootstrap, which includes the compiler. Some autoconf scripts
check stderr to determine a failure, so this causes builds to fail. This is
why it's (partially) disabled for this stage.

Bootstrap the rest of the system:
USE=-seccomp emerge -O1 app-misc/pax-utils
cd /var/db/repos/gentoo/scripts
mkdir -p /var/run
ln -s /bootstrap/bin/python3 /usr/bin/python
rm -f /etc/profile.env
USE=-nls FEATURES=-usersandbox BOOTSTRAP_RAP=1 ./bootstrap.sh

Reinitialize the environment:
env-update
exec bash -l
export PATH=$PATH:/bootstrap/bin:/bootstrap/sbin

In the following stages, you'll get tons of LD_PRELOAD libsandbox.so errors
from programs still linked against /bootstrap. This is safe to ignore, and
will gradually disappear as programs are rebuilt.

It'd be nice to remove the bootstrap tools now, but among wget/openssl, gawk,
gzip, grep, and other tools, they're still required for some reason.

From this point onward, you're pretty much on your own to beat portage into
performing an `emerge -e @system` and fixing any unspecified dependencies you
encounter along the way. This can be done in a whole number of ways, and will
differ on any given day, since this is painfully unsupported by gentoo devs.
Because the order in which you install everything can differ wildly, and
consequently affect the output of your build, it is advised to do a second
`emerge -e @system` afterwards, for sanity. Below I detail my most recent
procedure, which, I can't stress enough, may or may not work at the time
you're reading this.

Fix circular dependencies:
USE=-filecaps emerge -O1 sys-libs/pam
USE='-acl -xattr' emerge -O1 sys-apps/shadow  # required by acct-group/acct-user

Verify there's no circular deps left, for sanity:
emerge -fe @system

Build everything:
emerge -e @system

Clean up bootstrap tools/configs:
rm -rf /bootstrap /sources
rm -f /etc/portage/make.conf

Reinitialize environment:
env-update
exec bash -l

Consider rebuilding everything again without /bootstrap, since a lot of
packages couldn't be properly stripped, and a bunch of other tools have been
upgraded, during the first emerge -e @system:
emerge -be @system

To install everything into a clean root:
ROOT=/final emerge -K @system

Prepare files for catalyst:
mkdir -p /var/tmp/catalyst/snapshots
wget -O gentoo-latest.tar.xz http://distfiles.gentoo.org/snapshots/gentoo-latest.tar.xz
mkdir -p /var/tmp/catalyst/builds/default
cd /var/tmp/catalyst/builds/default
tar -C /final -cf stage3-amd64-latest.tar .
xz -9v stage3-amd64-latest.tar
rm -rf /final
wget -O stage1.spec "https://gitweb.gentoo.org/proj/releng.git/plain/releases/specs/amd64/stage1.spec"
wget -O stage2.spec "https://gitweb.gentoo.org/proj/releng.git/plain/releases/specs/amd64/stage2.spec"
wget -O stage3.spec "https://gitweb.gentoo.org/proj/releng.git/plain/releases/specs/amd64/stage3.spec"
sed -i -e 's/@TIMESTAMP@/latest/' -e '/^portage_confdir: /d' *.spec

Build the stage3:
emerge catalyst
catalyst -f stage1.spec

Clean up chroot:
umount -R proc sys dev etc/resolv.conf