Portability test surface: cross + selfhost over one support set

This doc is the spec for kit's portability test surface — the harmonized top-level make targets and the scripts beneath them that answer two questions:

1. cross — can kit, running on the dev host, cross-compile a correct executable for every target in its support set?
2. selfhost — can kit be built to run on each target, and then compile + run a program there?

It supersedes the old scattered targets (test-hosted*, test-libc*, test-freebsd*, test-coff-windows-*, test-toy-*-vm, test-link-x64, test-parse-rv64-wide, bootstrap-linux*, bootstrap-freebsd, the standalone windows_cross.sh, and the freestanding smokes). See "Migration" at the end.

Related: BOOTSTRAP.md (the 3-stage self-build mechanism), SYSROOTS.md (sysroot provisioning), windows.md.

Model: two modes over one matrix

                         support set (one canonical list in scripts/hosted.sh)
                ┌──────────────────────────────────────────────────────┐
   mode cross   │  host kit builds for T   →  artifact runs correctly on T │
   mode selfhost│  build a kit that runs on T → it builds + runs a program │
                │                              on T                         │
                └──────────────────────────────────────────────────────┘
   substrate (native / qemu-user / podman / VM / qemu-system) = IMPLICIT,
   resolved per target by the exec seam; never named in a target.

The support set, one token grammar everywhere: <os>[-<libc>]-<arch>.

cross matrix (16):
  linux-{glibc,musl}-{aa64,x64,rv64}     (6)
  freebsd-{aarch64,amd64,riscv64}        (3)
  windows-{aarch64,x64}                  (2)
  macos-aarch64                          (1)
  freestanding-{aa64,x64,rv64,rv32}      (4)   ← rv32 lives here only

selfhost matrix (12): the same, minus freestanding

Freestanding has no OS, so there is nothing to run a compiler on: it participates in cross only. selfhost of a freestanding-* target is a hard error.

Arch tokens are the short forms aa64 / x64 / rv64 (plus rv32, freestanding-only). The long forms (aarch64/amd64/x86_64/riscv64) are accepted as input aliases by scripts/hosted.sh.

Top-level make targets

make test-port                                   # test-cross + test-selfhost
make test-cross    [TARGET=…] [DEPTH=…] [KIT_VM=…] [RUN=0|1]
make test-selfhost [TARGET=…] [DEPTH=…] [KIT_VM=…]
make kit-cross     [TARGET=…] [CROSS_CC=kit|clang] [VERIFY=0|1]
make provision     [TARGET=…]            [KIT_VM=…]

Defaults: TARGET ?= all, DEPTH ?= smoke, KIT_VM ?= 1, RUN ?= 1, CROSS_CC ?= kit, VERIFY ?= 0.

test-port is not part of the default make test (it is heavy and needs provisioning). The native 3-stage bootstrap + test-bootstrap-toy stay in the default suite as the fast self-reproduction check.

TARGET= selector grammar

Resolved centrally by scripts/hosted.sh expand <selector> [--mode=cross|selfhost]:

Selector	Expands to
all	every support-set token (mode- and KIT_VM-filtered)
linux / freebsd / windows / macos / freestanding	all configs for that OS
linux-musl / linux-glibc	that libc, all 3 arches
linux-glibc-x64, freebsd-aarch64, freestanding-rv32, …	one exact config
a,b,c	comma list of any of the above

--mode=selfhost drops freestanding-* from the expansion. KIT_VM=0 drops freebsd-* and windows-*.

DEPTH

test-cross accepts:

• coarse — cross-compile and link the smoke artifacts across the expanded support set, but do not execute them. This is the named form of make test-cross RUN=0.

• smoke (default) — two cases, building up platform coverage:

  1. exit — an #include-free exit-code program (test/cross/cases/exit.c, returns 42): toolchain + link + crt + the exit-code path; on freestanding the bare-metal exit oracle. Runs on every config (the only case freestanding can run — no libc there).
  2. hello — the libc hello-world (hello.c, prints + returns 0): the sysroot headers + libc + stdout. Hosted configs only.

  Each case runs once per link mode (link_modes): musl exercises both static (libc.a) and dynamic (libc.so) — the two ways the sysroot is consumed; lanes are suffixed :static/:dynamic. FreeBSD is static; glibc is dynamic-only (static glibc is discouraged / NSS-fragile); macOS/Windows use their single default shape. So a musl target yields four smoke lanes (exit/hello × static/dynamic), a glibc target two, freestanding one.

• full — the two smoke cases + the toy and parse corpora (cross-compiled and run on the target) + the libc cases (linux only). Orchestrates the existing mature test/{toy,parse,libc} runners with the right arch/tag; it does not reimplement them.

smoke is the default to honor "prefer targeted runs over mass runs" (CLAUDE.md). DEPTH=full is the deliberate heavy run, usually with a scoped TARGET.

test-selfhost accepts smoke and full; it has no build-only coarse lane because its purpose is to run the newly built compiler on the target.

kit-cross — cross-build the compiler itself

test-cross asks "can kit cross-compile a program for a target?"; kit-cross asks "can kit (or clang) cross-compile kit itself to run on a target?". It is the general form of the old windows_cross.sh: pick any hosted token and a backend, and it produces a runnable kit binary.

make kit-cross TARGET=windows-x64                 # kit.exe for windows-x64, via kit
make kit-cross TARGET=linux CROSS_CC=clang VERIFY=1

• CROSS_CC=kit (default) dogfoods kit as the cross-compiler; CROSS_CC=clang uses an independent clang + lld (and llvm-mingw's compiler-rt for Windows) toolchain — a useful differential check, since clang's strictness surfaces source issues kit's frontend tolerates.
• VERIFY=1 runs the freshly built kit on the target through the same exec seam the tests use (kit_cross_verify.sh: bare kit prints its banner and exits 0 — exercises load + dynamic-linker + libc init + main).
• freestanding-* is excluded (kit needs an OS to run on): the selector uses expand --mode=selfhost.
• Output: build/kit-cross/<backend>/<target>/kit[.exe]. The per-target engine is scripts/kit_cross.sh; the makefile loop is in mk/port.mk.

The two backends differ only in toolchain wiring, not source: the Windows sources are plain-clang-clean (no __try, no _environ extern, __thread rather than __declspec(thread)), and kit cc auto-appends .exe to an extension-less -o for a Windows target exactly as gcc/clang-mingw do, so both backends write kit.exe directly. The GC flag is the one linker-specific knob: GNU/lld and kit ld take --gc-sections, Apple ld (clang on darwin) takes -dead_strip.

Architecture

Source of truth — scripts/hosted.sh

Already owns the support set and the triple/path/tag resolvers. Extended with:

• hosted.sh list [selector] — emit tokens (one per line).
• hosted.sh expand <selector> [--mode=…] — the selector grammar above.

The Makefile reads it via $(shell …); nothing else re-encodes the matrix.

Two orchestrators

• scripts/cross_test.sh <selector> [DEPTH] — expand (mode=cross) → fail-fast provisioning pre-flight → compile/link all smoke artifacts → optional execution phase (DEPTH=coarse, RUN=0, or KIT_CROSS_RUN=0 disables it) → aggregate report.
• scripts/selfhost.sh <selector> [DEPTH] — expand (mode=selfhost) → pre-flight → per-token self-host (dispatch by OS) → on-target corpus → report.

Both share test/lib/kit_sh_report.sh for the verdict/summary layer.

Provision-or-error

Provisioning (network + VM prepare) is separate from running, and a requested target whose sysroot / image / VM is missing is a hard error, never a silent skip. The orchestrators do an aggregated pre-flight: they collect all missing provisioning for the expanded TARGET set, print each with the exact make provision TARGET=… (or host-tool install) to fix, and exit non-zero before running anything.

make provision [TARGET=…] wraps hosted.sh prepare, make test-images, the glibc run images, the Windows UCRT sysroots, and VM prepare — scoped by TARGET and KIT_VM. rt archives are local build artifacts (not network): the runners build them on demand via idempotent make rt-….

One exec front door, three backends

test/lib/exec_target.sh is the single tag-dispatched front door. A tag is <arch>-<os>[-<libc>]. It routes to one of three backends:

exec_target.sh
├── stateless          os ∈ {linux, macos}     → native / qemu-user / podman
├── exec_vm.sh         os ∈ {freebsd, windows}  → boot VM, run
└── exec_bare.sh       os == freestanding       → qemu-system bare-metal   (NEW)

exec_bare.sh is the consolidated owner of all per-arch bare-metal scaffolding (reset stub + linker script + exit-code oracle + qemu-system invocation), generalizing the old exec_rv32_bare.sh to four arches and absorbing the stubs that were inlined in freestanding_system.sh. It exposes two contracts:

• exec_bare_run_image <arch> <kernel.elf> <out> <err> <rc> — run a ready bootable image (what test/link's kernel_image cases need; exec_kernel.sh is now a thin shim over this).
• exec_bare_setup <arch> <work> + exec_bare_run <arch> <obj> <work> <rc> — link a corpus .o (entry main, returns the exit code) with the per-arch stub
  • rt into a bootable image, then run it (what toy/parse need).

Exit-code oracle per arch (so callers compare rc == expected uniformly):

arch	mechanism	decode
aa64	ARM semihosting hlt #0xf000 + ADP_Stopped_ApplicationExit	qemu rc = guest code
rv64/rv32	SiFive test finisher MMIO at 0x100000	stub writes 0x3333\|(code<<16) (or 0x5555 for 0); qemu rc = code
x64	isa-debug-exit (iobase 0x501)	qemu rc = (code<<1)\|1 → code = (rc-1)>>1

Self-host shapes (the substrate asymmetry)

selfhost.sh dispatches by OS, because "build a kit that runs on T" differs:

• macos → native 3-stage bootstrap here, then the corpus through stage3.
• linux → native 3-stage in a podman container (emulated for non-host arch), then the corpus. Generalizes scripts/linux_bootstrap.sh to (arch, libc). rv64 uses a hybrid seed: an in-container clang stage1 would itself run emulated (slow), so the stage1 seed is cross-built on the host with clang (kit_cross.sh linux-musl-rv64 --cc=clang, native speed) and fed to mk/bootstrap.mk via BOOTSTRAP_SEED; only stages 2/3 run emulated. They are still byte-identical because the seed and stage2 are the same kit source, hence functionally identical compilers. Emulated builds write objects to container-local storage (the :Z virtiofs mount flakes new-file creates under sustained emulated write load) and copy the stage kits back.
• freebsd → native 3-stage in the VM, then the corpus (scripts/freebsd_bootstrap.sh).
• windows → cross-build kit.exe on the host (the VM has no seed compiler), then run it on the VM to cc + run a program (scripts/windows_cross.sh).

Every selfhost leaf ends with the same contract: a kit running on the target compiled and ran a program there.

Status & phased backlog

cross:

Config	coarse	smoke	full
linux-{glibc,musl}-{aa64,x64,rv64}	ready	ready	ready (toy X + parse E + libc)
macos-aarch64	ready	ready (native)	ready (native toy/parse)
freebsd-{aarch64,amd64,riscv64}	ready	ready (VM)	ready (toy via VM)
windows-{aarch64,x64}	ready	ready (VM)	ready (toy via VM)
freestanding-rv32	ready	ready (bare)	ready (toy X + parse E, bare)
freestanding-{aa64,x64,rv64}	ready	ready (bare smoke)	deferred

selfhost: macos (native), linux musl/glibc aarch64 (container), linux x64/rv64 (container under emulation), freebsd aarch64/amd64 (VM), windows aa64/x64 (cross

• VM).

Backlog: freestanding DEPTH=full for aa64 / x64 / rv64

Running the toy/parse corpora bare-metal on these three arches is new capability beyond the smoke payload. The smoke stubs in exec_bare.sh are minimal; the corpus exercises TLS, soft-float, large frames, i128, etc., which need hardened per-arch stubs (the rv32 path already has this — its reset stub seeds a static TLS image and enables the FPU). Work items:

1. Harden the aa64/rv64 exec_bare stubs (TLS image seed + thread-pointer setup, matching the rv32 stub and the toy X-lane start.c).
2. Build the x64 long-mode corpus stub (the rv32/aa64 reset path + a full exit-code oracle via isa-debug-exit).
3. Wire a bare lane into test/{toy,parse}/run.sh for aa64/x64/rv64 (the rv32 "X / V" path generalized), gated on qemu-system-<arch>.
4. Flip the table rows above from deferred to ready and drop this section.

Until then, test-cross TARGET=freestanding-{aa64,x64,rv64} DEPTH=full runs the smoke payload and logs that corpus-depth is not yet wired for the arch (it does not silently claim full coverage).