kit

kit
git clone https://git.ryansepassi.com/git/kit.git
Log | Files | Refs | README

Cross-compilation sysroots

This document is the design and implementation plan for packaging and distributing minimal sysroots that let kit users cross-compile for every platform in the release support set from any host.

Goals

Support set

Triple pattern libc variant Notes
{x86_64,aarch64,riscv64}-linux-musl musl static + stub .so Compiled from musl source
{x86_64,aarch64,riscv64}-linux-gnu glibc stub .so Stubs generated from reference glibc
{x86_64,aarch64}-windows-gnu mingw-w64 UCRT Import stubs + headers, open source
aarch64-apple-macos{14,15} libSystem stub .tbd stubs; headers by hash
{x86_64,aarch64,riscv64}-freebsd FreeBSD libc stub Stubs + headers, BSD license
{x86_64,aarch64,riscv64,riscv32}-none freestanding kit rt/ only; no sysroot kpkg

Freestanding targets require no sysroot kpkg; libkit_rt.a and rt/include/ serve that role and are already in-tree.

Sysroot kpkg layout

Single-root layout note (see ../RELEASE.md): kit consolidates everything it manages under one root, $KIT_HOME (default $XDG_DATA_HOME/kit ≡ ~/.local/share/kit). Installed sysroots therefore belong at $KIT_HOME/sysroots/<triple>/. The build/test provisioning today caches them under $XDG_CACHE_HOME/kit/sysroots (KIT_SYSROOTS_DIR in mk/env.mk); the SYSROOTS implementation should adopt the $KIT_HOME/sysroots location when it lands as a user-facing feature.

Each installed sysroot unpacks into $KIT_HOME/sysroots/<triple>/:

<triple>/
  include/          system headers
  lib/
    crt1.o          C runtime entry (where applicable)
    crti.o
    crtn.o
    libc.a          static libc (musl only; absent on stub-only platforms)
    libc.so         stub or real shared library
    libm.so
    libpthread.so
    ...             other platform libs
  tbd/              macOS .tbd stubs (macOS targets only)

The cc driver's -target flag resolves the sysroot by triple lookup under ~/.kit/sysroots/. Explicit --sysroot=<path> overrides this.

Library stubs

ELF stub .so (Linux glibc, FreeBSD)

A stub .so contains only what the linker needs: SONAME, exported symbol table, and GNU version info (.gnu.version, .gnu.version_r). No PT_LOAD segment, no .text, no .data. The file is ~10–50 KB instead of hundreds of KB for the real library.

Kit will include a stub generator (src/obj/elf_stub.c, exposed as kit sysroot gen-elf-stub) that:

  1. Reads a real .so (dynsym + versioning sections).
  2. Emits a minimal ELF with the same SONAME, all exported symbols as STT_NOTYPE/STB_GLOBAL/SHN_ABS, and the original .gnu.version_r chain.
  3. Records the source library's build-id in a .note.kit.stub section for traceability.

Stubs are generated offline from a reference installation (a glibc or FreeBSD system) and committed into the sysroot kpkg. They must be regenerated when the upstream ABI version changes.

macOS .tbd stubs

Apple's Text-Based Stub Library (.tbd, YAML) is the native macOS stub format used inside Xcode SDKs. Kit's Mach-O linker already knows how to read .tbd files (required for linking against system frameworks). The macOS sysroot kpkg contains .tbd files for libSystem, libm, libpthread, libc++, and the core framework stubs.

The .tbd files are open/YAML; they can be extracted from any macOS SDK installation and are small.

Windows import stubs (.dll.a)

mingw-w64 provides import stub archives for the Windows SDK DLLs (kernel32, ucrtbase, etc.) under a MIT-compatible license. The Windows sysroot kpkg bundles these directly alongside the mingw-w64 UCRT headers. No generation step is needed; the mingw-w64 tree is the upstream source.

musl: compiled from source

For *-linux-musl targets kit compiles musl directly rather than shipping pre-built archives. Advantages: reproducible, architecture-correct, no redistribution complexity, and a good exercise of kit's self-compilation pipeline.

The musl source tree (a pinned release tarball, verified by BLAKE2b hash) is bundled in the kpkg for each architecture. kit sysroot install aarch64-linux-musl fetches the kpkg, extracts the musl source, and compiles it with kit cc -target aarch64-linux-musl into:

The compile is cached by the musl source hash, so reinstalling the same kpkg version is a no-op.

macOS SDK headers: CAS-referenced blobs

Apple's SDK headers may not be redistributed by third parties. The macOS sysroot kpkg solves this by referencing the header blobs by CAS hash without embedding them:

# excerpt from the macOS sysroot kpkg manifest
headers:
  required_blobs:
    - hash: "sha256:abc123..."   # usr/include/stdio.h
      install_path: include/stdio.h
    - hash: "sha256:def456..."   # usr/include/sys/types.h
      install_path: include/sys/types.h
    ...

When kit sysroot install aarch64-apple-macos14 runs:

  1. Kit resolves each required blob from the local CAS.
  2. If any blobs are missing, it prints the missing hashes and instructs the user to run kit sysroot extract-macos-headers.
  3. Once all blobs are present, the sysroot is materialized normally.

The blobs themselves are ordinary CAS objects — the user may host them on any CAS-compatible store (private server, S3, local NAS) and point kit at it via KIT_CAS_REMOTE or kit cas remote add.

kit sysroot extract-macos-headers

kit sysroot extract-macos-headers [--sdk-path=<path>] [--target=<triple>]

Walks the SDK's usr/include/ (and selected framework headers), ingests each file as a CAS blob, and prints a manifest fragment with the resulting hashes. This is run once per SDK version by the user on their Mac. It does not upload anything; it just populates the local CAS. The user may then push those blobs to their preferred remote.

The manifest fragment output is what kit's sysroot release process uses to build the macOS kpkg. For users who just want things to work locally, the extract step followed by kit sysroot install is sufficient.

kit sysroot command surface

kit sysroot list                         # show installed sysroots
kit sysroot available                    # show sysroots available in configured remotes
kit sysroot install <triple>             # fetch + install sysroot kpkg
kit sysroot install <triple> --from=<path>  # install from a local kpkg file
kit sysroot remove <triple>              # uninstall
kit sysroot update [<triple>]            # reinstall latest version
kit sysroot path <triple>                # print install path (for --sysroot= use)

kit sysroot extract-macos-headers [--sdk-path=<path>] [--target=<triple>]
kit sysroot gen-elf-stub <input.so> -o <stub.so>
kit sysroot gen-kpkg <triple> --config=<sysroot.toml> -o <output.kpkg>

gen-kpkg is the sysroot builder used offline by kit's release pipeline. It reads a sysroot.toml that declares the source for each component (musl source hash, upstream .so paths to stub-ify, mingw tree path, .tbd directory, macOS header manifest), runs the appropriate generators, and emits a signed kpkg.

Implementation work items

Infrastructure

Sysroot sources and generation

CI and distribution

Open questions