Kit: a BusyBox for compilation

A Unix C toolchain is already a collection of programs. cc coordinates several of them: preprocessing and compilation, often an assembler, and finally a linker with the appropriate startup files and system libraries. Around that is the binutils suite for archives, symbols, object inspection, stripping, copying, and disassembly.

That still stops at producing a program. JIT execution and WebAssembly runtimes usually arrive as separate systems. Build tools sit above the toolchain, package tools sit after it, and SDK acquisition is another concern again. Each layer has its own configuration, target vocabulary, cache, and distribution conventions.

BusyBox simplified a similar problem for basic Unix utilities: put one coherent implementation behind many familiar command names. Kit applies that model to a compilation toolchain.

Kit is one multi-call executable containing a C compiler, assembler, linker, JIT, debugger, binary utilities, build system, and package tools. It can be used through the kit command:

kit cc main.c -o app
kit ar rcs libapp.a app.o
kit objdump -h -t app

Or kit install can create the usual tool names as symlinks, with hard links on Windows:

cc main.c -o app
ar rcs libapp.a app.o
objdump -h -t app

The command set includes the usual compiler and binutils surface—cc, cpp, as, ld, ar, ranlib, nm, size, strip, objcopy, objdump, and addr2line—along with less conventional tools such as run, dbg, mc, disas, build, cas, pkg, and update.

The BusyBox analogy explains the interface, but not the more useful property: these are views of one library, not unrelated programs packed into one file.

One compilation model

The compiler, linker, object tools, and JIT share the same target registry, code-generation interfaces, object model, relocation machinery, and debug information. A source file can move through the same implementation and end up as an object file, a linked executable, an in-process JIT image, optimizer IR, or portable C.

kit cc -c add.c -o add.o
kit cc --emit=ir -O1 add.c -o add.ir
kit run add.c
kit run --no-jit add.c

At -O0, semantic code generation feeds a native backend directly. At -O1, kit records an IR, performs lightweight optimization and register allocation, and then replays into the same backend. -O2 currently aliases -O1; the planned SSA pipeline belongs to a future distinct -O2 implementation.

The target backend produces kit's format-neutral object model. That object can be written as ELF, Mach-O, PE/COFF, or Wasm, passed to the linker, inspected by the binary tools, or turned into a JIT image. The standalone assembler and inline assembler use the same machine-code emitter. mc exposes it directly:

$ kit mc -target aarch64-none-elf 'add x0, x0, #1'
add x0, x0, #1    # encoding: [0x00,0x04,0x00,0x91]

Cross-compilation uses the same binary. The compiled target registry covers x86-64, AArch64, RISC-V, Arm freestanding targets, and WebAssembly across ELF, Mach-O, COFF, and Wasm environments. kit targets reports both what a profile can do and what local provisioning it needs:

freestanding-rv64  riscv64-none-elf       elf    compile,assemble,link,execute [qemu]
macos-aa64         aarch64-apple-darwin   macho  compile,assemble,link,execute,selfhost [native]
windows-x64        x86_64-windows         coff   compile,assemble,link,execute,selfhost [vm]
wasm32-wasi        wasm32-wasi            wasm   compile,link,execute [sysroot]

A profile being present does not imply that its SDK, sysroot, emulator, or VM is installed. Hosted cross-compilation still needs the target's system files.

Rough performance

Kit's compile-speed target is closer to tcc than to a large optimizing compiler, while its generated code is intended to stay in the same general size range. A representative Apple M1 compile of the 9 MB SQLite amalgamation gives the following orientation:

Compiler	-O0 compile	-O1 compile	-O0 text	-O1 text	Speedup vs clang	Text vs clang
tcc	0.060 s	N/A	1.31 MiB	N/A	13x / N/A	1.3x / N/A
kit	0.160 s	0.88 s	1.29 MiB	0.95 MiB	5x / 7x	1.3x / 1.1x
clang	0.783 s	6.34 s	0.97 MiB	0.88 MiB	1x / 1x	1x / 1x

Ratio columns are -O0 / -O1; higher speedup and lower text ratio are better. Times are warm means: seven -O0 runs and three -O1 runs. Text is emitted machine code, not total object size. At -O0, kit is about 3 times slower than tcc and 5 times faster than clang; its text is 1% smaller than tcc's and 32% larger than clang's. -O1 shrinks text about 25% for kit and 10% for clang. Across the nine-source corpus, kit's -O1 text was about 6% larger. Results vary by workload and revision.

A C library, not a command-line wrapper

The executable is the reference host for libkit. The driver and language frontends consume the public headers under include/kit/; they cannot reach into compiler internals. Compilation, object construction, linking, JIT, debugging, hashing, compression, content storage, and packaging are all available through that library boundary.

Libkit is freestanding C11. It has no mutable global state and does not quietly reach into the operating system. The host supplies allocation, diagnostics, file I/O, clocks, executable memory, TLS, and debugger operations through explicit context structs and vtables. State belongs to a compiler, builder, link session, JIT session, or another appropriate context.

This is useful for embedding, but it also matters for bootstrapping. Many compiler bootstrap paths start with a small C compiler and eventually have to construct a C++ toolchain before they can build the compiler they actually want. Kit is implemented in C, including the compiler, assembler, linker, object tools, and runtime support, so the bootstrap can go directly from a capable C compiler to kit without a C++ stage.

The normal self-host build starts with a host C compiler, uses that kit to build kit again, and then repeats the build once more. The second and third kit binaries must be byte-identical. Replacing the initial host compiler with a smaller seed is a separate trust and engineering problem, but it does not require ascending from that seed into a C++ compiler first.

.kpkg: content identity plus trust

Software distribution often grows as a collection of conventions: a tarball, a checksum file, a detached signature, a package manifest, an update feed, and a separate cache format. Kit defines one package and trust model for these pieces.

The bottom layer is the content-addressed store exposed by kit cas. A blob is identified by the BLAKE2b-256 hash of its bytes. A tree is a canonical manifest of paths, modes, sizes, and blob IDs. Blobs and trees are self-verifying, so they can be copied from an untrusted cache or mirror.

The package layer adds claims that hashes alone cannot provide: this tree is a particular version of a named package, these paths are its executables or libraries, and this signer authorizes that manifest. A canonical kit-package manifest names one or more output trees and their artifact roles. Minisign-compatible Ed25519 signatures bind the manifest to a trusted key.

The native .kpkg format carries that model in a seekable container. A fat package embeds all trees and content; metadata and thin forms can externalize content into a CAS without changing the logical package identity. A portable .tar.gz representation carries the same signed manifest and objects for use with ordinary archive tooling.

Hosted SDKs are planned to use this distribution path as well. Packaging SDK files as signed .kpkg artifacts gives cross-toolchain provisioning the same content identity, mirrors, and trust model as other kit artifacts rather than a separate SDK mechanism.

The command-line workflow is explicit:

kit pkg keygen -o release-key
kit pkg create --name app --version 1 \
  -s release-key.key --root out -o app.kpkg
kit pkg verify -p release-key.pub app.kpkg
kit pkg unpack --verify -p release-key.pub app.kpkg -C installed

Trust can come from an explicit public key, a managed trusted-key store, or a deliberate trust-on-first-use operation. A public key bundled inside a package does not become trusted merely because it is present. Content mirrors remain untrusted: bytes must match their signed content IDs before use.

Kit uses the same mechanism to distribute itself. kit update authenticates a signed release-channel index and its selected .kpkg, installs the version under KIT_HOME, and atomically changes the active toolchain. Older versions remain available for offline rollback. The package format does not attempt to be a dependency solver, and network transport remains driver policy; it defines the artifact, identity, and trust boundary.

kit build is a build system

Once artifacts have a standard content identity, the same model can be applied to producing them. Putting the compiler and linker in one executable removes one layer of toolchain assembly, but it does not answer which commands should run, with which inputs, or when their results can be reused.

kit build is kit's answer. It is a full content-addressed build system built on the CAS and tree model used by .kpkg.

A package-local BUILD.kit maps target names to recipe executables:

kit-build 1
[target app]
recipe recipes/app.sh

The target can then be requested by label:

kit build //:app

A recipe is an ordinary executable, often a shell script. It receives a clean environment, writes its result below KIT_BUILD_OUT, and asks the coordinator for inputs through the recipe protocol. kit build source, glob, need, fetch, and config-get declare source files, filesystem selections, target dependencies, hash-pinned downloads, and configuration values. Arguments after -- form local target configuration and participate in the cache key.

The output is a canonical directory tree identified by its BLAKE2b-256 hash. A successful build prints both the tree ID and its materialized path. Configuration maps, argument vectors, source blobs, dependency closures, and output trees are also stored by content identity rather than timestamps.

The cache records two forms of trace. A deep trace is the fast path for the ordinary case where none of a target's transitive inputs changed. A shallow trace handles the more interesting case where something changed below a target but the direct dependency's output did not. If a comment-only edit causes a library recipe to run but produces the same library tree, its dependent application does not need to relink.

The build system also has workspace packages, canonical target labels, configuration profiles, external repositories pinned by content, test targets, and explicit dependency requests that can be submitted and awaited separately. The build language stays small; complicated policy lives in recipe programs rather than in a second general-purpose language embedded in BUILD.kit.

There are some deliberate limits. Recipes are responsible for declaring all inputs; filesystem hermeticity is not yet enforced. Workspace files are live rather than snapshotted during a build, the current hosted scheduler is sequential, and build-store garbage collection is not implemented. The cache is only as correct as the recipe's declared-input contract.

Scope

Kit is not intended to hide that it is a relatively small toolchain. -O1 is the current optimizer. Shared-library creation is an ELF feature. WebAssembly supports same-invocation source builds and partial WASI, but not general separate-object archive linking. Hosted cross targets require their SDKs or sysroots. Some debugger and execution paths necessarily depend on the host.

Those boundaries are easier to reason about because the system is coherent. A target is described once. An object is represented once. The linker and JIT use the same relocation machinery. Builds and packages agree on what a content tree is. The command-line tools exercise the same APIs available to an embedder.

BusyBox made a base Unix environment easier to carry by turning many utilities into one program. Kit takes the same approach to compiling, linking, building, and distributing code.