kit Design

kit is a freestanding C11 compiler multi-tool, written in C11. This document is the front door to the architecture: it states what kit is, the principles that shape it, the layered design, the primary data flows, and an index of the subsystem references. The task-oriented reader map is README.md. API signatures and struct layouts live in the headers under include/kit/.

What kit is

A single multi-call binary (kit) that bundles a complete C toolchain plus the machinery to JIT and debug what it produces. The compiled tool registry (kit --help) and shipped public headers are the current surface; forward-looking release work lives in plan/RELEASE.md. Development surfaces, such as the Toy frontend and unregistered emu command, remain documented for maintainers. Capabilities:

• C11 preprocessor, single-pass parser/type checker, and code generator.
• A JIT compiler, an in-process runner, and an interactive debugger.
• A linker (objects/archives/DSO inputs -> executable or shared image), with basic linker-script support and file-based incremental linking.
• A standalone assembler (GAS subset) and inline assembler sharing one emitter.
• A lightweight no-SSA optimizer behind -O1: recorded IR, CFG/local cleanup, machinization, and linear-scan register allocation. The maintained SSA mid-end is reserved for a future distinct -O2; the accepted -O2 spelling currently aliases -O1.
• Cross-compiling backends for aarch64, x86-64, riscv64, and WebAssembly, plus a portable C-source backend.
• Object read/write for ELF, Mach-O, and PE/COFF; a Wasm object form.
• DWARF debug-info production and consumption; a disassembler.
• A bytecode interpreter over the optimizer IR (run --no-jit).
• Signed, content-addressed code distribution (.kpkg).
• A content-addressed build system (kit build) with declared recipe inputs, output-tree identity, deep/shallow dependency traces, workspace packages, and signed trace sharing.
• A public target-profile registry (kit targets) that separates compiler capability from local SDK/sysroot/runner provisioning.
• A parser/lexer generator (gram): EBNF in, C parser / lexer / token-machine tables out, with allocation-free push runtimes for the generated code.
• Object/archive utilities: ar, ranlib, nm, size, strip, objcopy, objdump, addr2line, strings.

Design principles

• Freestanding C11. The compiler builds and runs without a hosted libc; it ships its own headers and runtime so it can compile itself and target bare metal. The implementation obeys the same constraints it imposes on its output.
• No global state. There are no mutable globals or hidden singletons. All state hangs off an explicit context — a KitCompiler or a subsystem handle (KitObjBuilder, KitLinkSession, KitJit, KitDebugSession, KitEmu, frontend state) — so the library is reentrant and embeddable.
• The host supplies all side effects. libkit itself touches no OS. The host injects every side effect through vtables: heap, diagnostics, file I/O, metrics, and a clock via KitContext (include/kit/core.h); executable memory and JIT thread-local storage via KitJitHost (jit.h); and debugger OS hooks — threads, events, signal handling, code patching — via KitDbgHost (dbg.h). This is what makes "no global state" enforceable and keeps the library portable across hosts.
• No VLAs. Stack growth is bounded and predictable; dynamic sizing goes through arenas/heaps so freestanding and constrained targets stay safe.
• Strict modular layering. Each layer depends only inward, and subsystem internals are private. The public header set is the entire contract; crossing a boundary the wrong way is a design bug, not a shortcut.
• Multi-arch & multi-platform via vtables, not #ifdef. Architecture, ABI, and object-format variation is expressed as runtime dispatch tables — ArchImpl (src/arch/arch.h), TargetABI (src/abi/abi.h), and ObjFormatImpl (src/obj/format.h) — selected from the target triple. New targets are new table instances, not new conditional branches scattered through the tree.
• Build-time component gating. include/kit/config.h defines KIT_*_ENABLED flags for archs, object formats, languages, subsystems, and tools. The build drops disabled units entirely, so a minimal embedding pays only for what it uses, and the tool registry in driver/main.c is gated by the same flags.

Layered architecture

From outside in, each layer depends only on the layer beneath it:

driver/            CLI policy + host I/O. Includes ONLY <kit/*.h>.
  lang/            Frontends. C/cpp/toy consume <kit/*.h> + their own private
                   headers; wasm has one documented shared-module edge.
    include/kit/ PUBLIC BOUNDARY. The library's entire stable contract.
      src/api/     Composition: public handles <-> internal subsystems.
        src/...    Internal subsystems. Share private headers among their own
                   TUs; expose nothing except through include/kit/.

• driver/ implements the multi-call binary. driver/main.c holds the central tool table; each public tool (cc, as, ld, ar, run, dbg, cas, pkg, …) translates command-line flags into public API calls and supplies the host vtables. The content-addressed store and .kpkg packaging (tar/deflate/lz4, BLAKE2b, ed25519/minisign) are a libkit subsystem (src/dist/, behind <kit/cas.h> / <kit/package.h>); cas/pkg are thin CLIs over it.
• lang/ holds the frontends. lang/c preprocesses (lang/cpp), parses, type-checks, manages C declarations, and drives the public CG API; lang/wasm is the Wasm/WAT frontend and lang/toy is an internal test frontend exercising the same boundary. Each registers a KitFrontendVTable per compiler and emits through KitCg; no frontend owns object formats or linker policy.
• include/kit/ is the public boundary — the only headers driver/ and lang/ may include.
• src/api/ is the composition layer: it implements the public handles (src/api/compile.c, link.c, object_builder.c, archive.c, disasm.c, …) and wires enabled frontends (src/api/lang_registry.c). It is the single place where public types meet internal subsystems.
• src/ subsystems do the work: core (arenas, vectors, buffers, symbol interning, diagnostics, hashing), abi, arch, asm, cg, opt, obj, link, jit, dbg, emu, interp, debug (DWARF), wasm, os, and dist (content-addressed store + signed .kpkg packaging).

The layering invariant: driver/ includes only <kit/*.h> and the one frontend-public C header; C/cpp/toy frontends likewise stay on the public surface. The Wasm frontend's single src/wasm/wasm.h edge is a documented shared-internal module contract with the Wasm object/backend peers, not general permission for frontends to include src/. Anything else a frontend or tool needs is promoted into public headers. Subsystem *_internal.h headers stay private to their own translation units. See INTERFACES.md.

Key abstractions

• KitCg (include/kit/cg.h) is the frontend-facing code-generation API: a typed stack-machine IR over which all frontends emit functions, control flow, data, calls, and inline asm. It is the largest public contract and the point frontends couple to hardest.
• Tiered backend. A CgTarget (src/cg/cgtarget.h) receives the lowered CG stream. At -O0 a shared NativeDirectTarget adapts the physical NativeTarget (src/arch/native_target.h) directly; at -O1 the optimizer wrapper (src/opt/) records IR, runs the no-SSA O1 schedule, and replays into the same NativeTarget. The separate SSA schedule is implemented but remains unreachable while -O2 aliases -O1. Physical machine bytes flow through one arch-neutral MCEmitter (src/arch/mc.h).
• ObjBuilder (src/obj/obj.h) is the canonical in-memory object model during compilation, assembly, linking, JIT, inspection, and DWARF emission — one section/symbol/relocation store, with format knowledge behind ObjFormatImpl.
• Symbols. KitSym is an interned spelling (an identity, not a definition). Object builders use object-scoped symbol ids so locals from different objects never collide; the linker builds a separate resolved-symbol table across all inputs.

Primary data flows

1. C source -> object

driver cc -> KitContext + KitCompiler -> kit_compile_* (src/api/compile.c)
  -> registered C frontend (lang/c): lex -> preprocess -> parse/type/decl
  -> KitCg (public CG API)
  -> CgTarget  ( -O0 NativeDirect | -O1 opt wrapper )
  -> NativeTarget -> MCEmitter
  -> ObjBuilder -> object writer

The driver loads source bytes and picks options; src/api/compile.c creates an ObjBuilder and dispatches to the frontend registered for the input language. Assembly (.s) takes a shortcut: the asm frontend feeds the MCEmitter/ ObjBuilder path directly, bypassing KitCg because it is already target-level.

2. File link -> executable

objects / object bytes / archives / DSO inputs
  -> kit_link_* (src/api/link.c -> src/link/)
  -> object/archive readers -> symbol resolution -> layout
  -> relocation -> executable (or incremental patch) writer

The linker owns archive member selection, symbol resolution, section/segment layout, relocation (per-arch fixups behind ArchImpl), build/image-id handling, and final emission, for any enabled object format.

3. Run / JIT in-process

source/object inputs -> compile/link to a JIT LinkImage
  -> kit_link_jit  (KitExecMem from KitJitHost maps + protects pages)
  -> KitJit / KitDebugSession
  -> run (invoke entry) | dbg (breakpoints, stepping, regs/mem via KitDbgHost)

The JIT shares the same compile/object/relocation machinery as file output; only the final sink differs. Mapping executable memory and installing TLS are delegated to the host through KitJitHost. run --no-jit instead attaches a bytecode InterpProgram (src/interp/) and executes the entry through the interpreter while still using the JIT image for real data/extern addresses.

4. Emulate a guest ELF

guest ELF bytes -> emu ELF loader (src/emu/)
  -> decode/lift guest basic blocks
  -> CgTarget -> JIT image
  -> emu runtime (syscall + memory model)

The emulator is a user-mode ELF runner that translates guest basic blocks into the same backend/JIT infrastructure used for native JIT, executing them under a guest memory and syscall model.

State and ownership

The host owns storage and side effects (heap, file I/O, executable memory, TLS, debugger OS hooks); libkit owns compilation, object construction, linking, JIT layout, and relocation policy. Public APIs take explicit options and handles; internal state hangs off KitCompiler, the subsystem handles, or frontend context structs. Compile inputs are caller-owned byte buffers that must outlive the call; builders returned by compile are owned by the compiler until consumers finish; object/archive/DSO bytes handed to link calls are borrowed for the call unless an API states otherwise.

Documentation index

Doc	Covers
DESIGN.md	This map: what kit is, principles, layering, data flows, index.
INTERFACES.md	Interface inventory and review checklist across all tiers (public, backend, subsystem, core, frontend).
FRONTENDS.md	The lang/ frontends — C (preprocess/parse/type/decl), cpp, toy, wasm — and the frontend vtable contract.
CODEGEN.md	The KitCg public CG API and the tiered CgTarget -> NativeDirect/opt -> NativeTarget lowering path.
IR.md	The recording/optimizer IR: instructions, types, and how CG operations become analyzable functions.
ARCH.md	Per-arch backends (aarch64/x86-64/riscv64), ArchImpl dispatch, MCEmitter, register files, and fixups.
ASM.md	The standalone + inline assembler, GAS-subset syntax, and the shared emitter.
OPT.md	The optimizer: current no-SSA -O1, maintained future SSA -O2, register allocation, cleanup, and native replay.
INTERPRETER.md	The bytecode interpreter over the optimizer IR used by run --no-jit.
OBJ.md	The format-neutral object model and ELF/Mach-O/COFF/Wasm read/write behind ObjFormatImpl.
LINK.md	Linking: symbol resolution, layout, relocation, linker scripts, and incremental linking.
KERNEL.md	The freestanding kernel build + image pipeline: build-obj/build-exe, the linker-script subset, kit image / objcopy -O binary, flat-kernel Image headers, and kit cpio initramfs archives.
JIT.md	The JIT image model, executable-memory and TLS host hooks, and publish/append/replace.
EMU.md	Internal user-mode guest-ELF emulator design and per-block JIT translation.
DWARF.md	DWARF debug-info production and the consumer used by the debugger and dumpers.
DBG.md	The debugger: breakpoints, single-step, displaced execution, register/memory access.
CBACKEND.md	The portable C-source backend (src/arch/c_target/).
WASM.md	The WebAssembly backend, object form, and host-import binding.
DISTRIBUTE.md	Signed .kpkg packaging and the content-addressed store (src/dist/, <kit/cas.h> / <kit/package.h>, cas/pkg tools).
BUILD_COORDINATOR.md	The content-addressed build coordinator: <kit/build_coord.h>, kit build, trace/deepset caching, recipe protocol, and shared traces.
RELEASE.md	Releasing and updating kit itself: VERSION/kit --version, make dist + scripts/release.sh signed artifacts, the kit-release channel index, and kit update.
EMBEDDING.md	Linking libkit.a, host callbacks, handle lifetimes, compile/link sessions, frontend registration, and ownership rules.
DRIVER.md	The multi-call binary, tool registry, and command-line policy.
RUNTIME.md	The freestanding headers and compiler-rt/libc-style support in rt/.
BUILD.md	The build system and KIT_*_ENABLED component gating.
TESTING.md	The test suites and harnesses under test/.
PORT.md	The portability test surface: the harmonized cross + selfhost make targets and the substrates beneath them, parametrized over one support-set matrix.
BENCHMARKING.md	How to measure compile speed and code size reproducibly: instruction counts, Linux callgrind, the make bench-cc scaling guard, and the vs-tcc/clang bars.
CODE_SIZE.md	Line counts per component (per-format/per-target split from core).

Planned work and roadmaps live under doc/plan/; speculative not-yet-committed designs live under doc/ideas/.