commit 5d4228cb2de9464069e87e01ca86b62df9ca2b0a
parent 45d883cd4bfea07e86ecb7bb678d74d461d708d1
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Sun, 14 Jun 2026 13:15:59 -0700
doc: consolidate linker compatibility roadmap
Merge the ELF as-needed and TLSDESC planning into a single support-set roadmap for system-linker compatibility.
Diffstat:
2 files changed, 241 insertions(+), 0 deletions(-)
diff --git a/doc/plan/LINKER-COMPAT.md b/doc/plan/LINKER-COMPAT.md
@@ -0,0 +1,240 @@
+# System-linker compatibility
+
+This roadmap tracks the remaining linker work needed for `kit ld` to act as the
+system linker over kit's supported target set. The near-term driver is Rust
+using `kit ld` through `rustc`, but the target is broader: C, Rust, assembler
+objects, archives, DSOs/dylibs/import libraries, PIE executables, shared
+libraries, and relocatable links should all behave like the platform linker for
+the supported arch/OS pairs.
+
+Related: [LINKER.md](LINKER.md), [../LINK.md](../LINK.md),
+[../OBJ.md](../OBJ.md), [../DRIVER.md](../DRIVER.md),
+[SYSROOTS.md](SYSROOTS.md).
+
+## Scope
+
+The support set is format-driven:
+
+- Mach-O: macOS `aarch64` and `x86_64`.
+- ELF: Linux glibc/musl `aarch64`, `x86_64`, `riscv64`; FreeBSD where Rust and
+ sysroot artifacts exist; freestanding ELF for `aarch64`, `x86_64`, `riscv64`,
+ and `riscv32`.
+- COFF/PE: Windows UCRT MinGW (`*-pc-windows-gnullvm`) for `x86_64` and
+ `aarch64`.
+
+The linker should preserve format boundaries. ELF `--as-needed` and TLSDESC
+rules must not leak into Mach-O dylib or COFF import-library semantics. COFF's
+archive search behavior, Mach-O atom/GOT/TLV behavior, and ELF dynamic-section
+rules stay target-specific policy behind shared object and relocation APIs.
+
+## Current state
+
+The Rust-driven compatibility pass established these paths:
+
+- `kit ld` accepts the common system-linker flags Rust passes, including GNU
+ `-Wl,`/`-z` forms, target/sysroot flags, linker scripts, rlibs, exact
+ `-l:name` libraries, `-static-pie`, `-nodefaultlibs`, hosted/no-startfile
+ combinations, PE/MinGW flags, and Mach-O version/platform flags.
+- Rust std links through `kit ld` for macOS `aarch64`/`x86_64`, Linux glibc
+ `aarch64`/`x86_64`/`riscv64`, Linux musl `aarch64`/`x86_64`/`riscv64`,
+ FreeBSD `x86_64`, and Windows UCRT MinGW `aarch64`/`x86_64`.
+- Rust no-std links through `kit ld` for `aarch64-unknown-none`,
+ `x86_64-unknown-none`, `riscv64gc-unknown-none-elf`, and
+ `riscv32imac-unknown-none-elf`.
+- ELF symbol-version imports can resolve explicit `name@VERSION` references,
+ which FreeBSD Rust uses for libc compatibility symbols.
+- COFF archive libraries use global fixed-point search, matching the behavior
+ Rust/MinGW expects from the library set.
+- RISC-V TLS-GD and x86_64 TLS-LD can relax local executable TLS cases; AArch64
+ ELF TLSDESC relocation spellings are accepted and currently relax only for
+ local defined TLS.
+
+This is a useful system-linker subset, not completion. The open work below is
+the gap between "Rust examples link" and "the platform linker can be replaced
+over the support set."
+
+## Workstreams
+
+### 1. Ordered input and dependency selection
+
+ELF `--as-needed` must become real linker semantics, not a driver/script
+special case.
+
+The linker needs to model, per ordered DSO input:
+
+- current link mode (`default`, `--as-needed`, `--no-as-needed`),
+- selected/not-selected state,
+- why it was selected,
+- SONAME or fallback identity for `DT_NEEDED`,
+- the exports made available to later inputs.
+
+ELF behavior:
+
+- default/no-as-needed DSOs are selected as explicit dependencies,
+- as-needed DSOs are selected only when they satisfy an eligible strong
+ undefined reference at their position,
+- unselected as-needed DSOs do not satisfy later references,
+- selected DSOs are the source of `DT_NEEDED`,
+- duplicate SONAMEs are suppressed while preserving first selected order.
+
+GNU linker scripts should lower into the same ordered input model:
+
+- `INPUT(...)` inserts normal inputs with the current mode,
+- `GROUP(...)` preserves archive group behavior,
+- `AS_NEEDED(...)` temporarily pushes as-needed mode for nested DSO or `-l`
+ tokens,
+- nested scripts inherit and restore mode correctly.
+
+Mach-O and COFF need separate parity checks rather than ELF semantics:
+
+- Mach-O dylib load commands and weak/imported symbol behavior should follow
+ Darwin linker expectations.
+- COFF import libraries, auto-imports, weak externals, and library-set archive
+ search should remain COFF-specific.
+
+### 2. ELF TLS model planning
+
+ELF TLS must be planned before final relocation application and dynamic-section
+synthesis. The planner should classify each TLS relocation or relocation
+sequence by:
+
+- symbol locality and visibility,
+- imported/preemptible vs. non-preemptible,
+- weak undefined behavior,
+- executable/PIE/shared/relocatable output,
+- static vs. dynamic link,
+- target ABI TLS variant and thread-pointer bias.
+
+The action should be explicit:
+
+- relax to local-exec,
+- synthesize initial-exec,
+- preserve or synthesize dynamic TLS/TLSDESC,
+- preserve relocations for `-r`,
+- reject with a precise diagnostic.
+
+This pass should own the decision; architecture code should own only the
+instruction byte rewrite once a sequence has been validated.
+
+### 3. TLSDESC completion
+
+The current AArch64 TLSDESC path accepts Rust/LLVM executable-local cases by
+relaxing the four instruction relocations to materialize the local-exec offset.
+Full TLSDESC support needs more:
+
+- complete per-architecture relocation surfaces in `RelocKind`, ELF mapping,
+ relocation names, descriptor metadata, object read/write, and `objdump`;
+- descriptor GOT/data allocation, normally two machine words per unique
+ `(symbol, addend)` descriptor;
+- TLSDESC dynamic relocation emission in `.rela.dyn`;
+- integration with `.dynsym`, symbol versions, `DT_NEEDED`, PIE address shifts,
+ section GC, and DSO selection;
+- checked sequence relaxation for AArch64, then x86_64, then other ELF
+ architectures as ABI support lands;
+- shared-library output that preserves or synthesizes dynamic TLS where final
+ executable layout is not known.
+
+Imported TLSDESC references are real DSO uses and must select their provider
+under `--as-needed`.
+
+### 4. Architecture relocation and relaxation parity
+
+Each supported architecture needs both relocation coverage and checked
+relaxations for the code shapes produced by modern toolchains.
+
+Open areas include:
+
+- AArch64 ELF: complete TLSDESC dynamic path, validate local sequence
+ relaxation as a unit, cover host-toolchain GOT/TLS/unwind relocation shapes.
+- x86_64 ELF: complete TLS-LD/GD/IE/LE relaxation behavior and add TLSDESC
+ once modeled.
+- RISC-V ELF: keep expanding TLS-GD/LD pattern coverage conservatively, handle
+ compressed-instruction variants, and preserve dynamic TLS paths where local
+ relaxation is illegal.
+- Mach-O AArch64/x86_64: finish GOT/TLV/unwind relocation parity against
+ Apple-produced objects and static archives.
+- COFF AArch64/x86_64: continue matching MinGW/LLVM relocation spellings,
+ especially SECREL, unwind, pdata/xdata, and CRT object patterns.
+
+Relocation tests should pin both the descriptor table and representative byte
+patches so enum additions are covered immediately.
+
+### 5. Shared libraries and relocatable links
+
+Replacing a system linker requires more than executable links:
+
+- `-shared` should emit correct dynamic symbols, relocations, SONAME/install
+ names/import tables, and TLS behavior for each format.
+- `-r` should preserve relocations and avoid making final TLS/layout decisions.
+- PIE and non-PIE executable behavior must stay distinct.
+- Copy relocations, protected visibility, weak imports, COMDAT/section groups,
+ init/fini arrays, unwind tables, and build IDs must compose with GC and
+ dynamic linking.
+
+The near-term Rust path mostly exercises final executables; shared-library and
+relocatable coverage should be broadened deliberately.
+
+### 6. Runtime/sysroot interoperability
+
+System-linker replacement depends on finding the same runtime inputs as the
+platform toolchain:
+
+- crt objects and default startup ordering,
+- libc/libm/libpthread/librt/libdl/libutil and libc-specific linker scripts,
+- compiler builtins and unwind libraries (`libgcc_s`, compiler-rt, Rust
+ `compiler_builtins`),
+- platform dynamic loaders/interpreters,
+- MinGW UCRT CRT and import libraries,
+- FreeBSD versioned libc and auxiliary runtime libraries.
+
+The driver should prefer explicit command-line inputs, then sysroot/target
+layout, and finally host defaults only where that is safe. Runtime validation
+should distinguish linker failure from a too-minimal test container, such as an
+Alpine image missing `libgcc_s.so.1`.
+
+### 7. Diagnostics and tracing
+
+The linker needs precise diagnostics for unsupported system-linker cases:
+
+- target, input file/member, symbol, relocation kind, and output mode;
+- why a DSO was selected or skipped;
+- which symbol caused an as-needed DSO to be kept;
+- why a TLS access cannot be relaxed or represented dynamically;
+- which runtime library or sysroot path was searched.
+
+Use `KIT_TRACE` for opt-in structured tracing. Do not add global state.
+
+## Validation matrix
+
+For each supported arch/OS/libc family, track these lanes:
+
+- Rust std executable link,
+- Rust std runtime execution where a runner exists,
+- Rust no-std/freestanding link,
+- C/assembler object corpus link,
+- archive order and group behavior,
+- DSO/dylib/import-library dependency selection,
+- TLS local and imported access,
+- shared-library output,
+- relocatable `-r` output,
+- debug/unwind preservation.
+
+The first priority remains the verified Rust support set, because it is a good
+system-linker driver and produces real toolchain objects. The second priority is
+small focused fixtures for every feature above, so failures are explainable and
+do not depend on large external runtimes.
+
+## Acceptance criteria
+
+`kit ld` is complete enough to be the system linker for a target when:
+
+- Rust and C toolchains can use it directly with only target/sysroot/runtime
+ configuration, not target-specific wrapper rewrites;
+- executables and shared libraries run under the target runtime loader;
+- `readelf`/`otool`/`llvm-readobj` dynamic metadata matches the platform
+ linker's dependency and relocation shape for covered cases;
+- relocatable links preserve the relocation surface correctly;
+- local TLS relaxations are legal and checked, and dynamic TLS is emitted when
+ required;
+- archive/DSO/import-library selection follows target format semantics;
+- negative cases fail early with actionable diagnostics.
diff --git a/doc/plan/README.md b/doc/plan/README.md
@@ -12,6 +12,7 @@ shrinks to whatever remains open.
| [OPTIMIZER.md](OPTIMIZER.md) | Completing the O2 SSA mid-end, expanded inlining, -O0/-O1 performance work, machine register-constraint improvements. | [../OPT.md](../OPT.md) |
| [PERF.md](PERF.md) | Making kit the fastest `-O0` C compiler with code as dense as tcc: current compile-speed + code-size standings, how to reproduce them (macOS instruction counts, Linux callgrind, the `make bench-cc` scaling guard), and the ranked forward-looking levers on both axes. | [../ARCH.md](../ARCH.md) |
| [LINKER.md](LINKER.md) | Incremental linking: the file-based object-link redesign and remaining non-ELF format coverage. | [../LINK.md](../LINK.md) |
+| [LINKER-COMPAT.md](LINKER-COMPAT.md) | Completing system-linker compatibility across the support set: ordered DSO selection, ELF TLS/TLSDESC, shared libraries, relocatable links, runtime/sysroot interoperability, and Rust/toolchain validation. | [../LINK.md](../LINK.md), [../OBJ.md](../OBJ.md), [../DRIVER.md](../DRIVER.md) |
| [JIT.md](JIT.md) | Function-level hot reload, Go-runtime-style codegen support, and remaining JIT host-portability work. | [../JIT.md](../JIT.md) |
| [DEBUG.md](DEBUG.md) | The Windows debugger host adapter, x64/rv64 displaced single-step, profiling, and DWARF gaps. | [../DBG.md](../DBG.md), [../DWARF.md](../DWARF.md) |
| [INSTRUMENT.md](INSTRUMENT.md) | Debugging/profiling tools in the Valgrind/callgrind/asan/ubsan tradition: the emulator as a dynamic-binary-instrumentation substrate (the `EmuToolHooks` ABI, shadow planes, the memory seam, a guest debugger), plus the cross-frontend compiler-sanitizer story. | [../EMU.md](../EMU.md) |
