commit b4523172e9848697219b5bf500d4e025a4235c51
parent 7a1ff3b3618cd6590d0c4a7e63700f96c21b6975
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Wed, 17 Jun 2026 09:00:58 -0700
doc: retire DYNLIB.md plan — ELF shared library support shipped
Remove the forward-looking roadmap now that kit can produce ELF DSOs
(kit ld -shared / kit cc -shared -fPIC on Linux/aarch64). Move a brief
summary into the plan/README.md retired-docs list pointing at LINK.md
and OBJ.md.
Diffstat:
2 files changed, 4 insertions(+), 346 deletions(-)
diff --git a/doc/plan/DYNLIB.md b/doc/plan/DYNLIB.md
@@ -1,343 +0,0 @@
-# Dynamic/shared libraries (Linux first)
-
-Goal: make kit produce loadable shared libraries, with Linux/aarch64 as the
-first supported target. The useful first result is:
-
-```
-kit ld -shared -soname libfoo.so.1 -o libfoo.so foo.pic.o
-kit cc -shared -fPIC -Wl,-soname,libfoo.so.1 -o libfoo.so foo.c
-```
-
-where `readelf -h` reports `ET_DYN`, there is no required entry point, no
-`PT_INTERP`, `DT_SONAME` is present when requested, exported symbols are in
-`.dynsym`, unresolved imports become runtime relocations, and a normal Linux
-loader can use the `.so` as a dependency of an executable.
-
-## Current state
-
-kit already has most of the dynamic-executable substrate:
-
-- The public link API has `KIT_LINK_OUTPUT_SHARED`, `soname`, rpath/runpath,
- exports, and `allow_undefined` fields in `KitLinkSessionOptions`.
-- `driver/cmd/{cc,ld,build}.c` parse `-shared` / `-soname` but reject shared
- output with "creating dynamic/shared libraries is not yet supported".
-- `src/api/link.c` maps `KIT_LINK_OUTPUT_SHARED` to `link_set_pie(l, 1)`, but
- warns that `-soname`, rpaths, and explicit exports are ignored.
-- `src/obj/elf/link_dyn.c` synthesizes `.dynsym`, `.dynstr`, `.gnu.hash`,
- `.rela.dyn`, `.rela.plt`, `.plt`, `.got.plt`, and `.dynamic`, but it is
- explicitly shaped as ET_DYN executable support. It always chooses an
- interpreter path when dynamic state exists.
-- `src/obj/elf/link.c` emits `ET_DYN` when `img->pie` is set, but requires
- `img->entry_sym` and writes `e_entry` from it. It emits dynamic program
- headers as `PT_PHDR`, `PT_INTERP`, `PT_DYNAMIC`, and `PT_GNU_STACK` whenever
- `pie && img->dyn`.
-- The linker can already consume DSO inputs: `LINK_INPUT_DSO_BYTES` contributes
- exported dynsym symbols for resolution and creates `DT_NEEDED` records.
-- AArch64 already has PLT emission in `src/arch/aa64/link.c` and GOT-using
- relocation rows in `src/arch/aa64/reloc.c`.
-- kit-generated aarch64 code uses `obj_symbol_extern_via_got()` for undefined
- extern data when the target is ELF PIC/PIE, so some PIC codegen policy is
- already centralized in `src/obj/obj_secnames.c`.
-
-The missing part is not "all dynamic linking"; it is a distinct DSO output
-mode. PIE executables and DSOs are both `ET_DYN`, but they differ in entry
-requirements, interpreter headers, exported-symbol policy, undefined-symbol
-defaults, dynamic tags, relocation legality, and driver behavior.
-
-## Scope
-
-### Linux/aarch64 MVP
-
-- ELF64 only.
-- Target triples like `aarch64-linux-*`.
-- Work backward from a known-good loader fixture: verify the shared-library
- test harness with all-clang first, then swap in `kit ld -shared` while
- keeping clang PIC objects, then swap in `kit cc -fPIC` so the final lane is
- all kit.
-- Support regular function/data exports, DSO inputs, `DT_NEEDED`,
- `DT_SONAME`, `DT_RPATH` / `DT_RUNPATH`, `.gnu.hash`, `.rela.dyn`,
- `.rela.plt`, AArch64 PLT/GOT, `R_AARCH64_RELATIVE`, `GLOB_DAT`, and
- `JUMP_SLOT`.
-- Default shared-library undefined-symbol policy should match ELF linkers:
- allow unresolved non-weak references by default in `-shared`, unless a
- stricter flag is requested.
-
-### Explicitly later
-
-- x86-64 and riscv64 DSOs.
-- TLS models beyond the currently practical local-exec / initial-exec subset.
-- Copy relocations, symbol version definitions, version scripts, GNU
- `--dynamic-list`, `--exclude-libs`, `-Bsymbolic*`, `--as-needed` parity, and
- lazy binding.
-- Mach-O dylib and COFF/PE DLL production.
-- Runtime-library policy for building libkit_rt itself as a DSO.
-
-## Implementation checklist
-
-### 1. Split PIE-executable mode from DSO mode
-
-- Add an internal `emit_shared` / `output_shared` bit on `Linker` and mirror it
- on `LinkImage`, instead of representing shared output only as `emit_pie`.
-- Keep `emit_pie` for dynamic/static PIE executable behavior. Let shared output
- imply image base 0 and dynamic-section synthesis without implying an
- executable entry or interpreter.
-- Add `link_set_shared(Linker*, int)` beside `link_set_pie`.
-- In `kit_link_session_new`, for `KIT_LINK_OUTPUT_SHARED`, set shared mode,
- enable dynamic layout, and do not call the static-exe setup.
-- Update comments in `src/link/link.h`, `src/link/link_internal.h`, and
- `doc/LINK.md` after the implementation so "PIE/DSO" is not conflated with
- "PIE executable".
-
-Acceptance:
-
-- A shared link reaches `kit_link_session_resolve` without looking up `_start`.
-- Existing PIE executable tests keep their current behavior.
-
-### 2. Entry and ELF header behavior
-
-- Change `link_resolve_entry` / ELF emit so `KIT_LINK_OUTPUT_SHARED` permits
- `entry_sym == LINK_SYM_NONE`.
-- In `link_emit_elf`, write `e_type = ET_DYN` for shared output and
- `e_entry = 0` unless an explicit entry was intentionally accepted.
-- Keep `ET_EXEC` and PIE executable behavior unchanged.
-- Ensure link maps / symbol side-files tolerate no entry.
-
-Acceptance:
-
-- `kit ld -shared ...` emits an ELF header with `Type: DYN` and entry address
- `0x0`.
-- Missing `_start` is not a diagnostic for shared output.
-
-### 3. Program headers for DSOs
-
-- For shared output, do not synthesize `.interp` and do not emit `PT_INTERP`.
- DSOs are loaded by another program's interpreter.
-- Decide whether `PT_PHDR` is emitted for DSOs. It is valid and useful when the
- phdr table is in a loadable segment, but it must not drag in executable-only
- assumptions.
-- Continue emitting `PT_DYNAMIC` and `PT_GNU_STACK`.
-- Keep `PT_TLS` behavior if TLS sections are present, but gate unsupported TLS
- reloc models before producing a bogus library.
-- Preserve page/file congruence and the existing headers PT_LOAD behavior; do
- not disturb scripted/freestanding output.
-
-Acceptance:
-
-- `readelf -l libfoo.so` shows `DYNAMIC` and no `INTERP`.
-- Program headers' `p_vaddr % p_align == p_offset % p_align`.
-
-### 4. Dynamic section tags
-
-- Plumb `KitLinkSessionOptions.soname`, `rpaths`, `runpaths`, and `exports`
- into `Linker` / `LinkDynState`.
-- Add `DT_SONAME` when `-soname` is set.
-- Add `DT_RPATH` or `DT_RUNPATH` entries according to the existing driver
- `--enable-new-dtags` / `--disable-new-dtags` policy.
-- Keep `DT_NEEDED` for DSO inputs already used by the link.
-- Do not emit `DT_FLAGS_1 = DF_1_NOW` unconditionally for DSOs unless the link
- explicitly requests now binding. For the MVP, either omit it or share the
- current eager-binding behavior intentionally and document it.
-- Rework `.dynstr` construction so soname/path strings are included during
- layout, not appended at emit time.
-
-Acceptance:
-
-- `readelf -d` shows requested `SONAME`, `NEEDED`, and `RUNPATH`/`RPATH`.
-- No warning from `src/api/link.c` for supported shared-library flags.
-
-### 5. Exported dynamic symbols
-
-- Define the default export set for shared output as all defined non-local,
- non-hidden, non-section, non-file symbols. Preserve weak and protected
- bindings correctly.
-- Honor symbol visibility from the object model (`SV_DEFAULT`, `SV_HIDDEN`,
- `SV_PROTECTED`) in `.dynsym.st_other`.
-- Treat explicit `exports` as a later filtering/augmentation feature unless the
- first implementation wires it completely; do not silently ignore it.
-- Keep executable `--export-dynamic` separate from DSO exports. A shared library
- is not an executable with `-E`.
-- Ensure LTO preservation keeps exported DSO symbols alive. The existing
- `KIT_LINK_OUTPUT_SHARED` preserve-all-nonlocal path in `src/api/link.c` is
- the right starting point.
-
-Acceptance:
-
-- Hidden symbols are absent from `.dynsym` or marked non-preemptible as
- appropriate; default-visible globals are exported.
-- A consumer executable can resolve an exported function/data symbol from the
- produced `.so`.
-
-### 6. Undefined symbols and imports
-
-- For shared output, unresolved global references should become dynsym imports
- even when they were not satisfied by a link-time DSO input, unless the caller
- requests `--no-undefined` / `-z defs`.
-- Add a distinct "runtime import with no provider DSO" state or allow
- `imported` symbols with `dso_input_id == LINK_INPUT_NONE`; make
- `collect_needed` skip provider-less imports while still emitting dynsym and
- relocations.
-- Keep executable behavior strict by default: unresolved references in
- executable links remain errors unless `allow_undefined` or an external
- resolver applies.
-- Support weak undefined symbols as zero/null without requiring dynamic
- relocations where the psABI expects that behavior.
-
-Acceptance:
-
-- `kit ld -shared foo.o` can leave `bar` undefined and emits `bar` in
- `.dynsym` as `UND`.
-- A final executable link fails or succeeds according to its own unresolved
- policy, independent of the library production step.
-
-### 7. Relocation legality and runtime relocations
-
-- Audit `apply_all_relocs` for shared-output semantics. For DSOs, text
- relocations should be rejected unless an explicit textrel option exists.
-- AArch64 MVP:
- - calls/jumps to imported functions route through PLT and `R_AARCH64_JUMP_SLOT`;
- - absolute data/function pointer slots against imports emit `GLOB_DAT`;
- - absolute slots against internal non-ABS symbols emit `R_AARCH64_RELATIVE`;
- - GOT entries for preemptible/default-visible definitions must be handled
- correctly, not only undefined externals;
- - direct `ADR_PREL_PG_HI21` / `ADD_ABS_LO12_NC` references to preemptible
- default-visible DSO definitions must be rejected or emitted through a
- DSO-safe form.
-- Add a shared-output validation pass that catches reloc kinds the dynamic
- loader cannot apply for DSOs. This should diagnose the input object and
- symbol, not panic late in emit.
-- Keep `R_*_COPY` out of DSO output. Copy relocs belong to executable links.
-
-Acceptance:
-
-- Non-PIC absolute/text reloc inputs fail with a clear diagnostic under
- `-shared`.
-- PIC object fixtures with function calls, data imports, function-pointer
- initializers, and local address constants load and run through a consumer.
-
-### 8. AArch64 PIC codegen from kit
-
-- Validate that `-fPIC` sets `target.pic = KIT_PIC_PIC` for all compile paths
- that can feed a shared link.
-- Audit aarch64 backend symbol-address lowering:
- - undefined extern data currently uses GOT via `obj_symbol_extern_via_got`;
- - default-visible definitions in the same DSO may still be preemptible under
- ELF interposition rules, so data accesses may also need GOT-indirect forms
- unless hidden/protected or `-Bsymbolic` applies;
- - direct calls can remain direct for local/hidden definitions but external
- default-visible calls need PLT-compatible relocations when interposable.
-- Make `KIT_CG_OUTPUT_SHARED` and `KIT_CG_INTERPOSITION_*` drive these choices
- for both `-O0` and `-O1`.
-- Keep local hidden/internal symbols direct so PIC does not pessimize
- everything.
-
-Acceptance:
-
-- `kit cc -fPIC -c foo.c` produces relocations accepted by `kit ld -shared`.
-- `kit cc -shared -fPIC foo.c` produces the same class of working `.so` as
- linking equivalent external PIC objects.
-
-### 9. Driver behavior
-
-- Remove the `-shared` rejection in `driver/cmd/ld.c` after the linker support
- is in place.
-- Remove the `-shared` rejection in `driver/cmd/cc.c` only after kit-generated
- PIC objects are safe; before that, `kit ld -shared` can be enabled while
- `kit cc -shared` remains gated.
-- For `kit cc -shared`, default compilation to `-fPIC` if the user did not
- explicitly choose PIC/PIE/non-PIC. Reject `-shared -fno-pic`.
-- Set default output names consistently (`a.out` is wrong for `-shared`; use
- user `-o` or a library-shaped fallback only if the driver already has one).
-- `-Wl,-soname` should require `-shared`, as it does now, but become useful.
-- Decide and document `-nostdlib` / runtime archive behavior for shared
- libraries. Avoid pulling executable startup objects into a DSO.
-- Keep `build-lib -dynamic` gated until the lower-level `ld` and `cc` paths are
- working; then route it through `KIT_LINK_OUTPUT_SHARED`.
-
-Acceptance:
-
-- The all-clang harness lane is green before either driver gate is removed.
-- `kit ld -shared` is enabled after it passes with clang PIC objects.
-- `kit cc -shared -fPIC` is enabled after kit-generated PIC objects pass the
- same DSO fixture and no `_start` or crt executable startup is included.
-
-### 10. Test sequencing
-
-Prefer red-green targeted tests. Add small ELF/aarch64 cases before broad
-matrix runs. The sequencing is part of the design: prove the test harness with
-the system toolchain first, then replace one kit component at a time.
-
-- Build one reusable `test/link/elf-dso-aa64/` fixture or equivalent. It should
- produce a small `libfoo.so` and a consumer executable that calls an exported
- function, reads exported data, exercises an undefined import supplied by the
- consumer or another DSO, and checks hidden/default visibility. The fixture has
- three lanes:
- - **Harness baseline, all clang:** clang compiles PIC objects, clang links
- the shared library, and clang links the consumer. This verifies the source
- fixture, sysroot, runtime loader, qemu/VM lane, `readelf` checks, and run
- harness before kit is involved.
- - **Linker swap, kit ld only:** clang still compiles all PIC objects, but
- `kit ld -shared` links `libfoo.so`; clang or the system linker may still
- link the consumer. This isolates ELF DSO production, dynamic tags,
- exports/imports, PLT/GOT, and runtime relocations from kit codegen.
- - **Compiler swap, all kit:** `kit cc -fPIC` compiles the library objects,
- `kit ld -shared` links the DSO, and `kit cc`/`kit ld` links the consumer.
- This is the first lane that validates kit-generated PIC and DSO
- interposition choices.
-- Each lane should run the same observable checks:
- - header: ET_DYN, e_entry 0, no PT_INTERP, has PT_DYNAMIC;
- - dynamic tags: SONAME, NEEDED, RUNPATH/RPATH;
- - export set: default-visible exported, hidden not exported;
- - undefined import allowed in DSO and appears in `.dynsym`;
- - non-PIC relocation rejected with a precise diagnostic;
- - DSO consumed by a small executable under the Linux/aarch64 lane.
-- Keep object-level tests on the linker-swap lane until they are green; do not
- debug kit codegen and ELF DSO layout in the same red test.
-- Add kit-codegen tests for `kit cc -fPIC -c` relocation shape only after the
- clang-object linker-swap lane is stable; then add `kit cc -shared`.
-- Use targeted commands such as `make test-link TARGET=aarch64-linux` or the
- closest existing selector. Redirect output to a file and inspect/tail that
- file.
-- Once aarch64 is stable, repeat the same fixtures for x86-64 and riscv64 only
- after their PLT/GOT relocation coverage is audited.
-
-## Likely milestone order
-
-1. **All-clang harness baseline:** write the aarch64 shared-library fixture and
- prove it with clang compile, clang `-shared`, clang/system consumer link,
- `readelf` assertions, and runtime execution. No kit linker/codegen work is
- considered validated until this lane is green.
-2. **ELF linker skeleton:** internal shared mode, no entry, no interpreter,
- `ET_DYN` DSO headers.
-3. **Dynamic metadata:** SONAME/RPATH/RUNPATH plumbing and dynstr layout.
-4. **Swap in `kit ld -shared`:** keep clang PIC library objects, but produce
- the `.so` with kit. Make imports/exports, PLT/GOT, `DT_NEEDED`, and runtime
- relocations work in this isolated lane.
-5. **Diagnostics:** reject non-PIC/textrel inputs clearly under
- `kit ld -shared`.
-6. **Enable `kit ld -shared`:** remove the driver gate for object/archive/DSO
- inputs once the clang-object linker-swap lane is green.
-7. **Swap in kit codegen:** compile the same library sources with
- `kit cc -fPIC`, audit/fix aarch64 interposition choices, and keep the DSO
- linked by kit. This creates the all-kit library path.
-8. **Enable `kit cc -shared`:** compile and link source to `.so` through the
- driver, with no executable startup objects.
-9. **All-kit consumer lane:** link the consumer with kit as well, so the full
- compile/link/run path is kit-produced except for the host loader/sysroot.
-10. **Broaden:** x86-64/riscv64, build-lib dynamic mode, and additional ELF
- linker compatibility flags.
-
-## Open design decisions
-
-- Should shared links default to `--allow-shlib-undefined`, with `-z defs` /
- `--no-undefined` as the strict mode? This matches common ELF behavior and
- should be the default unless kit intentionally chooses stricter diagnostics.
-- Should `DF_1_NOW` remain the default for DSOs? Current PIE executable output
- uses eager binding. DSOs can work that way, but it is a policy choice rather
- than a requirement.
-- How much ELF interposition should codegen honor in v1? A conservative MVP can
- require `-fvisibility=hidden` for kit-generated DSOs until default-visible
- definition preemption is fully correct, but `kit ld -shared` for external PIC
- inputs should not impose that restriction.
-- Do we want a `-Bsymbolic` subset early? It can simplify same-DSO references,
- but it should be an explicit semantic change, not a hidden implementation
- shortcut.
diff --git a/doc/plan/README.md b/doc/plan/README.md
@@ -11,7 +11,6 @@ shrinks to whatever remains open (and is deleted once nothing remains open).
| [RELEASE.md](RELEASE.md) | Cross-cutting initial-release punchlist: remaining release blockers and per-subsystem validation gaps (completed items are removed, not checked off). | — |
| [OPTIMIZER.md](OPTIMIZER.md) | Completing the O2 SSA mid-end, live-range splitting/coalescing, the residual O1 generated-code-quality gaps that need SSA, -O0 quality, machine register-constraint improvements, and broader inlining. Carries the known non-deficiencies (what *not* to chase). | [../OPT.md](../OPT.md) |
| [LINKER.md](LINKER.md) | Incremental linking (JIT append + the unbuilt file-based "m2" redesign) **and** system-linker compatibility (ordered DSO/`--as-needed`, ELF TLS planning + TLSDESC, shared/relocatable links, sysroot interop, toolchain validation). | [../LINK.md](../LINK.md), [../OBJ.md](../OBJ.md) |
-| [DYNLIB.md](DYNLIB.md) | Producing ELF shared libraries, Linux/aarch64 first: splitting DSO output from PIE executables, dynamic tags, exports/imports, relocation legality, PIC codegen, driver enablement, and targeted tests. | [../LINK.md](../LINK.md), [../OBJ.md](../OBJ.md) |
| [DEBUG.md](DEBUG.md) | The interactive JIT debugger + DWARF: x64/rv64 session parity, displaced-step instruction coverage, unit/smoke tests, REPL polish, Toy/C REPL frontends, and DWARF producer/consumer gaps (loclists, CFI register recovery, composite locations). | [../DBG.md](../DBG.md), [../DWARF.md](../DWARF.md) |
| [PROF.md](PROF.md) | The not-yet-built host-native sampling profiler (`kit prof`): SIGPROF frame-pointer walk reusing the debugger's signal infrastructure, the `KitProfBuf`/`KitProfWriter` API, and folded/flat output. Complementary to (not rebased on) the emulator callgrind in INSTRUMENT.md. | [../DBG.md](../DBG.md) |
| [INSTRUMENT.md](INSTRUMENT.md) | Debugging/profiling tools in the Valgrind/callgrind/asan/ubsan tradition: the emulator as a dynamic-binary-instrumentation substrate (the `EmuToolHooks` ABI, shadow planes, the memory seam, a guest debugger), plus the cross-frontend compiler-sanitizer story. | [../EMU.md](../EMU.md) |
@@ -31,5 +30,7 @@ worklist and frontend/CG redesigns (→ [../OPT.md](../OPT.md),
`build-lib`/`build-obj` (→ [../DRIVER.md](../DRIVER.md)); the freestanding kernel/
image pipeline (→ [../KERNEL.md](../KERNEL.md)); the bootstrap fixed point
(→ [../BUILD.md](../BUILD.md)); aarch64 Windows self-host (→ [../WINDOWS.md](../WINDOWS.md));
-the portability test surface (→ [../PORT.md](../PORT.md)); and the compile-speed/
-code-size benchmarking methodology (→ [../BENCHMARKING.md](../BENCHMARKING.md)).
+the portability test surface (→ [../PORT.md](../PORT.md)); the compile-speed/
+code-size benchmarking methodology (→ [../BENCHMARKING.md](../BENCHMARKING.md)); and
+ELF shared library production — DSO output mode, dynamic tags, exports/imports,
+PIC codegen, and driver enablement (→ [../LINK.md](../LINK.md), [../OBJ.md](../OBJ.md)).
