commit 70592470957e05df49ed33592c6f9fd68f8c3421
parent 4e590650228592d27104243d8512af3ed0cb0c18
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Mon, 8 Jun 2026 18:44:57 -0700
link: root DSO back-referenced symbols under --gc-sections (freebsd -O1 fixed point)
A dynamic executable defines symbols its shared libraries reference back into it
— on FreeBSD, the crt defines `__progname`/`environ` and libc.so.7 has undefined
references to them. kit's --gc-sections liveness pass only rooted the entry,
init/fini, and retained sections, so when nothing in the executable referenced
those crt symbols they were garbage-collected out of the dynsym, and the
resulting `-O1` kit failed to load:
ld-elf.so.1: /lib/libc.so.7: Undefined symbol "__progname"
read_elf_dso now records each DSO's undefined-symbol names on its ObjImage
(obj_image_add_undef), and link_gc_compute roots the executable's definitions of
them — GNU ld's default "keep what shared libraries need" behaviour, surgical
(only DSO-referenced symbols, not all globals, so GC stays effective) and
monotonic (only adds roots). The DSO ObjImage is now always created (previously
only when versioned) so undef refs are captured for unversioned DSOs (musl) too;
versioning is unchanged (musl still emits no .gnu.version_r, glibc still does).
With this, the **aarch64-freebsd `-O1` (release) self-build reaches the
byte-identical fixed point** — FreeBSD now self-hosts at both -O0 and -O1
(Toy 1378/2/39 through stage3; the 2 fails are the pre-existing JIT-TLS .tdata
R-lane gap). test-link/test-elf/test-asm green; musl/glibc --gc-sections links
unaffected. Builds on the deferred-tombstone fix in the previous commit.
Diffstat:
5 files changed, 121 insertions(+), 64 deletions(-)
diff --git a/doc/plan/BOOTSTRAP.md b/doc/plan/BOOTSTRAP.md
@@ -67,62 +67,63 @@ Done on aarch64-freebsd (ELF), run natively inside the FreeBSD aarch64 VM from
the macOS host (`scripts/freebsd_bootstrap.sh aarch64`; see "Bootstrapping a
Linux target from a non-Linux host" — the FreeBSD VM path is the same shape):
-- **`-O0` (debug) chain reaches the fixed point**: `cmp stage2/kit stage3/kit`
- is byte-identical. The bootstrapped stage3 runs the Toy corpus at
- 1371 pass / 9 fail / 39 skip — the 9 failures are the same non-bootstrap gaps
- seen on aarch64-linux (Mach-O-tuned `.objdump` golden substrings on ELF, the
- JIT-TLS `.tdata`-init `R`-lane discrepancy, and one C-backend case the host
- clang rejects), not codegen issues. Reaching the fixed point required:
+- **Both the `-O0` (debug) and `-O1` (release) chains reach the fixed point**:
+ `cmp stage2/kit stage3/kit` is byte-identical in both modes. The bootstrapped
+ stage3 runs the Toy corpus at 1378 pass / 2 fail / 39 skip in both chains; the
+ 2 failures are the JIT-TLS `.tdata`-init `R`-lane discrepancy
+ (`141_threadlocal_mutate`) — a non-bootstrap gap shared with aarch64-linux
+ (the in-process JIT does not set up a per-thread TLS block / copy the `.tdata`
+ initializer; the native-link path is already `.link.skip`-gated), not a
+ codegen issue.
+
+ Four fixes were needed, in the order they surfaced:
- kit `cc` accepting `-rdynamic` (FreeBSD's `HOST_ENV_LDFLAGS` passes it; the
- other ELF hosts do not), and
- - **ELF symbol-version (Verneed/Versym) emission** in the linker. FreeBSD's
- INO64 transition left `stat`/`fstat`/... as two incompatible `struct stat`
- ABIs behind a hidden `FBSD_1.0` (compat) and the default `FBSD_1.5`; kit
- used to emit unversioned undefined references, so the runtime bound the
- compat version and read `st_size` at the wrong offset — stage2 then failed
- to read its own source files. The linker now reads each DSO's
- `.gnu.version_d`/`.gnu.version` and emits a matching `.gnu.version_r` +
- `.gnu.version` (gated on the DSO carrying versions, so musl/static links are
- unchanged; glibc links now also carry correct `GLIBC_*` requirements).
-- **`-O1` (release) chain does not yet reach the fixed point**, but the
- original blocker is fixed and a second, distinct one is now isolated.
- - **Fixed — deferred-symbol globalization (assembler).** The `-O1` *deferred*
+ other ELF hosts do not).
+ - **ELF symbol-version (Verneed/Versym) emission.** FreeBSD's INO64 transition
+ left `stat`/`fstat`/... as two incompatible `struct stat` ABIs behind a
+ hidden `FBSD_1.0` (compat) and the default `FBSD_1.5`; kit emitted
+ *unversioned* undefined references, so the runtime bound the compat version
+ and read `st_size` at the wrong offset — stage2 then failed to read its own
+ source files. The linker now reads each DSO's `.gnu.version_d`/`.gnu.version`
+ and emits a matching `.gnu.version_r` + `.gnu.version`, gated on the DSO
+ carrying versions (musl/static links unchanged; glibc links gain correct
+ `GLIBC_*` requirements).
+ - **(`-O1`) Deferred-symbol globalization in the assembler.** `-O1` *deferred*
anonymous const-data / jump-table symbols (`.Lkit_ro.N` / `.Lkit_jt.N`) are
- created as LOCAL tombstones (`obj_symbol_defer`, `removed=1`) and only
- materialized at `opt_whole_module_finalize`. The assembler's
- `promote_undef_externs` (`src/asm/asm.c`) — which globalizes undefined LOCAL
- externs — walked every symbol slot *including tombstones*, the only
- `obj_symiter` consumer that did not honor the `removed` contract (`obj.h`),
- so it flipped those tombstones to defined GLOBALs. It bit FreeBSD because
- its `<stdlib.h>` injects a file-scope `__asm__(".symver …")` (via
- `__sym_compat`) whose replay runs that pass *before* the deferred data is
- materialized; the four hosted `driver/env/*.o` then each defined a global
- `.Lkit_ro.0` and the stage2 link aborted with `duplicate definition of
- global symbol '.Lkit_ro.0'`. Not FreeBSD-codegen-specific — any TU with a
- file-scope `asm` plus a deferred const-data symbol at `-O1` reproduces it on
- any target. Fix: skip `removed` tombstones in `promote_undef_externs`.
- - **Open — `--gc-sections` drops crt symbols a DSO needs.** With that fixed,
- the stage2 link now succeeds, but the resulting `-O1` `kit` fails to *load*:
- `ld-elf.so.1: /lib/libc.so.7: Undefined symbol "__progname"`. The release
- chain links with `-Wl,--gc-sections`, and kit's section-GC liveness pass
- (`src/link/link_resolve.c`) does not root executable-defined symbols that a
- linked DSO references — so `__progname` and `environ` (defined in the
- FreeBSD crt, referenced by `libc.so.7`) get garbage-collected out of the
- dynsym. The `-O0` chain has no `--gc-sections` and is unaffected; reproduces
- by cross-linking any hosted FreeBSD exe with `-rdynamic -Wl,--gc-sections`.
- This is the next thing to fix for the `-O1` fixed point.
-
-This gives three fully self-hosting configurations (aarch64-macos, plus
-aarch64-linux under musl and glibc) and a fourth at `-O0` (aarch64-freebsd).
-The remaining work is breadth: the other native targets, the aarch64-freebsd
-`-O1` `--gc-sections`/DSO-root fix, and guarding the property over time.
+ LOCAL tombstones (`obj_symbol_defer`, `removed=1`) until
+ `opt_whole_module_finalize` materializes them. `promote_undef_externs`
+ (`src/asm/asm.c`) — which globalizes undefined LOCAL externs — walked every
+ slot *including tombstones* (the only `obj_symiter` consumer not honoring the
+ `removed` contract) and flipped them to defined GLOBALs. It bit FreeBSD
+ because `<stdlib.h>` injects a file-scope `__asm__(".symver …")` whose replay
+ runs that pass *before* the deferred data is materialized, so the four hosted
+ `driver/env/*.o` each defined a global `.Lkit_ro.0` → `duplicate definition
+ of global symbol '.Lkit_ro.0'`. Fix: skip `removed` tombstones. Not
+ FreeBSD-specific — any TU with a file-scope `asm` + a deferred const-data
+ symbol at `-O1` reproduces it on any target.
+ - **(`-O1`) `--gc-sections` rooting of DSO back-references.** With the above
+ fixed the stage2 link succeeded, but the `-O1` `kit` failed to *load*
+ (`ld-elf.so.1: /lib/libc.so.7: Undefined symbol "__progname"`): the release
+ chain links `-Wl,--gc-sections`, and kit's section-GC liveness
+ (`src/link/link_resolve.c`) did not root executable definitions that a
+ linked DSO references, so `__progname`/`environ` (crt-defined, needed by
+ `libc.so.7`) were collected out of the dynsym. `read_elf_dso` now records
+ each DSO's undefined-symbol names and the GC pass roots the executable's
+ definitions of them — matching GNU ld's default "keep what shared libraries
+ need" behaviour (`-O0` has no `--gc-sections` and is unaffected).
+
+This gives four fully self-hosting configurations — aarch64-macos,
+aarch64-linux (musl + glibc), and aarch64-freebsd — each at both `-O0` and
+`-O1`. The remaining work is breadth: the other native targets (x86-64, rv64),
+and guarding the property over time.
## Open problems and next steps
### Widen target and platform coverage
-The fixed point holds for aarch64-macos and aarch64-linux (musl + glibc). The
-bootstrap should hold for every supported native target and object format. Until
+The fixed point holds for aarch64-macos, aarch64-linux (musl + glibc), and
+aarch64-freebsd. The bootstrap should hold for every supported native target
+and object format. Until
each is green it is an open question whether its backend + object writer are
fully deterministic and self-consistent.
diff --git a/src/link/link_resolve.c b/src/link/link_resolve.c
@@ -748,6 +748,33 @@ void link_gc_compute(Linker* l, LinkImage* img, GcLive* g) {
}
}
+ /* Keep executable definitions that a linked shared library references but
+ * nothing in the executable does (e.g. FreeBSD libc.so.7's back-references
+ * to crt-defined `environ` / `__progname`). Without rooting these, GC drops
+ * the defining section and the resulting dynamic exe fails to load
+ * ("Undefined symbol"). read_elf_dso records each DSO's undef names. */
+ for (ii = 0; ii < LinkInputs_count(&l->inputs); ++ii) {
+ LinkInput* in = LinkInputs_at(&l->inputs, ii);
+ const ObjImage* dim;
+ u32 u, nu;
+ if (in->kind != LINK_INPUT_DSO_BYTES || !in->obj) continue;
+ dim = obj_image(in->obj);
+ nu = obj_image_nundefs(dim);
+ for (u = 0; u < nu; ++u) {
+ LinkSymId id = symhash_get(&img->globals, obj_image_undef(dim, u));
+ u32 tii;
+ ObjSecId tsid;
+ ObjAtomId taid;
+ if (id == LINK_SYM_NONE) continue;
+ if (gc_def_site(img, l, id, &tii, &tsid, &taid)) {
+ if (taid != OBJ_ATOM_NONE)
+ gc_mark_atom(g, &q, h, tii, taid);
+ else
+ gc_mark(g, &q, h, tii, tsid);
+ }
+ }
+ }
+
while (q.n > 0) {
u64 v = q.items[--q.n];
u32 cii = GC_II(v);
diff --git a/src/obj/elf/read.c b/src/obj/elf/read.c
@@ -1117,10 +1117,11 @@ ObjBuilder* read_elf_dso(Compiler* c, const char* name, const u8* data,
ObjBuilder* ob = obj_new(c);
if (!ob) compiler_panic(c, SRCLOC_NONE, "read_elf_dso: obj_new failed");
- /* Symbol versioning: when the DSO carries .gnu.version(_d), attach an
- * ObjImage whose dynsyms record each export's default version so the linker
- * can emit a matching .gnu.version_r requirement (see build_versions in
- * link_dyn.c). Absent versioning (musl) leaves im NULL and changes nothing. */
+ /* The DSO always gets an ObjImage: its dynsyms record each export's default
+ * version (so the linker can emit a matching .gnu.version_r — see
+ * build_versions in link_dyn.c, harmless/empty for unversioned DSOs like
+ * musl), and its undef list records the symbols this DSO references so
+ * --gc-sections keeps the executable's definitions of them alive. */
u32 verdef_max = 0;
Sym* verdef_tbl = read_elf_verdefs(c, data, len, shdrs, e_shnum, &verdef_max);
const u8* versym = NULL;
@@ -1132,8 +1133,7 @@ ObjBuilder* read_elf_dso(Compiler* c, const char* name, const u8* data,
nversym = (u32)(shdrs[i].sh_size / 2u);
break;
}
- ObjImage* im =
- (versym && verdef_tbl) ? obj_image_ensure(ob, OBJ_KIND_DYN) : NULL;
+ ObjImage* im = obj_image_ensure(ob, OBJ_KIND_DYN);
if (im && soname) obj_image_set_soname(im, soname);
u32 nsyms = (u32)(sh->sh_size / ELF64_SYM_SIZE);
@@ -1144,18 +1144,24 @@ ObjBuilder* read_elf_dso(Compiler* c, const char* name, const u8* data,
u8 st_info = p[4];
u8 st_other = p[5];
u16 st_shndx = rd_u16_le(p + 6);
-
- /* Skip the DSO's own undefined imports — they don't satisfy any
- * undef in our consumer. Locals (STB_LOCAL) likewise aren't
- * exported and would only confuse the resolver. */
- if (st_shndx == SHN_UNDEF) continue;
u32 e_bind = ELF64_ST_BIND(st_info);
- if (e_bind == STB_LOCAL) continue;
-
u32 nlen;
- const char* nm = strtab_lookup(strtab, strtab_sz, st_name, &nlen);
+ const char* nm;
+ Sym sn;
+
+ /* Locals are neither exports nor reference dependencies we track. */
+ if (e_bind == STB_LOCAL) continue;
+ nm = strtab_lookup(strtab, strtab_sz, st_name, &nlen);
if (!nlen) continue;
- Sym sn = pool_intern_slice(c->global, (Slice){.s = nm, .len = nlen});
+ sn = pool_intern_slice(c->global, (Slice){.s = nm, .len = nlen});
+
+ /* The DSO's own undefined references: not exports, but if the executable
+ * defines one (e.g. libc.so.7's `environ` / `__progname`, defined by the
+ * crt) the static linker must keep that definition under --gc-sections. */
+ if (st_shndx == SHN_UNDEF) {
+ obj_image_add_undef(im, sn);
+ continue;
+ }
u32 e_type_field = ELF64_ST_TYPE(st_info);
u16 bind = elf_bind_to_obj(e_bind);
diff --git a/src/obj/obj.c b/src/obj/obj.c
@@ -189,6 +189,11 @@ struct ObjImage {
u32 ndynsyms, cap_dynsyms;
ObjImageReloc* dynrelocs;
u32 ndynrelocs, cap_dynrelocs;
+ /* Undefined symbol names a DSO references (interned). Used by the linker's
+ * --gc-sections pass to keep executable-defined symbols a shared library
+ * needs (e.g. libc.so.7's `environ` / `__progname`) from being collected. */
+ Sym* undefs;
+ u32 nundefs, cap_undefs;
};
static void obj_image_free_(ObjBuilder* ob) {
@@ -209,6 +214,8 @@ static void obj_image_free_(ObjBuilder* ob) {
if (im->dynrelocs)
im->heap->free(im->heap, im->dynrelocs,
sizeof(*im->dynrelocs) * im->cap_dynrelocs);
+ if (im->undefs)
+ im->heap->free(im->heap, im->undefs, sizeof(*im->undefs) * im->cap_undefs);
ob->heap->free(ob->heap, im, sizeof(*im));
ob->image = NULL;
}
@@ -273,6 +280,11 @@ void obj_image_add_dynreloc(ObjImage* im, const ObjImageReloc* rel) {
return;
im->dynrelocs[im->ndynrelocs++] = *rel;
}
+void obj_image_add_undef(ObjImage* im, Sym name) {
+ if (!im || !name) return;
+ if (VEC_GROW(im->heap, im->undefs, im->cap_undefs, im->nundefs + 1)) return;
+ im->undefs[im->nundefs++] = name;
+}
ObjKind obj_image_kind(const ObjImage* im) {
return im ? im->kind : OBJ_KIND_REL;
@@ -302,6 +314,10 @@ u32 obj_image_ndynrelocs(const ObjImage* im) { return im ? im->ndynrelocs : 0; }
const ObjImageReloc* obj_image_dynreloc(const ObjImage* im, u32 idx) {
return (im && idx < im->ndynrelocs) ? &im->dynrelocs[idx] : NULL;
}
+u32 obj_image_nundefs(const ObjImage* im) { return im ? im->nundefs : 0; }
+Sym obj_image_undef(const ObjImage* im, u32 idx) {
+ return (im && idx < im->nundefs) ? im->undefs[idx] : 0;
+}
void obj_ext_set(ObjBuilder* ob, ObjExtKind kind, void* payload,
ObjExtFreeFn free_fn) {
diff --git a/src/obj/obj.h b/src/obj/obj.h
@@ -960,6 +960,11 @@ void obj_image_add_dep(ObjImage*, const ObjImageDep*);
void obj_image_add_rpath(ObjImage*, Sym rpath);
void obj_image_add_dynsym(ObjImage*, const ObjImageSym*);
void obj_image_add_dynreloc(ObjImage*, const ObjImageReloc*);
+/* Undefined symbol names a DSO references (interned). The linker's
+ * --gc-sections pass roots executable definitions of these so a shared
+ * library's back-references (e.g. libc.so.7 → `environ` / `__progname`)
+ * survive section GC. */
+void obj_image_add_undef(ObjImage*, Sym name);
/* Image read-side queries (object_file.c glue, objdump). */
ObjKind obj_image_kind(const ObjImage*);
@@ -977,6 +982,8 @@ u32 obj_image_ndynsyms(const ObjImage*);
const ObjImageSym* obj_image_dynsym(const ObjImage*, u32 idx);
u32 obj_image_ndynrelocs(const ObjImage*);
const ObjImageReloc* obj_image_dynreloc(const ObjImage*, u32 idx);
+u32 obj_image_nundefs(const ObjImage*);
+Sym obj_image_undef(const ObjImage*, u32 idx);
/* ---- file format emitters ---- */
void emit_elf(Compiler*, ObjBuilder*, Writer*);
