commit 91a660f11818a3d87a1e8510360b843f670efa01
parent 58f31a5e6f4252d8eb52be87c6e1d226f3ff6e17
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Mon, 11 May 2026 12:17:03 -0700
STAGE2.md plan update
Diffstat:
| M | doc/STAGE2.md | | | 248 | ++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------- |
1 file changed, 163 insertions(+), 85 deletions(-)
diff --git a/doc/STAGE2.md b/doc/STAGE2.md
@@ -1,15 +1,12 @@
# Stage-2 self-host
What's missing to make `make self` produce a stage-2 `cfree` built by stage-1
-cfree itself. Companion to `DESIGN.md`. Snapshot taken by compiling every
-`src/**/*.c` and `driver/*.c` individually with stage-1 cfree-cc.
+cfree itself. Companion to `DESIGN.md`.
-Result at snapshot: **60 of 104 files compile clean. 44 fail.** The failures
-collapse into ~10 root causes, listed below in roughly the order a fix would
-unblock the most files.
-
-B1–B6 have landed; re-run the audit recipe at the bottom of this file to
-refresh the count.
+Latest snapshot: **104 / 106 files compile clean** (92/92 `src/**/*.c`,
+12/14 `driver/*.c`). The two remaining driver failures (`env.c`, `ld.c`)
+are both blocked by A2 — system-header ingest. Everything in `src/` builds
+under stage 1.
## Build configuration
@@ -20,107 +17,159 @@ cfree-stage1 cc -isystem rt/include -isystem rt/include/libc -Iinclude -Isrc
```
`-isystem rt/include/libc` is required so the hosted libc headers are
-visible (top-level `rt/include/` only ships the freestanding set). The SDK
-include path is deliberately *not* on the search path — stage 2 should
-resolve everything through `rt/include` + `rt/include/libc`.
+visible (top-level `rt/include/` only ships the freestanding set).
-`DEPFLAGS` is empty for stage 2 until B0 lands.
+`DEPFLAGS` is empty for stage 2 today; B0 has landed but the recipe has
+not been switched back on.
## Checklist
### Preprocessor / lexer
-- [ ] **A1.** `#include "x.h"` doesn't search the source file's directory.
- C99 §6.10.2 requires quoted includes to look in the including file's
- directory first, then fall back to the bracketed-include search list. Today
- cfree's pp jumps straight to the search list. Repro:
- `echo '#include "foo.h"' > /tmp/dir/use.c && cfree cc -c /tmp/dir/use.c`
- fails to find `/tmp/dir/foo.h`. _Blocks all 13 `driver/*.c` files._
-- [ ] **A2.** Expand `rt/include/libc/` to cover the POSIX/Mach surface the
- driver uses. Missing today: `sys/stat.h`, `sys/mman.h`, `sys/syscall.h`,
- `fcntl.h`, `unistd.h`, `signal.h`, `pthread.h`, `dlfcn.h`, `mach/mach.h`,
- `mach/mach_vm.h`, `mach/vm_map.h`. Scope question (vs. dropping the
- dependency in the driver source); not a compiler bug. _Blocks most of
- `driver/`._
+- [x] **A1.** Quoted `#include "x.h"` now searches the includer's
+ directory first per C99 §6.10.2 (commit c9baaf8). Was blocking every
+ `driver/*.c` file.
+- [ ] **A2.** System-header ingest. The driver pulls a POSIX/Mach surface
+ (`sys/stat.h`, `sys/mman.h`, `sys/syscall.h`, `fcntl.h`, `unistd.h`,
+ `signal.h`, `pthread.h`, `dlfcn.h`, `mach/mach.h`, `mach/mach_vm.h`,
+ `mach/vm_map.h`) that today's `rt/include/libc/` doesn't ship.
+ **Direction: ingest the real SDK headers** rather than growing
+ `rt/include/libc/`. With `-isystem $SDK/usr/include` and the right host
+ predefines, the SDK parses up to a small set of constructs cfree
+ doesn't yet handle. Each sub-item below is the minimal feature needed.
+
+ - [ ] **A2-S1.** Asm-label on function declarators:
+ `T fn(args) __asm__("name");`. GCC asm-label rename extension; what
+ `__DARWIN_ALIAS` / `__DARWIN_ALIAS_C` / `__DARWIN_INODE64` /
+ `__DARWIN_EXTSN` expand to. Blocks `sys/stat.h`, `sys/mman.h`,
+ `unistd.h`, `_string.h`, `_stdio.h`.
+ - [ ] **A2-S2.** Asm-label on global variables:
+ `extern T name __asm__("name");`. Same extension, declarator position
+ differs from S1. Blocks `_time.h` (→ `<time.h>`, `<signal.h>`,
+ `<pthread.h>`).
+ - [ ] **A2-S3.** Unknown `#pragma` accepted as no-op (full semantics
+ not required for ingest). Today fatal "expected declaration". Blocks
+ `sys/fcntl.h`, `mach/vm_types.h`. Same root cause as R2 below.
+ - [ ] **A2-S4.** `__has_include`, `__has_feature`, `__has_extension`
+ as preprocessor builtins inside `#if`. (`__has_attribute` already
+ works.) Blocks `Availability.h` and the `__enum_decl` feature-detect
+ branch.
+ - [ ] **A2-S5.** `__uint128_t` declared type. Declare-only is enough
+ to parse `mach/arm/_structs.h` (signal.h, ucontext); full codegen
+ is a bigger lift.
+ - [ ] **A2-S6.** `#warning` accepted as non-fatal. Today cfree errors
+ on the directive itself; `sys/cdefs.h`'s
+ `#warning "Unsupported compiler"` aborts any SDK ingest unless
+ `-D__GNUC__` is also passed.
+ - [ ] **A2-S7.** Predefine macOS-host macros (`__APPLE__`, `__MACH__`,
+ `__arm64__`/`__aarch64__`, `__LITTLE_ENDIAN__`, `__GNUC__`,
+ `__GNUC_MINOR__`) automatically when targeting macOS, so callers
+ don't need to hand-pass `-D`.
+
+ After S6+S7+S1+S2+S3+S4, both blocked driver files should ingest the
+ SDK without any growth of `rt/include/libc/`. S5 only needed for
+ signal.h/ucontext paths.
### Driver — dep emission
-- [ ] **B0.** Implement `cfree_dep_iter_new` / `_next` (today both stubs in
- `src/api/stubs.c:106-115`). PP needs to record header-include edges so the
- iterator can drain them. Until then, stage 2 strips `-MMD -MP` via
- `DEPFLAGS=''`. Also: change the failure path in `driver/cc.c:1264` so a
- NULL iter doesn't surface as `"out of memory"` — that error message hid
- this for the whole first investigation. _Quality-of-life; stage 2 builds
- fine without dep files for now._
+- [x] **B0.** `cfree_dep_iter_new` / `_next` implemented over
+ SourceManager (commit 8919185). Stage 2 can re-enable `-MMD -MP`
+ whenever the recipe drops `DEPFLAGS=''`.
### Parser / sema
-- [x] **B1.** Recognize `__alignof__` as an alias for `_Alignof`. Routed
- through `ident_kw`/`is_kw`; every `KW_ALIGNOF` consumer accepts both
- spellings.
-- [x] **B2.** Implement `__builtin_ctz`. Added `cg_intrinsic_unary_to_int`
- wrapper; lowers via the existing `INTRIN_CTZ` path (already implemented in
- all three backends — aa64 `rbit; clz`, x64 `bsf`, rv64 `ctz`).
-- [x] **B3.** No actual fix required — `parse_array_bound` (parse.c:3948-3958)
- already routes `SEK_ENUM_CST` through `eval_const_int`. The STAGE2 repro
- failed because it combined this with B4 (string literal in static init);
- fixing B4 unblocked the example. Regression case added.
-- [x] **B4.** `try_parse_addr_const` now accepts `TOK_STR`: it mints a
- rodata symbol via `emit_string_to_rodata` and emits a reloc against the
- pointer slot.
-- [x] **B5.** `try_parse_addr_const` admits `SEK_FUNC` identifiers (same
- `v.sym` shape as `SEK_GLOBAL`). Diagnostic reworded — the old message
- "static initializer requires object with static storage" was misleading
- for functions, which do have static storage duration.
-- [x] **B6.** The brace-tracker was a red herring. Root cause: file-scope
- `T name[] = {...}` skipped `complete_incomplete_array`, so the static-init
- walker saw `arr.count=0` and tripped "too many initializers" on the
- first element. Block-scope path already handled this; file-scope path now
- mirrors it. Affects every file-scope incomplete-array-of-aggregate init,
- not just the inner-array-field shape called out in the original repro.
+- [x] **B1.** `__alignof__` aliased to `_Alignof` (type-name form).
+- [x] **B2.** `__builtin_ctz` lowered through `INTRIN_CTZ`.
+- [x] **B3.** `parse_array_bound` already routed `SEK_ENUM_CST` through
+ `eval_const_int` — original repro was actually B4. Regression case
+ added.
+- [x] **B4.** `try_parse_addr_const` accepts string literals via
+ `emit_string_to_rodata`.
+- [x] **B5.** `try_parse_addr_const` admits `SEK_FUNC` identifiers.
+- [x] **B6.** File-scope `T name[] = {...}` now calls
+ `complete_incomplete_array` to match the block-scope path.
+- [x] **B7.** `__alignof__` accepts a **unary-expression** operand
+ (`__alignof__(*ptr)`), not just a type-name. Required by the
+ `VEC_GROW` macro in `src/core/vec.h`; previously blocked 8 files in
+ `src/debug/` and `src/link/`.
+- [x] **B8.** `sizeof` accepts the no-parens **unary-expression** form
+ in constant-expression contexts (e.g. file-scope initializers). C99
+ §6.5.3.4 standard, not an extension. Blocked `src/arch/aa64_isa.c`
+ and `src/arch/aa64_regs.c`.
+- [x] **B9.** Block-scope `static T name[] = {...}` now completes the
+ incomplete array, mirroring B6's file-scope fix. Was blocking
+ `src/pp/pp.c`.
### Codegen — aarch64 backend
-- [ ] **C1.** Argument lowering: handle `OPK_INDIRECT` source operands in
- both the INT and FP paths at `src/arch/aarch64.c:2073-2129`. Today only
- `OPK_IMM` / `OPK_REG` / `OPK_LOCAL` are wired; an indirect source (e.g.,
- passing `ptr->field` by value, or `arr[i].field` where the addressing was
- lowered to a base+offset load) panics with
- `aarch64 call: arg storage kind 4 unsupported`. The fix mirrors the
- existing `OPK_LOCAL` case but loads from `[base + part->src_offset]`
- instead of `[fp - slot_off + src_offset]`. _6 files: `src/arch/mc.c`,
- `src/cg/cg.c`, `src/decl/{decl,decl_attrs}.c`, `src/opt/opt.c`,
- `src/pp/pp.c`._
-- [ ] **C2.** Same `OPK_INDIRECT` gap in the indirect-return path
- (separate panic string: `aarch64 ret indirect: storage kind 4
- unsupported`). _`src/api/pipeline.c`, `src/parse/parse_asm.c`._
+- [x] **C1.** `OPK_INDIRECT` source operands handled in INT and FP arg
+ paths (commit f2d3e01).
+- [x] **C2.** `OPK_INDIRECT` on the indirect-return path (commit
+ f2d3e01).
+- [x] **C0.** Stage-1 regalloc "no spillable victim (class 0)" panic
+ fixed — was choking on the complex functions in `src/arch/aarch64.c`,
+ `src/arch/rv64.c`, `src/cg/cg.c`, and `src/opt/opt.c`. Not a feature
+ gap; a regalloc bug surfaced by self-host pressure.
### Codegen — x64 backend
-- [ ] **C3.** Mirror C1/C2 on x64. The same panics exist at
- `src/arch/x64.c:1761,1798,1817,1827,1904`. Doesn't block aarch64
- self-host but blocks x64 self-host once that's attempted.
+- [ ] **C3.** Mirror C1/C2 on x64
+ (`src/arch/x64.c:1761,1798,1817,1827,1904`). Doesn't block aarch64
+ self-host; blocks x64 self-host when that's attempted.
### Linker
-- [ ] **D1.** Stage 2 currently relies on `$(CC) -o $@ ... $(LIB_AR)` to do
- the final link — for stage 2 that's `cfree-stage1 cc`, which in turn
- shells out to the host linker. Once stage 2 builds, the `$(BIN)` recipe
- should be reviewed to confirm the produced binary is genuinely a
- stage-1-emitted object linked through cfree's own ld path, not falling
- back to clang/ld silently.
+- [ ] **D1.** Stage 2 currently relies on `$(CC) -o $@ ... $(LIB_AR)`
+ for the final link — for stage 2 that's `cfree-stage1 cc`, which in
+ turn shells out to the host linker. Once stage 2 builds, verify the
+ produced binary is genuinely a stage-1-emitted object linked through
+ cfree's own ld path, not falling back to clang/ld silently.
### Hosted libc shim
-- [ ] **E1.** Today `libcfree_hosted_macos.a` is built but not threaded into
- the `$(BIN)` link. For a "self-host on rt libc" milestone (separate from
- this checklist's primary goal of "stage 2 builds at all"), the `$(BIN)`
- rule on macOS should consume the hosted shim and route libc calls through
- it instead of clang's default `-lSystem` glue.
-
-## How to re-run the audit
-
-After landing any fix, regenerate the failure list with:
+- [ ] **E1.** `libcfree_hosted_macos.a` is built but not threaded into
+ the `$(BIN)` link. For a "self-host on rt libc" milestone (separate
+ from "stage 2 builds at all"), the macOS `$(BIN)` rule should consume
+ the hosted shim and route libc calls through it instead of clang's
+ default `-lSystem` glue.
+
+## Runtime — `rt/lib/*` ingest
+
+Separate from stage-2 self-host: can cfree compile `libcfree_rt.a`?
+Probed on the `aarch64-apple-darwin` variant — 8 sources, freestanding,
+no system headers. Result: **2 / 8 clean** today. Flags must drop
+`-std=c11 -Wpedantic -Wall -Wextra -Werror -ffreestanding -fno-builtin` —
+cfree rejects all of these. (`-fno-builtin` is the only one not already
+on the stage-2 drop list.)
+
+- [ ] **R1.** Accept `__inline` and `__inline__` as keyword aliases for
+ `inline`. One-line lexer/keyword-table change. Blocks `int/int.c`,
+ `fp/fp.c`, `int64/int64.c` via `rt/lib/include/lp64_le/int_lib.h:83`
+ (`static __inline …`).
+- [ ] **R2.** Unknown `#pragma` accepted as no-op. Same root cause as
+ A2-S3 — one fix, two payoffs. Blocks
+ `atomic/atomic_freestanding.c` (`#pragma clang diagnostic …`).
+- [ ] **R3.** Fold `__builtin_offsetof(T, m)` as a constant expression.
+ cfree already computes struct layout; this is plumbing for the
+ constant-evaluator. Blocks `coro/aarch64.c` (`offsetof` inside
+ `_Static_assert`).
+- [ ] **R4.** `_Alignas(N)` on a **struct member** must raise the
+ containing aggregate's alignment (C11 §6.7.5). cfree honors
+ `__attribute__((aligned(N)))` on the struct itself, but member-level
+ `_Alignas` doesn't propagate. Blocks `coro/coro.c` whose
+ `_Alignof(coro_t)` assertion silently evaluates to less than 16.
+- [ ] **R5.** `__int128` keyword. Latent — `int_lib.h` declares the
+ type via `__attribute__((mode(TI)))` which parses, but full codegen
+ correctness on `__int128` operations hasn't been exercised. Will
+ matter for ABI variants that hit the int128 paths.
+
+After R1+R2+R3+R4, all 8 sources of the `aarch64-apple-darwin` variant
+should compile. The same fixes apply to the linux variants (same
+`int_lib.h`, same `coro.c`, same Apache-2.0 atomic shim).
+
+## How to re-run the audits
+
+Stage-2 audit (src + driver):
```sh
make && cp build/cfree build/cfree-stage1
@@ -135,4 +184,33 @@ for f in $(find driver -name '*.c' | sort); do
done
```
+System-header ingest probe (after A2 work):
+
+```sh
+SDK=$(xcrun --show-sdk-path)
+DEFS="-D__GNUC__=4 -D__GNUC_MINOR__=2 -D__arm64__=1 -D__aarch64__=1 \
+ -D__LITTLE_ENDIAN__=1 -D__APPLE__=1 -D__MACH__=1"
+for h in sys/stat.h sys/mman.h sys/syscall.h fcntl.h unistd.h signal.h \
+ pthread.h dlfcn.h mach/mach.h mach/mach_vm.h mach/vm_map.h \
+ stdio.h stdlib.h string.h; do
+ echo "#include <$h>" > /tmp/h.c
+ $BIN cc $DEFS -isystem rt/include -isystem "$SDK/usr/include" \
+ -c /tmp/h.c -o /tmp/h.o 2>&1 | head -1 | sed "s|^|$h: |"
+done
+```
+
+rt ingest probe (`aarch64-apple-darwin` variant):
+
+```sh
+SRCS="lib/int/int.c lib/fp/fp.c lib/mem/mem.c \
+ lib/atomic/atomic_freestanding.c lib/cfree/ifunc_init.c \
+ lib/int64/int64.c lib/coro/aarch64.c lib/coro/coro.c"
+FLAGS="-target aarch64-apple-darwin -DHAS_INT128=1 \
+ -Irt/lib/include/common -Irt/lib/impl \
+ -Irt/lib/include/lp64_le -Irt/include"
+for f in $SRCS; do
+ $BIN cc $FLAGS -c "rt/$f" -o /dev/null 2>&1 | head -1 | sed "s|^|rt/$f: |"
+done
+```
+
Then `make self` to confirm a clean stage-2 build end-to-end.
