commit e51e9c845ab7a87b875bd188fe1b90bff39705cc
parent 54cd542c9e1c6acb458dfe3ff604e99f1926554a
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Mon, 15 Jun 2026 09:00:29 -0700
doc: update cg+frontend plans
Diffstat:
2 files changed, 500 insertions(+), 142 deletions(-)
diff --git a/doc/plan/CG-STACK-API.md b/doc/plan/CG-STACK-API.md
@@ -10,12 +10,14 @@ use.
This is not a plan to replace the stack API with free-floating value handles. It
is a plan to (a) make stack slots rich enough for a language frontend to use them
as its expression stack, (b) move all effective-address folding behind the CG
-API, (c) make codegen suppression a CG mode that also folds constants, so the
-parser has one uniform path and one constant evaluator, and (d) **delete the
-`pcg` adapter layer** so the parser drives the CG API directly.
+API, (c) make codegen suppression a CG mode that preserves stack shape,
+frontend slot facts, and constant facts, and (d) **delete the `pcg` adapter
+layer** so the parser drives the CG API directly.
-This is a **clean-break rewrite, no backcompat**. The spec below describes the
-end state; §7 is a direct cutover, not an incremental dual-stack migration.
+This is a **clean-break rewrite, no backcompat**. The implementation is phased
+only where subsystems are genuinely separable; each phase is a one-time cutover
+with the old path removed for that subsystem. There is no dual-stack
+compatibility mode and no byte-identity gate.
---
@@ -25,7 +27,7 @@ end state; §7 is a direct cutover, not an incremental dual-stack migration.
|---|---|---|
| D1 | Keep the stack API, add a stack-owned frontend slot sidecar | yes |
| D2 | Free `KitCgValue`/`KitCgPlace` handles independent of stack lifetime | **rejected** — loses the "on stack == live" invariant |
-| D3 | Where the type/flags stack lives during suppression | **CG unevaluated mode** — KitCg maintains a valueless type+sidecar stack; the parser drives CG uniformly with no `if (emit)` forks |
+| D3 | Where the type/flags stack lives during suppression | **CG unevaluated mode** — KitCg maintains stack shape, sidecar facts, and constant facts while target emission is off |
| D4 | Effective-address folding | **Automatic behind the CG API.** The parser builds places naively (`deref`, `field_at`, `elem`); CG decides whether to fold into the fused `[base + index*scale + offset]` operand or materialize. **No deferral or folding logic in the parser.** |
| D5 | Success gate | **Correctness + determinism** (full suites pass, output deterministic). Emitted bytes MAY change vs today; byte-identity is *not* a gate. |
| D6 | Fused `store_void` / `store_keep` as CG ops | **in scope** |
@@ -33,7 +35,9 @@ end state; §7 is a direct cutover, not an incremental dual-stack migration.
| D8 | Call-arg stack-mark API (old §4.4) | **dropped** — C already passes args on the CG stack (`kit_cg_call`), so a mark API adds nothing |
| D9 | `KitCgSlotInfo` + new ops are public API | **forced** — `lang/` builds only against `include/kit/cg.h` (no `src/cg` includes) |
| D10 | Sidecar is a parallel array, not inlined into `ApiSValue` | **forced** — `ApiSValue` is `_Static_assert`'d `<= 64` bytes and is near the limit (`src/cg/value.c:14`) |
-| D11 | C integer-constant-expression evaluation | **moves into CG.** Unevaluated mode also **folds constants** (not just types); the parser evaluates an ICE by parsing it once through the normal path under unevaluated mode and reading the folded value. The parser's duplicate `cexpr_*` grammar and `cint_*` arithmetic engine are **deleted**. |
+| D11 | C integer-constant-expression evaluation | **moves onto a CG constant-value model, with parser-owned C legality tracking.** The parser evaluates an ICE by parsing it once through the normal path under unevaluated mode, checking the syntactic/semantic ICE guard, and reading the folded value from CG. The parser's duplicate `cexpr_*` grammar and `cint_*` arithmetic engine are **deleted in the final ICE phase**, not during the stack/place cutover. |
+| D12 | Constant payload shape | **stack-owned side payload, not `OPK_IMM` alone.** `kit_cg_top_const_int` today only answers for <=64-bit immediate operands; the new model carries known integer bits + width + type independently of whether the emitted operand would be an immediate/local/runtime call. |
+| D13 | `__int128` constant policy | **support 128-bit integer folding in CG, including div/rem.** CG already has i128/u128 types and emitted i128 ops call runtime helpers (`__divti3`, `__udivti3`, `__modti3`, `__umodti3`). Factor the runtime two-limb arithmetic into a shared word-int implementation compiled into `libkit.a`, with runtime ABI wrappers and CG constant folding both calling it through sensible `{lo,hi}` signatures. Do not include `rt/lib/int64/int64.c` wholesale into `libkit` and do not make compile-time folding depend on target runtime helper symbols. |
---
@@ -96,9 +100,15 @@ shadow-stack edit.
CgTarget contract — **no backend changes**.
6. **The hot `ApiSValue` node stays `<= 64` bytes.** The sidecar is a parallel
array (D10).
-7. **No global state.** The sidecar and the unevaluated counter hang off `KitCg`.
+7. **No global state.** The sidecar, constant payload, and unevaluated counter
+ hang off `KitCg`.
8. **The sidecar is opt-in.** It is allocated only for a frontend that enables
it; toy/wasm leave it off and pay nothing.
+9. **Constant facts are stack-owned CG facts, not backend facts.** The constant
+ payload shadows the CG stack in lockstep, carries known/unknown integer bits
+ plus width/type, and has no `CgTarget`, local, temp, or native-backend
+ lifetime. Frontends read it only through public CG APIs; they never inspect
+ `ApiSValue` to decide whether a value is constant.
---
@@ -110,7 +120,9 @@ A parallel array `KitCg.lang_side`, the same length as `KitCg.stack`, allocated
lazily when the frontend calls `kit_cg_lang_enable`. Entry *i* shadows
`stack[i]`. CG moves sidecar entries in exact lockstep with the value stack
(§6). NULL `lang_side` means "no sidecar"; all sidecar ops degrade to no-ops /
-zeroed reads.
+zeroed reads. A separate constant payload shadows the same stack depths whenever
+constant tracking is enabled; it is owned by CG and follows the same lockstep
+rules as the language sidecar, but it is not a C type/flags store.
```c
typedef struct KitCgSlotInfo {
@@ -233,13 +245,22 @@ in both forms.
### 4.5 Unevaluated mode: fold + types, zero emission (D3, D11)
-Unevaluated mode serves two needs that share one machine: (1) compute C result
-types/flags while emitting nothing (`sizeof` operand, `_Generic` controlling
-expr, statically-dead `&&`/`||`/`?:` arms, `extern inline` bodies, constant
-initializer paths), and (2) **fold a constant expression** so the parser can read
-its value (case labels, array sizes incl. file scope, enum values, bit-field
-widths, `alignas`, `_Static_assert`; see §4.7). It is the single mechanism that
-replaces both `suppress_codegen` and the parser's `cexpr_*`/`cint_*` evaluator.
+Unevaluated mode is a CG execution mode, not the full definition of C
+"unevaluated operand" legality. It serves two implementation needs that share
+one stack machine:
+
+- **Type/diagnostic-only parsing:** `sizeof` non-VLA operands, `_Generic`
+ controlling expressions and inactive associations, statically-dead expression
+ arms, `extern inline` bodies when the TU only needs diagnostics, and other
+ parse paths that must compute C result types/flags without emitting code.
+- **Constant-value parsing:** ICE and static arithmetic-constant contexts where
+ the parser must parse the real expression grammar, suppress target emission,
+ then read the folded value from CG (§4.7).
+
+These uses share the same CG mechanics, but not the same C legality rules. CG
+answers "what value/type would this expression stack produce if it can be folded";
+the parser's constant guard (§4.7) answers "is this syntax permitted in this C
+constant-expression category."
```c
void kit_cg_unevaluated_push(KitCg*); /* enter fold+type-only mode (nestable) */
@@ -248,33 +269,48 @@ void kit_cg_unevaluated_pop(KitCg*);
Contract while `unevaluated > 0`:
-- **Maintained:** stack depth, the sidecar (lang_type/lang_flags), and
- **constant folding** of immediate operands — `push_int`/`binop`/`unop`/`cmp`
- fold through the existing C-correct core (`api_try_fold_int_binop/unop/cmp`,
- `api_fold_result`, `src/cg/fold.c`), so `kit_cg_top_const_int` answers
- correctly. Structural ops move sidecar entries normally.
-- **Skipped:** every `CgTarget`/emission call, temp/local allocation, const-local
- store/load tracking, `-O0` ref accounting, and the aggregate-place validation
- in `api_push`. A producing op that *cannot* fold to an immediate pushes a
- value-less node (operand = none/poison) — its `kit_cg_top_const_int` is simply
- false, which is the "not a constant expression" signal (§4.7).
-- **No forced CG-type lowering for the type-only need.** The parser reads C
- facts from the sidecar, not the node `type`; push ops tolerate
- `KIT_CG_TYPE_NONE`. (Constant folding *does* use the supplied CG type for
- width/signedness, so an ICE parse passes real types — cheap, cached.)
-- **Function-independent.** Folding allocates nothing and touches no function
- state, so the mode is valid at file scope (no open function). Handle-returning
- control ops (`label_new`, `scope_begin`, `scope_*_label`) return benign
- dummies; `label_place`/`jump`/`branch_*`/`scope_end` tolerate them. The
- `&&`/`||`/`?:` parsers fold their all-constant cases via `kit_cg_top_const_int`
- (which now works here) and their branch/temp emission no-ops away.
-
-This generalizes suppression from "skip the value op" to "skip *all* of CG,"
-letting the parser **delete its `if (emit)` forks** entirely (e.g. the dual-path
-loop in `parse_stmt.c:85-118` collapses to one path). `suppress_codegen` moves
-out of `Parser` into `KitCg`. Parser-side semantic side-effects in suppressed
-regions (VLA-size slot recording for `sizeof`, `_Generic` association
-collection, diagnostics) still run, because the parser code path is unchanged.
+- **Maintained:** stack depth, `KitCgSlotInfo` sidecar facts, and the constant
+ payload. Every producing operation leaves a stack slot of the requested CG
+ type when a type is available; structural ops move/copy both side payloads in
+ lockstep with the value stack.
+- **Folded:** integer `push`/`cast`/`unop`/`binop`/`cmp` update the constant
+ payload using the CG fold core. A producing op that cannot fold pushes an
+ unknown constant payload, not an emitted value. Unknown is the value-level
+ "not a constant" signal; it is distinct from "illegal C ICE syntax."
+- **Skipped:** every `CgTarget`/emission call, temp allocation, local allocation,
+ const-local store/load tracking, `-O0` ref accounting, and aggregate-place
+ validation that exists only to protect emitted operands. Handle-returning APIs
+ that the parser reaches while suppressed return benign dummy handles; those
+ handles exist only to keep one parse path shaped correctly and cannot be used
+ to recover emitted values later.
+- **File-scope safe:** the mode touches no function state, so it works before a
+ function is open. This is required for file-scope array bounds, enum values,
+ bit-field widths, and static initializers.
+- **Type-only tolerant:** pure type/diagnostic paths may use
+ `KIT_CG_TYPE_NONE` when no CG type is needed. Constant folding still requires
+ real CG integer types so CG knows width and signedness.
+
+`&&`, `||`, and `?:` need explicit handling; they are not solved by making
+labels and branches no-op. In emitting mode they keep their normal lowering
+through labels, branches, and temporaries. In unevaluated mode the parser's
+control-shape helpers parse each syntactic operand once, compute the C result
+type/flags from the sidecar, and push a known or unknown result constant:
+
+- `&&` and `||` can fold from the known left operand without requiring the
+ short-circuited operand to be value-known, but the skipped operand is still
+ parsed for types and diagnostics under the correct C "not evaluated" guard.
+- `?:` folds when the condition is known and the selected arm is known; the
+ unselected arm is still parsed for type merging and diagnostics under the
+ correct guard. When the condition is unknown, the result constant is unknown
+ even if both arms happen to have the same value, unless CG later grows a
+ deliberate meet rule.
+
+The important boundary is that the parser may keep semantic branches for C
+control-shape constructs. What disappears is parser-owned emission bookkeeping:
+`Parser.suppress_codegen`, `pcg_emit_enabled`, duplicate shadow-stack updates,
+and forks whose only purpose was "emit this CG op or do not emit it." Parser-side
+semantic side-effects in suppressed regions (VLA-size bookkeeping, `_Generic`
+association collection, diagnostics) still run on the normal parse path.
### 4.6 Calls (D8)
@@ -291,43 +327,120 @@ grammar* (`cexpr_mul`/`add`/`shift`/`rel`/`eq`/`band`/`bxor`/`bor`/`land`/…) p
a *second constant-arithmetic engine* (`cint_*`, with 128-bit lo/hi, casts,
conversions) — and it re-implements `offsetof` / `__builtin_constant_p` /
enum-constant handling separately from the real parser, a standing drift hazard.
-All of it is **deleted**.
+All of it is **deleted in the ICE cutover phase**.
-The replacement is one helper over the normal path:
+"A real constant-value model" means CG no longer treats "constant" as a property
+of `ApiSValue.operand == OPK_IMM`. The current `kit_cg_top_const_int` can only
+answer for <=64-bit immediate operands. The new model is a parallel stack-owned
+payload: every CG stack slot has known/unknown state plus integer bits, width,
+signedness, and result CG type. It can represent a folded value even when normal
+emission would have produced a local, a wide value lowered through a runtime
+helper, or no emitted value at all because unevaluated mode is active.
+Proposed public shape:
+
+```c
+typedef struct KitCgConstInt {
+ uint64_t lo; /* low bits, always truncated to width */
+ uint64_t hi; /* high bits for width > 64 */
+ uint16_t width; /* 1..128 for integer constants */
+ uint8_t is_signed;
+ uint8_t known;
+} KitCgConstInt;
+
+int kit_cg_top_const_int_ex(KitCg*, KitCgConstInt* out);
+int kit_cg_top_const_i64(KitCg*, int64_t* out); /* convenience */
+void kit_cg_push_const_int(KitCg*, KitCgTypeId type, const KitCgConstInt* v);
```
-i64 / CConstInt eval_const_int(p, loc):
- kit_cg_unevaluated_push(cg);
- parse_conditional(p); /* the REAL grammar, folding as it goes */
- ok = kit_cg_top_const_int(cg, &v); /* (or the i128 variant below) */
- info = kit_cg_slot_info(cg, 0); /* C result type from the sidecar */
- kit_cg_drop(cg);
- kit_cg_unevaluated_pop(cg);
- if (!ok) perr("constant expression required");
- return { v, info.lang_type };
-```
-Consequences:
+The exact names can change, but the surface must be width-complete and must not
+require frontends to inspect `ApiSValue` or know which operands are immediates.
+
+Fold coverage required before the ICE cutover:
+
+- **<=64-bit integers:** add/sub/mul, div/rem, bitwise ops, shifts, comparisons,
+ integer casts, and booleanization. Div/rem by zero must fail cleanly with a
+ diagnostic path the parser can report; it must not silently produce an
+ arbitrary constant.
+- **128-bit integers:** CG already has i128/u128 types and emitted i128
+ arithmetic uses `libkit_rt` helpers. For compile-time folding, create a small
+ shared word-int implementation compiled into `libkit.a` and used by CG through
+ signatures like:
-- One grammar, one fold engine. The constant value comes from the same
- C-correct fold core used at runtime (`src/cg/fold.c`); the C *type* of the
- result comes from the sidecar (the normal path ran usual-arithmetic
- conversions). `sizeof`, `_Alignof`, `offsetof`, `_Generic`, and enum constants
- are handled once, by the real parser — a correctness win.
-- **New API:** a width-complete constant query for i128 ICE, since
- `kit_cg_top_const_int` is 64-bit:
```c
- int kit_cg_top_const_i128(KitCg*, uint64_t* lo, uint64_t* hi); /* or a bytes form */
+ typedef struct KitInt128Words { uint64_t lo, hi; } KitInt128Words;
+
+ int kit_int128_udivmod(KitInt128Words n, KitInt128Words d,
+ KitInt128Words* q, KitInt128Words* r);
+ int kit_int128_sdivmod(KitInt128Words n, KitInt128Words d,
+ KitInt128Words* q, KitInt128Words* r);
```
-- **Diagnostics tradeoff:** failure to fold yields a generic "constant
- expression required" rather than `cexpr_*`'s targeted messages, and ICE
- constraints are enforced semantically (non-foldable → reject) rather than
- syntactically (e.g. the comma operator is foldable yet is not an ICE). A light
- syntactic guard can be added where a precise diagnostic is required.
-- Address/static-initializer constants (`&x + 4`, string-literal addresses,
- designated-init relocations) are a *separate* constant category handled in
- `parse_init.c`; they are **out of scope** here (a later unification could fold
- them onto a CG constant-data path, but not in this plan).
+
+ The implementation should factor the same `ut_add`/`ut_sub`/`ut_mul`/
+ `ut_udivmod` algorithms currently in `rt/lib/int64/int64.c`. The runtime
+ `__*ti3` ABI functions become thin wrappers around that shared implementation
+ (or include the same shared `.inc`), so CG and runtime stay bit-for-bit
+ aligned without duplicating logic. Keep the shared surface over explicit
+ `{lo,hi}` words so the compiler does not depend on being linked against
+ `libkit_rt`, on target ABI helper names, or on host support for TI-mode
+ integers.
+
+The replacement parser helper is one pass over the normal grammar plus a guard:
+
+```c
+CConstEval eval_const_int(Parser* p, SrcLoc loc, CConstKind kind) {
+ CConstGuardMark mark = c_const_guard_push(p, kind);
+
+ kit_cg_unevaluated_push(p->cg);
+ parse_conditional(p); /* the real grammar */
+ int ok_const = kit_cg_top_const_int_ex(p->cg, &value);
+ KitCgSlotInfo info = kit_cg_slot_info(p->cg, 0);
+ kit_cg_drop(p->cg);
+ kit_cg_unevaluated_pop(p->cg);
+
+ int ok_guard = c_const_guard_pop(p, mark);
+ if (!ok_guard || !ok_const || !c_type_is_integer(info.lang_type))
+ perr(loc, "integer constant expression required");
+ return (CConstEval){ value, info.lang_type };
+}
+```
+
+The syntactic/semantic guard is mandatory. A foldable result is not the same
+thing as a legal C integer constant expression:
+
+- `(0, 1)` folds to `1`, but the comma operator is not permitted in an ICE when
+ it is evaluated.
+- `(x = 1, 1)` and `f(), 1` can have foldable tails, but assignment and function
+ call syntax still make the evaluated expression illegal.
+- `sizeof(x++)` can be valid when the operand is non-VLA because the increment is
+ not evaluated; the guard must understand these not-evaluated exceptions rather
+ than reject tokens blindly.
+- `enum` constants are ICE operands; ordinary objects are not, even if some
+ optimization or local-const tracking could know their value.
+- Floating constants are only allowed in the narrow C cases such as the immediate
+ operand of a cast to integer type; the fold payload alone cannot encode that
+ syntactic permission.
+
+Implement the guard as parser mode/counter state, not a second expression
+grammar. Normal parse routines call small note functions when active, for
+example "comma operator in integer constant expression" or "assignment in static
+initializer constant." The guard also has explicit "not evaluated" submodes for
+`sizeof` non-VLA operands and short-circuited `&&`/`||`/`?:` arms. At the end,
+both checks must pass: the guard says the syntax is legal for the requested C
+constant category, and CG says the stack top has a known value of the required
+type.
+
+Keep the categories separate:
+
+- **C ICE:** integer result plus the C ICE guard; used by case labels, enum
+ values, bit-field widths, array bounds that require ICE, `alignas`, and
+ `_Static_assert`.
+- **Static arithmetic constants:** can reuse the CG payload and normal parser
+ path, but may have different C legality from ICE.
+- **Static address/relocation constants:** `&x + 4`, string-literal addresses,
+ and designated-init relocations remain in `parse_init.c` for this plan. A
+ later project can move them onto a CG constant-data model, but this plan does
+ not.
---
@@ -340,11 +453,11 @@ Consequences:
| `PcgSlot.flags` (C value flags) | CG sidecar `lang_flags` |
| `Parser.cg_slot_stack` / `cg_type_sp` / `cg_type_cap` | gone — the CG stack is the typed stack |
| `PcgLvAux` (offset/scale/base_kind/bit_*) + `pcg_materialize_lv_to_ptr` / `pcg_lv_to_memop_place` / `pcg_lv_member` / `pcg_lv_subscript` / `pcg_decay_array` | gone — CG place ops (`deref`/`field_at`/`elem`/`addr`/`field_bits`) fold automatically |
-| `Parser.suppress_codegen` + `pcg_emit_enabled` forks | CG unevaluated mode |
+| `Parser.suppress_codegen` + `pcg_emit_enabled` forks | CG unevaluated mode; parser keeps only semantic forks required by C control-shape constructs |
| `pcg_dup`/`swap`/`drop` (mirror onto shadow stack) | direct `kit_cg_dup`/`swap`/`drop` (sidecar moves automatically) |
| assignment `dup`/`rot3`/`swap` sequences | `kit_cg_store` / `kit_cg_store_keep` |
| `cg_adapter.c` / `cg_adapter.h` / the `pcg_*` layer | **deleted** — parser drives `kit_cg_*` directly |
-| `cexpr_*` grammar + `cint_*` constant-arith engine (`parse_expr.c`) | **deleted** — ICE folds on CG under unevaluated mode (§4.7) |
+| `cexpr_*` grammar + `cint_*` constant-arith engine (`parse_expr.c`) | **deleted in the ICE cutover** — parser owns the legality guard; CG owns constant-value folding (§4.7) |
| C conversions, usual-arithmetic-conversions, binop/cmp/atomic op maps, lvalue-legality, null-ptr/register/bitfield C rules | plain helper *functions* the parser calls (no stack mechanics, no `pcg_` prefix) |
After the cutover there is **no adapter layer**. The parser calls `kit_cg_*`
@@ -356,12 +469,13 @@ directly, with no parallel stack and no EA state.
---
-## 6. Op-by-op: the sidecar lockstep set
+## 6. Op-by-op: the sidecar + constant lockstep set
-Every stack-mutating CG op maintains `lang_side` so depths stay in sync. Grouped
-by effect:
+Every stack-mutating CG op maintains both `lang_side` and the constant payload so
+depths stay in sync. Grouped by effect:
-- **Producers (push a fresh slot, clear its sidecar):** `push_int`, `push_float`,
+- **Producers (push a fresh slot, clear sidecar, set known/unknown const):**
+ `push_int`, `push_float`,
`push_null`, `push_local`, `push_local_addr`, `push_symbol_addr`,
`push_label_addr`, `load`, `addr`, `deref`, `field`/`field_at`/`elem`/
`elem_scaled` (consume then push a place — clear), `alloca`, `vararg_next`,
@@ -369,70 +483,93 @@ by effect:
`intrinsic` result, overflow-builtin result, `inline_asm` outputs.
- **Retypers (1→1, keep depth):** `trunc`/`sext`/`zext`/`bitcast`/`fpext`/
`fptrunc`/`int<->float`/`ptr<->int`, `int_unop`/`fp_unop`. Result sidecar
- cleared; parser retags.
+ cleared; parser retags. The constant payload is converted/folded when CG can
+ do so, otherwise marked unknown.
- **Combiners (N→1):** `int_binop`/`fp_binop`/`int_cmp`/`fp_cmp` (2→1),
`field`/`elem` (consume base/index), `store_keep` (2→1), `call`/`call_symbol`
(N→0/1), `va_copy` (2→0), `atomic_store` (2→0), `atomic_cmpxchg` (3→…).
-- **Pure structural:** `dup` (copy top sidecar), `dup2`, `swap`, `rot3`, `drop`,
- `store`/`store_void` (2→0). These move sidecar entries with no semantic change.
+ Integer combiners fold the constant payload when operands are known and the op
+ is supported; otherwise the result payload is unknown.
+- **Pure structural:** `dup` (copy top sidecar + constant payload), `dup2`,
+ `swap`, `rot3`, `drop`, `store`/`store_void` (2→0). These move side payloads
+ with no semantic change.
- **Consumers (→0):** `branch_true`/`branch_false`/`switch`/`computed_goto`,
`ret`.
-- **Scope edges:** `api_scope_store_results`/`api_scope_push_results` move the
- sidecar with carried results when `lang_side` is active. (C uses only void
- scopes, so this is inert for C, but must be correct for any future
- result-carrying frontend that enables the sidecar.)
+- **Scope edges:** `api_scope_store_results`/`api_scope_push_results` move both
+ side payloads with carried results when active. (C uses only void scopes, so
+ this is inert for C, but must be correct for any future result-carrying
+ frontend that enables the sidecar.)
Implementation choke points: `api_push`/`api_pop` (`src/cg/value.c:161/187`) and
the structural ops `kit_cg_dup`/`dup2`/`swap`/`drop`/`rot3`
-(`src/cg/memory.c:598-735`). Routing the sidecar move through `api_push`/`api_pop`
-(which every producer/consumer already calls) covers most ops automatically; the
-structural ops and scope-edge movers need explicit sidecar handling.
+(`src/cg/memory.c:598-735`). Routing side-payload movement through
+`api_push`/`api_pop` (which every producer/consumer already calls) covers most
+ops automatically; the structural ops and scope-edge movers need explicit
+handling.
---
## 7. Cutover plan (clean break, gate = correctness + determinism)
-No dual-stack interim, no byte-identity gate. Land in dependency order:
-
-1. **CG sidecar + queries.** Add `KitCgSlotInfo`, `lang_side`,
- `kit_cg_lang_enable`, `kit_cg_slot_info`, `kit_cg_retag_top`/`_at`,
- `kit_cg_set_top_flags`; wire lockstep through `api_push`/`api_pop` + the
- structural ops + scope-edge movers. No frontend change yet.
-2. **CG unevaluated mode.** Add `kit_cg_unevaluated_push`/`pop` with the
- fold-constants + maintain-stack + skip-emission/skip-alloc contract (4.5),
- valid at file scope.
-3. **CG place + store + const ops.** Add `kit_cg_field_at`, `kit_cg_store_keep`,
- and the i128 constant query (`kit_cg_top_const_i128`); confirm
- `deref`/`elem`/`field_bits` compose to the fused operand without parser help.
-4. **Rewrite the C frontend onto the new API.** Delete `cg_slot_stack` /
- `cg_type_sp` / `cg_type_cap` / `PcgSlot` / `PcgLvAux`; drive `kit_cg_*`
- directly + retag; replace EA folding with eager places; replace assignment
- choreography with `store`/`store_keep`; call `kit_cg_lang_enable` at
- function/unit setup; move `suppress_codegen` to the CG unevaluated counter and
- drop the parser `if (emit)` forks.
-5. **Delete the adapter.** Remove `cg_adapter.c`/`.h` and the `pcg_` prefix;
- relocate the C-semantic helpers (op maps, conversion lattice, inc/dec,
- atomic/intrinsic/inline-asm shaping) as ordinary parse-module functions
- (D7). Update `doc/FRONTENDS.md` to describe the durable parser → CG contract.
-6. **Move ICE onto CG (last).** Reimplement `eval_const_int` over the normal
- parse path under unevaluated mode (§4.7); delete the `cexpr_*` grammar and the
- `cint_*` engine. Sequenced last so the stack rewrite is validated before the
- constant subsystem is removed. Re-run every ICE context: case labels, array
- sizes (block + file scope), enum values, bit-field widths, `alignas`,
- `_Static_assert`, designated-init indices, and i128 constant expressions.
-
-Gate at each step:
-
-- `make test-cg-api test-parse test-pp test-toy` (frontend + CG API),
-- `make test-cross` / targeted smoke for codegen correctness across arches,
-- determinism: same input → identical output across repeated runs,
-- toy/wasm unaffected (sidecar off): `test-toy`, wasm front suite.
-
-Targeted codegen tests for the place rewrite: local/global load+store; `s.f`,
-`p->f`, `a[i]`, `a[i].f`; bit-field load/store; assignment-expression value
-preservation (`a=b=c`, `if((x=f()))`); `++`/`--` and compound assignment;
-`&` of each lvalue form; struct lvalue / sub-object rvalue; volatile and atomic
-accesses; `sizeof`/`_Generic`/dead-arm suppression returning the right types.
+No dual-stack interim, no byte-identity gate. Each phase is either CG
+infrastructure with no C frontend cutover, or a one-time frontend cutover that
+removes the old path for that subsystem. Do not land a phase where the parser's
+shadow stack and the CG sidecar both authoritatively model live expressions.
+
+1. **CG sidecar + slot queries.** Add `KitCgSlotInfo`, `lang_side`,
+ `kit_cg_lang_enable`, `kit_cg_slot_info`, `kit_cg_retag_top`/`_at`, and
+ `kit_cg_set_top_flags`; wire lockstep through `api_push`/`api_pop`,
+ structural ops, and scope-edge movers. Add focused CG API tests for
+ producer/consumer/structural/scope lockstep. No C frontend change yet.
+2. **CG place + store ops.** Add `kit_cg_field_at` and `kit_cg_store_keep`; move
+ the SEXT-load hint decision into `kit_cg_load`; confirm `deref`/`elem`/
+ `field_at`/`field_bits` compose to fused operands without parser help. Add
+ targeted place tests for local/global load+store, `s.f`, `p->f`, `a[i]`,
+ `a[i].f`, bit-fields, displacement overflow, volatile/atomic accesses,
+ `&` of each lvalue form, assignment value preservation, `++`/`--`, and
+ compound assignment.
+3. **CG unevaluated + constant payload substrate.** Add
+ `kit_cg_unevaluated_push`/`pop`, the stack-side constant payload, the
+ width-complete constant query/push API, <=64 div/rem folding, and the required
+ 128-bit folds including div/rem via the shared word-int unit compiled into
+ `libkit.a`. Add runtime wrapper tests or existing runtime tests to confirm the
+ `__*ti3` ABI helpers still use the same implementation. No C frontend cutover
+ yet. Add CG API tests for
+ file-scope/no-function use, unknown constants, dummy handles, no target/local
+ calls, nested unevaluated mode, and side-payload lockstep under suppression.
+4. **C frontend one-stack cutover.** Enable the lang sidecar for C; replace
+ parser stack reads/writes with `kit_cg_slot_info` + retag; replace EA folding
+ with eager places; replace assignment choreography with `store`/`store_keep`;
+ move `Parser.suppress_codegen` uses to CG unevaluated mode for type-only
+ paths; delete `cg_slot_stack`, `cg_type_sp`, `cg_type_cap`, `PcgSlot`,
+ `PcgLvAux`, and parser-owned effective-address folding. The existing
+ `cexpr_*`/`cint_*` subsystem may remain only because ICE is a later subsystem;
+ it must not keep or resurrect a live expression shadow stack.
+5. **Adapter deletion and helper relocation.** Remove `cg_adapter.c`/`.h` and
+ the `pcg_` prefix. Move the C-semantic helpers (op maps, conversion lattice,
+ inc/dec, atomic/intrinsic/inline-asm shaping) into ordinary parse-module
+ functions that call `kit_cg_*` directly. Update `doc/FRONTENDS.md` to describe
+ the durable parser → CG contract.
+6. **ICE cutover.** Add the parser `CConstGuard`; reimplement `eval_const_int`
+ over the normal parse path under unevaluated mode (§4.7); delete the
+ `cexpr_*` grammar and `cint_*` engine. Re-test every ICE context: case labels,
+ array sizes (block + file scope), enum values, bit-field widths, `alignas`,
+ `_Static_assert`, designated-init indices, invalid ICE syntax that still
+ folds, `sizeof` not-evaluated exceptions, short-circuit exceptions, and i128
+ constant expressions including div/rem and divide-by-zero diagnostics.
+
+Use targeted gates and redirect output to files per `AGENTS.md`:
+
+- Sidecar/constant substrate: `make test-cg-api test-toy`.
+- C frontend cutover: `make test-cg-api test-parse test-pp test-toy`.
+- Codegen-sensitive place/store changes: `make test-cg-api test-isa
+ test-aa64-inline` plus targeted native smoke cases for x64/aa64/rv64 as
+ appropriate.
+- Cross/portability: run `make test-cross TARGET=<selector> DEPTH=smoke` after
+ the one-stack C cutover is green on native targets, and a broader portability
+ pass before final handoff. `make test-port` is not a per-phase gate.
+- Determinism: same input → identical output across repeated runs; byte identity
+ against the old implementation is not required.
---
@@ -446,10 +583,13 @@ accesses; `sizeof`/`_Generic`/dead-arm suppression returning the right types.
- R5 No backend (`CgTarget`/NDT/NT/MC) change; place ops lower through existing
contracts. (§3.5)
- R6 `ApiSValue` stays `<= 64` bytes; sidecar is a parallel array. (§3.6, D10)
-- R7 No global state; sidecar + unevaluated counter hang off `KitCg`. (§3.7)
+- R7 No global state; sidecar, constant payload, and unevaluated counter hang off
+ `KitCg`. (§3.7)
- R8 Sidecar opt-in; toy/wasm pay nothing. (§3.8)
-- R9 Suppression: type+flags maintained, zero emission, zero temp/local alloc, no
- forced CG-type lowering; parser loses its `if (emit)` forks. (§4.5)
+- R9 Suppression: type+flags+constant facts maintained, zero emission, zero
+ temp/local alloc, no forced CG-type lowering for type-only paths; parser loses
+ emission-bookkeeping forks, not semantic forks required by C control-shape
+ constructs. (§4.5)
- R10 Eager places; all EA folding (const offset, dynamic index, displacement
overflow, SEXT-load hint) decided inside CG; `PcgLvAux` deleted. (§4.3, D4)
- R11 Bit-field geometry rides the CG place (`field_bits`); C keeps a semantic
@@ -458,19 +598,26 @@ accesses; `sizeof`/`_Generic`/dead-arm suppression returning the right types.
`lang_flags`. (§4.1)
- R13 `store` / `store_keep` replace assignment dup/rot3/swap. (§4.4)
- R14 No call-mark API; C args stay on the CG stack. (§4.6, D8)
-- R15 ICE evaluation stays parser-side and suppression-independent. (§4.5)
+- R15 ICE evaluation moves to CG's constant-value model plus a parser-owned
+ syntactic/semantic C constant guard. (§4.7, D11)
- R16 Gate = correctness + determinism, not byte-identity. (D5)
- R17 `cg_adapter.c`/`.h` and the `pcg_` prefix deleted; parser drives `kit_cg_*`
directly; C-semantic helpers survive as plain functions. (§5, D7)
- R18 inc/dec, compound assignment, va_arg, atomics, intrinsics, inline-asm,
calls, returns all re-expressed on the new place ops + slot queries. (§6)
-- R19 Unevaluated mode folds constants (not just types) and is valid at file
- scope; `&&`/`||`/`?:` fold their all-constant cases under it. (§4.5)
-- R20 `eval_const_int` reimplemented over the normal parse path; the `cexpr_*`
- grammar and `cint_*` engine deleted; all ICE contexts re-tested. (§4.7, §7.6)
-- R21 i128 constant query (`kit_cg_top_const_i128`) added; ICE diagnostics
- preserved adequately ("constant expression required" + optional syntactic
- guards). (§4.7)
+- R19 `&&`/`||`/`?:` have explicit unevaluated fold paths; they do not rely on
+ no-op labels/branches to leave the right stack result. (§4.5)
+- R20 `eval_const_int` is reimplemented over the normal parse path with
+ `CConstGuard`; the `cexpr_*` grammar and `cint_*` engine are deleted; all ICE
+ contexts and invalid foldable-but-not-ICE cases are re-tested. (§4.7, §7.6)
+- R21 CG exposes a width-complete integer constant payload API; frontends do not
+ inspect `ApiSValue` or `OPK_IMM` to determine constant-ness. (§4.7, D12)
+- R22 <=64-bit and 128-bit div/rem folding move into CG with clean
+ divide-by-zero failure; the 128-bit path uses a shared `{lo,hi}` word-int
+ implementation compiled into `libkit.a`, with runtime `__*ti3` helpers as
+ wrappers or include-site users of the same implementation. (§4.7, D13)
+- R23 Cutovers are one-time subsystem moves with old paths removed; no dual-stack
+ or backcompat phase is introduced. (§7)
---
diff --git a/doc/plan/FRONTEND-SHAPE.md b/doc/plan/FRONTEND-SHAPE.md
@@ -235,7 +235,218 @@ fusion). What *is* available:
posture: bank the scanner lever (−12 %), continue the touch-once grind, treat "within
1.5× of tcc" as a multi-campaign destination — unless the modularity constraints are
relaxed (a tcc-style monolithic hot path), which is explicitly out of scope. **The
-structural hunt (§D) is closed; the path forward is incremental.**
+structural hunt (§D) is closed; the path forward is incremental.** *(Reopened
+2026-06-15 — §E: direction to relax the no-fusion constraint, the one lever §D.8
+named as able to move the uniform-density factor.)*
+
+---
+
+## E. Direction (2026-06-15): front-half fusion around a shared tagged cell
+
+§D.8 is the premise, not a contradiction: the gap is a **uniform ~2.8×
+per-operation density** factor — kit and tcc spend instructions in nearly the same
+*proportions*, kit is ~2.8× denser **everywhere** — so there is no one phase to
+crush, and the only lever that reduces density *across every phase at once* is the
+one §D.2 named and then declined: **fusion**. It removes the per-token / per-op
+*overhead* — inter-stage call/return per token, the 32 B token copied through
+slots, the parallel value stacks, the public value-stack API layer, the CG
+type-universe re-query, growable-container bounds checks — which is exactly the
+pervasive tax §D.8 measured. The landed lean-token boundary (LEX-PP-API.md) is the
+confirming negative: a lean *representation that still copies through separate
+layers* nets ≈ −1.3 % (drain-measured lex+pp still **2.1× tcc**), precisely as
+§D.6/§D.8 predicted a representation-only change would. **Representation was never
+the lever; the handoff and the per-op layering are.** This section relaxes the
+no-fusion constraint under explicit direction and specifies the fused design.
+
+### E.1 What is relaxed; what stays invariant
+
+**Relaxed.** The front half — **lexer + preprocessor + parser** — fuses around one
+shared, in-place **cell** and one value stack. The parser stops being a module that
+consumes an abstract token stream *by value*; it shares the cell representation with
+the pp output and drives emit off it directly. The standing "leave the modules as
+separable as they are" rule (§D.3) is dropped *for the front half*.
+
+**Invariant — two hard constraints the fusion must preserve:**
+1. **The preprocessor stays independently drainable.** `cpp` / `cc -E` /
+ `KIT_PP_DRAIN` must run lexer+pp to a token stream with **no parser attached**.
+ Mechanically: the cell's *lexeme* fields are self-contained, the pp reads/writes
+ only those, and `pp_next` remains the standalone pull. The parser's *semantic*
+ tags live in a region the pp never touches.
+2. **The `CgTarget`→`NativeTarget` backend seam + seven backends stay** (§4.3). The
+ fused value stack drives emission *through* the kept seam (its two indirect calls
+ per primitive are not the cost — §D.8). We fuse the *front half*, not the backend.
+
+One line: **one record flows bytes→token→value with no copy and no re-derivation;
+the pp can still be drained at the token stage; the backend seam is untouched.**
+
+### E.2 The shared cell
+
+One record serves the whole front half. The lexer/pp fill the **lexeme** view; the
+parser resolves a primary **in place** into the **value** view, at which point the
+cell *is* a value-stack entry (kit's `ApiSValue`, tcc's `SValue`). The two views
+overlap — a resolved value no longer needs spelling/`aux` — so the record stays
+small; `loc` and `flags` survive both. Exact layout is an implementation detail to
+size against the < 64 B value-node discipline; the shape:
+
+```c
+typedef struct Cell {
+ u16 kind; /* token kind in lexeme view; value-node kind once resolved */
+ u16 flags; /* lexeme: BOL/space/suffix/encoding · value: lvalue/const/...
+ * (disjoint bit ranges — the two views do not co-live) */
+ u32 loc; /* byte offset; kept through to emit for diagnostics + DWARF */
+ union {
+ struct { /* LEXEME — lexer/pp own; the pp drain reads ONLY this */
+ u32 aux; /* IDENT: interned Sym · PUNCT/#/##: code · PARAM: idx */
+ u32 text; /* spelling ref: (len @loc) for source, or a Sym, tagged */
+ } t;
+ struct { /* VALUE — parser owns; the pp NEVER reads/writes this */
+ u32 type; /* int-bitmask scalar type, or a CG type-id (fallback) */
+ u32 r; /* storage class / value-stack location (local/reg/const)*/
+ i64 c; /* immediate when constant-known (parse-time fold) */
+ } v;
+ } u;
+} Cell;
+```
+
+- **`type` is an int-bitmask for the scalar common case** (signedness × width ×
+ float/ptr/bool), carried on the cell so the hot path never calls
+ `api_unalias_type`/`pred_bits`/`class`/`size` — the §D.2 "classify-once" facts,
+ now *inline on the flowing value* rather than a cache beside the heavy `Type`.
+ Anything a bitmask can't carry (aggregate, bitfield, VLA, `_Atomic`, wide /
+ `__int128`) sets a `FALLBACK` bit and `type` indexes the full CG type universe;
+ the slow path is unchanged. This is the §D "int-bitmask type on a lean value slot,
+ with fallback" made concrete on the unified cell.
+- **No separate `PcgSlot` + `ApiSValue`.** Today the front half keeps two parallel
+ value stacks in lockstep (§0 Pillar 3). The cell *is* the single stack entry;
+ Tier 3's "CG-owns-one-stack-with-an-opaque-aux-slot" is subsumed — the aux *is*
+ the value view of the cell.
+
+### E.3 Data flow (explicit, end to end)
+
+```
+ source bytes
+ │ scan (cclass / word-at-a-time) ── LEXER ──
+ ▼
+ Cell.t ← {kind, flags(BOL/space/suffix), loc=off, aux=Sym|punct, text=span}
+ │ written into a small ring slot (cur / next / pending); NOT returned by value
+ ▼
+ ── PREPROCESSOR ── reads/writes only Cell.t
+ │ · directive recognition, #if skip, include push (source stack)
+ │ · macro expansion: bodies are Cell[] streams, replayed by pointer;
+ │ hidesets ride a side-channel keyed by stream position (never on the cell)
+ │ · __LINE__/__FILE__/paste/stringize synthesize Cell.t (text = interned Sym)
+ ▼
+ pp_next(pp, &slot) ── the drain boundary ──
+ ├───────────────► cpp / cc -E / KIT_PP_DRAIN : consume Cell.t, serialize/discard.
+ │ No parser, no Cell.v ever touched.
+ ▼
+ ── PARSER ── tags the slot IN PLACE, then it becomes a value-stack entry
+ │ primary IDENT: resolve Sym → binding (BindingTab, O(1)) → write Cell.v
+ │ {type(bitmask|fallback), r=storage, c=const?}; push.
+ │ primary literal: decode text → Cell.v {type, c}; push.
+ │ operator: reduce the top Cell.v's on the value stack, driving emit.
+ ▼
+ ── CG-DRIVE ── the value-stack Cell.v's call the CgTarget seam directly
+ │ (no public ApiSValue push/pop API in between; the cell carries r/type/c)
+ ▼
+ CgTarget → NativeTarget → MCEmitter (KEPT seam; emits bytes)
+```
+
+The copies that exist today and disappear: (a) `lex_next` sret → `pp` slot → parser
+`fetch_tok` by-value return → `p->cur` (three hops → one in-place write); (b) token
+→ separate `PcgSlot` *and* `ApiSValue` derivation (→ the primary's cell *is* the
+value entry); (c) the per-op `Type`-universe queries (→ bitmask on the cell). The
+remaining unavoidable move is `next → cur` promotion for LL(2) lookahead (one struct
+move), as today.
+
+### E.4 Tagging: token → value, in place
+
+This is the fused step the boundary rewrite set up but did not take. When the parser
+accepts a **primary**:
+
+- **Identifier.** `kind==IDENT`, `u.t.aux` is the interned `Sym`. Resolve once:
+ keyword via `kw_map`; else binding via `BindingTab` (already O(1), §0 Pillar 2 —
+ the kit-native `TokenSym.sym_identifier`). Overwrite the cell's value view:
+ `u.v.type` = the binding's int-bitmask type (or fallback id), `u.v.r` = its storage
+ (local frame slot / global sym / enum const → `c`). The cell is now an SValue;
+ push it. **No second lookup, no `PcgSlot`+`ApiSValue` pair, no `Type`-universe
+ call** for the scalar case.
+- **Literal.** Decode `u.t.text` (via the text helper) into `u.v.c` + `u.v.type`
+ from the suffix/encoding `flags`; push. (Decode stays lazy — only literals that
+ reach a value are decoded, as now.)
+
+Operators never "tag a token" — they consume value-cells off the stack and call the
+seam. So "the parser tags the token with symbol/type" is precisely: **a primary's
+lexeme cell is resolved in place into the value cell it becomes**, fusing the
+token stream and the value stack into one representation with one resolution.
+
+Diagnostics/DWARF: `loc` rides the cell into `u.v`, so `pcg_set_loc` reads it off the
+live value entry; line/col still materialize lazily through the pp (LEX-PP-API.md).
+
+### E.5 How pp drainability survives the fusion
+
+- The **lexeme view (`Cell.t`) is closed under lex+pp**: every pp operation
+ (directives, macro expansion, paste/stringize, `__LINE__`) produces and consumes
+ only `Cell.t`. Nothing in the pp reads `Cell.v`.
+- `pp_next(pp, &slot)` is the **single standalone entry**: `cpp`, `cc -E`, and
+ `KIT_PP_DRAIN` call it in a loop over a slot they own and never tag. The parser is
+ just *another* caller that happens to then tag the slot. Fusing the pp's internal
+ pull chain (`lex_next`→`src_next_raw_into`→`pp_pull_into` → one tight `pp_next`,
+ tcc's `next_nomacro`+`next` shape) is **internal** to the pp module and does not
+ touch this boundary.
+- So "fused" here means **shared representation + in-place tagging + a collapsed
+ internal pull**, *not* one mega-function. The pp stays a module with a drain entry;
+ the parser stays a module that drives the seam. What's gone is the *by-value
+ handoff and the duplicate value representation* between them.
+
+### E.6 Sequenced build (each byte-identity-gated)
+
+Order matters: land the cheap copy/handoff collapses first (they're the Tier 1/2/3
+work below, now repurposed as fusion precursors), then the two genuinely-fused steps.
+
+```
+E-pre Tier 1B + 1A (macro-arg copies; newline→BOL) — relay slimming, byte-id
+ Tier 2 (out-pointer pull into parser ring) — kills the by-value copy
+E-a collapse the pp internal pull chain into one pp_next (next_nomacro shape);
+ pp stays drainable. Internal-only; byte-id.
+E-b introduce Cell; lexer/pp fill Cell.t; parser ring holds Cells; pp drain
+ reads Cell.t. (Still derives PcgSlot/ApiSValue separately — no value fusion
+ yet.) Byte-id; this is the representation swap, isolated.
+E-c value fusion: a primary's Cell.t resolves in place to Cell.v; Cell IS the
+ single value-stack entry; delete cg_slot_stack + the ApiSValue duplicate;
+ drive the CgTarget seam off Cell.v. Byte-id; the big structural step.
+E-d int-bitmask type on Cell.v for scalars + fallback bit to the CG type
+ universe. Byte-id; this is the type-density lever.
+```
+
+Every step keeps the emitted object **bit-for-bit identical** (it changes how data
+flows, not what is emitted) — same gate as the boundary rewrite: `make perf-golden`
+→ edit → `make perf-gate` (incl. the splice/`#line`/diagnostic battery), full
+pp/parse suites, and verify on **x64 + rv64** for the value-stack steps (shared NDT
+infra, §5). `KIT_PP_DRAIN` + the tcc `-bench` drain (PERF.md §3) measure lex+pp after
+each step to confirm the gap is actually closing.
+
+### E.7 Prototype-first; payoff; the floor
+
+Per §D.4's discipline, **validate the compounding before building the full lane.**
+The decisive prototype is **E-c on a reduced grammar slice** (scalar locals + arith +
+calls, end to end bytes→emit on the unified cell, no fallback paths) measured
+instr/byte against tcc — because §D.6 proved representation alone is ~6 %, the
+*fusion* (removed handoff + removed dual stack + removed type re-query) is the
+untested variable. If a fused scalar slice does not move materially toward tcc's
+density, fusion is also marginal and ~2.5× is the floor for kit's engineering style
+(the honest §D.4 stop). If it does, build E-a..E-d in order.
+
+Payoff and floor, stated honestly:
+- This attacks the **pervasive** factor §D.8 found, so unlike the −12 % scanner lever
+ it is *not* bounded to one phase — it is the credible path toward 1.5× tcc.
+- It is **unproven and expensive** (a front-half rewrite, larger than the boundary
+ rewrite), and it **spends the front-half modularity** (the parser fuses with the
+ pp output; only pp-drain + the backend seam remain as boundaries).
+- The **kept floor**: the `CgTarget` seam's two indirect calls per primitive and the
+ seven-backend generality remain by §4.3 decision — full tcc parity (a single
+ monolithic hot path through emission) is still out of scope; the target is *1.5×*,
+ not parity.
---
