commit f2841b12dee3f171dfab2888e0ebc619060a50b3
parent 4fa49d55d7bc1ccb2918c81505c9d0f26002e18c
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Thu, 11 Jun 2026 14:53:47 -0700
docs(perf): 16B Tok is a compile regression, not neutral — dead end
Built + merged the 16B-Tok foundation onto HEAD (byte-identical, gate
clean) and measured it on a quiet machine (best-of-15): it SPLITS —
pp-macro -E +3.3% but type-decl -7.1%, body-size -3.4%, fn-count -2.8%.
The earlier 'neutral' was load-noise. The compile path is loc-heavy, so
every token pays the LocId indirection on both ends (lexer interns a 12B
store, parser resolves via a cross-TU call), outweighing the smaller-Tok
copy saving; -E wins only by passing tokens through without resolving.
The planned lexer inline removes only the intern call overhead, not the
store/resolve work that IS the regression, and the clean inline would
require exposing libkit's loc-table to lang/cpp (boundary violation
anyway). Tok is shared by -c and -E and -c dominates -> net loss.
Recorded as a dead end.
Diffstat:
1 file changed, 18 insertions(+), 9 deletions(-)
diff --git a/doc/plan/PERF.md b/doc/plan/PERF.md
@@ -199,15 +199,24 @@ design). Ranked by (impact × confidence) / risk:
reloc/ABI handling and variable-length x86 encoding). Worth it only as a
deliberate big bet on the dominant bucket. (Spike + numbers persisted under
`build/perf-nonbid-analysis/codegen-copypatch-spike.md`.)
-2. **16-byte `Tok` (register return)** — *implemented and byte-identical* (a
- lossless 32-bit `LocId` side-table replaces the inline 12 B `SrcLoc`, so
- diagnostics stay exact and it passes the full byte-id gate), and it does
- achieve the `x0:x1` register-return ABI (no x8 sret). But it measured
- **neutral**: the two per-token `kit_loc_intern` calls it adds cost about as
- much as the sret copy they remove. To flip it positive, **inline the loc-table
- append into the lexer's hot loop** (publish the append cursor so a
- static-inline fast path does one bounds-compare + store, out-of-line only on
- grow). The clean 16 B `Tok` foundation is the prerequisite and is done.
+2. **16-byte `Tok` (register return)** — *implemented, byte-identical, and
+ rejected on measurement.* The lossless 32-bit `LocId` side-table makes `Tok`
+ 16 B (exact diagnostics, passes the full byte-id gate) and achieves the
+ `x0:x1` register-return ABI. But on a **quiet** machine (best-of-15; the
+ earlier "neutral" was load-noise) it **splits**: pp-macro `-E` **+3.3 %** but
+ type-decl **−7.1 %**, body-size **−3.4 %**, fn-count **−2.8 %**. Reason: the
+ compile path is loc-heavy (a loc per statement + `.loc` on every AST node), so
+ each token pays the `LocId` indirection on *both* ends — the lexer interns
+ each loc (a 12 B store into the table) and the parser resolves it (a table
+ lookup + a cross-TU `kit_loc_resolve`) — which costs more than the smaller-`Tok`
+ copy saves; `-E` wins only because the pp passes tokens through without
+ resolving. The planned "inline the loc append into the lexer" does **not** fix
+ this: it removes only the intern *call overhead*, not the per-token store work
+ or the parser-side resolve cost, which are the regression. And the clean inline
+ is blocked anyway — it would require exposing libkit's loc-table layout to the
+ `lang/cpp` frontend, a boundary violation. `Tok` is one type shared by `-c` and
+ `-E`, so it is all-or-nothing, and `-c` dominates → net loss. **Dead end; do
+ not revive.**
3. **Keyword classify-once** — *partly landed.* The real residual the `KwMap`
missed was `parse_decl_specs` re-deciding keyword-ness ~27×/token (26 `is_kw`
+ a trailing `ident_kw`); a single alias-aware `classify_kw()` at the loop top
