kit

commit bfa30f771ee3069362c448511d73d1910a200510
parent d953be125bc6d503b325833cce8d7cafb0e7fb47
Author: Ryan Sepassi <rsepassi@gmail.com>
Date:   Sat, 13 Jun 2026 10:22:43 -0700

perf(pp): drain non-directive newlines in place — skip per-newline arena reset

The cc parser feed (pp_next -> pp_pull_into with skip_nl=1) discards every
non-directive TOK_NEWLINE, which is ~51% of all produced tokens. §C.3a dropped
them one frame lower with a `continue`, but each dropped newline still paid a
full outer-loop iteration: a kit_arena_reset() call (out-of-line) to reclaim
the expansion scratch plus the SRC_BUF fast-path probe, both redundant for a
newline (the source stack is unchanged between a newline and the token after
it; a SRC_BUF only ever sits above the SRC_LEX a file newline came from, so it
is never on top here, and raw newline reads allocate nothing in xarena).

Drain a run of newlines in place via src_next_raw_into instead of re-entering
the outer loop, removing one arena-reset call + probe per dropped newline. The
drained token re-enters the normal directive/macro flow, so a following `#` or
content token is handled exactly as before.

Byte-identical: 60/60 perf identity gate PASS (incE/ppmE/spliceE -E
categories, diagnostics, splice battery, objects/-g/linked exe), test-parse
4920/0, test-pp 110/0; sqlite -c object bit-for-bit identical to golden.

Diffstat:
M
lang/cpp/pp/pp_expand.c
 | 
26
++++++++++++++++++++++----

1 file changed, 22 insertions(+), 4 deletions(-)
diff --git a/lang/cpp/pp/pp_expand.c b/lang/cpp/pp/pp_expand.c
@@ -981,11 +981,29 @@ static void pp_pull_into(Pp* pp, Tok* out, int skip_nl) {
       src_next_raw_into(pp, out, &hs, &src_kind);
     }
     if (out->kind == TOK_EOF) return;
-    /* §C.3a: absorb the parser-facing non-directive newline drop here so it
-     * never round-trips up through pp_next's sret. A newline is whitespace,
+    /* §C.3a/§C.3b: absorb the parser-facing non-directive newline drop here so
+     * it never round-trips up through pp_next's sret. A newline is whitespace,
      * not a directive introducer, so dropping it before the directive / macro
-     * checks below is the same as pp_next dropping it after the return. */
-    if (skip_nl && out->kind == TOK_NEWLINE) continue;
+     * checks below is the same as pp_next dropping it after the return.
+     *
+     * §C.3b drains a run of newlines in place via src_next_raw_into rather than
+     * `continue`-ing the outer loop per newline. The outer loop's only
+     * per-iteration work before this point is the xarena reclaim
+     * (kit_arena_reset, a real call) and the SRC_BUF fast-path probe; for a
+     * newline both are redundant — the source stack is unchanged between a
+     * newline and the token that follows it (a newline never pushes a source,
+     * and a SRC_BUF can only sit *above* the SRC_LEX a file newline came from,
+     * so it is not on top here). Skipping them is byte-identical to the §C.3a
+     * `continue` while removing one kit_arena_reset call + probe per dropped
+     * newline (51% of produced tokens). The drained token re-enters the normal
+     * directive / macro flow below, so a `#` or content token after the
+     * newline is handled exactly as before. */
+    if (skip_nl && out->kind == TOK_NEWLINE) {
+      do {
+        src_next_raw_into(pp, out, &hs, &src_kind);
+      } while (out->kind == TOK_NEWLINE);
+      if (out->kind == TOK_EOF) return;
+    }
     if (out->kind == TOK_PP_HASH && (out->flags & TF_AT_BOL) &&
         src_kind == SRC_LEX) {
       process_directive(pp, out->loc);