commit f8f4e3001e5e7b7283e99b8b0e2209b738c167f0
parent 277edfa3d286c43fc3e80ffc1f0ad0c5795321d3
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Tue, 16 Jun 2026 08:24:51 -0700
doc/plan/PERF.md: sqlite -O1 optimizer callgrind profile (opt_regalloc_place 24.5%); from disasm-audit run, artifacts in build/linux-prof/
Diffstat:
| M | doc/plan/PERF.md | | | 72 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
1 file changed, 72 insertions(+), 0 deletions(-)
diff --git a/doc/plan/PERF.md b/doc/plan/PERF.md
@@ -246,6 +246,51 @@ instruction metric (Apple malloc is cheap) but it corrects the reset-vs-free
semantics and cuts heap pressure/fragmentation. Heap counters reuse the
`KitProfiler` machinery (embedder counter range), opt-in via `KIT_METRICS=1`.
+**SQLite `-O1` optimizer profile** (Linux callgrind, self `Ir`,
+optimizer-finalize only). Full sqlite `-O1` callgrind collection on the default
+2 GiB Podman VM overflows Valgrind's brk segment and dies with rc 137, so the
+useful profile toggles collection at `opt_on_finalize` and forces glibc malloc
+onto `mmap` (`MALLOC_MMAP_THRESHOLD_=1 MALLOC_ARENA_MAX=1`). That captures the
+whole-program O1 sweep (reachability/internalization, CGIR lowering, inlining,
+per-function O1 pipeline, native emit) but **excludes frontend recording**, so it
+is a distribution profile, not a total comparable to the `-O0` full-compile
+profile above. Command and raw output are in
+`build/linux-prof/cg.sqlite_o1.opt_finalize.mmap.*`.
+
+Profiled command:
+`kit cc -O1 -c sqlite3.c -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -lc`.
+Total collected inside `opt_on_finalize`: **15.496 B Ir**.
+
+| self % | function(s) | subsystem |
+|--:|---|---|
+| **24.5** | `opt_regalloc_place` | physical register placement |
+| **18.7** | `opt_func_from_cg_ir` | CGIR -> optimizer Func lowering |
+| **7.8** | `alloc_assign_group_hard` | hard-register assignment grouping |
+| **7.4** | `ir_note_emit` | IR/lowering record construction |
+| 4.5 | `opt_build_loop_tree` | loop analysis |
+| 3.2 / 1.7 / 1.6 / 1.5 | `opt_bitset_has` / `opt_bitset_union` / `opt_bitset_copy` / `opt_bitset_union_and_not` | liveness/range bitsets |
+| 2.4 | `opt_live_ranges_build` | live-range construction |
+| 2.1 / 0.8 | `msort_with_tmp` / `u32_cmp` | sorting in optimizer hot paths |
+| 1.6 / 0.3 | `__GI_memset` / `__GI_memcpy` | libc memory |
+| 1.5 / 0.2 | `inline_call_site` / `opt_inline` | whole-program inliner |
+| 1.2 / 1.0 / 0.3 / 0.2 | `metrics_count` / `metrics_scope_from_name` / `metrics_scope_from_name'2` / `metrics_count'2` | profiling string-name lookup |
+| 1.1 | `arena_alloc` | arena bump path |
+| 1.0 | `opt_addr_xform_pregs` | address transform |
+| 0.1 | `opt_emit_native` | native emit self-time |
+
+Interpretation: sqlite `-O1` compile time is dominated by the optimizer's
+middle/back half, not native emission. Register allocation alone is at least
+**32 %** (`opt_regalloc_place` + `alloc_assign_group_hard`, excluding the shared
+bitset/range helpers), while CGIR lowering plus record construction is another
+**26 %**. Built-in metrics on the same TU report **2,522 functions**,
+**121,075 blocks**, **98,516 PRegs**, **1,049,079 ranges**, and **29.1 M live
+bitset words touched**, which explains why the liveness/range bitset helpers are
+visible. A cheap-looking cleanup also shows up: `metrics_scope_from_name` and
+`metrics_count` burn ~2.7 % of O1 finalization even with `KIT_METRICS` unset,
+because each hot scope/counter call string-decodes the name before discovering
+the profiler sink is null. The O1 hot path should use enum-specific wrappers or
+check the profiler before name decoding.
+
### Code size is still locally spill-bound
Even with aggregate `.text` now smaller than tcc, the remaining per-mnemonic
@@ -325,6 +370,33 @@ scripts/perf_callgrind.sh run <tag> # builds kit in-container, runs callgrind,
# → build/linux-prof/cg.<tag>.annot.txt (self Ir + callers)
```
+For the sqlite `-O1` optimizer-finalize profile, use `podman machine` explicitly
+to give the VM enough memory for Valgrind, and force malloc away from brk:
+
+```sh
+podman machine stop || true
+podman machine set --memory 3072
+podman machine start
+podman run --rm --platform linux/arm64 -v "$PWD":/work:Z kit-prof sh -c '
+ set -eu
+ cd /work/tmp/projects/sqlite-amalg
+ env MALLOC_MMAP_THRESHOLD_=1 MALLOC_ARENA_MAX=1 \
+ valgrind --tool=callgrind --cache-sim=no --branch-sim=no --dump-instr=no \
+ --collect-jumps=no --collect-atstart=no --toggle-collect=opt_on_finalize \
+ --callgrind-out-file=/work/build/linux-prof/cg.sqlite_o1.opt_finalize.mmap.out \
+ /work/build/linux-prof/kit cc -O1 -c sqlite3.c \
+ -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -lc -o /tmp/sqlite_o1.o \
+ 2>/work/build/linux-prof/cg.sqlite_o1.opt_finalize.mmap.vg.log
+ callgrind_annotate --threshold=99.5 \
+ /work/build/linux-prof/cg.sqlite_o1.opt_finalize.mmap.out \
+ >/work/build/linux-prof/cg.sqlite_o1.opt_finalize.mmap.annot.txt
+'
+# restore the default local VM shape after the run
+podman machine stop
+podman machine set --memory 2048
+podman machine start
+```
+
Gotchas baked into the script (do not relearn): **(a)** pass `-lc` so kit finds
the glibc sysroot even for `-c`; use **bookworm** (glibc 2.36), not ubuntu 24.04
(glibc 2.39's `bits/math-vector.h` uses a vector-typedef attribute the C frontend
