commit 86ef4a0326e9cda798fafd2a5a04900881e5b86b
parent 8b8690a117ea8e74ec13bbd64458f29402317ed1
Author: Ryan Sepassi <rsepassi@gmail.com>
Date: Fri, 19 Jun 2026 17:57:54 -0700
Fix build deep config projection
Diffstat:
11 files changed, 423 insertions(+), 102 deletions(-)
diff --git a/doc/BUILD_COORDINATOR.md b/doc/BUILD_COORDINATOR.md
@@ -57,8 +57,8 @@ the resolution algorithm that drives it.
| **Base inputs** | The *leaves* of the dependency graph, dynamically requested by recipes: **config values**, **source files**, **globs**, and hash-pinned **fetch blobs**. |
| **Target dep** | A dependency of one target on the *output tree* of another. The graph's interior edges, created by `need`. |
| **Shallow trace** | A record of one build: its direct base inputs + its direct target deps (by output `tree-id`) → its output `tree-id`. |
-| **Deep trace** | A record of one build: `(root config-id, argv)` + a pointer to the transitive input closure (a **deepset**) → its output `tree-id`. |
-| **Deepset** | One node of the transitive input-closure DAG (`deep-set-id` = `BLAKE2b(canonical body)`): a target's recipe-id + its direct source/glob/blob leaves + its children's `deep-set-id`s. A self-verifying CAS blob; shared structurally across ancestors. |
+| **Deep trace** | A record of one build: `(argv)` + a pointer to the transitive input closure (a **deepset**) → its output `tree-id`. Config matching is driven by the deepset's scope-projected config observations, not by whole-config equality. |
+| **Deepset** | One node of the transitive input-closure DAG (`deep-set-id` = `BLAKE2b(canonical body)`): a target's recipe-id + its direct config/source/glob/blob leaves + its children's `deep-set-id`s. A self-verifying CAS blob; shared structurally across ancestors. |
| **Tree cache** | On-disk *materialized* output directories, keyed by `tree-id`, ready to hand back as a filesystem path. |
The base-input kinds are exactly the dynamically-requested leaves: a recipe asks
@@ -154,11 +154,12 @@ build.)
### Why two trace kinds
-The deep trace answers *"is this exact configuration unchanged and has any input
-moved?"* by refreshing the deepset closure — a DAG walk with structural sharing
-and an id-equality short-circuit, not a re-resolution. When it matches, the output
-is determined and we skip straight to materialization. It is the inner dev-loop
-fast path: edit code, rebuild under the same config.
+The deep trace answers *"are the config values this closure read from this
+request's config scope unchanged, and has any other input moved?"* by refreshing
+the deepset closure — a DAG walk with structural sharing and an id-equality
+short-circuit, not a re-resolution. When it matches, the output is determined and
+we skip straight to materialization. It is the inner dev-loop fast path: edit
+code, rebuild under the same relevant config.
The shallow trace handles everything the deep path defers — a changed source that
might not actually move a dep's output, and *any* configuration or argv change. It
@@ -242,16 +243,16 @@ The `recipe <recipe-id>` scalar is the only definition-derived input (recomputed
through the live definition on refresh); the definition file itself is *not* a
source leaf, so editing an unrelated target's stanza does not bust this trace.
-**Deep trace** — the `(root config-id, argv)` binding to an output, plus a single
-pointer to the transitive input closure (held as a **deepset** DAG, below), *no*
-`[dep]` and *no* per-key `[config]` section:
+**Deep trace** — the `(argv)` binding to an output, plus a single pointer to the
+transitive input closure (held as a **deepset** DAG, below), *no* `[dep]` and
+*no* inline per-key `[config]` section. Config observations live on deepset nodes
+so they can be checked with the correct propagated-config scope:
```
kit-build-deep 1
target //app:server
recipe <recipe-id>
output <output-tree-id>
-root-config <config-id> ; this target's effective propagated config (full map blob)
argv <argv-id> ; local config (serialized vector blob)
deepset <deep-set-id> ; the transitive input closure, as a deepset DAG (a CAS blob)
```
@@ -281,7 +282,6 @@ kit-build-test-deep 1
target //pkg:unit
recipe <recipe-id>
result <result-tree-id>
-root-config <config-id>
argv <argv-id>
deepset <deep-set-id>
```
@@ -300,6 +300,8 @@ ancestor:
kit-build-deepset 1
target //app:server ; carried so refresh can recompute this node's recipe-id
recipe <recipe-id> ; the node's recipe content-hash (the only definition-derived input)
+[config] ; this node's scope-projected propagated config observations
+opt <present|unset/default...> <value-hash-or->
[source] ; this node's DIRECT source leaves, sorted by path
src/main.c <blob-id>
config.h - ; an ABSENT leaf: "-" = the path was absent when read
@@ -313,16 +315,20 @@ src/*.c <glob-result-hash>
`deep-set-id = BLAKE2b(canonical deepset body)`, which is exactly its CAS blob id.
-- `config <config-id>` and `argv <argv-id>` reference the **serialized** effective
- config map and argv vector — canonical CAS blobs, kept by value and bundled with
- shared traces — so the target's invocation and every `need` it issues replay
- *exactly*, not merely check (see [Configuration model](#configuration-model)).
- `[config]` lists consumed propagated-key observations (consumed whether set
*or* unset), including the presence bit and any recipe-supplied default hash.
Presence is part of the match: a key consumed while unset does not match a
later explicit value, even if that value equals the default the recipe returned.
When both sides are present, matching compares the value hash; when both are
unset, the observation matches.
+- During deep refresh, those observations are interpreted relative to the root
+ query's propagated-config scope. A descendant observation for key `K` is a
+ root-scope observation only if no `need` edge on the path from the root to that
+ descendant overlaid `K`. If a recipe reaches `need X --config K=V`, then `X`'s
+ reads of `K` are shielded from the parent's root-scope match: unchanged parent
+ recipe code and unchanged parent inputs reproduce the overlay. The same reads
+ are still recorded in `X`'s own deepset and are checked when `X` is queried as
+ the root under its effective config.
- `<blob-id>` = the source file's CAS blob id (`kit_blob_info`); source and glob
leaves are kept by hash because they are *verified*, never replayed. An absent
source is recorded as `<path> -` (a lone `-`), so *creating* the file later
@@ -352,21 +358,19 @@ src/*.c <glob-result-hash>
together with the dep's own serialized config/argv blobs this re-resolves each
dep under the exact propagated config *and* argv it used, overlays included.
-**Why the deep trace needs only `root-config` + the deepset closure.** If the
-request's `config-id` equals `root-config`, its `argv-id` equals the recorded
-`argv`, *and* refreshing the deepset finds nothing moved — every source/glob/blob leaf
-still matches the live workspace **and** every node's `recipe-id` still recomputes
-the same through the live definition — then every recipe in the closure has
-byte-identical inputs *and* identical code, so every `need` overlay it computes is
-identical, every effective config downstream is identical, and by determinism the
-output is identical. Any input that could perturb a downstream overlay (a source a
-recipe branches on, a config value) is itself either a recorded source/glob/blob leaf,
-a per-node `recipe-id`, or folded into `root-config`, so nothing escapes the
-check. Using the whole `root-config` (rather than just
-consumed keys) is a deliberate, conservative simplification: an *unconsumed*
-config change busts the deep fast path, but the [shallow path](#resolution-algorithm)
-recovers — it re-runs only the recipes whose output actually changed, so unrelated
-subtrees re-verify by hash without re-running.
+**Why the deep trace needs argv + the deepset closure.** At Phase 1 query time
+the coordinator has only the requested target's effective propagated config and
+local argv; it has not re-run the recipe, so it cannot recompute dynamic `need`
+overlays. A deep match is valid when the request's `argv-id` equals the recorded
+`argv`, and refreshing the deepset finds nothing moved: every source/glob/blob
+leaf still matches the live workspace, every node's `recipe-id` still recomputes
+the same through the live definition, and every scope-projected config
+observation still matches the current root config. Then every root-scope value
+that could perturb a recipe branch or downstream overlay is unchanged. Values
+introduced by recorded `need` overlays are not compared to the parent's root
+config; their stability follows from the unchanged recipe code and unchanged
+inputs that generated those overlays. By determinism, every effective downstream
+config is identical where it matters, so the output is identical.
### Target record (mutable, the only mutable object)
@@ -473,8 +477,8 @@ A request is `resolve(T, cfg, argv, chain)` where `cfg` is the effective
propagated config (with `cfg.id` its config-id) and `argv` is the effective local
argv (with `argv.id` its argv-id — the request's argv, or empty when none is
supplied). The config/argv-dependence is recorded in and verified against the
-traces; the *deep* path checks whole-config-id and argv-id equality, the *shallow*
-path checks consumed keys and argv-id.
+traces; the *deep* path checks argv-id plus the deepset's scope-projected config
+observations, while the *shallow* path checks consumed keys and argv-id.
```
resolve(T, cfg, argv, chain):
@@ -486,11 +490,11 @@ resolve(T, cfg, argv, chain):
if record(T) is empty and trace_remotes configured: # clean checkout: try shared traces
pull_once(T) # verify+install signed bundle, then re-scan
- # ---- Phase 1: deep fast path (same config+argv, did anything move?) ---
+ # ---- Phase 1: deep fast path (same argv, did any observed input move?) ---
for D in deep_traces(T), newest-first:
- if D.root_config == cfg.id and D.argv == argv.id:
+ if D.argv == argv.id:
node = deepset_load(D.deepset) # load DAG from CAS; absent => skip (fail-safe)
- if node and refresh(node) all match live: # rehash/reglob + recompute recipe-id, memoized
+ if node and refresh(node, cfg) all match live: # config observations + rehash/reglob + recipe-id
p = materialize(D.output) # pure: cache -> CAS -> remote; ERR => fall through
if p ok: return done(D.output, p, node)
@@ -502,13 +506,14 @@ resolve(T, cfg, argv, chain):
if any consumed key in S.config-keys differs between cfg and M: continue
if any direct source/glob/blob leaf of S changed: continue
ok = true ; child_nodes = []
- for (dep, dep_cfg_id, dep_argv_id, recorded_tree) in S.deps: # may run in parallel
- r = resolve(dep, config_by_id(dep_cfg_id), argv_by_id(dep_argv_id), chain') # replay the need by id
+ for (dep, overlay_id, dep_argv_id, recorded_tree) in S.deps: # may run in parallel
+ dep_cfg = overlay(cfg, config_by_id(overlay_id)) # replay the need overlay
+ r = resolve(dep, dep_cfg, argv_by_id(dep_argv_id), chain')
if r.output != recorded_tree: ok = false; break
child_nodes.push(r.leafset)
if ok:
- node = union(direct{recipe_id, source/glob/blob leaves} of S, child_nodes) # builds+stores the deepset
- write_deep_trace(T, cfg.id, argv.id, node.id, S.output) # refresh the deep trace
+ node = union(direct{recipe_id, config/source/glob/blob leaves} of S, S.deps, child_nodes)
+ write_deep_trace(T, argv.id, node.id, S.output) # refresh the deep trace
p = materialize(S.output) # pure; ERR => fall through
if p ok: return done(S.output, p, node)
@@ -535,11 +540,13 @@ tree), and removes the resolve→runner edge.
This is the brief's flow, sharpened by config:
-- *Phase 1* is the deep fast path: identical config-id **and argv-id** and an
- all-clear deepset refresh — every source/glob/blob leaf unchanged **and** every
- node's `recipe-id` still recomputing the same — ⇒ reuse. `refresh` hits the
- per-process memos and the deep-set-id validity cache, so a shared subtree is
- checked once and an unchanged subtree is skipped by id equality.
+- *Phase 1* is the deep fast path: identical **argv-id** and an all-clear deepset
+ refresh — every scope-projected config observation matches the current root
+ config, every source/glob/blob leaf is unchanged, and every node's `recipe-id`
+ still recomputes the same — ⇒ reuse. `refresh` hits the per-process memos and
+ the deep-set-id validity cache, so a shared subtree is checked once and an
+ unchanged subtree is skipped by id equality when its already-validated config
+ projection is the same.
- *Phase 2* is the shallow path, taken when Phase 1 finds nothing (config or argv
changed, or a source moved). It guards on the target's own argv-id, consumed
config, and direct leaves, then re-resolves each recorded dep **under the dep's
@@ -558,6 +565,64 @@ whose recipe maps it to an unchanged output (`tree-id` identical), the dep compa
passes and `T`'s recipe is skipped. The deep trace is the cheap "nothing moved"
check; the shallow trace is the "did the churn actually reach me?" check.
+### Diagnostic resolution trace
+
+Tests sometimes need to assert *which target* took which resolution path, not
+just aggregate counters. A diagnostic trace mode should emit a stable,
+line-oriented event stream describing resolution decisions. It is not cache
+identity, not bundled, and not intended for normal quiet builds; it is an
+assertion/debug surface for humans and tests.
+
+The implemented target-level events are:
+
+```
+resolve-start action=<build|test> target=<T> root=<0|1> config=<config-id> argv=<argv-id>
+deep-candidate target=<T> trace=<trace-id>
+deep-miss target=<T> reason=<no-match|malformed>
+deep-hit target=<T> output=<tree-id>
+shallow-candidate target=<T> trace=<trace-id>
+shallow-miss target=<T> reason=<no-match>
+shallow-hit target=<T> output=<tree-id>
+recipe-run target=<T>
+```
+
+When implementing the scope-projected deep config logic, extend the trace so deep
+config checks make scope projection visible:
+
+```
+deep-config target=<T> key=<K> scope=root result=<match|mismatch>
+deep-config target=<T> child=<U> key=<K> scope=overlay-shielded overlay=<overlay-config-id>
+```
+
+This lets tests distinguish "the requested target deep-hit" from "a child
+deep-hit while the requested target shallow-hit", which aggregate
+`deep_hits`/`shallow_hits` cannot express.
+
+Expected event shapes for the config-scope cases:
+
+- Plain inheritance: `A -> X -> Y`, `Y` reads `mode`, no overlays. Rebuilding
+ `A` with changed `mode` emits `deep-miss target=A reason=config` with
+ `deep-config ... key=mode scope=root result=mismatch`.
+- Parent overlay: `A -> X --config mode=forced`, `X` reads `mode`, and `A` does
+ not read `mode`. Rebuilding `A` with root `mode` changed emits
+ `deep-config target=A child=X key=mode scope=overlay-shielded ...`, then
+ `deep-hit target=A`.
+- Parent read plus overlay: `A` reads `mode` and also needs
+ `X --config mode=forced`; `X` reads `mode`. Rebuilding `A` with root `mode`
+ changed emits a root-scope mismatch for `A`'s own read and a shielded child
+ observation for `X`, then `deep-miss target=A reason=config`.
+- Overlay one key, inherit another: `A -> X --config mode=forced`; `X` reads
+ `mode` and `flavor`. Changing root `mode` should still emit
+ `deep-hit target=A`; changing root `flavor` should emit
+ `deep-miss target=A reason=config`.
+- Intermediate overlay: `A -> X -> Y --config mode=forced`; `Y` reads `mode`.
+ Rebuilding `A` with root `mode` changed emits a shielded observation for the
+ `X -> Y` edge and `deep-hit target=A` if no root-scope observations changed.
+- Overlay derived from root config: `A` reads `mode` and then overlays
+ `child_mode=$mode` into `X`; `X` reads `child_mode`. Changing root `mode`
+ emits `deep-miss target=A reason=config` because `A`'s own root-scope read
+ changed; `X`'s `child_mode` observation remains overlay-shielded for `A`.
+
### Test caching
`kit build test` is first-class test execution in the coordinator, not a separate
@@ -612,11 +677,11 @@ run_recipe(T, cfg, argv, chain):
on success:
output = kit_cas_add_tree_from_dir(out)
put serialized cfg map (cfg.id) and argv vector (argv.id) into the CAS
- node = union(direct{recipe_id, source/glob/blob/absent leaves}, child deepset nodes)
+ node = union(direct{recipe_id, config/source/glob/blob/absent leaves}, child deepset nodes)
# builds + stores the root deepset as a CAS blob, yields its deep-set-id
write_shallow_trace(T, recipe_id, argv.id, cfg.id, consumed key names,
direct leaves, dep edges with their config-ids and argv-ids, output)
- write_deep_trace (T, cfg.id, argv.id, node.id, output) # references the deepset
+ write_deep_trace (T, argv.id, node.id, output) # references the deepset
prepend both to T's target record (dedup, truncate to cap)
install out in build/cache/<output>/
return done({output, path}, node)
@@ -762,7 +827,7 @@ Build execution:
```
kit build [--store DIR] [--root DIR] [--def FILE]
[--config K=V|env.NAME]... [--env NAME[=V]]...
- [--verify] [--stats] TARGET [-- ARG...]
+ [--verify] [--stats] [--trace] TARGET [-- ARG...]
```
Test execution:
@@ -770,7 +835,7 @@ Test execution:
```
kit build test [--store DIR] [--root DIR] [--def FILE]
[--config K=V|env.NAME]... [--env NAME[=V]]...
- [--verify] [--stats] TARGET [-- ARG...]
+ [--verify] [--stats] [--trace] TARGET [-- ARG...]
```
`kit build` prints `<output-tree-hex> <path>` and exits 0 on success. `kit build
@@ -778,7 +843,9 @@ test` prints `<status> <result-tree-hex> <path>`, where status is `PASS` or
`FAIL`; it exits 0 for PASS, 1 for FAIL, and 2 for bad CLI usage. Infrastructure
errors also exit nonzero and print diagnostics on stderr. `--stats` prints
cumulative coordinator counters, including `test_runs`, `test_cache_hits`, and
-`test_failures`, to stderr.
+`test_failures`, to stderr. `--trace` prints target-level resolution decisions
+to stderr, using the [diagnostic resolution trace](#diagnostic-resolution-trace)
+event names.
Recipe-side shell helper commands are available as `kit build <verb> ...` inside
a running recipe (`$KIT_BUILD_SOCK` set), or explicitly outside a recipe as
@@ -908,9 +975,9 @@ deterministically.
Both trace kinds written for both targets. `build/cache/A0/` materialized, path
returned.
2. **No-op rebuild (same `C0`).** Phase 1: `//app:server`'s deep trace has
- `root-config c0` and `argv a0` (both match) and refreshes `opt` via config-id, `src/*.c`,
- `src/main.c`, and the folded `lib/core.c` — all memoized, all match. `A0` is in
- the tree cache. Returns instantly, nothing re-run, no graph walk.
+ `argv a0` and refreshes the root-scope `opt` observation, `src/*.c`,
+ `src/main.c`, and the folded `lib/core.c` — all memoized, all match. `A0` is
+ in the tree cache. Returns instantly, nothing re-run, no graph walk.
3. **Comment-only edit to `lib/core.c`.** `lib/core.c`'s blob-id moved → the deep
trace fails Phase 1 (a folded source leaf changed). Phase 2: direct leaves
(`opt`, `src/*.c`, `src/main.c`) unchanged, so probe the one dep —
@@ -918,13 +985,14 @@ deterministically.
byte-identical `L0` (comment stripped). `L0 == L0` ⇒ `//app:server`'s recipe is
**skipped**, `A0` restored, a refreshed deep trace written so step 2's fast path
returns next time.
-4. **Flip `opt` (now `C1`, `config-id = c1`).** Phase 1 is skipped outright:
- `//app:server`'s deep trace has `root-config c0 ≠ c1`. Phase 2 under `c1`:
- `opt` is a consumed key whose value differs between `c1` and the trace's built
- map → no shallow trace holds → Phase 3 re-runs `//app:server`. Its `need //lib:core` carries no `opt` override,
- so the dep resolves as `(//lib:core, c1, a0)`; but `//lib:core` never
- consumes `opt`, so its shallow trace under `c1` matches by argv-id (`a0`),
- consumed-keys (empty), and direct leaves, and `L0` is reused without
+4. **Flip `opt` (now `C1`, `config-id = c1`).** Phase 1 loads the deep trace
+ because `argv a0` still matches, then fails while refreshing the root-scope
+ `opt` observation. Phase 2 under `c1`: `opt` is a consumed key whose value
+ differs between `c1` and the trace's built map → no shallow trace holds →
+ Phase 3 re-runs `//app:server`. Its `need //lib:core` carries no `opt`
+ override, so the dep resolves as `(//lib:core, c1, a0)`; but `//lib:core`
+ never consumes `opt`, so its shallow trace under `c1` matches by argv-id
+ (`a0`), consumed-keys (empty), and direct leaves, and `L0` is reused without
re-running. Only the one recipe that actually depends on `opt` re-ran.
## Limits and deferred work
diff --git a/driver/cmd/build_coord.c b/driver/cmd/build_coord.c
@@ -37,6 +37,7 @@ typedef struct BuildCli {
size_t nargs;
int verify;
int stats;
+ int trace;
int test_mode;
int has_config_default;
int source_optional;
@@ -80,6 +81,7 @@ void driver_help_build(void) {
" --config env.N Seed env.N from current $N\n"
" --env N[=V] Shorthand for --config env.N[=V]\n"
" --verify Verify the returned output tree in the CAS\n"
+ " --trace Print target-level resolution decisions to stderr\n"
" --stats Print build-resolution counters to stderr\n");
}
@@ -643,6 +645,8 @@ static int build_parse_args(BuildCli* cli, int argc, char** argv, int first_arg,
cli->args[cli->nargs++] = kit_slice_cstr(argv[++i]);
} else if (driver_streq(a, "--stats")) {
cli->stats = 1;
+ } else if (driver_streq(a, "--trace")) {
+ cli->trace = 1;
} else if (driver_streq(a, "--verify")) {
cli->verify = 1;
} else if (a[0] == '-') {
@@ -662,6 +666,11 @@ static int build_parse_args(BuildCli* cli, int argc, char** argv, int first_arg,
return 0;
}
+static void build_trace_stderr(void* user, KitSlice line) {
+ (void)user;
+ driver_errf(BUILD_TOOL, "trace %.*s", KIT_SLICE_ARG(line));
+}
+
static void build_print_stats(const KitBuildStats* s) {
if (!s) return;
driver_errf(BUILD_TOOL,
@@ -767,6 +776,7 @@ int driver_build(int argc, char** argv) {
opts.build_def_path = kit_slice_cstr(cli.def);
opts.jobs = 1;
opts.verify = cli.verify;
+ if (cli.trace) opts.trace = build_trace_stderr;
st = kit_build_coordinator_open(&ctx, &bh.host, kit_slice_cstr(abs_store),
&opts, &coord);
diff --git a/include/kit/build_coord.h b/include/kit/build_coord.h
@@ -246,6 +246,8 @@ typedef struct KitBuildTraceRemote {
int tofu; /* trust-on-first-use */
} KitBuildTraceRemote;
+typedef void (*KitBuildTraceFn)(void* user, KitSlice line);
+
/* ------------------------------------------------------------------ *
* Coordinator lifecycle and requests
* ------------------------------------------------------------------ */
@@ -255,6 +257,8 @@ typedef struct KitBuildOptions {
KitSlice build_def_path; /* build-definition file (tracked as a source) */
int jobs; /* max concurrent recipes; <=0 => host default */
int verify; /* verify mode: re-run cache hits and compare */
+ KitBuildTraceFn trace; /* optional diagnostic resolution trace sink */
+ void* trace_user;
/* Untrusted content mirrors, tried in order (trustless, hash-verified). */
const KitBuildObjectRemote* object_remotes;
size_t n_object_remotes;
diff --git a/src/build/coord.c b/src/build/coord.c
@@ -2,6 +2,7 @@
#include "bundle.h"
+#include <stdarg.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
@@ -341,6 +342,25 @@ void build_coord_stat_bump(KitBuildCoordinator* c, BuildStatField f) {
}
}
+void build_coord_tracef(KitBuildCoordinator* c, const char* fmt, ...) {
+ char line[1024];
+ KitSlice slice;
+ va_list ap;
+ int n;
+ if (!c || !c->opts.trace || !fmt) return;
+ va_start(ap, fmt);
+ n = vsnprintf(line, sizeof line, fmt, ap);
+ va_end(ap);
+ if (n < 0) return;
+ if ((size_t)n >= sizeof line) {
+ line[sizeof line - 1u] = '\0';
+ n = (int)strlen(line);
+ }
+ slice.s = line;
+ slice.len = (size_t)n;
+ c->opts.trace(c->opts.trace_user, slice);
+}
+
KitStatus build_coord_open(const KitContext* ctx, const KitBuildHost* host,
KitSlice store_root, const KitBuildOptions* opts,
KitBuildCoordinator** out) {
diff --git a/src/build/coord.h b/src/build/coord.h
@@ -31,8 +31,9 @@
/* The in-memory deepset node: one node of the transitive input-closure DAG that
* a resolution returns and the targets memo caches (trace.h BuildDeepSet is its
- * on-disk/CAS-blob form). It carries this node's DIRECT source/glob/blob leaves, its
- * target + recipe-id, and pointers to its children's (interned) nodes. `id` is
+ * on-disk/CAS-blob form). It carries this node's DIRECT source/glob/blob leaves,
+ * its target-scope projected config observations, its target + recipe-id, and
+ * pointers to its children's (interned) nodes. `id` is
* the deep-set-id — the CAS blob id of the emitted node — so an unchanged
* subtree is recognized by id equality without descending. Folded into a parent
* by build_leafset_union and refreshed (against the live workspace) by the deep
@@ -42,7 +43,7 @@ typedef struct BuildLeafSet {
uint8_t id[BUILD_HASH_LEN]; /* deep-set-id == CAS blob id of this node */
char target[BUILD_TARGET_MAX];
uint8_t recipe[BUILD_HASH_LEN]; /* recompute via defn to detect a repoint */
- BuildConfigLeaf* configs; /* DIRECT config observations */
+ BuildConfigLeaf* configs; /* target-scope config observations */
size_t n_configs;
BuildSourceLeaf* sources; /* DIRECT leaves of this node */
size_t n_sources;
@@ -132,6 +133,7 @@ typedef enum BuildStatField {
BUILD_STAT_TEST_FAILURE,
} BuildStatField;
void build_coord_stat_bump(KitBuildCoordinator*, BuildStatField);
+void build_coord_tracef(KitBuildCoordinator*, const char* fmt, ...);
/* Open/close. open loads + parses the build definition, opens the CAS and
* store, allocates the memos, and sizes the jobs semaphore (clamped to 1 when
diff --git a/src/build/resolve.c b/src/build/resolve.c
@@ -61,6 +61,85 @@ static int config_observation_match(const BuildConfig* cfg,
return build_id_eq(value_hash, obs->value_hash);
}
+static int config_leaf_eq(const BuildConfigLeaf* a, const BuildConfigLeaf* b) {
+ return a && b && strcmp(a->key, b->key) == 0 && a->present == b->present &&
+ a->has_default == b->has_default &&
+ build_id_eq(a->value_hash, b->value_hash) &&
+ build_id_eq(a->default_hash, b->default_hash);
+}
+
+static int config_map_has_key(const BuildConfig* cfg, const char* key) {
+ KitSlice value;
+ int present = 0;
+ if (!cfg || !key) return 0;
+ if (build_config_get(cfg, kit_slice_cstr(key), &value, &present) != BUILD_OK)
+ return 0;
+ return present;
+}
+
+static int projected_config_append(KitBuildCoordinator* c,
+ BuildConfigLeaf** rows, size_t* n,
+ size_t* cap,
+ const BuildConfigLeaf* row) {
+ size_t i, newcap, old_size, new_size;
+ BuildConfigLeaf* fresh;
+ if (!c || !rows || !n || !cap || !row) return BUILD_ERR;
+ for (i = 0; i < *n; ++i) {
+ if (config_leaf_eq(&(*rows)[i], row)) return BUILD_OK;
+ }
+ if (*n == *cap) {
+ newcap = *cap ? *cap * 2u : 16u;
+ if (newcap < *cap || newcap > ((size_t)-1) / sizeof **rows)
+ return BUILD_ERR;
+ old_size = *cap * sizeof **rows;
+ new_size = newcap * sizeof **rows;
+ fresh = (BuildConfigLeaf*)c->ctx->heap->realloc(
+ c->ctx->heap, *rows, old_size, new_size, _Alignof(BuildConfigLeaf));
+ if (!fresh) return BUILD_ERR;
+ *rows = fresh;
+ *cap = newcap;
+ }
+ (*rows)[(*n)++] = *row;
+ return BUILD_OK;
+}
+
+static int projected_config_collect(KitBuildCoordinator* c,
+ const BuildLeafSet* direct,
+ const BuildDepEdge* deps,
+ const BuildLeafSet* const* children,
+ size_t nchildren,
+ BuildConfigLeaf** out_rows,
+ size_t* out_n, size_t* out_cap) {
+ size_t i, j;
+ if (!c || !direct || !out_rows || !out_n || !out_cap) return BUILD_ERR;
+ *out_rows = NULL;
+ *out_n = 0;
+ *out_cap = 0;
+ for (i = 0; i < direct->n_configs; ++i) {
+ if (projected_config_append(c, out_rows, out_n, out_cap,
+ &direct->configs[i]) != BUILD_OK)
+ return BUILD_ERR;
+ }
+ for (i = 0; i < nchildren; ++i) {
+ BuildConfigEntry overlay_entries[64];
+ BuildConfig overlay_cfg;
+ if (!deps || !children || !children[i]) return BUILD_ERR;
+ build_config_init(&overlay_cfg, overlay_entries,
+ sizeof overlay_entries / sizeof overlay_entries[0]);
+ if (build_coord_config_by_id(c, deps[i].overlay_id, &overlay_cfg) !=
+ BUILD_OK)
+ return BUILD_ERR;
+ for (j = 0; j < children[i]->n_configs; ++j) {
+ if (config_map_has_key(&overlay_cfg, children[i]->configs[j].key))
+ continue;
+ if (projected_config_append(c, out_rows, out_n, out_cap,
+ &children[i]->configs[j]) != BUILD_OK)
+ return BUILD_ERR;
+ }
+ }
+ return BUILD_OK;
+}
+
typedef struct BuildSeenId {
uint8_t id[BUILD_HASH_LEN];
} BuildSeenId;
@@ -549,6 +628,14 @@ int build_resolve(KitBuildCoordinator* c, KitSlice target, const BuildConfig* cf
if (build_config_id(c->ctx->heap, cfg, config_id) != BUILD_OK ||
build_argv_id(c->ctx->heap, argv, argv_id) != BUILD_OK)
return BUILD_ERR;
+ if (c->opts.trace) {
+ char cfg_hex[BUILD_HEX_LEN], argv_hex[BUILD_HEX_LEN];
+ kit_hex_encode(cfg_hex, config_id, BUILD_HASH_LEN);
+ kit_hex_encode(argv_hex, argv_id, BUILD_HASH_LEN);
+ build_coord_tracef(c, "resolve-start action=build target=%.*s root=%d config=%s argv=%s",
+ KIT_SLICE_ARG(target), chain ? 0 : 1, cfg_hex,
+ argv_hex);
+ }
if (build_chain_extend(c, chain, BUILD_ACTION_BUILD, target, config_id,
argv_id, &child, err, sizeof err) != BUILD_OK) {
build_diagf(c->ctx, "build: %s", err);
@@ -581,6 +668,12 @@ scan_record:
if (build_store_get_trace(&c->store, rec.rows[i].trace_id, &fd) !=
BUILD_OK)
continue;
+ if (c->opts.trace) {
+ char trace_hex[BUILD_HEX_LEN];
+ kit_hex_encode(trace_hex, rec.rows[i].trace_id, BUILD_HASH_LEN);
+ build_coord_tracef(c, "deep-candidate target=%.*s trace=%s",
+ KIT_SLICE_ARG(target), trace_hex);
+ }
memset(&dt, 0, sizeof dt);
err[0] = '\0';
if (build_deep_parse(fd.data, fd.size, &dt, NULL, 0) == BUILD_OK &&
@@ -600,10 +693,20 @@ scan_record:
build_store_release(&c->store, &fd);
return BUILD_ERR;
}
+ if (c->opts.trace) {
+ char out_hex[BUILD_HEX_LEN];
+ kit_hex_encode(out_hex, dt.output, BUILD_HASH_LEN);
+ build_coord_tracef(c, "deep-hit target=%.*s output=%s",
+ KIT_SLICE_ARG(target), out_hex);
+ }
build_store_release(&c->store, &fd);
build_coord_stat_bump(c, BUILD_STAT_DEEP_HIT);
return BUILD_OK;
}
+ if (c->opts.trace)
+ build_coord_tracef(c, "deep-miss target=%.*s reason=%s",
+ KIT_SLICE_ARG(target),
+ err[0] ? "malformed" : "no-match");
if (err[0])
KIT_LOGD("skip deep trace for %.*s: %s", KIT_SLICE_ARG(target), err);
err[0] = '\0';
@@ -625,6 +728,12 @@ scan_record:
if (build_store_get_trace(&c->store, rec.rows[i].trace_id, &fd) !=
BUILD_OK)
continue;
+ if (c->opts.trace) {
+ char trace_hex[BUILD_HEX_LEN];
+ kit_hex_encode(trace_hex, rec.rows[i].trace_id, BUILD_HASH_LEN);
+ build_coord_tracef(c, "shallow-candidate target=%.*s trace=%s",
+ KIT_SLICE_ARG(target), trace_hex);
+ }
rows = count_lines(fd.data, fd.size);
memset(&st, 0, sizeof st);
st.configs = configs;
@@ -648,14 +757,24 @@ scan_record:
build_store_release(&c->store, &fd);
return BUILD_ERR;
}
+ if (c->opts.trace) {
+ char out_hex[BUILD_HEX_LEN];
+ kit_hex_encode(out_hex, st.output, BUILD_HASH_LEN);
+ build_coord_tracef(c, "shallow-hit target=%.*s output=%s",
+ KIT_SLICE_ARG(target), out_hex);
+ }
build_store_release(&c->store, &fd);
build_coord_stat_bump(c, BUILD_STAT_SHALLOW_HIT);
return BUILD_OK;
}
+ if (c->opts.trace)
+ build_coord_tracef(c, "shallow-miss target=%.*s reason=no-match",
+ KIT_SLICE_ARG(target));
build_store_release(&c->store, &fd);
}
}
}
+ build_coord_tracef(c, "recipe-run target=%.*s", KIT_SLICE_ARG(target));
return build_run_recipe(c, target, cfg, argv, child, out);
}
@@ -674,6 +793,14 @@ int build_test_resolve(KitBuildCoordinator* c, KitSlice target,
if (build_config_id(c->ctx->heap, cfg, config_id) != BUILD_OK ||
build_argv_id(c->ctx->heap, argv, argv_id) != BUILD_OK)
return BUILD_ERR;
+ if (c->opts.trace) {
+ char cfg_hex[BUILD_HEX_LEN], argv_hex[BUILD_HEX_LEN];
+ kit_hex_encode(cfg_hex, config_id, BUILD_HASH_LEN);
+ kit_hex_encode(argv_hex, argv_id, BUILD_HASH_LEN);
+ build_coord_tracef(c, "resolve-start action=test target=%.*s root=%d config=%s argv=%s",
+ KIT_SLICE_ARG(target), chain ? 0 : 1, cfg_hex,
+ argv_hex);
+ }
if (build_chain_extend(c, chain, BUILD_ACTION_TEST, target, config_id,
argv_id, &child, err, sizeof err) != BUILD_OK) {
build_diagf(c->ctx, "build test: %s", err);
@@ -706,6 +833,12 @@ scan_record:
if (build_store_get_trace(&c->store, rec.rows[i].trace_id, &fd) !=
BUILD_OK)
continue;
+ if (c->opts.trace) {
+ char trace_hex[BUILD_HEX_LEN];
+ kit_hex_encode(trace_hex, rec.rows[i].trace_id, BUILD_HASH_LEN);
+ build_coord_tracef(c, "deep-candidate target=%.*s trace=%s",
+ KIT_SLICE_ARG(target), trace_hex);
+ }
memset(&dt, 0, sizeof dt);
err[0] = '\0';
if (build_test_deep_parse(fd.data, fd.size, &dt, NULL, 0) == BUILD_OK &&
@@ -728,10 +861,20 @@ scan_record:
build_store_release(&c->store, &fd);
return BUILD_ERR;
}
+ if (c->opts.trace) {
+ char out_hex[BUILD_HEX_LEN];
+ kit_hex_encode(out_hex, dt.output, BUILD_HASH_LEN);
+ build_coord_tracef(c, "deep-hit target=%.*s output=%s",
+ KIT_SLICE_ARG(target), out_hex);
+ }
build_store_release(&c->store, &fd);
build_coord_stat_bump(c, BUILD_STAT_TEST_CACHE_HIT);
return BUILD_OK;
}
+ if (c->opts.trace)
+ build_coord_tracef(c, "deep-miss target=%.*s reason=%s",
+ KIT_SLICE_ARG(target),
+ err[0] ? "malformed" : "no-match");
if (err[0])
KIT_LOGD("skip test deep trace for %.*s: %s", KIT_SLICE_ARG(target),
err);
@@ -754,6 +897,12 @@ scan_record:
if (build_store_get_trace(&c->store, rec.rows[i].trace_id, &fd) !=
BUILD_OK)
continue;
+ if (c->opts.trace) {
+ char trace_hex[BUILD_HEX_LEN];
+ kit_hex_encode(trace_hex, rec.rows[i].trace_id, BUILD_HASH_LEN);
+ build_coord_tracef(c, "shallow-candidate target=%.*s trace=%s",
+ KIT_SLICE_ARG(target), trace_hex);
+ }
nrows = count_lines(fd.data, fd.size);
memset(&st, 0, sizeof st);
st.configs = configs;
@@ -779,14 +928,24 @@ scan_record:
build_store_release(&c->store, &fd);
return BUILD_ERR;
}
+ if (c->opts.trace) {
+ char out_hex[BUILD_HEX_LEN];
+ kit_hex_encode(out_hex, st.output, BUILD_HASH_LEN);
+ build_coord_tracef(c, "shallow-hit target=%.*s output=%s",
+ KIT_SLICE_ARG(target), out_hex);
+ }
build_store_release(&c->store, &fd);
build_coord_stat_bump(c, BUILD_STAT_TEST_CACHE_HIT);
return BUILD_OK;
}
+ if (c->opts.trace)
+ build_coord_tracef(c, "shallow-miss target=%.*s reason=no-match",
+ KIT_SLICE_ARG(target));
build_store_release(&c->store, &fd);
}
}
}
+ build_coord_tracef(c, "recipe-run target=%.*s", KIT_SLICE_ARG(target));
return build_run_test_recipe(c, target, cfg, argv, child, out, 1);
}
@@ -821,11 +980,15 @@ int build_dispatch(KitBuildCoordinator* c, KitSlice target,
}
int build_leafset_union(KitBuildCoordinator* c, const BuildLeafSet* direct,
+ const BuildDepEdge* deps,
const BuildLeafSet* const* children, size_t nchildren,
const BuildLeafSet** out) {
BuildDeepSet ds;
BuildLeafSet leaf;
BuildLeafSet check_root;
+ BuildConfigLeaf* projected_configs = NULL;
+ size_t n_projected_configs = 0;
+ size_t cap_projected_configs = 0;
uint8_t(*child_ids)[BUILD_HASH_LEN] = NULL;
const BuildLeafSet** child_ptrs = NULL;
KitWriter* w = NULL;
@@ -836,6 +999,11 @@ int build_leafset_union(KitBuildCoordinator* c, const BuildLeafSet* direct,
int ok = BUILD_ERR;
if (!c || !direct || !out) return BUILD_ERR;
if (nchildren && !children) return BUILD_ERR;
+ if (nchildren && !deps) return BUILD_ERR;
+ if (projected_config_collect(c, direct, deps, children, nchildren,
+ &projected_configs, &n_projected_configs,
+ &cap_projected_configs) != BUILD_OK)
+ goto out;
if (nchildren) {
child_ids = (uint8_t(*)[BUILD_HASH_LEN])c->ctx->heap->alloc(
c->ctx->heap, nchildren * sizeof *child_ids, _Alignof(uint8_t));
@@ -852,8 +1020,8 @@ int build_leafset_union(KitBuildCoordinator* c, const BuildLeafSet* direct,
memset(&check_root, 0, sizeof check_root);
snprintf(check_root.target, sizeof check_root.target, "%s", direct->target);
memcpy(check_root.recipe, direct->recipe, BUILD_HASH_LEN);
- check_root.configs = direct->configs;
- check_root.n_configs = direct->n_configs;
+ check_root.configs = projected_configs;
+ check_root.n_configs = n_projected_configs;
check_root.sources = direct->sources;
check_root.n_sources = direct->n_sources;
check_root.globs = direct->globs;
@@ -870,8 +1038,8 @@ int build_leafset_union(KitBuildCoordinator* c, const BuildLeafSet* direct,
memset(&ds, 0, sizeof ds);
snprintf(ds.target, sizeof ds.target, "%s", direct->target);
memcpy(ds.recipe, direct->recipe, BUILD_HASH_LEN);
- ds.configs = direct->configs;
- ds.n_configs = direct->n_configs;
+ ds.configs = projected_configs;
+ ds.n_configs = n_projected_configs;
ds.sources = direct->sources;
ds.n_sources = direct->n_sources;
ds.globs = direct->globs;
@@ -893,8 +1061,8 @@ int build_leafset_union(KitBuildCoordinator* c, const BuildLeafSet* direct,
build_deepset_id(bytes, len, leaf.id);
snprintf(leaf.target, sizeof leaf.target, "%s", direct->target);
memcpy(leaf.recipe, direct->recipe, BUILD_HASH_LEN);
- leaf.configs = direct->configs;
- leaf.n_configs = direct->n_configs;
+ leaf.configs = projected_configs;
+ leaf.n_configs = n_projected_configs;
leaf.sources = direct->sources;
leaf.n_sources = direct->n_sources;
leaf.globs = direct->globs;
@@ -913,11 +1081,16 @@ out:
if (child_ptrs)
c->ctx->heap->free(c->ctx->heap, child_ptrs,
nchildren * sizeof *child_ptrs);
+ if (projected_configs)
+ c->ctx->heap->free(c->ctx->heap, projected_configs,
+ cap_projected_configs * sizeof *projected_configs);
return ok;
}
-int build_leafset_refresh(KitBuildCoordinator* c, const BuildLeafSet* leafset,
- const BuildConfig* cfg, int* all_match) {
+static int build_leafset_refresh_inner(KitBuildCoordinator* c,
+ const BuildLeafSet* leafset,
+ const BuildConfig* cfg,
+ int check_config, int* all_match) {
size_t i;
KitSlice target;
uint8_t recipe[BUILD_HASH_LEN];
@@ -927,8 +1100,10 @@ int build_leafset_refresh(KitBuildCoordinator* c, const BuildLeafSet* leafset,
if (build_coord_recipe_id(c, target, recipe) != BUILD_OK ||
!build_id_eq(recipe, leafset->recipe))
goto done;
- for (i = 0; i < leafset->n_configs; ++i) {
- if (!config_observation_match(cfg, &leafset->configs[i])) goto done;
+ if (check_config) {
+ for (i = 0; i < leafset->n_configs; ++i) {
+ if (!config_observation_match(cfg, &leafset->configs[i])) goto done;
+ }
}
for (i = 0; i < leafset->n_sources; ++i) {
uint8_t blob[BUILD_HASH_LEN];
@@ -955,8 +1130,8 @@ int build_leafset_refresh(KitBuildCoordinator* c, const BuildLeafSet* leafset,
}
for (i = 0; i < leafset->n_children; ++i) {
int child_match = 0;
- if (build_leafset_refresh(c, leafset->children[i], cfg, &child_match) !=
- BUILD_OK)
+ if (build_leafset_refresh_inner(c, leafset->children[i], cfg, 0,
+ &child_match) != BUILD_OK)
return BUILD_ERR;
if (!child_match) goto done;
}
@@ -964,3 +1139,8 @@ int build_leafset_refresh(KitBuildCoordinator* c, const BuildLeafSet* leafset,
done:
return BUILD_OK;
}
+
+int build_leafset_refresh(KitBuildCoordinator* c, const BuildLeafSet* leafset,
+ const BuildConfig* cfg, int* all_match) {
+ return build_leafset_refresh_inner(c, leafset, cfg, 1, all_match);
+}
diff --git a/src/build/resolve.h b/src/build/resolve.h
@@ -14,9 +14,10 @@
* leafset}, doing the least work necessary. Three phases (see doc/BUILD_COORDINATOR.md
* §Resolution algorithm):
*
- * Phase 1 deep fast path - same config-id AND argv-id, and an all-clear
- * refresh of the deepset closure (every source/glob/blob
- * leaf unchanged AND every node's recipe-id still
+ * Phase 1 deep fast path - same argv-id, matching target-scope projected
+ * config observations, and an all-clear refresh of
+ * the deepset closure (every source/glob/blob leaf
+ * unchanged AND every node's recipe-id still
* recomputes the same) => reuse, no graph walk.
* Phase 2 shallow path - config, argv, and/or sources moved: guard on the
* recipe-id, argv-id, consumed keys + direct leaves,
@@ -29,11 +30,11 @@
* On a local-record miss, resolution first attempts one lazy trace-remote pull
* (build_coord_trace_remote_pull_once) and re-scans before Phase 3. The config-
* dependence is recorded in and verified against the traces; the deep path
- * checks whole-config-id + argv-id equality, the shallow path checks consumed
- * keys + argv-id. Recipe identity rides as a per-node scalar in the deepset
- * (Phase 1) and as the recipe-id guard (Phase 2); there is NO whole-definition
- * dependency. Cross-path dedup is the coordinator's targets memo; per-path cycle
- * detection is the build chain below.
+ * checks target-scope projected consumed keys + argv-id, the shallow path checks
+ * direct consumed keys + argv-id. Recipe identity rides as a per-node scalar in
+ * the deepset (Phase 1) and as the recipe-id guard (Phase 2); there is NO
+ * whole-definition dependency. Cross-path dedup is the coordinator's targets
+ * memo; per-path cycle detection is the build chain below.
*/
/* One frame of the per-path build chain: the (target, config-id, argv-id) at one
@@ -132,20 +133,26 @@ int build_materialize(KitBuildCoordinator*,
const uint8_t tree_id[BUILD_HASH_LEN], char* path_out,
size_t cap);
-/* Build a parent's deepset node from its DIRECT leaves and its children's nodes.
- * `direct` carries this node's target, recipe-id, and direct source/glob/blob leaves
- * (its children fields empty); `children` are the deps' resolved nodes. Computes
+/* Build a parent's deepset node from its DIRECT leaves, awaited dep edges, and
+ * child nodes. `direct` carries this node's target, recipe-id, and direct leaves
+ * (its children fields empty); `deps` and `children` are aligned arrays for the
+ * awaited deps. The node stores this target's projected config closure: direct
+ * config observations plus child config observations whose keys were not
+ * overlaid by that dep edge. Computes
* the deep-set-id, stores the canonical node as a CAS blob (kit_cas_add_blob —
* the children's blobs were stored when they were built, so this is incremental
* and deduped), and returns the interned node. This IS the closure union —
- * "concat direct leaves + child ids, hash" — never a flat N-way merge. */
+ * "concat direct leaves + projected config leaves + child ids, hash" — never a
+ * flat N-way merge. */
int build_leafset_union(KitBuildCoordinator*, const BuildLeafSet* direct,
+ const BuildDepEdge* deps,
const BuildLeafSet* const* children, size_t nchildren,
const BuildLeafSet** out);
/* Refresh a deepset node against the live workspace: per node, rehash its direct
* source leaves (absent stays absent), reglob its glob leaves, check its fetched
- * blobs are still present in the CAS, recompute its
+ * blobs are still present in the CAS, check the root node's projected config
+ * observations against the query config, recompute each node's
* recipe-id through the LIVE definition (catching a dep repointed to a
* different-content recipe, treating same-content as unchanged), and recurse
* into children — all through the per-process memos and the deep-set-id validity
diff --git a/src/build/runner.c b/src/build/runner.c
@@ -878,8 +878,8 @@ int build_runner_record_traces(KitBuildCoordinator* c, KitSlice target,
direct.n_blobs = log->n_blobs;
direct.children = log->child_leafsets;
direct.n_children = log->n_children;
- if (build_leafset_union(c, &direct, log->child_leafsets, log->n_children,
- out_leafset) != BUILD_OK)
+ if (build_leafset_union(c, &direct, log->deps, log->child_leafsets,
+ log->n_children, out_leafset) != BUILD_OK)
return BUILD_ERR;
memset(&shallow, 0, sizeof shallow);
@@ -969,8 +969,8 @@ int build_runner_record_test_traces(KitBuildCoordinator* c, KitSlice target,
direct.n_blobs = log->n_blobs;
direct.children = log->child_leafsets;
direct.n_children = log->n_children;
- if (build_leafset_union(c, &direct, log->child_leafsets, log->n_children,
- out_leafset) != BUILD_OK)
+ if (build_leafset_union(c, &direct, log->deps, log->child_leafsets,
+ log->n_children, out_leafset) != BUILD_OK)
return BUILD_ERR;
memset(&shallow, 0, sizeof shallow);
diff --git a/src/build/runner.h b/src/build/runner.h
@@ -44,7 +44,7 @@ typedef struct BuildPendingNeed {
* the live pending-needs table (submitted, not yet awaited). Converted on success
* into a shallow trace (direct inputs + dep edges) and a deep trace (a single
* pointer to the root deepset node, built by build_leafset_union over these direct
- * leaves + the child nodes). Sections hang off caller/arena storage (no VLAs). */
+ * leaves + dep edges + child nodes). Sections hang off caller/arena storage (no VLAs). */
typedef struct BuildDepLog {
BuildConfigLeaf* configs; /* direct propagated-config observations */
size_t n_configs, cap_configs;
@@ -103,7 +103,7 @@ int build_runner_service(KitBuildCoordinator*, KitBuildConn*, KitSlice target,
/* Convert a completed dep-log + output into the canonical persisted state:
* - build the root deepset node (build_leafset_union over the log's direct
- * leaves + recipe-id + child nodes), which stores it as a CAS blob and yields
+ * leaves + recipe-id + dep edges + child nodes), which stores it as a CAS blob and yields
* its deep-set-id;
* - emit + store the SHALLOW trace (direct inputs + dep edges, by hash) and the
* DEEP trace (scalars + the root deep-set-id), and prepend both to the target
diff --git a/src/build/trace.h b/src/build/trace.h
@@ -122,11 +122,13 @@ typedef struct BuildDeepTrace {
/* One node of the transitive input-closure DAG — the structural form of the old
* flat leafset ("compact deep traces"). One node per target build: its DIRECT
- * source/glob/blob leaves, its recipe-id, and the deep-set-ids of its DIRECT deps.
- * The fully-flattened closure is the recursive union, NEVER materialized: a
+ * source/glob/blob leaves, its recipe-id, its target-scope projected config
+ * observations, and the deep-set-ids of its DIRECT deps. The fully-flattened
+ * source/glob/blob closure is the recursive union, NEVER materialized: a
* subtree reached through many parents is one node, stored once — a Merkle DAG
* that structurally shares like the CAS. Building a parent is "concat my direct
- * leaves + child ids, hash"; that IS the union. deep-set-id = BLAKE2b(canonical
+ * leaves + projected config leaves + child ids, hash"; that IS the union.
+ * deep-set-id = BLAKE2b(canonical
* body), which is exactly a CAS blob id: a deepset is SELF-VERIFYING content,
* stored in cas/blob/ (trustless, fetchable, bundled as a plain blob), unlike a
* trace, which is a CLAIM in build/trace/. `target` is carried so refresh can
@@ -137,7 +139,7 @@ typedef struct BuildDeepTrace {
typedef struct BuildDeepSet {
char target[BUILD_TARGET_MAX];
uint8_t recipe[BUILD_HASH_LEN];
- BuildConfigLeaf* configs; /* this node's DIRECT config observations */
+ BuildConfigLeaf* configs; /* this node's target-scope config observations */
size_t n_configs;
size_t cap_configs;
BuildSourceLeaf* sources; /* this node's DIRECT source leaves */
diff --git a/test/buildcoord/run.sh b/test/buildcoord/run.sh
@@ -592,6 +592,19 @@ test_assert_fail_result() {
fi
}
+not_contains() {
+ name=$1
+ file=$2
+ needle=$3
+ if grep -F "$needle" "$file" >/dev/null 2>&1; then
+ { printf 'unexpected text: %s\n' "$needle"; sed 's/^/file: /' "$file"; } \
+ > "$work/$name.diag"
+ not_ok "$name" "$work/$name.diag"
+ else
+ ok "$name"
+ fi
+}
+
tree_path_from() {
awk 'NF >= 2 {print $2; exit}' "$1"
}
@@ -897,10 +910,25 @@ build_assert_ok buildcoord-config-override-cold --stats --config mode=debug \
override_debug_path=$(tree_path_from "$work/buildcoord-config-override-cold.out")
contains "buildcoord-config-override-debug-output" \
"$override_debug_path/mode.txt" "mode:forced"
-build_assert_ok buildcoord-config-override-change --stats --config mode=release \
- //scope:parent-override
-contains "buildcoord-config-override-change-shallow" \
- "$work/buildcoord-config-override-change.err" "shallow_hits=1"
+build_assert_ok buildcoord-config-override-leaf-direct-deep --trace --stats \
+ --config mode=forced //scope:leaf
+contains "buildcoord-config-override-leaf-direct-deep-hit" \
+ "$work/buildcoord-config-override-leaf-direct-deep.err" "deep_hits=1"
+contains "buildcoord-config-override-leaf-direct-deep-target-hit" \
+ "$work/buildcoord-config-override-leaf-direct-deep.err" \
+ "trace deep-hit target=//scope:leaf"
+contains "buildcoord-config-override-leaf-direct-deep-no-run" \
+ "$work/buildcoord-config-override-leaf-direct-deep.err" "recipes_run=0"
+build_assert_ok buildcoord-config-override-change --trace --stats \
+ --config mode=release //scope:parent-override
+contains "buildcoord-config-override-change-deep" \
+ "$work/buildcoord-config-override-change.err" "deep_hits=1"
+contains "buildcoord-config-override-change-parent-deep" \
+ "$work/buildcoord-config-override-change.err" \
+ "trace deep-hit target=//scope:parent-override"
+not_contains "buildcoord-config-override-change-parent-not-shallow" \
+ "$work/buildcoord-config-override-change.err" \
+ "trace shallow-hit target=//scope:parent-override"
contains "buildcoord-config-override-change-no-run" \
"$work/buildcoord-config-override-change.err" "recipes_run=0"
override_release_path=$(tree_path_from "$work/buildcoord-config-override-change.out")
