kit

kit
git clone https://git.ryansepassi.com/git/kit.git
Log | Files | Refs | README

commit 2f9f6ad15b39f93e6ffef11a1824d580dc780ab2
parent 938e0f309ec6aa31b78fe343c287756f3a26107c
Author: Ryan Sepassi <rsepassi@gmail.com>
Date:   Wed, 17 Jun 2026 13:49:19 -0700

doc: add durable doc/RELEASE.md (release + update mechanism); retire completed doc/plan/SELFDIST.md; fix cross-refs

Diffstat:
Mdoc/DESIGN.md | 1+
Mdoc/DISTRIBUTE.md | 2+-
Adoc/RELEASE.md | 195+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mdoc/plan/README.md | 1-
Ddoc/plan/SELFDIST.md | 555-------------------------------------------------------------------------------
Mdoc/plan/SYSROOTS.md | 2+-
6 files changed, 198 insertions(+), 558 deletions(-)

diff --git a/doc/DESIGN.md b/doc/DESIGN.md @@ -221,6 +221,7 @@ unless an API states otherwise. | [CBACKEND.md](CBACKEND.md) | The portable C-source backend (`src/arch/c_target/`). | | [WASM.md](WASM.md) | The WebAssembly backend, object form, and host-import binding. | | [DISTRIBUTE.md](DISTRIBUTE.md) | Signed `.kpkg` packaging and the content-addressed store (`src/dist/`, `<kit/cas.h>` / `<kit/package.h>`, `cas`/`pkg` tools). | +| [RELEASE.md](RELEASE.md) | Releasing and updating kit itself: `VERSION`/`kit --version`, `make dist` + `scripts/release.sh` signed artifacts, the `kit-release` channel index, and `kit update`. | | [DRIVER.md](DRIVER.md) | The multi-call binary, tool registry, and command-line policy. | | [RUNTIME.md](RUNTIME.md) | The freestanding headers and compiler-rt/libc-style support in `rt/`. | | [BUILD.md](BUILD.md) | The build system and `KIT_*_ENABLED` component gating. | diff --git a/doc/DISTRIBUTE.md b/doc/DISTRIBUTE.md @@ -187,7 +187,7 @@ pkgid does not match. This binds the signature to the exact manifest content, not merely to a name. Trust anchors live in a trusted-keys file. Under the single-root layout -(see [plan/SELFDIST.md](plan/SELFDIST.md)) the path is `$KIT_TRUSTED_KEYS`, +(see [RELEASE.md](RELEASE.md)) the path is `$KIT_TRUSTED_KEYS`, else `$KIT_HOME/config/trusted_keys` (with `$KIT_HOME` defaulting to `$XDG_DATA_HOME/kit` ≡ `~/.local/share/kit`, `%LOCALAPPDATA%\kit` on Windows); the legacy `$HOME/.config/kit/trusted_keys` is still read when the preferred diff --git a/doc/RELEASE.md b/doc/RELEASE.md @@ -0,0 +1,195 @@ +# Releasing and updating kit + +kit ships itself. A kit release is just a **signed kit package**: the same +content-addressed `.kpkg`/`.tar.gz` machinery that packages user code +([DISTRIBUTE.md](DISTRIBUTE.md)) is dogfooded onto the toolchain, and a running +kit verifies and installs a newer kit with `kit update`. There is no second +mechanism and no new trusted-byte surface beyond the existing minisign/Ed25519 +path. This document is the operational reference; the cross-cutting release +punchlist lives in [plan/RELEASE.md](plan/RELEASE.md). + +## Versioning + +The single source of truth is the committed top-level **`VERSION`** file holding +a CalVer string `YYYY.MINOR.PATCH` (`YYYY` = release-series year, `MINOR` = +release within the year, `PATCH` = fix release). Nothing derives the version +from git tags; an annotated tag `v<VERSION>` is created at release time *from* +`VERSION`, not the reverse. + +`kit --version` prints the version plus provenance, and every tool answers it +the same way (so toolchain probes like `cc --version` work): + +``` +$ kit --version +kit 2026.6.0 (a3f91c2, aarch64-macos) +$ cc --version +kit cc 2026.6.0 (a3f91c2, aarch64-macos) +``` + +- `2026.6.0` — contents of `VERSION`. +- `a3f91c2` — short git hash (the build id), `unknown` / `<hash>-dirty` when git + state is unavailable or dirty. +- `aarch64-macos` — the host triple this binary targets (compile-time). + +Mechanism: `mk/flags.mk` reads `VERSION` + the git short hash once per `make` +invocation and injects them as `-D` defines into the single TU `driver/version.c` +(`kit_version_string` / `kit_build_id` / `kit_host_triple`), gated on a dedicated +`.version-config` sentinel so a version or commit change recompiles only that one +object. There is **no** `__DATE__`/`__TIME__` anywhere, so a given tree always +embeds the same bytes and the self-host byte-identity gate is preserved. + +## Building release artifacts + +Two layers, so the everyday target needs no secret key. + +### `make dist` — native host + +Builds `RELEASE=1`, stages a self-contained `kit/` tree, and emits — for the +native host triple — a signed fat `.kpkg`, a signed portable `.tar.gz`, and a +detached `.tar.gz.minisig`: + +``` +build/dist/ + kit-2026.6.0-aarch64-macos.kpkg native fat, signed + kit-2026.6.0-aarch64-macos.tar.gz portable, signed (manifest sig embedded) + kit-2026.6.0-aarch64-macos.tar.gz.minisig detached sig over the tarball (bootstrap) +``` + +The staging tree is `bin/kit`, `lib/libkit.a`, `include/`, `support/rt/{include,lib}/`, +`VERSION`, and README/license. Signing uses `KIT_SIGN_KEY=<path>` if set, +otherwise the in-tree **NON-RELEASE** test key (`test/dist/keys/nonrelease.key`) +with a loud warning — so the full `make dist` → verify → `kit update` pipeline +runs hermetically in CI. The `.kpkg` is the canonical update unit; the `.tar.gz` +carries its signed manifest internally, and the extra detached `.minisig` lets a +user with only stock `minisign` verify before unpacking. + +### `scripts/release.sh` — the multi-target matrix + +For each **hosted** (runnable) triple in `scripts/hosted.sh`'s support set, +release.sh cross-builds the kit binary + per-target rt archives (via +`kit_cross.sh` and, for foreign libc, the sysroots of [plan/SYSROOTS.md](plan/SYSROOTS.md)), +stages and packages each exactly as `make dist` does, signs everything with the +**real** release key (`KIT_SIGN_KEY`, held in CI/secret, never in-tree), and +emits + signs the channel index. It asserts a clean git tree and refuses the +in-tree test key. Freestanding targets have no host to run on and produce no +artifact. Knobs: `KIT_SIGN_KEY` (required), `KIT_RELEASE_URL_BASE` (mirror base +URLs), `KIT_RELEASE_TARGETS`, `KIT_VM=0` (drop FreeBSD/Windows on a VM-less +runner), `KIT_CHANNEL`. + +Detached signatures over arbitrary files (the `.tar.gz.minisig` and the index +signature) are produced by `kit pkg sign -s KEY [-o OUT] FILE`, which writes a +stock-`minisign`-compatible signature. + +## Channel index and hosting + +Each channel (`stable` for v1) is described by a small signed, byte-stable text +file, `kit-release 1`, emitted/parsed by libkit (`src/dist/release.c`, +`kit_release_index_*`): + +``` +kit-release 1 +channel = stable +version = 2026.6.0 +hash = blake2b-256 + +[host] +target = aarch64-macos +kpkg = <64-hex package-id of the .kpkg> +targz = <64-hex package-id of the .tar.gz> +size = 9311461 +url = https://github.com/<org>/kit/releases/download/v2026.6.0/kit-2026.6.0-aarch64-macos.kpkg +url = https://mirror.example/kit/2026.6.0/kit-2026.6.0-aarch64-macos.kpkg +``` + +Hosting is not a fixed location: each artifact is identified by its **content +id** (package-id), and the repeated `url` lines are an ordered, untrusted +**mirror list**. The fetcher tries them in order and accepts bytes only after +they match the signed content id and the package signature, so any host — +GitHub Releases, a CDN, an S3/R2 bucket, a private mirror — is interchangeable. +The index may be served over plain HTTP. + +## Trust, keys, and rotation + +Release artifacts are signed with a dedicated minisign (Ed25519) keypair whose +secret is held in CI/secret and never enters the tree. The binary embeds an +**array of trusted release public keys** (`driver/release_key.c`); a signature +from **any** embedded key verifies an update, which gives key rotation for free: +ship release N+1 signed by a key already embedded in N, and introduce the next +key in a later overlap window. Verifying an update requires the signature to +validate against an embedded (or `--key`-supplied) key **and** the fetched +package-id to match the signed channel-index entry **and** the version to pass +policy (below). + +The published release public key + the detached `.tar.gz.minisig` give the +**bootstrap** path when no kit exists yet: + +``` +$ minisign -Vm kit-2026.6.0-aarch64-macos.tar.gz -P <published kit release pubkey> +$ tar xzf kit-2026.6.0-aarch64-macos.tar.gz && kit/bin/kit --version +``` + +From then on the installed kit's embedded anchor verifies all future updates. +(The in-tree default anchor is the NON-RELEASE test key; the real release pubkey +replaces/augments it at release time.) + +## Installing and updating + +`kit update` manages a single self-contained root, `$KIT_HOME` (default +`$XDG_DATA_HOME/kit` ≡ `~/.local/share/kit`; `%LOCALAPPDATA%\kit` on Windows): + +``` +$KIT_HOME/ + versions/2026.6.0/{bin,lib,include,support,VERSION} + current -> versions/2026.6.0 atomic active-toolchain pointer + bin/ PATH links -> ../current/bin/kit + config/trusted_keys pkg trust anchors + cache/downloads/ fetched kpkgs / scratch +``` + +Put `$KIT_HOME/bin` on `PATH` once. An update verifies the package, unpacks it +into a temp dir, renames it into `versions/<new>`, flips `current` atomically, +and refreshes the `bin/` links — the live binary is reached through `current`, +so the swap never mutates the running image. All versions are retained, so any +prior one is an instant **offline** flip. + +``` +kit update # latest on the tracked channel (network) +kit update <file.kpkg> # offline: verify + install a local package +kit update --version 2026.6.0 # offline flip if installed, else fetch +kit update --check # report available vs installed; change nothing +kit update --rollback # flip to the previous installed version +kit update --list # list installed versions, mark current +kit update --prune # remove all non-current installed versions +kit update --key <pubkey> # verify against an explicit key (mirrors/forks) +kit update --allow-downgrade # permit a channel-driven downgrade +kit update --dry-run # print the plan; change nothing +``` + +Networked updates resolve the channel index from `--index <url>` or +`$KIT_UPDATE_INDEX_URL` and download through a fetch hook in the driver +(`driver_fetch_url`, which shells to `curl`/`wget`); libkit itself stays +network-free, and since packages are signed the transport is untrusted. + +**Version policy**: channel-driven updates are **monotonic** by default — `kit +update` refuses a version older than the running one. Deliberate moves bypass +the guard: `--version <X>`, `--rollback`, local-file installs, and +`--allow-downgrade` (all still fully verified). CalVer compare is numeric per +field. + +Self-update only ever writes under the user-writable `$KIT_HOME`; it never +touches a system/root-owned install (e.g. under `/usr/local`) — there, +re-acquire or use the OS package manager. + +## macOS Gatekeeper + +A downloaded Mach-O is quarantined; v1 ships without Apple notarization, so +remove the quarantine flag once: + +``` +$ xattr -dr com.apple.quarantine kit/bin/kit +$ kit/bin/kit --version +``` + +kit ad-hoc-signs its own Mach-O output, so the binary is valid code; this is +only the download-quarantine prompt. Developer ID signing + notarization is a +later milestone. diff --git a/doc/plan/README.md b/doc/plan/README.md @@ -18,7 +18,6 @@ shrinks to whatever remains open (and is deleted once nothing remains open). | [ARM32.md](ARM32.md) | 32-bit ARM (`arm-none-eabi`, ARMv7-M/ARMv7E-M Thumb-2, Cortex-M3/M4/M7) freestanding backend: Phase 1 (walking skeleton) is landed; Phase 2 tracks the remaining ops, the -O1 known-frame path, 64-bit, atomics, TLS, and the `qemu-system-arm` cross-test lane. | [../ARCH.md](../ARCH.md), [../PORT.md](../PORT.md) | | [MCU.md](MCU.md) | Microcontroller development workflow for STM32 Cortex-M (ARMv7-M/E-M) and Espressif RISC-V (ESP32-C/H/P): the last-mile gaps over the working backends — `objcopy` ihex/srec, reusable startup/vector-table + starter linker script, the rv32 soft-float-default footgun, SDK interop, and on-target debug. Assessment stage; references ARM32.md for ARM-core follow-ons. ARMv6-M/Cortex-M0 and Xtensa are out of scope. | [ARM32.md](ARM32.md), [../RUNTIME.md](../RUNTIME.md), [LINKER.md](LINKER.md) | | [SYSROOTS.md](SYSROOTS.md) | Cross-compile sysroot packaging: minimal per-target stubs/headers/CRT objects distributed via `.kpkg` for the support set. Design complete, implementation not yet started. | — | -| [SELFDIST.md](SELFDIST.md) | Self-distribution: versioning kit (CalVer in `VERSION`, `kit --version`), `make dist` → signed per-host `.kpkg` + `.tar.gz` via kit's own pkg/cas, a signed channel index, and `kit update` (offline-first, pluggable curl/wget fetch, atomic versioned install root). Design, not yet built. | [../DISTRIBUTE.md](../DISTRIBUTE.md) | | [BUILD.md](BUILD.md) | A new content-addressed build coordinator (Bazel/Nix-style incremental builds layered on the CAS) — storage state machine, caching algorithm, recipe protocol. Design, not yet built. Distinct from `../BUILD.md` (kit's own Makefile build). | — (new subsystem) | | [TODO.md](TODO.md) | Open deferred fixes and code smells, plus terse backlog folded from retired plan docs (arch-backend parity, Wasm object backend, Windows x64 self-host, bootstrap breadth). Completed items are removed instead of checked off. A current backlog, not a roadmap. | — | diff --git a/doc/plan/SELFDIST.md b/doc/plan/SELFDIST.md @@ -1,555 +0,0 @@ -# Self-distribution (kit ships kit) - -This is the design and implementation plan for distributing the **kit toolchain -itself** — versioning it, building signed release artifacts for every supported -host, and updating an installed kit in place. It is the detailed design behind -the "release packaging: version, …, `kit install` default tool set" line in -[RELEASE.md](RELEASE.md). All design decisions below are settled; the remaining -work is implementation (see *Work items*). - -It deliberately reuses the signed content-addressed packaging subsystem already -in the tree (see [../DISTRIBUTE.md](../DISTRIBUTE.md)) rather than inventing a -second mechanism: a kit release is *just a signed kit package* whose payload -happens to be the toolchain. The asymmetry today is that `kit pkg` / `kit cas` -package **user** code, while kit's own `make dist` is an unsigned, unversioned, -native-host-only `tar czf` (`mk/dist.mk`). This plan closes that gap by -dogfooding the package pipeline onto kit's own release. - -## Goals - -- A single canonical **version** for kit, embedded in the binary - (`kit --version`) and carried as the package version of every release - artifact, with one committed source of truth. -- `make dist` (and a multi-target release driver) produce, per host triple, a - **signed** native `.kpkg` *and* a **signed** portable `.tar.gz`, both built - with kit's own pkg/cas tooling. -- A running kit can **verify and install** a newer kit with one command, against - a trust anchor it already carries, with no new trusted-byte surface beyond the - existing minisign/Ed25519 path. -- First acquisition (no kit yet) is verifiable with stock `minisign` and a - published public key — no chicken-and-egg. -- Reproducibility and the self-host byte-identity gate are preserved: nothing in - a release artifact depends on wall-clock time or build host. - -## Non-goals / deferred to later milestones - -- A built-in TLS/HTTP stack in libkit. Transport is untrusted; integrity comes - from the signature, so the first fetch implementation delegates to - `curl`/`wget`. A native minimal HTTP/1.1 fetcher can replace the hook later - (still no TLS needed). -- A dependency solver or multi-package release graph. A kit release is one - package; sysroots ([SYSROOTS.md](SYSROOTS.md)) are separate packages. -- Background/automatic updates or a daemon. `kit update` is explicit. -- Delta/binary-diff and thin/external-CAS packages. v1 ships **fat** per-host - packages; the thin shape + per-blob delta updates are a later evolution the - format already accommodates. -- `beta` / `nightly` channels. v1 is `stable` only; the index format reserves a - `channel` field so adding them is non-breaking. -- Apple Developer ID notarization. v1 documents quarantine removal (see - *macOS Gatekeeper*); notarization is a later milestone. - -## Relationship to existing subsystems - -| Concern | Where it lives | This plan | -|---|---|---| -| Content store, blobs, trees, signing, `.kpkg`/`.tar.gz` | [../DISTRIBUTE.md](../DISTRIBUTE.md), `src/dist/`, `include/kit/{cas,package}.h` | reused unchanged as the mechanism | -| Per-host build / cross-compile matrix | `scripts/hosted.sh`, [../PORT.md](../PORT.md) | drives the multi-target release build | -| Cross-compile sysroots (needed to build kit for a foreign host) | [SYSROOTS.md](SYSROOTS.md) | a build-time dependency of the release pipeline; its `~/.kit/sysroots` path becomes `$KIT_HOME/sysroots` under the single-root layout below | -| Release punchlist | [RELEASE.md](RELEASE.md) | this is the design for its "version/packaging" item | -| Install layout / tool symlinks | `driver/cmd/install.c` | extended with a versioned single-root install (`$KIT_HOME/bin` links) | - -libkit stays the **mechanism** (verify, unpack, materialize, hash, sign, plus -the new index format + version compare); the driver owns the **policy** (version -semantics, channel selection, trust anchor, fetch, atomic swap). This matches -the existing split where "argument parsing and trusted-keys path/pin policy stay -in the driver" (DISTRIBUTE.md). - -## Versioning - -### Source of truth: CalVer in `VERSION` - -A committed top-level `VERSION` file holds a CalVer string `YYYY.MINOR.PATCH`: - -``` -2026.6.0 -``` - -- `YYYY` — calendar year of the release series. -- `MINOR` — release within the year, incremented per shipped release. -- `PATCH` — fix release on a given `YYYY.MINOR`. - -CalVer signals cadence over API-compat promises, which fits a pre-1.0 -self-hosting toolchain better than semver. The file is the single source of -truth; nothing computes the version from git tags. (A matching annotated git tag -`v2026.6.0` is still created at release time for provenance, but it is *derived -from* `VERSION`, not the other way around.) - -### Build id and `kit --version` - -`kit --version` prints the version plus provenance: - -``` -$ kit --version -kit 2026.6.0 (a3f91c2, aarch64-macos) -``` - -- `2026.6.0` — contents of `VERSION`. -- `a3f91c2` — short git commit hash (the **build id**), injected at build time - (`unknown` / `<hash>-dirty` when git state is unavailable or dirty). -- `aarch64-macos` — the host triple this binary targets, known at compile time. - -Mechanism: the Makefile reads `VERSION` and the git short hash once and passes -them as `-DKIT_VERSION_STR=…` / `-DKIT_BUILD_ID=…` to a single small translation -unit `driver/version.c` exposing `kit_version_string()`, `kit_build_id()`, and -`kit_host_triple()`. Only that TU recompiles when `VERSION` or the commit -changes, so version churn does not rebuild the world. - -### `--version` everywhere - -`--version` / `version` routing slots into `kit_driver_main_ex` -(`driver/main.c`) alongside the existing `--help` / `help <tool>` handling. Every -**tool entry point** answers `--version` with the same banner, prefixed by the -tool name, so toolchain probes like `cc --version` work uniformly: - -``` -$ cc --version -kit cc 2026.6.0 (a3f91c2, aarch64-macos) -$ ld --version -kit ld 2026.6.0 (a3f91c2, aarch64-macos) -``` - -Compiler identity for configure scripts comes from predefined macros, not this -string, so a clear kit banner is correct and not a compatibility risk. - -### Reproducibility contract - -The self-host byte-identity gate compiles kit with kit from one tree and -compares stage2 vs stage3. To keep it intact: - -- The version comes from a **committed** constant, so both stages embed the same - bytes. -- The build id is injected **once** per `make` invocation (from `git rev-parse`) - and passed identically to every stage, so it does not differ stage-to-stage. -- **No** `__DATE__` / `__TIME__` anywhere in the binary. If a build date is ever - embedded it must honor `SOURCE_DATE_EPOCH`. -- Release builds assert a clean git tree. - -### Where the version flows - -`VERSION` → (1) the binary's `--version`; (2) the -channel index entries; (3) the git tag at release time. - -## Release artifacts (`make dist` → signed packages) - -### One package model, per-host fat artifacts - -A kit release is a package named `kit`, version `= cat VERSION`, produced **once -per host triple** as a self-contained **fat** package (everything embedded — no -external CAS). The package manifest's per-`[output]` `target` triple records the -host; the native host's output is flagged `default`. (The `kit-package 3` -manifest already models per-output target triples, a default flag, and -`exe`/`lib`/`data` artifact overlays — DISTRIBUTE.md — so no format change is -needed.) - -Per-host fat packages keep the model simple: each download is self-contained, -`kit update <file.kpkg>` works fully offline, and the download is only the -user's own host. Common headers/rt-source are duplicated across the ~dozen -server-side artifacts, but that storage is cheap; the thin/external-CAS shape -(cross-host dedup, delta updates) is a later evolution. - -Staging tree (what gets hashed into the package), aligned with today's -`mk/dist.mk` and `kit install`'s expectations: - -``` -kit/ - bin/kit the multi-call binary (exe) - lib/libkit.a static library (lib) - include/ public headers (data) - support/rt/include/ freestanding + kit rt headers (data) - support/rt/lib/ kit rt sources (data) - VERSION the release version (data) - README.txt, LICENSE.txt, NOTICE.txt (data) -``` - -### Both formats, both signed - -For each host triple the pipeline emits, all signed by the release key: - -``` -build/dist/ - kit-2026.6.0-aarch64-macos.kpkg native fat, signed (manifest+descriptor) - kit-2026.6.0-aarch64-macos.tar.gz portable, signed (embedded manifest.minisig) - kit-2026.6.0-aarch64-macos.tar.gz.minisig detached sig over the tarball bytes (bootstrap) - …one set per triple… - stable.index signed channel index (see below) - stable.index.minisig -``` - -- The `.kpkg` is the canonical update unit. -- The `.tar.gz` already carries the signed manifest internally - (`kit/package.manifest.minisig`), so it is signed by construction. The - **additional detached `.tar.gz.minisig`** over the whole archive lets a user - with only stock `minisign` (and no kit) verify before unpacking — the bootstrap - trust path. -- Both representations verify the same logical package/tree/blob content - (DISTRIBUTE.md determinism invariants), so a user may acquire either and end up - bit-identical. - -### `make dist` vs the release pipeline - -Two layers, so the everyday target needs no secret key: - -- **`make dist`** (rework of `mk/dist.mk`): native host only. Builds - `RELEASE=1`, stages the tree, and runs the in-tree pkg pipeline to emit the - signed `.kpkg` + `.tar.gz` + detached `.tar.gz.minisig` for the host triple. - Signing key is `KIT_SIGN_KEY=<path>` if set; **otherwise it signs with a - clearly-labeled, in-tree NON-RELEASE test keypair** (the same key the - end-to-end tests use) and prints a loud warning. Artifacts are therefore always - verifiable, and the full `make dist → verify → update` pipeline runs - hermetically in CI. -- **`scripts/release.sh`** (new): the full matrix. For each hosted triple in - `scripts/hosted.sh`'s support set it cross-builds the kit binary and the - per-target rt archives (leaning on the existing `kit-cross` / `windows_cross` - / bootstrap machinery and, for foreign libc, the sysroots of - [SYSROOTS.md](SYSROOTS.md)), stages, packages, and signs each with the **real** - release key (held in CI/secret, never in-tree), then emits and signs the - channel index. Freestanding targets have no host to run on and produce no kit - release artifact; the release set is the **hosted** triples only. - -Artifacts are named by the `scripts/hosted.sh` selector triples (e.g. -`aarch64-macos`, `x86_64-linux-gnu`) for one consistent naming scheme across the -build matrix, the index, and the file names. - -## Hosting and the mirror model - -Hosting is **not** a single fixed location. The signed channel index identifies -each artifact by its **content id** (package-id) and carries an *ordered list of -mirror URLs* for it; the fetcher tries them in order and accepts bytes only after -they match the signed content id and signature. Because identity is the hash, it -does not matter who serves the bytes — GitHub Releases, a CDN, an S3/R2 bucket, -or a private mirror are all interchangeable, and any subset can be listed. - -For v1 fat packages the "blob" is the whole `.kpkg` file. The same mechanism -generalizes to per-blob/per-chunk mirror lists when thin packages arrive later — -same principle, finer grain. GitHub Releases is a fine default first mirror -(durable URLs, free, signatures make its trust irrelevant), but the design pins -no host. - -## Manifest, signing, and trust anchor - -### The release key set - -Release artifacts are signed with a dedicated minisign (Ed25519) keypair whose -secret is held in CI/secret and never enters the tree. The binary embeds an -**array of trusted release public keys** (`driver/release_key.c`, -`static const KitReleaseKey kit_release_keys[]`); a signature from **any** -embedded key verifies an update. The public key(s) are also **published** out of -band for the stock-minisign bootstrap path. - -Verifying an update requires (a) the signature validates against an embedded (or -`--key`-supplied) key, **and** (b) the fetched package-id matches the channel -index entry, **and** (c) the version is acceptable per policy (below). The -existing `pkgid=` trusted-comment binding (DISTRIBUTE.md) ties the signature to -exact manifest bytes. - -### Key rotation: overlap windows - -Rotation needs no extra protocol — just overlapping embedded sets: - -``` -release N embeds {A, B}, signed by A → users now trust both A and B -release N+1 signed by B → trusted (users from N hold B) -release N+2 embeds {B, C}, signed by B → introduces C the same way -``` - -A user always validates a new artifact against a key their already-trusted kit -holds. Hard rotation after a key compromise falls back to the published -bootstrap key (re-acquire + verify with stock minisign). A data-driven -(TUF-style) key set in the index was considered and rejected as over-built for -v1. - -### Bootstrap trust (first install) - -No kit yet, so kit cannot verify itself. The published release key + the detached -`.tar.gz.minisig` let the user verify with stock minisign: - -``` -$ minisign -Vm kit-2026.6.0-aarch64-macos.tar.gz -P <published kit release pubkey> -$ tar xzf kit-2026.6.0-aarch64-macos.tar.gz && kit/bin/kit --version -``` - -From then on the installed kit's embedded anchor verifies all future updates. - -## Channel index format - -A small, signed, byte-stable text file per channel (`stable` for v1), in the -same canonical-text + detached-minisig idiom as the package manifest, and emitted -/ parsed by libkit (`src/dist/release.c`). Proposed `kit-release 1`: - -``` -kit-release 1 -channel = stable -version = 2026.6.0 -hash = blake2b-256 - -[host] -target = aarch64-macos -kpkg = <64-hex package-id of the .kpkg> -targz = <64-hex package-id of the .tar.gz> -size = 31457280 -url = https://github.com/<org>/kit/releases/download/v2026.6.0/kit-2026.6.0-aarch64-macos.kpkg -url = https://dl.kit.dev/2026.6.0/kit-2026.6.0-aarch64-macos.kpkg -url = https://mirror.example/kit/2026.6.0/kit-2026.6.0-aarch64-macos.kpkg - -[host] -target = x86_64-linux-gnu -… -``` - -- Signed by the release key; the signature's trusted comment binds the index id, - exactly like packages. -- Repeated `url` lines are an ordered **mirror list** for that host's `.kpkg`; - all are untrusted fetch hints (integrity = signed `kpkg` id + package - signature), so the index may be served over plain HTTP and mirrored freely. -- `kit update` resolves `(channel, host-triple)` → entry, fetches the kpkg from - the first reachable mirror, checks the fetched package-id equals the index's, - then runs full `kit_pkg_verify` against the embedded key. - -## Installation layout (single root) - -Everything kit manages for a user lives under **one self-contained root**, with -each concern in a subdir — kit never scatters files across separate -config/cache/data trees. The root defaults to `$XDG_DATA_HOME/kit` (i.e. -`~/.local/share/kit` when `XDG_DATA_HOME` is unset), so it is a single tree under -the conventional data path rather than a bare dotdir in `$HOME`; `$KIT_HOME` -overrides it outright. On Windows the default is `%LOCALAPPDATA%\kit`. - -``` -$KIT_HOME/ (default $XDG_DATA_HOME/kit ≡ ~/.local/share/kit) - versions/ - 2026.5.3/{bin,lib,include,support,VERSION,…} - 2026.6.0/{bin,lib,include,support,VERSION,…} - current -> versions/2026.6.0 atomic active-toolchain pointer - bin/ PATH links -> ../current/bin/kit - config/ - config tracked channel, retention, prefs - trusted_keys pkg trust anchors (kit pkg trust) - sysroots/ cross-compile sysroots (SYSROOTS.md) - cache/ - downloads/ fetched kpkgs / scratch -``` - -Consolidating `config/` and `cache/` *inside* the root is a deliberate deviation -from XDG's three-root split, in favor of one tree that is trivial to locate, back -up, or delete. The user puts `$KIT_HOME/bin` on `PATH` once (cargo/rustup style); -`kit install <dir>` still lays links into any other directory on demand. - -Two reconciliations of already-shipped/sketched design follow from the single -root: - -- **trusted_keys** moves from `~/.config/kit/trusted_keys` (shipped — see - DISTRIBUTE.md / `src/dist/trust.c`) to `$KIT_HOME/config/trusted_keys`. The - `$KIT_TRUSTED_KEYS` override is still honored, and the legacy path is read when - present so existing setups keep working. -- **sysroots** move from `~/.kit/sysroots` (SYSROOTS.md) to `$KIT_HOME/sysroots`. - -Self-update operates only on this user-writable root. A system-wide / root-owned -install (e.g. under `/usr/local`) is out of scope for self-update v1; there, -`kit update` reports that the install is not user-managed and points to -re-acquisition or the OS package manager. - -## Updating (`kit update`) - -### Flow and atomic swap - -1. Resolve target version (channel index, explicit `--version`, or local file) - and confirm it matches the running host triple. -2. If the target version is already installed under `versions/`, skip to the - flip (offline). Otherwise fetch the `.kpkg` (see fetch model). -3. `kit_pkg_verify` against the embedded (or `--key`) release key; confirm id - matches the index; confirm version policy. -4. `kit_pkg_unpack` into a temp dir under the data root, then rename into - `versions/<new>` (rename within a filesystem is atomic). -5. **Flip `current`** to the new version dir atomically (symlink swap on POSIX). -6. Refresh `$KIT_HOME/bin` links via the existing `kit install` logic so new - tools appear and removed ones go away. - -Because the live binary is reached through `current`, the swap does not mutate -the running image. **All** installed versions are retained by default -(`kit update --prune` removes the non-current ones), so any prior version is an -instant **offline** flip via `kit update --version <X>` or `--rollback`. - -Platform notes: -- **POSIX**: symlink flip is atomic; the running inode stays mapped during a swap. -- **Windows**: no default symlinks and a running `.exe` is locked. `current` is a - junction (or copy); install links are hard links (already the `kit install` - Windows default); a tool already running keeps its handle to the old file. - -### Fetch model: offline-first, pluggable, curl/wget first - -- **Offline-first**: `kit update <file.kpkg>` (or `--from <file>`) verifies and - installs a locally present package with **no network code**. This is the whole - update mechanism minus the download, and ships first. -- **Pluggable fetch hook**: networked `kit update` resolves the index and - downloads through a host fetch vtable in the driver env layer - (`driver/env/`), keeping libkit network-free. The **first implementation shells - out to `curl`/`wget`** (the driver already spawns external processes for - linkers), so kit ships no HTTP/TLS of its own. Because packages are signed, a - native fetcher (later) never needs TLS. - -### Version policy - -Default is **monotonic** for channel-driven updates: `kit update` refuses to move -to a version older than the running one. Intentional moves bypass the guard: -`kit update --version <X>` (flip to an installed version), `--rollback` (flip to -the immediately-previous installed version), and local-file installs are -deliberate and allowed (still fully verified); `--allow-downgrade` permits a -channel-driven downgrade. `--rollback` reverts the **active version only** — -channel tracking is untouched, and the command warns that the next -`kit update` will re-upgrade unless the user pins with `--version`. CalVer -compare is numeric per field (`YYYY`, then `MINOR`, then `PATCH`). - -### Command surface - -``` -kit update # latest on the tracked channel (default: stable) -kit update --version 2026.6.0 # specific version: offline flip if installed, else fetch -kit update <file.kpkg> # offline: verify + install a local package -kit update --from <file.kpkg> # explicit local-file form -kit update --check # report available vs installed; change nothing -kit update --rollback # flip to the previous installed version (warns re-upgrade) -kit update --list # list installed versions, mark current -kit update --prune # remove all non-current installed versions -kit update --key <pubkey> # verify against an explicit key (mirrors/forks) -kit update --allow-downgrade # permit a channel-driven downgrade -kit update --dry-run # print the plan; change nothing -``` - -## macOS Gatekeeper - -A downloaded Mach-O is quarantined and Gatekeeper blocks it on first run unless -notarized. v1 ships **without Apple notarization** and documents the one-time -removal: - -``` -$ xattr -dr com.apple.quarantine kit/bin/kit -$ kit/bin/kit --version -``` - -kit already ad-hoc-signs its own Mach-O output (required to run on Apple -Silicon), so the binary itself is valid code; this is purely the -download-quarantine prompt. Developer ID signing + notarization is a later -milestone once a signing identity exists. - -## Security / threat model - -- **Trust root**: the embedded release public-key set. Compromise of a release - secret key is the catastrophic case; mitigated by overlap-window rotation and - the out-of-band published key for hard recovery. -- **Untrusted transport / mirrors**: any mirror may tamper freely; a forged or - truncated download fails signature or package-id verification and is rejected - before any swap. -- **No partial install**: verification completes before the atomic `current` - flip; a crash mid-update leaves the old `current` intact. -- **Downgrade attack**: defended by the monotonic default on channel updates; an - attacker cannot push an older signed-but-vulnerable kit without an explicit - `--allow-downgrade`/`--version`. -- **Index substitution**: the index is signed and binds package-ids, so a swapped - index cannot redirect to an unsigned or mismatched package. - -## Code boundaries - -- **libkit (mechanism)** — the existing CAS/package/sign/verify/unpack path - (unchanged) plus a new `src/dist/release.c`: `kit-release` index emit/parse and - CalVer compare, byte-stable and hermetically tested in `test/dist` alongside - manifest/tree/kpkg. Surfaced via the dist subsystem (gating consistent with - `KIT_PKG_ENABLED`). -- **driver (policy)** — `driver/version.c` (version/build-id/triple), - `driver/release_key.c` (embedded key set), `driver/cmd/update.c` (channel - selection, install-root management, atomic swap, rollback/prune/list), and the - `driver/env/` fetch hook. Argument parsing and trust/path policy stay here, per - the existing split. - -## Work items - -**Status: implemented.** Everything below is in the tree and gated green -(`make test-release-index` + `make test-selfdist` + `make test-dist`; `make -dist` produces a verifiable signed kpkg/tar.gz/minisig). Two items are -deliberately deferred to release time, not code: (1) generating the **real** -release keypair offline and embedding/publishing its pubkey — the in-tree -default anchor is the clearly-labelled NON-RELEASE test key, replaced at release -time in `driver/release_key.c`; (2) running `scripts/release.sh` itself, which -needs the real key + provisioned cross sysroots/VMs. - -### Versioning -- [x] Add top-level `VERSION` (CalVer `YYYY.MINOR.PATCH`). -- [x] Makefile: read `VERSION` + git short hash; inject `-DKIT_VERSION_STR` / - `-DKIT_BUILD_ID` into a new `driver/version.c` - (`kit_version_string`/`kit_build_id`/`kit_host_triple`). Injected via a - `version.o`-only target rule keyed on a dedicated `.version-config` - sentinel, so version/commit churn recompiles only that one TU. -- [x] Wire `--version` / `version` into `kit_driver_main_ex` and every tool entry - point (shared `kit <tool> X.Y.Z (build, triple)` banner). Done by - intercepting `--version` in `dispatch()` (covers symlink + subcommand - forms); `update` is exempt because its `--version <ver>` is a value flag. -- [x] Reproducibility: no `__DATE__`/`__TIME__`; build id injected once; - clean-tree assertion in release builds (`scripts/release.sh` refuses a - dirty tree). The build id appends `-dirty`/`unknown` only as a marker; the - version/triple come from committed/compile-time data, so the self-host - byte-identity gate is unaffected. - -### Release artifacts -- [x] Rework `mk/dist.mk`: stage the tree (binary at `bin/kit`, libkit.a, - headers, rt headers + sources, `VERSION`, README/license), then emit signed - fat `.kpkg` + `.tar.gz` + detached `.tar.gz.minisig` for the native triple. - Key from `KIT_SIGN_KEY`, else the in-tree NON-RELEASE test key + loud - warning. -- [x] Add the in-tree test keypair (clearly labeled) under `test/dist/keys/`; - used for the hermetic test and as the `make dist` fallback; its pubkey is - the embedded `kit update` anchor. -- [x] `scripts/release.sh`: cross-build + package + sign every hosted triple in - `scripts/hosted.sh`'s support set with the real release key; emit + sign - the channel index. Documents the sysroot dependency for foreign-libc - targets. (Maintainer-run; not exercised by CI here.) - -### Channel index + trust -- [x] `src/dist/release.c` + tests: `kit-release 1` emit/parse (ordered `url` - mirror lists) and CalVer compare (`test/api/release_index_test.c`). -- [x] `driver/release_key.c`: embedded release public-key set (array → any key - verifies, for overlap-window rotation); `kit update --key` override. The - real release keypair is generated offline at release time (secret never in - tree); the in-tree default is the NON-RELEASE test pubkey. - -### `kit update` -- [x] `driver/cmd/update.c` (+ `KIT_TOOL_UPDATE_ENABLED`): offline verify+install - first; then channel/index resolution (`--index` / `$KIT_UPDATE_INDEX_URL`) - and the full command surface. -- [x] Single-root install layout (`$KIT_HOME`, default `$XDG_DATA_HOME/kit`): - `versions/`, `current`, `bin/`, `config/`, `cache/`. Unpack → atomic - `current` flip → refresh `$KIT_HOME/bin` links; keep-all retention; - `--rollback` / `--list` / `--prune`. Self-update only ever writes under the - user-writable `$KIT_HOME`, never a system install (e.g. `/usr/local`). -- [x] Reconcile paths to the single root: `trusted_keys` → - `$KIT_HOME/config/trusted_keys` (honor `$KIT_TRUSTED_KEYS` + legacy - `~/.config/kit/trusted_keys`); driver path policy in `driver/cmd/pkg.c`, - `src/dist/trust.c` format helpers unchanged; DISTRIBUTE.md updated. Noted - the `$KIT_HOME/sysroots` move in SYSROOTS.md. -- [x] Host fetch hook in `driver/env/` (`driver_fetch_url`): execs `curl` - (falling back to `wget`) with the URL as a distinct argv element (no - shell); signature-gated, transport-agnostic, mirror-list fallback. -- [x] Monotonic/downgrade policy (channel updates refuse older than current - unless `--allow-downgrade`; `--version`/local-file are deliberate and - exempt). Networked host match is by the channel-index `[host]` lookup on - `kit_host_triple()`. - -### Tests -- [x] Hermetic: CalVer compare + index emit/parse byte-stability - (`test/api/release_index_test.c`, `make test-release-index`). -- [x] End-to-end (`test/dist/run.sh`, `make test-selfdist`, 62 cases): verify + - `kit update` into a temp `KIT_HOME`; asserts atomic swap, link refresh, - offline `--version` flip, `--rollback`, `--prune`, `--list`. (Builds the - package with `kit pkg create` — the operation `make dist` performs — to - stay fast + hermetic.) -- [x] Bootstrap: detached `.tar.gz.minisig` verifies with stock minisign. -- [x] Tamper: flipped bytes / wrong key / channel-downgrade are all rejected - pre-swap (the downgrade case via a `file://` channel index). diff --git a/doc/plan/SYSROOTS.md b/doc/plan/SYSROOTS.md @@ -35,7 +35,7 @@ Freestanding targets require no sysroot kpkg; `libkit_rt.a` and ## Sysroot kpkg layout -> Single-root layout note (see [SELFDIST.md](SELFDIST.md)): kit consolidates +> Single-root layout note (see [../RELEASE.md](../RELEASE.md)): kit consolidates > everything it manages under one root, `$KIT_HOME` (default `$XDG_DATA_HOME/kit` > ≡ `~/.local/share/kit`). Installed sysroots therefore belong at > `$KIT_HOME/sysroots/<triple>/`. The build/test provisioning today caches them