kit

kit
git clone https://git.ryansepassi.com/git/kit.git
Log | Files | Refs | README

commit 93d60d7d4a25b700ec107367f8b0d7765f67e2bf
parent 3142c122a8d1a3ed829736bc52450b470a66c452
Author: Ryan Sepassi <rsepassi@gmail.com>
Date:   Tue, 16 Jun 2026 17:11:16 -0700

cpio: add `kit cpio` for SVR4 newc initramfs archives

New byte-utility tool (sibling of `ar`) that packages and inspects the
SVR4 "newc" cpio archives the Linux kernel unpacks as initramfs:

- create (`-o`) from files/dirs (recursed), list (`-t`), extract (`-i`);
  regular files, directories, and symlinks.
- deterministic output: members sorted by path (a sorted DFS so each dir
  precedes its children), uid/gid 0, mtime 0, sequential inode, mode =
  type | perms keyed on the source exec bit; closing TRAILER!!! + 512-byte
  tail pad. Identical inputs => byte-identical archive.
- gzip/lz4 compression (`--compress=`, `-z`, `--lz4`) via kit/compress.h;
  `-d` + always-on auto-detect on read; specific diagnostic for zstd/xz.
- concatenated archives accepted on read (early-init segments).
- 070701 (newc) and 070702 (crc) magics; create defaults to newc, `-H crc`
  emits the data-sum variant.
- security: `..`/absolute member names refused on both create and extract.

The newc codec lives driver-local in driver/cmd/cpio.c (the driver has no
-Isrc; only this tool consumes it), mirroring src/dist/tar.c's stateless
append/finish/iter shape. Two additive host shims were needed: driver_readlink
(symlink targets) and driver_path_lstat (no-follow operand classification + the
source executable bit). Gated by KIT_TOOL_CPIO_ENABLED.

Verified bidirectionally against host bsdcpio. test/cpio/run.sh covers
round-trip, determinism, newc shape, gzip/lz4/crc, concatenation, and the
security/usage negatives (+ optional KIT_CPIO_TEST_HOST=1 host cross-check).

Closes the initramfs/cpio checklist in doc/plan/KERNEL.md.

Diffstat:
Mdoc/plan/KERNEL.md | 42+++++++++++++++++++++++++++++-------------
Adriver/cmd/cpio.c | 1124+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mdriver/driver.h | 2++
Mdriver/env.h | 18++++++++++++++++++
Mdriver/env/posix.c | 24++++++++++++++++++++++++
Mdriver/env/windows.c | 21+++++++++++++++++++++
Mdriver/main.c | 5+++++
Minclude/kit/config.h | 1+
Mmk/driver_srcs.mk | 1+
Mmk/test.mk | 6+++++-
Atest/cpio/run.sh | 170+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
11 files changed, 1400 insertions(+), 14 deletions(-)

diff --git a/doc/plan/KERNEL.md b/doc/plan/KERNEL.md @@ -528,19 +528,35 @@ metadata accuracy, and cc/ld report parity). The checklist below is closed out. Added on top of the closed-out Phase 1-5 work; not yet started. -Initramfs / cpio (archive packaging, sibling to `ar`): - -- [ ] `kit cpio` (newc/SVR4 only, magic `070701`/`070702`): create / list (`-t`) - / extract (`-i`) with deterministic ordering and normalized metadata, closing - `TRAILER!!!`; reuse `src/dist/tar.c` patterns. -- [ ] Archive concatenation for early-init segments (build + accept concatenated - inputs). -- [ ] Compression as a `kit cpio` flag (`--compress=gzip|lz4`, `-z`/`--lz4`; - `-d`/auto-detect on read) — gzip + lz4 only, clear diagnostic for zstd/xz. An - initramfs is just a compressed newc archive, so no separate `initramfs` tool. -- [ ] Tool gating in `driver/main.c` (`KIT_TOOL_CPIO_ENABLED`) + a round-trip - fixture (pack → list → unpack, byte-deterministic) and a kernel-unpack shape - check. +Initramfs / cpio (archive packaging, sibling to `ar`) — landed as `kit cpio`. +The newc codec lives driver-local in `driver/cmd/cpio.c` (only the tool consumes +it; the driver has no `-Isrc`, so it mirrors `tar.c`'s stateless append/finish/ +iter shape rather than living in the dist subsystem). Create needed two +additive host shims: `driver_readlink` (read symlink targets) and +`driver_path_lstat` (no-follow operand classification + the source executable +bit). Bidirectionally interop-verified against host `bsdcpio`. + +- [x] `kit cpio` (newc/SVR4 only, magic `070701`/`070702`): create (`-o`) / list + (`-t`) / extract (`-i`) with deterministic ordering (members sorted by path, + a sorted DFS so each directory precedes its children) and normalized metadata + (uid/gid 0, mtime 0, sequential inode, mode = type | perms keyed on the source + exec bit), closing `TRAILER!!!` + a 512-byte tail pad. Regular files, + directories, and symlinks; `..`/absolute names refused on both create and + extract. +- [x] Archive concatenation for early-init segments: the reader continues past a + `TRAILER!!!`, skips inter-segment zero padding, and resumes on the next + `070701`/`070702` magic (trailing non-cpio data is noted, not fatal). Building + a concatenation is shell `cat` of 512-padded archives. +- [x] Compression as a `kit cpio` flag (`--compress=gzip|lz4`, `-z`/`--lz4`; + `-d` + always-on auto-detect on read) via the public `kit/compress.h` codecs — + gzip + lz4 only, with a specific diagnostic for a zstd/xz magic or + `--compress=zstd|xz`. An initramfs is just a compressed newc archive, so no + separate `initramfs` tool. +- [x] Tool gating in `driver/main.c` (`KIT_TOOL_CPIO_ENABLED`) + the + `test/cpio/run.sh` round-trip fixture (pack → list → extract, byte- + deterministic, newc-shape and 512-pad asserts, gzip/lz4/crc/concat coverage, + security + usage negatives; optional `KIT_CPIO_TEST_HOST=1` cross-check against + the host `cpio`). Flat kernel `Image` header (arm64 / riscv64) — landed via the `--image-header` modifier on `--format bin`/`rom` (not separate formats); it overlays the first diff --git a/driver/cmd/cpio.c b/driver/cmd/cpio.c @@ -0,0 +1,1124 @@ +#include <kit/compress.h> +#include <kit/core.h> +#include <stddef.h> +#include <stdint.h> +#include <string.h> + +#include "driver.h" +#include "env.h" + +/* `kit cpio` — create / list / extract SVR4 "newc" cpio archives, the format + * the Linux kernel unpacks as its initramfs (magic 070701, or 070702 for the + * CRC variant). This is an archive packager in the byte-utility family, a + * sibling of `ar`; it does not boot, mount, or interpret the archive. + * + * The newc framing is plain container bookkeeping (110-byte ASCII-hex headers, + * 4-byte-aligned name and data, a closing TRAILER!!! record, a 512-byte tail + * pad), so it lives here driver-local rather than in libkit — only this tool + * consumes it. Compression rides on the public kit/compress.h codecs. + * + * Determinism: members are sorted by archived path; metadata is normalized + * (mode = type | perms with perms keyed on the source executable bit, uid/gid + * 0, mtime 0, sequential inode). Identical inputs yield byte-identical output. + */ + +#define CPIO_TOOL "cpio" + +#define CPIO_HDR_LEN 110u +#define CPIO_MAGIC_NEWC "070701" +#define CPIO_MAGIC_CRC "070702" + +/* mode type bits (the octal S_IF* values, target-independent). */ +#define CPIO_S_IFMT 0170000u +#define CPIO_S_IFREG 0100000u +#define CPIO_S_IFDIR 0040000u +#define CPIO_S_IFLNK 0120000u + +/* Largest symlink target / read buffer for an extracted link. */ +#define CPIO_LINK_MAX 4096u + +/* --------------------------------------------------------------------------- + * Low-level field codec + * ------------------------------------------------------------------------- */ + +static void cpio_put_hex8(uint8_t* p, uint32_t v) { + static const char hx[] = "0123456789ABCDEF"; + int i; + for (i = 7; i >= 0; --i) { + p[i] = (uint8_t)hx[v & 0xFu]; + v >>= 4; + } +} + +static int cpio_get_hex8(const uint8_t* p, uint32_t* out) { + uint32_t v = 0; + int i; + for (i = 0; i < 8; ++i) { + int d = driver_hex_nibble((char)p[i]); + if (d < 0) return 1; + v = (v << 4) | (uint32_t)d; + } + *out = v; + return 0; +} + +static size_t cpio_round4(size_t n) { return (n + 3u) & ~(size_t)3u; } + +static uint8_t cpio_ft_from_mode(uint32_t mode) { + switch (mode & CPIO_S_IFMT) { + case CPIO_S_IFDIR: + return 3; + case CPIO_S_IFLNK: + return 7; + case CPIO_S_IFREG: + return 4; + default: + return 0; + } +} + +/* --------------------------------------------------------------------------- + * Writer side: stream newc records to a KitWriter, tracking the running + * archive offset so both 4-byte record alignment and the final 512-byte pad + * are measured from the start of the archive. + * ------------------------------------------------------------------------- */ + +typedef struct CpioOut { + KitWriter* w; + uint64_t total; /* bytes written so far */ + uint32_t next_ino; + int crc; /* emit 070702 with a data-byte-sum checksum */ + int err; +} CpioOut; + +static int cpio_emit(CpioOut* o, const void* p, size_t n) { + if (o->err) return 1; + if (n && kit_writer_write(o->w, p, n) != KIT_OK) { + o->err = 1; + return 1; + } + o->total += n; + return 0; +} + +/* Pad the archive up to the next 4-byte boundary. Record starts are always + * 4-aligned, so this lands data (after the name) and the next header (after + * data) on 4-byte boundaries as newc requires. */ +static int cpio_pad4(CpioOut* o) { + static const uint8_t z[4] = {0, 0, 0, 0}; + size_t pad = (size_t)((4u - (o->total & 3u)) & 3u); + return pad ? cpio_emit(o, z, pad) : 0; +} + +static int cpio_emit_record(CpioOut* o, const char* name, size_t namelen, + uint32_t mode, uint32_t nlink, uint32_t ino, + const uint8_t* data, uint32_t size) { + uint8_t hdr[CPIO_HDR_LEN]; + uint32_t namesize = (uint32_t)namelen + 1u; /* includes the trailing NUL */ + uint32_t check = 0; + static const uint8_t nul = 0; + + if (o->crc && data) { + uint32_t i; + for (i = 0; i < size; ++i) check += data[i]; + } + + memcpy(hdr, o->crc ? CPIO_MAGIC_CRC : CPIO_MAGIC_NEWC, 6); + cpio_put_hex8(hdr + 6, ino); + cpio_put_hex8(hdr + 14, mode); + cpio_put_hex8(hdr + 22, 0u); /* uid */ + cpio_put_hex8(hdr + 30, 0u); /* gid */ + cpio_put_hex8(hdr + 38, nlink); + cpio_put_hex8(hdr + 46, 0u); /* mtime */ + cpio_put_hex8(hdr + 54, size); + cpio_put_hex8(hdr + 62, 0u); /* devmajor */ + cpio_put_hex8(hdr + 70, 0u); /* devminor */ + cpio_put_hex8(hdr + 78, 0u); /* rdevmajor */ + cpio_put_hex8(hdr + 86, 0u); /* rdevminor */ + cpio_put_hex8(hdr + 94, namesize); + cpio_put_hex8(hdr + 102, check); + + if (cpio_emit(o, hdr, CPIO_HDR_LEN)) return 1; + if (cpio_emit(o, name, namelen)) return 1; + if (cpio_emit(o, &nul, 1)) return 1; + if (cpio_pad4(o)) return 1; + if (size && cpio_emit(o, data, size)) return 1; + if (cpio_pad4(o)) return 1; + return 0; +} + +/* Append one member, assigning the next sequential inode. */ +static int cpio_append(CpioOut* o, const char* name, size_t namelen, + uint32_t mode, uint32_t nlink, const uint8_t* data, + uint32_t size) { + if (cpio_emit_record(o, name, namelen, mode, nlink, o->next_ino, data, size)) + return 1; + o->next_ino++; + return 0; +} + +/* Write the TRAILER!!! record (inode 0, nlink 1, no data) then pad the whole + * archive to a 512-byte boundary so concatenated segments stay aligned. */ +static int cpio_finish(CpioOut* o) { + if (cpio_emit_record(o, "TRAILER!!!", 10u, 0u, 1u, 0u, NULL, 0u)) return 1; + { + static const uint8_t z[64] = {0}; + size_t pad = (size_t)((512u - (o->total & 511u)) & 511u); + while (pad) { + size_t n = pad > sizeof z ? sizeof z : pad; + if (cpio_emit(o, z, n)) return 1; + pad -= n; + } + } + return 0; +} + +/* --------------------------------------------------------------------------- + * Reader side: parse newc records (handling concatenated segments) and hand + * each non-trailer entry to a visitor. + * ------------------------------------------------------------------------- */ + +typedef struct CpioEntry { + const char* name; /* NUL-terminated, aliases the input buffer */ + uint32_t name_len; + uint32_t mode; + uint8_t filetype; /* derived from mode */ + const uint8_t* data; + uint32_t size; +} CpioEntry; + +/* Returns nonzero to stop the walk with an error. */ +typedef int (*CpioVisit)(void* user, const CpioEntry* e); + +static int cpio_is_magic(const uint8_t* p) { + return memcmp(p, CPIO_MAGIC_NEWC, 6) == 0 || memcmp(p, CPIO_MAGIC_CRC, 6) == 0; +} + +static int cpio_parse(const uint8_t* data, size_t len, CpioVisit fn, + void* user) { + size_t off = 0; + int saw_trailer = 0; + + while (off + CPIO_HDR_LEN <= len) { + const uint8_t* h = data + off; + uint32_t mode, filesize, namesize; + size_t name_off, data_off; + const char* name; + + if (!cpio_is_magic(h)) { + driver_errf(CPIO_TOOL, + off == 0 ? "not a cpio newc archive (bad magic)" + : "corrupt cpio header (bad magic mid-stream)"); + return 1; + } + if (cpio_get_hex8(h + 14, &mode) || cpio_get_hex8(h + 54, &filesize) || + cpio_get_hex8(h + 94, &namesize)) { + driver_errf(CPIO_TOOL, "corrupt cpio header (non-hex field)"); + return 1; + } + if (namesize == 0) { + driver_errf(CPIO_TOOL, "corrupt cpio header (zero name size)"); + return 1; + } + name_off = off + CPIO_HDR_LEN; + if (name_off + namesize > len) { + driver_errf(CPIO_TOOL, "truncated cpio archive (name runs past end)"); + return 1; + } + name = (const char*)(data + name_off); + if (name[namesize - 1u] != '\0') { + driver_errf(CPIO_TOOL, "corrupt cpio header (name not NUL-terminated)"); + return 1; + } + data_off = cpio_round4(name_off + namesize); + + if (namesize == 11u && memcmp(name, "TRAILER!!!", 11) == 0) { + saw_trailer = 1; + off = data_off; /* filesize is 0 for the trailer */ + /* Skip inter-segment zero padding; a following non-zero run that is a + * cpio magic starts a concatenated archive, otherwise it is trailing + * data we do not parse (e.g. an appended compressed image). */ + while (off < len && data[off] == 0) ++off; + if (off >= len) break; + if (off + 6u > len || !cpio_is_magic(data + off)) { + driver_errf(CPIO_TOOL, + "note: ignoring %lu trailing byte(s) after TRAILER (not a " + "concatenated cpio archive)", + (unsigned long)(len - off)); + break; + } + continue; + } + + if (data_off + filesize > len) { + driver_errf(CPIO_TOOL, "truncated cpio archive (data runs past end)"); + return 1; + } + { + CpioEntry e; + e.name = name; + e.name_len = namesize - 1u; + e.mode = mode; + e.filetype = cpio_ft_from_mode(mode); + e.data = data + data_off; + e.size = filesize; + if (fn(user, &e)) return 1; + } + off = cpio_round4(data_off + filesize); + } + + if (!saw_trailer) { + driver_errf(CPIO_TOOL, "warning: archive has no TRAILER!!! record"); + } + return 0; +} + +/* --------------------------------------------------------------------------- + * Member collection for `-o` create. + * ------------------------------------------------------------------------- */ + +typedef struct CpioMember { + char* name; /* archived path (cleaned), heap-owned */ + size_t name_alloc; + uint32_t name_len; + char* src; /* host path to read, heap-owned, or NULL */ + size_t src_alloc; + uint8_t filetype; /* 3 dir, 4 regular, 7 symlink */ + int executable; + uint64_t size; +} CpioMember; + +typedef struct CpioBuild { + DriverEnv* env; + CpioMember* items; + size_t count; + size_t cap; +} CpioBuild; + +static char* cpio_strdup(DriverEnv* env, const char* s, size_t* out_alloc) { + size_t n = driver_strlen(s) + 1u; + char* p = (char*)driver_alloc(env, n); + if (p) memcpy(p, s, n); + if (out_alloc) *out_alloc = n; + return p; +} + +static int cpio_build_grow(CpioBuild* b) { + size_t nc = b->cap ? b->cap * 2u : 16u; + CpioMember* nv = + (CpioMember*)driver_alloc(b->env, nc * sizeof(CpioMember)); + if (!nv) return 1; + if (b->items) { + memcpy(nv, b->items, b->count * sizeof(CpioMember)); + driver_free(b->env, b->items, b->cap * sizeof(CpioMember)); + } + b->items = nv; + b->cap = nc; + return 0; +} + +static int cpio_build_add(CpioBuild* b, const char* arch, const char* src, + uint8_t ft, int exe, uint64_t size) { + CpioMember* m; + if (b->count >= b->cap && cpio_build_grow(b)) return 1; + m = &b->items[b->count]; + memset(m, 0, sizeof *m); + m->name = cpio_strdup(b->env, arch, &m->name_alloc); + if (!m->name) return 1; + m->name_len = (uint32_t)(m->name_alloc - 1u); + if (src) { + m->src = cpio_strdup(b->env, src, &m->src_alloc); + if (!m->src) return 1; + } + m->filetype = ft; + m->executable = exe; + m->size = size; + b->count++; + return 0; +} + +static void cpio_build_free(CpioBuild* b) { + size_t i; + for (i = 0; i < b->count; ++i) { + if (b->items[i].name) driver_free(b->env, b->items[i].name, + b->items[i].name_alloc); + if (b->items[i].src) driver_free(b->env, b->items[i].src, + b->items[i].src_alloc); + } + if (b->items) driver_free(b->env, b->items, b->cap * sizeof(CpioMember)); + b->items = NULL; + b->count = b->cap = 0; +} + +/* Walk one source path (`src`) recording it (and, for directories, its + * contents) under the archived name `arch`. An empty `arch` means "emit the + * contents only" (used for a "." operand), so no entry is written for the + * root itself. Symlinks are recorded, never followed. */ +static int cpio_collect(CpioBuild* b, const char* src, const char* arch) { + DriverEnv* env = b->env; + uint64_t size = 0; + uint8_t ft = 0; + int exe = 0; + int rc = driver_path_lstat(src, &size, &ft, &exe); + if (rc != 0) { + driver_errf(CPIO_TOOL, "cannot stat: %s", src); + return 1; + } + + if (ft == 4) return cpio_build_add(b, arch, src, 4, exe, size); + if (ft == 7) return cpio_build_add(b, arch, src, 7, 0, 0); + if (ft != 3) { + driver_errf(CPIO_TOOL, "skipping unsupported file type: %s", src); + return 0; /* not fatal — special/device nodes are out of scope */ + } + + /* Directory: emit its own entry (unless this is the contents-only root), + * then recurse over a snapshot of its children. */ + if (arch[0] != '\0' && cpio_build_add(b, arch, NULL, 3, 0, 0)) return 1; + { + DriverDirHandle* h = driver_open_dir(env, src); + uint64_t i; + if (!h) { + driver_errf(CPIO_TOOL, "cannot read directory: %s", src); + return 1; + } + for (i = 0;; ++i) { + const char* nm; + uint32_t nl; + uint64_t ino, sz, mt; + uint8_t cft; + char* csrc; + char* carch; + size_t csrc_sz = 0, carch_sz = 0; + int crc; + if (driver_read_dir_entry(h, i, &nm, &nl, &ino, &sz, &mt, &cft) != 0) + break; + csrc = driver_path_join(env, src, nm, &csrc_sz); + carch = driver_path_join(env, arch, nm, &carch_sz); + if (!csrc || !carch) { + if (csrc) driver_free(env, csrc, csrc_sz); + if (carch) driver_free(env, carch, carch_sz); + driver_close_dir(env, h); + return 1; + } + crc = cpio_collect(b, csrc, carch); + driver_free(env, csrc, csrc_sz); + driver_free(env, carch, carch_sz); + if (crc) { + driver_close_dir(env, h); + return 1; + } + } + driver_close_dir(env, h); + } + return 0; +} + +static int cpio_name_cmp(const CpioMember* a, const CpioMember* b) { + const unsigned char* x = (const unsigned char*)a->name; + const unsigned char* y = (const unsigned char*)b->name; + size_t i = 0; + while (x[i] && y[i]) { + if (x[i] != y[i]) return (int)x[i] - (int)y[i]; + ++i; + } + return (int)x[i] - (int)y[i]; +} + +/* Bottom-up merge sort by archived name. Stable and O(n log n); a plain + * lexicographic order places every directory before its descendants (a + * parent name is a strict prefix of "parent/child"), so the emitted stream + * is a valid sorted DFS. */ +static int cpio_sort(CpioBuild* b) { + size_t n = b->count, width; + CpioMember* src = b->items; + CpioMember* tmp; + if (n < 2) return 0; + tmp = (CpioMember*)driver_alloc(b->env, n * sizeof(CpioMember)); + if (!tmp) return 1; + for (width = 1; width < n; width *= 2) { + size_t i; + for (i = 0; i < n; i += 2 * width) { + size_t l = i; + size_t mid = i + width < n ? i + width : n; + size_t r = i + 2 * width < n ? i + 2 * width : n; + size_t a = l, c = mid, k = l; + while (a < mid && c < r) + tmp[k++] = cpio_name_cmp(&src[a], &src[c]) <= 0 ? src[a++] : src[c++]; + while (a < mid) tmp[k++] = src[a++]; + while (c < r) tmp[k++] = src[c++]; + } + { + CpioMember* t = src; + src = tmp; + tmp = t; + } + } + if (src != b->items) { + memcpy(b->items, src, n * sizeof(CpioMember)); + tmp = src; + } + driver_free(b->env, tmp, n * sizeof(CpioMember)); + return 0; +} + +/* Strip a leading "./" run and any leading '/'. Returns the cleaned pointer + * into the original string and reports whether an absolute prefix was + * dropped. A trailing '/' is handled separately by the caller. */ +static const char* cpio_clean_name(const char* raw, int* stripped_abs) { + *stripped_abs = 0; + while (raw[0] == '.' && raw[1] == '/') raw += 2; + while (raw[0] == '/') { + raw += 1; + *stripped_abs = 1; + } + return raw; +} + +/* Whether `n` contains a ".." path component. Used to refuse archiving an + * operand whose name would escape the archive root (and which our own extract + * would then reject). */ +static int cpio_name_has_dotdot(const char* n) { + size_t i = 0, comp = 0; + for (;; ++i) { + char c = n[i]; + if (c == '/' || c == '\0') { + if (i - comp == 2u && n[comp] == '.' && n[comp + 1u] == '.') return 1; + comp = i + 1u; + if (c == '\0') return 0; + } + } +} + +/* --------------------------------------------------------------------------- + * Options + dispatch + * ------------------------------------------------------------------------- */ + +typedef struct CpioOpts { + int mode; /* 'o' create, 't' list, 'i' extract */ + const char* file; /* -F archive path, or NULL for stdin/stdout */ + int crc; /* emit/accept 070702 */ + int verbose; /* -v */ + int compress; /* create: compress the output */ + KitCompressFormat cfmt; + int decompress; /* -d (read); auto-detect is always on regardless */ +} CpioOpts; + +static int cpio_parse_format(const char* s, int* crc) { + if (driver_streq(s, "newc")) { + *crc = 0; + return 0; + } + if (driver_streq(s, "crc") || driver_streq(s, "newcrc") || + driver_streq(s, "sv4crc")) { + *crc = 1; + return 0; + } + return 1; +} + +static int cpio_parse_compress(const char* s, KitCompressFormat* fmt) { + if (driver_streq(s, "gzip") || driver_streq(s, "gz")) { + *fmt = KIT_COMPRESS_GZIP; + return 0; + } + if (driver_streq(s, "lz4")) { + *fmt = KIT_COMPRESS_LZ4_FRAME; + return 0; + } + return 1; /* zstd/xz/unknown */ +} + +void driver_help_cpio(void) { + driver_printf( + "%.*s", + KIT_SLICE_ARG(KIT_SLICE_LIT( + "kit cpio — create / list / extract SVR4 newc cpio archives\n" + "\n" + "USAGE\n" + " kit cpio -o [-F FILE] [-H newc|crc] [-z|--lz4] [-v] PATH...\n" + " kit cpio -t [-F FILE] [-v]\n" + " kit cpio -i [-F FILE] [-v]\n" + "\n" + "DESCRIPTION\n" + " Packages the SVR4 \"newc\" cpio format (magic 070701, or 070702\n" + " with -H crc) the Linux kernel unpacks as initramfs. Create reads\n" + " the given files and directories (recursed); list and extract read\n" + " the archive from -F FILE or stdin. Regular files, directories,\n" + " and symlinks are supported; special/device nodes are not.\n" + "\n" + " Output is deterministic: members are sorted by path and metadata\n" + " is normalized (uid/gid 0, mtime 0, mode by type and exec bit).\n" + "\n" + "MODES\n" + " -o, --create create an archive from PATH operands\n" + " -t, --list list the members of an archive\n" + " -i, --extract extract the members of an archive\n" + "\n" + "OPTIONS\n" + " -F, --file FILE archive file (default: stdout on -o, stdin " + "else)\n" + " -H, --format FMT newc (default) | crc (070702 checksum variant)\n" + " -z gzip-compress the created archive\n" + " --lz4 LZ4-frame-compress the created archive\n" + " --compress=C compress with C = gzip | lz4\n" + " -d decompress on read (auto-detected regardless)\n" + " -v, --verbose list/announce each member\n" + " -h, --help show this help\n" + "\n" + " Compression is gzip or lz4 only; zstd/xz are rejected. On read a\n" + " gzip/lz4 archive is decompressed automatically. Concatenated\n" + " archives (early-init segments) are accepted on list/extract.\n" + "\n" + "EXIT CODES\n" + " 0 success 1 I/O or format error 2 bad usage\n"))); +} + +/* ---- list ---- */ + +typedef struct CpioListCtx { + int verbose; +} CpioListCtx; + +static int cpio_list_visit(void* user, const CpioEntry* e) { + CpioListCtx* c = (CpioListCtx*)user; + if (!c->verbose) { + driver_printf("%.*s\n", (int)e->name_len, e->name); + return 0; + } + if (e->filetype == 7) { + driver_printf("%06o %8u %.*s -> %.*s\n", (unsigned)(e->mode & 07777u), + (unsigned)e->size, (int)e->name_len, e->name, (int)e->size, + (const char*)e->data); + } else { + driver_printf("%06o %8u %.*s\n", (unsigned)(e->mode & 07777u), + (unsigned)e->size, (int)e->name_len, e->name); + } + return 0; +} + +/* ---- extract ---- */ + +typedef struct CpioExtractCtx { + DriverEnv* env; + KitContext* ctx; + int verbose; +} CpioExtractCtx; + +/* Reject absolute paths, ".." components, embedded NUL, and empty names so a + * crafted archive cannot escape the destination directory. */ +static int cpio_name_safe(const char* n, uint32_t len) { + uint32_t i, comp = 0; + if (len == 0 || n[0] == '/') return 0; + for (i = 0; i <= len; ++i) { + char ch = (i < len) ? n[i] : '/'; + if (i < len && ch == '\0') return 0; + if (ch == '/') { + uint32_t cl = i - comp; + if (cl == 2u && n[comp] == '.' && n[comp + 1u] == '.') return 0; + comp = i + 1u; + } + } + return 1; +} + +/* mkdir -p the parent directory of `name` (which has no trailing slash). */ +static int cpio_make_parents(DriverEnv* env, const char* name) { + size_t len = driver_strlen(name); + size_t i = len; + char* p; + int rc; + while (i > 0 && name[i - 1u] != '/') --i; + if (i <= 1u) return 0; /* no parent component */ + p = (char*)driver_alloc(env, i); /* i-1 chars + NUL */ + if (!p) return 1; + memcpy(p, name, i - 1u); + p[i - 1u] = '\0'; + rc = driver_mkdir_p(env, p); + driver_free(env, p, i); + return rc; +} + +static int cpio_extract_visit(void* user, const CpioEntry* e) { + CpioExtractCtx* c = (CpioExtractCtx*)user; + DriverEnv* env = c->env; + const char* name = e->name; + + if (!cpio_name_safe(name, e->name_len)) { + driver_errf(CPIO_TOOL, "refusing unsafe member name: %.*s", + (int)e->name_len, name); + return 1; + } + + if (e->filetype == 3) { + if (driver_mkdir_p(env, name)) { + driver_errf(CPIO_TOOL, "cannot create directory: %s", name); + return 1; + } + } else if (e->filetype == 4) { + KitWriter* w = NULL; + if (cpio_make_parents(env, name)) { + driver_errf(CPIO_TOOL, "cannot create parent of: %s", name); + return 1; + } + if (c->ctx->file_io->open_writer(c->ctx->file_io->user, name, &w) != + KIT_OK) { + driver_errf(CPIO_TOOL, "cannot create file: %s", name); + return 1; + } + if (e->size) (void)kit_writer_write(w, e->data, e->size); + if (kit_writer_status(w) != KIT_OK) { + driver_writer_abort(w); + kit_writer_close(w); + driver_errf(CPIO_TOOL, "write failed: %s", name); + return 1; + } + kit_writer_close(w); + if (e->mode & 0111u) (void)driver_mark_executable_output(name); + } else if (e->filetype == 7) { + char tgt[CPIO_LINK_MAX]; + if (e->size >= sizeof tgt) { + driver_errf(CPIO_TOOL, "symlink target too long: %s", name); + return 1; + } + if (cpio_make_parents(env, name)) { + driver_errf(CPIO_TOOL, "cannot create parent of: %s", name); + return 1; + } + memcpy(tgt, e->data, e->size); + tgt[e->size] = '\0'; + (void)driver_remove_file(name); /* replace any stale entry */ + if (driver_create_symlink(tgt, name) != 0) { + driver_errf(CPIO_TOOL, "cannot create symlink: %s", name); + return 1; + } + } else { + driver_errf(CPIO_TOOL, "skipping unsupported member type: %.*s", + (int)e->name_len, name); + return 0; + } + + if (c->verbose) driver_printf("%.*s\n", (int)e->name_len, name); + return 0; +} + +/* --------------------------------------------------------------------------- + * Create + * ------------------------------------------------------------------------- */ + +static int cpio_write_members(KitContext* ctx, CpioBuild* b, CpioOut* o) { + size_t i; + for (i = 0; i < b->count; ++i) { + CpioMember* m = &b->items[i]; + if (m->filetype == 3) { + if (cpio_append(o, m->name, m->name_len, CPIO_S_IFDIR | 0755u, 2u, NULL, + 0u)) + return 1; + } else if (m->filetype == 7) { + char tgt[CPIO_LINK_MAX]; + if (driver_readlink(m->src, tgt, sizeof tgt) != 0) { + driver_errf(CPIO_TOOL, "cannot read symlink target: %s", m->src); + return 1; + } + if (cpio_append(o, m->name, m->name_len, CPIO_S_IFLNK | 0777u, 1u, + (const uint8_t*)tgt, (uint32_t)driver_strlen(tgt))) + return 1; + } else { /* regular file */ + KitFileData fd = {0}; + uint32_t mode = CPIO_S_IFREG | (m->executable ? 0755u : 0644u); + int wrc; + if (ctx->file_io->read_all(ctx->file_io->user, m->src, &fd) != KIT_OK) { + driver_errf(CPIO_TOOL, "cannot read file: %s", m->src); + return 1; + } + if (fd.size > 0xFFFFFFFFu) { + ctx->file_io->release(ctx->file_io->user, &fd); + driver_errf(CPIO_TOOL, "file too large for newc (>= 4 GiB): %s", + m->src); + return 1; + } + wrc = cpio_append(o, m->name, m->name_len, mode, 1u, fd.data, + (uint32_t)fd.size); + ctx->file_io->release(ctx->file_io->user, &fd); + if (wrc) return 1; + } + } + return cpio_finish(o); +} + +static int cpio_create(DriverEnv* env, KitContext* ctx, const CpioOpts* opt, + const char** ops, size_t nops) { + CpioBuild b; + CpioOut co; + KitWriter* target = NULL; /* where records are written */ + KitWriter* mem = NULL; /* buffer when compressing */ + KitWriter* out = NULL; /* final output (file/stdout) */ + int owned_out = 0; + int rc = 1; + size_t i; + + memset(&b, 0, sizeof b); + b.env = env; + + for (i = 0; i < nops; ++i) { + int abs_stripped = 0; + const char* cleaned = cpio_clean_name(ops[i], &abs_stripped); + char* owned = NULL; + size_t owned_sz = 0; + size_t cl; + if (abs_stripped) { + driver_errf(CPIO_TOOL, + "warning: storing %s as relative (leading '/' stripped)", + ops[i]); + } + /* Drop a trailing slash so a directory operand archives as "dir" not + * "dir/". An empty cleaned name ("." / "/" / "./") means contents-only. */ + cl = driver_strlen(cleaned); + while (cl > 0 && cleaned[cl - 1u] == '/') --cl; + owned = (char*)driver_alloc(env, cl + 1u); + if (!owned) goto done; + owned_sz = cl + 1u; + memcpy(owned, cleaned, cl); + owned[cl] = '\0'; + if (cl > 0 && cpio_name_has_dotdot(owned)) { + driver_errf(CPIO_TOOL, + "operand escapes the archive root (.. component): %s; cd into " + "the directory and pass a relative path", + ops[i]); + driver_free(env, owned, owned_sz); + rc = 1; + goto done; + } + rc = cpio_collect(&b, ops[i], owned); + driver_free(env, owned, owned_sz); + if (rc) goto done; + } + + if (cpio_sort(&b)) { + driver_errf(CPIO_TOOL, "out of memory sorting members"); + rc = 1; + goto done; + } + + if (opt->compress) { + if (kit_writer_mem(env->heap, &mem) != KIT_OK) { + driver_errf(CPIO_TOOL, "out of memory"); + rc = 1; + goto done; + } + target = mem; + } else if (opt->file) { + if (ctx->file_io->open_writer(ctx->file_io->user, opt->file, &out) != + KIT_OK) { + driver_errf(CPIO_TOOL, "cannot open output: %s", opt->file); + rc = 1; + goto done; + } + owned_out = 1; + target = out; + } else { + out = driver_stdout_writer(env); + owned_out = 1; + target = out; + } + + memset(&co, 0, sizeof co); + co.w = target; + co.next_ino = 1u; + co.crc = opt->crc; + + if (cpio_write_members(ctx, &b, &co) || co.err) { + if (owned_out && out) driver_writer_abort(out); + rc = 1; + goto done; + } + + if (opt->compress) { + const uint8_t* bytes; + size_t blen = 0; + bytes = kit_writer_mem_bytes(mem, &blen); + if (opt->file) { + if (ctx->file_io->open_writer(ctx->file_io->user, opt->file, &out) != + KIT_OK) { + driver_errf(CPIO_TOOL, "cannot open output: %s", opt->file); + rc = 1; + goto done; + } + } else { + out = driver_stdout_writer(env); + } + owned_out = 1; + if (kit_compress(ctx, opt->cfmt, bytes, blen, out) != KIT_OK) { + driver_writer_abort(out); + rc = 1; + goto done; + } + } + + rc = 0; + +done: + if (owned_out && out) kit_writer_close(out); + if (mem) kit_writer_close(mem); + cpio_build_free(&b); + return rc; +} + +/* --------------------------------------------------------------------------- + * Read (list / extract) — load, optionally decompress, then parse. + * ------------------------------------------------------------------------- */ + +/* Recognize zstd / xz so we can reject them specifically instead of letting + * the cpio parser report a meaningless "bad magic". */ +static int cpio_is_unsupported_compressed(const uint8_t* d, size_t n) { + static const uint8_t zstd[4] = {0x28, 0xB5, 0x2F, 0xFD}; + static const uint8_t xz[6] = {0xFD, '7', 'z', 'X', 'Z', 0x00}; + if (n >= 4 && memcmp(d, zstd, 4) == 0) return 1; + if (n >= 6 && memcmp(d, xz, 6) == 0) return 1; + return 0; +} + +static int cpio_read(DriverEnv* env, KitContext* ctx, const CpioOpts* opt) { + const uint8_t* raw = NULL; + size_t rawlen = 0; + DriverLoad ld = {0}; + uint8_t* sbuf = NULL; + size_t sbuf_len = 0; + int loaded_file = 0, loaded_stdin = 0; + const uint8_t* payload; + size_t payload_len; + KitWriter* dmem = NULL; + KitCompressFormat cfmt; + int rc = 1; + + if (opt->file) { + KitSlice in; + if (driver_load_bytes(&env->file_io, CPIO_TOOL, opt->file, &ld, &in) != 0) + return 1; + loaded_file = 1; + raw = in.data; + rawlen = in.len; + } else { + if (!driver_read_stdin(env, &sbuf, &sbuf_len)) { + driver_errf(CPIO_TOOL, "failed to read stdin"); + return 1; + } + loaded_stdin = 1; + raw = sbuf; + rawlen = sbuf_len; + } + + payload = raw; + payload_len = rawlen; + + if (cpio_is_unsupported_compressed(raw, rawlen)) { + driver_errf(CPIO_TOOL, + "archive is zstd/xz-compressed; kit cpio supports gzip and " + "lz4 only"); + goto done; + } + if (kit_compress_detect(raw, rawlen, &cfmt) == KIT_OK) { + size_t dlen = 0; + if (kit_writer_mem(env->heap, &dmem) != KIT_OK) { + driver_errf(CPIO_TOOL, "out of memory"); + goto done; + } + if (kit_decompress(ctx, cfmt, raw, rawlen, dmem) != KIT_OK) goto done; + payload = kit_writer_mem_bytes(dmem, &dlen); + payload_len = dlen; + } else if (opt->decompress) { + driver_errf(CPIO_TOOL, "-d given but input is not gzip/lz4-compressed"); + /* not fatal: fall through and parse as a raw archive */ + } + + if (opt->mode == 't') { + CpioListCtx lc; + lc.verbose = opt->verbose; + rc = cpio_parse(payload, payload_len, cpio_list_visit, &lc); + } else { + CpioExtractCtx xc; + xc.env = env; + xc.ctx = ctx; + xc.verbose = opt->verbose; + rc = cpio_parse(payload, payload_len, cpio_extract_visit, &xc); + } + +done: + if (dmem) kit_writer_close(dmem); + if (loaded_file) driver_release_bytes(&env->file_io, &ld); + if (loaded_stdin && sbuf) driver_free(env, sbuf, sbuf_len); + return rc; +} + +/* --------------------------------------------------------------------------- + * Entry point + * ------------------------------------------------------------------------- */ + +static int cpio_opt_arg(int argc, char** argv, int* i, const char* tool, + const char* flag, const char** out) { + /* Accept "--flag=value", "-Fvalue", or a separate next argument. */ + const char* a = argv[*i]; + size_t flen = driver_strlen(flag); + if (driver_strneq(a, flag, flen) && a[flen] == '=') { + *out = a + flen + 1u; + return 0; + } + if (*i + 1 >= argc) { + driver_errf(tool, "%s requires an argument", flag); + return 1; + } + *out = argv[++(*i)]; + return 0; +} + +int driver_cpio(int argc, char** argv) { + DriverEnv env; + KitContext ctx; + CpioOpts o; + const char** ops = NULL; + size_t nops = 0; + int i, rc = 2; + + if (driver_argv_wants_help(argc, argv, 1)) { + driver_help_cpio(); + return 0; + } + + memset(&o, 0, sizeof o); + driver_env_init(&env); + ctx = driver_env_to_context(&env); + + ops = (const char**)driver_alloc(&env, sizeof(char*) * (size_t)(argc)); + if (!ops) { + driver_errf(CPIO_TOOL, "out of memory"); + rc = 1; + goto done; + } + + for (i = 1; i < argc; ++i) { + const char* a = argv[i]; + + if (driver_streq(a, "--")) { /* end of options */ + for (++i; i < argc; ++i) ops[nops++] = argv[i]; + break; + } + + if (driver_strneq(a, "--", 2)) { /* long option */ + if (driver_streq(a, "--create")) { + o.mode = 'o'; + } else if (driver_streq(a, "--list")) { + o.mode = 't'; + } else if (driver_streq(a, "--extract")) { + o.mode = 'i'; + } else if (driver_streq(a, "--verbose")) { + o.verbose = 1; + } else if (driver_streq(a, "--decompress")) { + o.decompress = 1; + } else if (driver_streq(a, "--lz4")) { + o.compress = 1; + o.cfmt = KIT_COMPRESS_LZ4_FRAME; + } else if (driver_streq(a, "--file") || driver_strneq(a, "--file=", 7)) { + if (cpio_opt_arg(argc, argv, &i, CPIO_TOOL, "--file", &o.file)) + goto done; + } else if (driver_streq(a, "--format") || + driver_strneq(a, "--format=", 9)) { + const char* v; + if (cpio_opt_arg(argc, argv, &i, CPIO_TOOL, "--format", &v)) goto done; + if (cpio_parse_format(v, &o.crc)) { + driver_errf(CPIO_TOOL, "unsupported format: %s (use newc or crc)", v); + goto done; + } + } else if (driver_streq(a, "--compress") || + driver_strneq(a, "--compress=", 11)) { + const char* v; + if (cpio_opt_arg(argc, argv, &i, CPIO_TOOL, "--compress", &v)) goto done; + if (cpio_parse_compress(v, &o.cfmt)) { + driver_errf(CPIO_TOOL, + "unsupported compressor: %s (kit cpio supports gzip and " + "lz4 only)", + v); + goto done; + } + o.compress = 1; + } else { + driver_errf(CPIO_TOOL, "unknown option: %s", a); + goto done; + } + continue; + } + + if (a[0] == '-' && a[1] != '\0') { /* short cluster, e.g. -tv, -Fout */ + int j; + for (j = 1; a[j]; ++j) { + char ch = a[j]; + if (ch == 'o') { + o.mode = 'o'; + } else if (ch == 't') { + o.mode = 't'; + } else if (ch == 'i') { + o.mode = 'i'; + } else if (ch == 'v') { + o.verbose = 1; + } else if (ch == 'd') { + o.decompress = 1; + } else if (ch == 'z') { + o.compress = 1; + o.cfmt = KIT_COMPRESS_GZIP; + } else if (ch == 'F' || ch == 'H') { + const char* v; + if (a[j + 1]) { + v = a + j + 1; /* -Fvalue */ + } else if (i + 1 < argc) { + v = argv[++i]; /* -F value */ + } else { + driver_errf(CPIO_TOOL, "-%c requires an argument", ch); + goto done; + } + if (ch == 'F') { + o.file = v; + } else if (cpio_parse_format(v, &o.crc)) { + driver_errf(CPIO_TOOL, "unsupported format: %s (use newc or crc)", + v); + goto done; + } + break; /* value consumed the rest of the cluster */ + } else { + driver_errf(CPIO_TOOL, "unknown option: -%c", ch); + goto done; + } + } + continue; + } + + ops[nops++] = a; /* operand (a file/dir, or "-") */ + } + + if (o.mode == 0) { + driver_errf(CPIO_TOOL, "one of -o, -t, or -i is required"); + goto done; + } + if (o.compress && o.mode != 'o') { + driver_errf(CPIO_TOOL, "compression flags apply to -o (create) only"); + goto done; + } + if (o.decompress && o.mode == 'o') { + driver_errf(CPIO_TOOL, "-d applies to -t/-i (read) only"); + goto done; + } + if (o.mode != 'o' && nops > 0) { + driver_errf(CPIO_TOOL, + "warning: member patterns are not supported; operating on the " + "whole archive"); + } + + if (o.mode == 'o') { + rc = cpio_create(&env, &ctx, &o, ops, nops); + } else { + rc = cpio_read(&env, &ctx, &o); + } + +done: + if (ops) driver_free(&env, ops, sizeof(char*) * (size_t)(argc)); + driver_env_fini(&env); + return rc; +} diff --git a/driver/driver.h b/driver/driver.h @@ -33,6 +33,7 @@ int driver_cpp(int argc, char** argv); int driver_as(int argc, char** argv); int driver_ld(int argc, char** argv); int driver_ar(int argc, char** argv); +int driver_cpio(int argc, char** argv); int driver_ranlib(int argc, char** argv); int driver_strip(int argc, char** argv); int driver_objcopy(int argc, char** argv); @@ -78,6 +79,7 @@ void driver_help_cpp(void); void driver_help_as(void); void driver_help_ld(void); void driver_help_ar(void); +void driver_help_cpio(void); void driver_help_ranlib(void); void driver_help_strip(void); void driver_help_objcopy(void); diff --git a/driver/env.h b/driver/env.h @@ -174,6 +174,17 @@ int driver_path_mtime_ns(const char* path, int64_t* out); int driver_path_stat(const char* path, uint64_t* out_size, uint64_t* out_mtime_ns, uint8_t* out_filetype); +/* Like driver_path_stat but does NOT follow a terminal symlink (lstat) and + * additionally reports whether the entry is an executable regular file (any + * execute bit set). *out_executable is 0 for non-regular entries and on hosts + * without an executable bit (Windows). Fills *out_size and *out_filetype with + * the same KIT_WASM_FILETYPE_* coding as driver_path_stat. Returns 0 on + * success, 1 if the path does not exist, 2 on any other error. Used by + * `cpio -o` to classify operands without dereferencing symlinks and to carry + * the source executable bit into the archived mode. */ +int driver_path_lstat(const char* path, uint64_t* out_size, + uint8_t* out_filetype, int* out_executable); + /* Opaque directory-enumeration handle. Holds a snapshot of all entries * (excluding "." and "..") taken at driver_open_dir time. */ typedef struct DriverDirHandle DriverDirHandle; @@ -210,6 +221,13 @@ int driver_self_exe_path(DriverEnv*, char** out, size_t* out_size); * on Windows). */ int driver_create_symlink(const char* target, const char* link_path); +/* Read the target of the symbolic link at `path` into `buf` (capacity `cap`, + * including room for the terminating NUL), NUL-terminating it. Returns 0 on + * success, nonzero on failure or when the target does not fit in `cap`. POSIX + * uses readlink(2); Windows is best-effort and may always fail (initramfs + * symlinks are a Linux artifact), in which case the caller diagnoses. */ +int driver_readlink(const char* path, char* buf, size_t cap); + /* Create a hard link named `link_path` referring to the same file as `target`. * Returns 0 on success. POSIX uses link(2); Windows uses CreateHardLinkW. Both * require `target` and `link_path` to live on the same filesystem/volume. */ diff --git a/driver/env/posix.c b/driver/env/posix.c @@ -528,6 +528,18 @@ int driver_path_stat(const char* path, uint64_t* out_size, return 0; } +int driver_path_lstat(const char* path, uint64_t* out_size, + uint8_t* out_filetype, int* out_executable) { + struct stat sb; + if (!path || lstat(path, &sb) != 0) + return (errno == ENOENT || errno == ENOTDIR) ? 1 : 2; + *out_size = (uint64_t)sb.st_size; + *out_filetype = posix_mode_to_wasm_filetype(sb.st_mode); + *out_executable = + S_ISREG(sb.st_mode) && (sb.st_mode & (S_IXUSR | S_IXGRP | S_IXOTH)) != 0; + return 0; +} + typedef struct DriverDirEntryRec { char* name; size_t name_alloc; @@ -715,6 +727,18 @@ int driver_create_symlink(const char* target, const char* link_path) { return symlink(target, link_path) == 0 ? 0 : 1; } +int driver_readlink(const char* path, char* buf, size_t cap) { + ssize_t n; + if (!path || !buf || cap == 0) return 1; + /* Read into the full buffer; readlink never NUL-terminates and reports the + * untruncated length, so n >= cap means the target did not fit. */ + n = readlink(path, buf, cap); + if (n < 0) return 1; + if ((size_t)n >= cap) return 1; + buf[n] = '\0'; + return 0; +} + int driver_create_hardlink(const char* target, const char* link_path) { if (!target || !link_path) return 1; return link(target, link_path) == 0 ? 0 : 1; diff --git a/driver/env/windows.c b/driver/env/windows.c @@ -706,6 +706,16 @@ int driver_path_stat(const char* path, uint64_t* out_size, return 0; } +int driver_path_lstat(const char* path, uint64_t* out_size, + uint8_t* out_filetype, int* out_executable) { + uint64_t mtime = 0; + /* GetFileAttributesExW reports a reparse point without following it, so + * driver_path_stat is already effectively no-follow on Windows; there is no + * POSIX-style execute bit. */ + if (out_executable) *out_executable = 0; + return driver_path_stat(path, out_size, &mtime, out_filetype); +} + typedef struct DriverDirEntryRec { char* name; size_t name_alloc; @@ -993,6 +1003,17 @@ int driver_create_symlink(const char* target, const char* link_path) { return ok ? 0 : 1; } +int driver_readlink(const char* path, char* buf, size_t cap) { + /* Reading a reparse point is materially more involved than POSIX + * readlink(2), and initramfs symlinks are a Linux artifact. The Windows + * build is best-effort and reports failure; the caller emits a clear + * diagnostic and skips the entry rather than archiving a bogus link. */ + (void)path; + (void)buf; + (void)cap; + return 1; +} + int driver_create_hardlink(const char* target, const char* link_path) { wchar_t* wtarget; wchar_t* wlink; diff --git a/driver/main.c b/driver/main.c @@ -75,6 +75,11 @@ static const DriverToolDesc driver_tools[] = { "Create / modify / list / extract POSIX `ar` archives", DRIVER_GROUP_TOOLCHAIN}, #endif +#if KIT_TOOL_CPIO_ENABLED + {"cpio", driver_cpio, NULL, driver_help_cpio, + "Create / list / extract SVR4 newc cpio (initramfs) archives", + DRIVER_GROUP_TOOLCHAIN}, +#endif #if KIT_TOOL_RANLIB_ENABLED {"ranlib", driver_ranlib, NULL, driver_help_ranlib, "Refresh the symbol index of an `ar` archive", DRIVER_GROUP_TOOLCHAIN}, diff --git a/include/kit/config.h b/include/kit/config.h @@ -114,6 +114,7 @@ #define KIT_TOOL_AS_ENABLED 1 #define KIT_TOOL_LD_ENABLED 1 #define KIT_TOOL_AR_ENABLED 1 +#define KIT_TOOL_CPIO_ENABLED 1 #define KIT_TOOL_RANLIB_ENABLED 1 #define KIT_TOOL_STRIP_ENABLED 1 #define KIT_TOOL_OBJCOPY_ENABLED 1 diff --git a/mk/driver_srcs.mk b/mk/driver_srcs.mk @@ -36,6 +36,7 @@ DRIVER_TOOL_SRCS = \ $(call tool-cmd,AS,as) \ $(call tool-cmd,LD,ld) \ $(call tool-cmd,AR,ar) \ + $(call tool-cmd,CPIO,cpio) \ $(call tool-cmd,RANLIB,ranlib) \ $(call tool-cmd,STRIP,strip) \ $(call tool-cmd,OBJCOPY,objcopy) \ diff --git a/mk/test.mk b/mk/test.mk @@ -65,6 +65,7 @@ TEST_TARGETS = \ test-dbg \ test-driver \ test-driver-ar \ + test-driver-cpio \ test-driver-cas \ test-driver-cc \ test-driver-build \ @@ -214,7 +215,7 @@ test-images: test-cf-corpus-selftest: @bash test/lib/kit_corpus_selftest.sh -test-driver: test-driver-cc test-driver-build test-driver-ar test-driver-cas test-driver-strip test-driver-objcopy test-driver-objdump test-driver-pkg test-driver-strings test-driver-tools test-driver-wasm +test-driver: test-driver-cc test-driver-build test-driver-ar test-driver-cpio test-driver-cas test-driver-strip test-driver-objcopy test-driver-objdump test-driver-pkg test-driver-strings test-driver-tools test-driver-wasm test-driver-cc: bin @KIT=$(abspath $(BIN)) sh test/driver/run.sh @@ -297,6 +298,9 @@ test-ar: $(AR_TEST_BIN) test-driver-ar: bin @KIT=$(abspath $(BIN)) test/ar/run.sh +test-driver-cpio: bin + @KIT=$(abspath $(BIN)) sh test/cpio/run.sh + test-driver-cas: bin @KIT=$(abspath $(BIN)) sh test/cas/run.sh diff --git a/test/cpio/run.sh b/test/cpio/run.sh @@ -0,0 +1,170 @@ +#!/bin/sh +# Driver-level checks for `kit cpio` (SVR4 newc initramfs archives). +# +# Self-checking (no golden files): create -> list -> extract round-trips, +# determinism, the newc shape (070701 magic, TRAILER!!! record, 512-byte tail +# pad), gzip/lz4 compression with read auto-detect, archive concatenation, the +# 070702 CRC variant, and the security / usage negatives are asserted inline +# via the shared kit_* verbs (ok/run_ok/run_fail/contains/same_file). +# +# Set KIT_CPIO_TEST_HOST=1 to additionally cross-check against the host `cpio`. + +set -u + +script_dir=$(cd "$(dirname "$0")" && pwd) +repo_root=$(cd "$script_dir/../.." && pwd) + +KIT="${KIT:-$repo_root/build/kit}" + +if [ ! -x "$KIT" ]; then + echo "cpio: kit binary not found at $KIT" >&2 + exit 2 +fi + +work=$(mktemp -d "${TMPDIR:-/tmp}/kit-cpio-test.XXXXXX") +trap 'rm -rf "$work"' EXIT + +KIT_KIT_DIR="$repo_root/test/lib" +. "$repo_root/test/lib/kit_sh_kit.sh" +kit_report_init + +# ---- fixture: a small tree exercising every supported member type ----------- +# regular file, executable regular file, zero-byte file, binary file, a nested +# subdirectory, an empty directory, and a symlink. +mkdir -p "$work/payload/root/sub" "$work/payload/root/emptydir" +printf 'hello cpio\n' > "$work/payload/root/a.txt" +printf '#!/bin/sh\necho hi\n' > "$work/payload/root/run.sh" +chmod +x "$work/payload/root/run.sh" +: > "$work/payload/root/empty.txt" +printf '\000\001\002\377\376zz' > "$work/payload/root/sub/b.bin" +ln -s a.txt "$work/payload/root/link" + +create() { ( cd "$work/payload" && "$KIT" cpio "$@" ); } + +# ---- create + list --------------------------------------------------------- +run_ok cpio-create create -o -F "$work/out.cpio" root +"$KIT" cpio -t -F "$work/out.cpio" > "$work/list.txt" 2> "$work/list.err" +for name in root root/a.txt root/run.sh root/empty.txt root/emptydir \ + root/link root/sub root/sub/b.bin; do + contains "cpio-list-$name" "$work/list.txt" "$name" +done + +# Deterministic ordering: directory entries precede their children, sorted. +printf 'root\nroot/a.txt\nroot/empty.txt\nroot/emptydir\nroot/link\nroot/run.sh\nroot/sub\nroot/sub/b.bin\n' \ + > "$work/list.want" +same_file cpio-list-sorted "$work/list.want" "$work/list.txt" + +# ---- newc shape ------------------------------------------------------------ +magic=$(dd if="$work/out.cpio" bs=1 count=6 2>/dev/null) +if [ "$magic" = "070701" ]; then ok cpio-magic-newc +else echo "magic=$magic" > "$work/magic.diag"; not_ok cpio-magic-newc "$work/magic.diag"; fi + +contains cpio-has-trailer "$work/out.cpio" "TRAILER!!!" + +sz=$(wc -c < "$work/out.cpio") +if [ "$((sz % 512))" -eq 0 ]; then ok cpio-512-pad +else echo "size=$sz" > "$work/pad.diag"; not_ok cpio-512-pad "$work/pad.diag"; fi + +# ---- determinism ----------------------------------------------------------- +run_ok cpio-create-again create -o -F "$work/out2.cpio" root +same_file cpio-deterministic "$work/out.cpio" "$work/out2.cpio" + +# ---- extract + content compare --------------------------------------------- +mkdir -p "$work/ex" +( cd "$work/ex" && "$KIT" cpio -i -F "$work/out.cpio" ) 2> "$work/ex.err" +same_file cpio-extract-a "$work/payload/root/a.txt" "$work/ex/root/a.txt" +same_file cpio-extract-bin "$work/payload/root/sub/b.bin" "$work/ex/root/sub/b.bin" +same_file cpio-extract-empty "$work/payload/root/empty.txt" "$work/ex/root/empty.txt" +assert_file_exists cpio-extract-run "$work/ex/root/run.sh" +is_executable cpio-extract-exec-bit "$work/ex/root/run.sh" +if [ -d "$work/ex/root/emptydir" ]; then ok cpio-extract-emptydir +else not_ok cpio-extract-emptydir; fi +if [ -L "$work/ex/root/link" ] && [ "$(readlink "$work/ex/root/link")" = a.txt ]; then + ok cpio-extract-symlink +else not_ok cpio-extract-symlink; fi + +# ---- gzip / lz4 round-trips (read auto-detects the codec) ------------------ +for z in "-z gzip" "--lz4 lz4"; do + set -- $z + flag=$1; tag=$2 + run_ok "cpio-create-$tag" create -o "$flag" -F "$work/out.$tag" root + "$KIT" cpio -t -F "$work/out.$tag" > "$work/list.$tag" 2> "$work/list.$tag.err" + same_file "cpio-$tag-list-matches" "$work/list.txt" "$work/list.$tag" + mkdir -p "$work/ex-$tag" + ( cd "$work/ex-$tag" && "$KIT" cpio -i -F "$work/out.$tag" ) 2>/dev/null + same_file "cpio-$tag-extract" "$work/payload/root/sub/b.bin" \ + "$work/ex-$tag/root/sub/b.bin" +done + +# ---- 070702 CRC variant ---------------------------------------------------- +run_ok cpio-create-crc create -o -H crc -F "$work/out.crc" root +crcmagic=$(dd if="$work/out.crc" bs=1 count=6 2>/dev/null) +if [ "$crcmagic" = "070702" ]; then ok cpio-magic-crc +else echo "magic=$crcmagic" > "$work/crc.diag"; not_ok cpio-magic-crc "$work/crc.diag"; fi +mkdir -p "$work/ex-crc" +( cd "$work/ex-crc" && "$KIT" cpio -i -F "$work/out.crc" ) 2>/dev/null +same_file cpio-crc-extract "$work/payload/root/a.txt" "$work/ex-crc/root/a.txt" + +# ---- concatenation: two archives back-to-back list as both segments -------- +cat "$work/out.cpio" "$work/out.cpio" > "$work/cat.cpio" +"$KIT" cpio -t -F "$work/cat.cpio" > "$work/cat.list" 2> "$work/cat.err" +n=$(grep -c '^root/a.txt$' "$work/cat.list" || true) +if [ "$n" = 2 ]; then ok cpio-concat-both-segments +else echo "count=$n" > "$work/cat.diag"; not_ok cpio-concat-both-segments "$work/cat.diag"; fi + +# ---- negatives ------------------------------------------------------------- +run_fail cpio-reject-zstd-flag create -o --compress=zstd -F "$work/z.cpio" root +printf '\050\265\057\375 fake zstd payload' > "$work/fake.zst" +run_fail cpio-reject-zstd-magic "$KIT" cpio -t -F "$work/fake.zst" +"$KIT" cpio -t -F "$work/fake.zst" > "$work/zst.out" 2>&1 || true +contains cpio-zstd-message "$work/zst.out" "gzip and lz4 only" +run_fail cpio-d-with-create create -o -d -F "$work/d.cpio" root +run_fail cpio-no-mode "$KIT" cpio -F "$work/out.cpio" +run_fail cpio-create-dotdot "$KIT" cpio -o -F "$work/dd.cpio" ../payload + +# ---- extract-time path-traversal safety ------------------------------------ +# Hand-craft a newc archive whose sole member is named "../escape" and confirm +# extract refuses it (no file is written outside the destination). The 110-byte +# header is built field-by-field (magic + 13 x 8-hex: ino mode uid gid nlink +# mtime filesize devmajor devminor rdevmajor rdevminor namesize check) so the +# field widths can't drift. +ev="$work/evil.cpio" +cpio_hdr() { printf '%s' "$1"; shift; for f in "$@"; do printf '%08X' "$f"; done; } +{ + # mode 0100644 = 33188; name "../escape\0" = 10 bytes; data "pwn" = 3 bytes. + cpio_hdr 070701 0 33188 0 0 1 0 3 0 0 0 0 10 0 + printf '../escape\000' # 10-byte name; 110+10=120 is 4-aligned + printf 'pwn\000' # 3 data bytes + 1 pad to 4-align + cpio_hdr 070701 0 0 0 0 1 0 0 0 0 0 0 11 0 + printf 'TRAILER!!!\000\000\000\000' # 11-byte name + 3 pad +} > "$ev" +mkdir -p "$work/exsafe" +( cd "$work/exsafe" && "$KIT" cpio -i -F "$ev" ) > "$work/safe.out" 2>&1 && safe_rc=0 || safe_rc=1 +if [ "$safe_rc" = 1 ] && [ ! -e "$work/escape" ] && [ ! -e "$work/exsafe/escape" ]; then + ok cpio-extract-refuses-dotdot +else + { echo "rc=$safe_rc"; cat "$work/safe.out"; } > "$work/safe.diag" + not_ok cpio-extract-refuses-dotdot "$work/safe.diag" +fi +# List still inspects the crafted name (read path is not blocked). +"$KIT" cpio -t -F "$ev" > "$work/evil.list" 2>/dev/null +contains cpio-list-shows-dotdot "$work/evil.list" "../escape" + +# ---- optional cross-check against the host cpio ---------------------------- +if [ "${KIT_CPIO_TEST_HOST:-0}" = 1 ] && command -v cpio >/dev/null 2>&1; then + if cpio -t < "$work/out.cpio" > "$work/host.list" 2>/dev/null; then + contains cpio-host-lists-kit "$work/host.list" "root/a.txt" + else + not_ok cpio-host-lists-kit + fi + mkdir -p "$work/hostex" + ( cd "$work/hostex" && cpio -id < "$work/out.cpio" ) 2>/dev/null + same_file cpio-host-extracts-kit "$work/payload/root/a.txt" \ + "$work/hostex/root/a.txt" +else + skip_test cpio-host-lists-kit "KIT_CPIO_TEST_HOST!=1 or host cpio absent" + skip_test cpio-host-extracts-kit "KIT_CPIO_TEST_HOST!=1 or host cpio absent" +fi + +kit_summary cpio-driver +kit_exit