kit

kit
git clone https://git.ryansepassi.com/git/kit.git
Log | Files | Refs | README

commit a292100f526e3ee83fd6f4e76eee1eb3a4694559
parent 296361e67f879675a2e8f178cf6f9850a1a3e5ce
Author: Ryan Sepassi <rsepassi@gmail.com>
Date:   Thu,  4 Jun 2026 11:36:24 -0700

windows: provision llvm-mingw UCRT sysroots for kit-only cross-compile

Add scripts/llvm_mingw_sysroot.sh to download the pinned mstorsjo/llvm-mingw
UCRT release and extract only each target's include/ and lib/ (headers, CRT
objects, import archives) — never the bundled clang/gcc/ld/lld tools. kit
cross-compiles and links x86_64-windows and aarch64-windows entirely on its
own against these sysroots.

- hosted Windows-mingw profile links libucrt.a (UCRT) instead of libmsvcrt.a
  and is renamed windows-mingw-ucrt; cc.c sysroot comment follows suit
- mk/rt.mk wires the x86_64 __chkstk stack-probe into the windows runtime
- mk/test.mk adds a windows-ucrt-sysroots target (provenance-marker driven)
  and points test-coff-windows-ucrt at both runtimes + both smoke scripts
- COFF smokes discover sysroots under build/llvm-mingw/*/ucrt, validate via
  libucrt.a, and gain an optional Windows-VM execution path (scripts/
  windows_vm.sh over SSH); they remain self-skipping with no sysroot
- doc/WINDOWS.md documents provisioning + VM execution

test-coff smokes use kit exclusively (kit cc to compile+link, kit objdump -p
to inspect): 78 + 126 link-level asserts pass for both arches; VM/Wine exec
paths self-skip when unconfigured.

Diffstat:
Adoc/WINDOWS.md | 94+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mdriver/cmd/cc.c | 6++----
Mdriver/lib/hosted.c | 6+++---
Mmk/rt.mk | 1+
Mmk/test.mk | 17++++++++++++++---
Ascripts/llvm_mingw_sysroot.sh | 281+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Ascripts/windows_vm.sh | 224+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mtest/coff/README.md | 11++++++-----
Mtest/coff/windows-system-dlls-smoke.sh | 27++++++++++++++++++++++++++-
Mtest/coff/windows-ucrt-hosted-smoke.sh | 27++++++++++++++++++++++++++-
10 files changed, 677 insertions(+), 17 deletions(-)

diff --git a/doc/WINDOWS.md b/doc/WINDOWS.md @@ -0,0 +1,94 @@ +# Windows Targets + +kit's Windows targets are PE/COFF, 64-bit only: + +- `x86_64-windows` +- `aarch64-windows` + +The hosted profile is MinGW-w64 UCRT via llvm-mingw. kit uses the target +headers, CRT objects, and import libraries from that sysroot; it does not use +llvm-mingw's compiler, assembler, or linker tools for the cross-compile path. + +## UCRT Sysroots + +The pinned sysroot source is mstorsjo/llvm-mingw release `20260602`. + +Provision both target sysroots: + +```sh +scripts/llvm_mingw_sysroot.sh prepare all +``` + +Downloads go under `${XDG_CACHE_HOME:-$HOME/.cache}/kit/llvm-mingw/20260602` +by default. Extracted target sysroots live under: + +```text +build/llvm-mingw/20260602/ucrt/x86_64-w64-mingw32 +build/llvm-mingw/20260602/ucrt/aarch64-w64-mingw32 +``` + +Only each target's `include/` and `lib/` directories are extracted. + +Direct compile examples: + +```sh +build/kit cc -target x86_64-windows \ + --sysroot "$(scripts/llvm_mingw_sysroot.sh path x64)" \ + test.c -o test-x64.exe + +build/kit cc -target aarch64-windows \ + --sysroot "$(scripts/llvm_mingw_sysroot.sh path aarch64)" \ + test.c -o test-arm64.exe +``` + +The opt-in test target provisions the sysroots and runs the hosted PE/COFF +smokes for both architectures: + +```sh +make test-coff-windows-ucrt +``` + +`make test-coff` remains self-skipping when no UCRT sysroot is present, so the +default suite does not download release archives. + +## Windows VMs + +Execution uses existing Windows VMs reachable by OpenSSH. Configure one or both: + +```sh +export KIT_WINDOWS_VM_X64=kit@127.0.0.1 +export KIT_WINDOWS_VM_X64_PORT=2225 + +export KIT_WINDOWS_VM_AARCH64=kit@127.0.0.1 +export KIT_WINDOWS_VM_AARCH64_PORT=2226 +``` + +Optional shared settings: + +```sh +export KIT_WINDOWS_VM_SSH_KEY=$HOME/.ssh/id_ed25519 +export KIT_WINDOWS_VM_SSH_OPTS="-o UserKnownHostsFile=/dev/null" +``` + +Probe a VM: + +```sh +scripts/windows_vm.sh doctor +scripts/windows_vm.sh smoke x64 +scripts/windows_vm.sh smoke aarch64 +``` + +Run a kit-produced executable: + +```sh +scripts/windows_vm.sh run x64 build/probe-x64.exe arg1 arg2 +scripts/windows_vm.sh run aarch64 build/probe-arm64.exe +``` + +The runner uploads the executable over SSH stdin into `%TEMP%`, runs it through +PowerShell, returns the guest exit code, and removes the temporary directory +unless `KIT_WINDOWS_VM_KEEP=1` is set. + +The COFF Windows smoke scripts prefer configured VMs. If no VM endpoint is set, +they fall back to the existing podman/Wine path and self-skip when that is not +available. diff --git a/driver/cmd/cc.c b/driver/cmd/cc.c @@ -830,11 +830,9 @@ static int cc_apply_env(CcOptions* o) { /* Append a default `<sysroot>/lib` to the library search path for * Windows targets. The llvm-mingw UCRT sysroot ships import archives - * such as libkernel32.a, libmsvcrt.a, and the UCRT API-set archives + * such as libkernel32.a, libucrt.a, and the UCRT API-set archives * under <sysroot>/lib; the user-supplied -L list is searched first, - * then this appended default. In this profile libmsvcrt.a is the - * UCRT-flavoured mingw compatibility archive, not a request to import - * literal msvcrt.dll. Sysroot resolution order: + * then this appended default. Sysroot resolution order: * 1. -isysroot / --sysroot on the command line (already in * o->sysroot at this point); * 2. KIT_SYSROOT env var (e.g. .../x86_64-w64-mingw32). diff --git a/driver/lib/hosted.c b/driver/lib/hosted.c @@ -460,7 +460,7 @@ static int hosted_resolve_windows_mingw(const DriverHostedRequest* req, "Windows hosted profile requires --sysroot or KIT_SYSROOT"); return 1; } - plan->profile_name = "windows-mingw"; + plan->profile_name = "windows-mingw-ucrt"; if (hosted_add_incdirs(plan, req->env, dirs) != 0) { driver_errf(req->tool, "out of memory"); return 1; @@ -484,7 +484,7 @@ static int hosted_resolve_windows_mingw(const DriverHostedRequest* req, "libmingwex.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || hosted_add_required_search( plan->after, &plan->nafter, DRIVER_HOSTED_MAX_AFTER, req, dirs, - "libmsvcrt.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || + "libucrt.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || hosted_add_required_search( plan->after, &plan->nafter, DRIVER_HOSTED_MAX_AFTER, req, dirs, "libadvapi32.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || @@ -508,7 +508,7 @@ static int hosted_resolve_windows_mingw(const DriverHostedRequest* req, "libmingwex.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || hosted_add_required_search( plan->after, &plan->nafter, DRIVER_HOSTED_MAX_AFTER, req, dirs, - "libmsvcrt.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || + "libucrt.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0 || hosted_add_required_search( plan->after, &plan->nafter, DRIVER_HOSTED_MAX_AFTER, req, dirs, "libkernel32.a", DRIVER_HOSTED_INPUT_ARCHIVE) != 0) diff --git a/mk/rt.mk b/mk/rt.mk @@ -127,6 +127,7 @@ RT_x86_64-pc-windows_ABI = llp64 RT_x86_64-pc-windows_INT128 = 1 RT_x86_64-pc-windows_CORO = x86_64_win RT_x86_64-pc-windows_HOSTED = 1 +RT_EXTRA_SRCS_x86_64-pc-windows = rt/lib/stack/chkstk_x86_64_win.c RT_i386-linux_TARGET = i386-linux-gnu RT_i386-linux_ABI = ilp32 diff --git a/mk/test.mk b/mk/test.mk @@ -169,7 +169,7 @@ DEFAULT_TEST_TARGETS = \ bootstrap \ test-bootstrap-toy -.PHONY: test $(TEST_TARGETS) +.PHONY: test $(TEST_TARGETS) windows-ucrt-sysroots test: $(DEFAULT_TEST_TARGETS) @@ -577,6 +577,8 @@ COFF_IMPORT_SMOKE_BIN = build/test/pe-import-smoke COFF_IMPORT_MINGW_BIN = build/test/pe-import-mingw COFF_DSO_FORWARDER_BIN = build/test/pe-dso-forwarder COFF_MIXED_ARCHIVE_BIN = build/test/pe-mixed-archive +LLVM_MINGW_SYSROOT_X64_MARKER = build/llvm-mingw/20260602/ucrt/x86_64-w64-mingw32/PROVENANCE +LLVM_MINGW_SYSROOT_AARCH64_MARKER = build/llvm-mingw/20260602/ucrt/aarch64-w64-mingw32/PROVENANCE JIT_RUNNER = build/test/jit-runner PARSE_RUNNER = build/test/parse-runner ASM_RUNNER = build/test/asm-runner @@ -631,6 +633,14 @@ $(COFF_MIXED_ARCHIVE_BIN): test/coff/pe-mixed-archive.c $(LIB_OBJS) @mkdir -p $(dir $@) $(CC) $(HARNESS_CFLAGS) -Isrc test/coff/pe-mixed-archive.c $(LIB_OBJS) -o $@ +$(LLVM_MINGW_SYSROOT_X64_MARKER): scripts/llvm_mingw_sysroot.sh + @bash scripts/llvm_mingw_sysroot.sh prepare x64 + +$(LLVM_MINGW_SYSROOT_AARCH64_MARKER): scripts/llvm_mingw_sysroot.sh + @bash scripts/llvm_mingw_sysroot.sh prepare aarch64 + +windows-ucrt-sysroots: $(LLVM_MINGW_SYSROOT_X64_MARKER) $(LLVM_MINGW_SYSROOT_AARCH64_MARKER) + $(LINK_EXE_RUNNER): test/link/harness/link_exe_runner.c $(LIB_AR) @mkdir -p $(dir $@) $(CC) $(HARNESS_CFLAGS) test/link/harness/link_exe_runner.c $(LIB_AR) -o $@ @@ -672,8 +682,9 @@ test-coff: lib bin rt-aarch64-windows $(ROUNDTRIP_BIN_COFF) $(COFF_IMPORT_SMOKE_ test-coff-mingw-import: lib $(COFF_IMPORT_MINGW_BIN) $(COFF_IMPORT_MINGW_BIN) -test-coff-windows-ucrt: bin rt-aarch64-windows - bash test/coff/windows-ucrt-hosted-smoke.sh +test-coff-windows-ucrt: bin rt-x86_64-pc-windows rt-aarch64-windows windows-ucrt-sysroots + KIT_SYSROOT=$(abspath build/llvm-mingw/20260602/ucrt) bash test/coff/windows-ucrt-hosted-smoke.sh + KIT_SYSROOT=$(abspath build/llvm-mingw/20260602/ucrt) bash test/coff/windows-system-dlls-smoke.sh # The parse/asm/macho harnesses select a cross-target via KIT_TEST_ARCH # (default aa64); the link rt dependency is resolved through the shared diff --git a/scripts/llvm_mingw_sysroot.sh b/scripts/llvm_mingw_sysroot.sh @@ -0,0 +1,281 @@ +#!/usr/bin/env bash +# Download the pinned llvm-mingw UCRT package into the user cache and extract +# only target headers/libs for kit's Windows cross-compile tests. + +set -eu + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +RELEASE="${KIT_LLVM_MINGW_RELEASE:-20260602}" +BASE_URL="${KIT_LLVM_MINGW_BASE_URL:-https://github.com/mstorsjo/llvm-mingw/releases/download/$RELEASE}" + +cache_root() { + if [ -n "${KIT_LLVM_MINGW_DOWNLOAD_DIR:-}" ]; then + printf '%s\n' "$KIT_LLVM_MINGW_DOWNLOAD_DIR" + return 0 + fi + if [ -n "${XDG_CACHE_HOME:-}" ]; then + printf '%s\n' "$XDG_CACHE_HOME/kit/llvm-mingw/$RELEASE" + return 0 + fi + if [ -n "${HOME:-}" ]; then + printf '%s\n' "$HOME/.cache/kit/llvm-mingw/$RELEASE" + return 0 + fi + printf 'llvm-mingw-sysroot: set XDG_CACHE_HOME, HOME, or KIT_LLVM_MINGW_DOWNLOAD_DIR\n' >&2 + exit 1 +} + +DL_ROOT="$(cache_root)" +OUT_ROOT="${KIT_LLVM_MINGW_ROOT:-$ROOT/build/llvm-mingw/$RELEASE/ucrt}" + +usage() { + cat <<EOF +usage: scripts/llvm_mingw_sysroot.sh <command> [arch] + +commands: + doctor print selected package and tool availability + fetch download and verify the host llvm-mingw UCRT package + prepare [arch|all] extract target include/ and lib/ dirs (default: all) + path <arch> print the extracted target sysroot path + env <arch> print a KIT_SYSROOT export for the target sysroot + +arches: + x64 | x86_64 | amd64 | aarch64 | arm64 | aa64 | all + +downloads: + $DL_ROOT + +extracted sysroots: + $OUT_ROOT +EOF +} + +die() { + printf 'llvm-mingw-sysroot: %s\n' "$*" >&2 + exit 1 +} + +canon_arch() { + case "${1:-all}" in + x64|x86_64|amd64) echo x64 ;; + aarch64|arm64|aa64) echo aarch64 ;; + all|"") echo all ;; + *) die "unknown arch '${1:-}'" ;; + esac +} + +target_triple() { + case "$(canon_arch "$1")" in + x64) echo x86_64-w64-mingw32 ;; + aarch64) echo aarch64-w64-mingw32 ;; + *) die "target_triple requires a concrete arch" ;; + esac +} + +host_package() { + local os mach + os="$(uname -s 2>/dev/null || echo unknown)" + mach="$(uname -m 2>/dev/null || echo unknown)" + case "$os" in + Darwin) + echo "llvm-mingw-$RELEASE-ucrt-macos-universal.tar.xz|d3310f9b86b368900850af8bc95da20648f9fc6b0be3bb64dbf8fb18d7c0894f|tar" + ;; + Linux) + case "$mach" in + x86_64|amd64) + echo "llvm-mingw-$RELEASE-ucrt-ubuntu-22.04-x86_64.tar.xz|9d191203f9768ead60662d3ae53cdf28e0a28b1e6d44b7f329b9202cb2add337|tar" + ;; + aarch64|arm64) + echo "llvm-mingw-$RELEASE-ucrt-ubuntu-22.04-aarch64.tar.xz|e71b61c968f65f94ed3878ca22ab663d7854d91c053c1b8a824ac2f1c9a18503|tar" + ;; + *) + die "unsupported Linux host architecture: $mach" + ;; + esac + ;; + MINGW*|MSYS*|CYGWIN*) + case "$mach" in + x86_64|amd64) + echo "llvm-mingw-$RELEASE-ucrt-x86_64.zip|3de3eda9377bbaf35f8c9001f190380f63b8ee981fa55d3ae9d7cce7c6ad7c70|zip" + ;; + aarch64|arm64) + echo "llvm-mingw-$RELEASE-ucrt-aarch64.zip|cb5c20fbe1808e31ada5cbe4efd9daa2fee19c91dac6ec5ca1ac46a9c7247e37|zip" + ;; + *) + die "unsupported Windows host architecture: $mach" + ;; + esac + ;; + *) + die "unsupported host OS: $os" + ;; + esac +} + +pkg_field() { + local n=$1 pkg=$2 + printf '%s\n' "$pkg" | awk -F'|' -v n="$n" '{ print $n }' +} + +sha256_file() { + if command -v shasum >/dev/null 2>&1; then + shasum -a 256 "$1" | awk '{ print $1 }' + return 0 + fi + if command -v sha256sum >/dev/null 2>&1; then + sha256sum "$1" | awk '{ print $1 }' + return 0 + fi + die "missing shasum or sha256sum" +} + +verify_archive() { + local path=$1 expect=$2 got + got="$(sha256_file "$path")" + [ "$got" = "$expect" ] || + die "checksum mismatch for $(basename "$path"): got $got want $expect" +} + +archive_path() { + local pkg name + pkg="$(host_package)" + name="$(pkg_field 1 "$pkg")" + printf '%s\n' "$DL_ROOT/$name" +} + +fetch_package() { + local pkg name digest url path + pkg="$(host_package)" + name="$(pkg_field 1 "$pkg")" + digest="$(pkg_field 2 "$pkg")" + url="$BASE_URL/$name" + path="$DL_ROOT/$name" + mkdir -p "$DL_ROOT" + if [ -f "$path" ]; then + verify_archive "$path" "$digest" + printf 'archive already verified: %s\n' "$path" + return 0 + fi + command -v curl >/dev/null 2>&1 || die "curl not found" + printf 'download: %s\n' "$url" + curl -fL --retry 3 --retry-delay 2 --continue-at - -o "$path.part" "$url" + mv "$path.part" "$path" + verify_archive "$path" "$digest" + printf 'verified: %s\n' "$path" +} + +archive_top() { + local path=$1 kind=$2 + case "$kind" in + tar) tar -tf "$path" | sed -n '1s#/.*##p' ;; + zip) unzip -Z1 "$path" | sed -n '1s#/.*##p' ;; + esac +} + +extract_arch() { + local arch triple pkg kind path top tmp out + arch="$(canon_arch "$1")" + [ "$arch" != all ] || die "extract_arch requires a concrete arch" + triple="$(target_triple "$arch")" + pkg="$(host_package)" + kind="$(pkg_field 3 "$pkg")" + path="$(archive_path)" + fetch_package + top="$(archive_top "$path" "$kind")" + [ -n "$top" ] || die "could not determine archive top directory" + tmp="$OUT_ROOT/.extract-$triple" + out="$OUT_ROOT/$triple" + rm -rf "$tmp" "$out.tmp" + mkdir -p "$tmp" "$OUT_ROOT" + case "$kind" in + tar) + tar -xf "$path" -C "$tmp" \ + "$top/$triple/include" \ + "$top/$triple/lib" \ + "$top/generic-w64-mingw32/include" + ;; + zip) + command -v unzip >/dev/null 2>&1 || die "unzip not found" + unzip -q "$path" \ + "$top/$triple/include*" \ + "$top/$triple/lib/*" \ + "$top/generic-w64-mingw32/include/*" \ + -d "$tmp" + ;; + esac + [ -r "$tmp/$top/$triple/include/windows.h" ] || + die "archive did not contain readable $triple/include/windows.h" + [ -d "$tmp/$top/$triple/lib" ] || + die "archive did not contain $triple/lib" + if [ -d "$tmp/$top/generic-w64-mingw32" ]; then + rm -rf "$OUT_ROOT/generic-w64-mingw32.tmp" + mv "$tmp/$top/generic-w64-mingw32" "$OUT_ROOT/generic-w64-mingw32.tmp" + rm -rf "$OUT_ROOT/generic-w64-mingw32" + mv "$OUT_ROOT/generic-w64-mingw32.tmp" "$OUT_ROOT/generic-w64-mingw32" + fi + mv "$tmp/$top/$triple" "$out.tmp" + rm -rf "$out" + mv "$out.tmp" "$out" + rm -rf "$tmp" + { + printf 'release=%s\n' "$RELEASE" + printf 'asset=%s\n' "$(basename "$path")" + printf 'url=%s/%s\n' "$BASE_URL" "$(basename "$path")" + printf 'sha256=%s\n' "$(pkg_field 2 "$pkg")" + printf 'target=%s\n' "$triple" + } > "$out/PROVENANCE" + printf 'sysroot ready: %s\n' "$out" +} + +prepare() { + local arch + arch="$(canon_arch "${1:-all}")" + if [ "$arch" = all ]; then + extract_arch x64 + extract_arch aarch64 + else + extract_arch "$arch" + fi +} + +sysroot_path() { + local arch triple + arch="$(canon_arch "$1")" + [ "$arch" != all ] || die "path requires a concrete arch" + triple="$(target_triple "$arch")" + printf '%s\n' "$OUT_ROOT/$triple" +} + +doctor() { + local pkg + pkg="$(host_package)" + printf 'host: %s/%s\n' "$(uname -s 2>/dev/null)" "$(uname -m 2>/dev/null)" + printf 'release: %s\n' "$RELEASE" + printf 'asset: %s\n' "$(pkg_field 1 "$pkg")" + printf 'download cache: %s\n' "$DL_ROOT" + printf 'extract root: %s\n' "$OUT_ROOT" + for tool in curl tar shasum sha256sum unzip; do + if command -v "$tool" >/dev/null 2>&1; then + printf ' OK %s (%s)\n' "$tool" "$(command -v "$tool")" + else + printf ' MISSING %s\n' "$tool" + fi + done + for arch in x64 aarch64; do + printf ' %-7s %s\n' "$arch" "$(sysroot_path "$arch")" + done +} + +cmd="${1:-}" +case "$cmd" in + doctor) doctor ;; + fetch) fetch_package ;; + prepare) shift; prepare "${1:-all}" ;; + path) [ $# -eq 2 ] || { usage; exit 2; }; sysroot_path "$2" ;; + env) + [ $# -eq 2 ] || { usage; exit 2; } + printf 'export KIT_SYSROOT=%s\n' "$(sysroot_path "$2")" + ;; + -h|--help|help|"") usage ;; + *) usage; exit 2 ;; +esac diff --git a/scripts/windows_vm.sh b/scripts/windows_vm.sh @@ -0,0 +1,224 @@ +#!/usr/bin/env bash +# Run kit-produced Windows executables inside configured Windows VMs over SSH. + +set -eu + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +POWERSHELL="${KIT_WINDOWS_VM_POWERSHELL:-powershell.exe}" + +usage() { + cat <<EOF +usage: scripts/windows_vm.sh <command> [args...] + +commands: + doctor print local tools and configured VM endpoints + smoke <arch> run a small PowerShell probe in the VM + run <arch> exe [args] upload exe to the VM, run it, then remove it + +arches: + x64 | x86_64 | amd64 | aarch64 | arm64 | aa64 + +env: + KIT_WINDOWS_VM_X64 SSH destination for the x64 Windows VM + KIT_WINDOWS_VM_AARCH64 SSH destination for the arm64 Windows VM + KIT_WINDOWS_VM_X64_PORT / KIT_WINDOWS_VM_AARCH64_PORT + optional SSH ports + KIT_WINDOWS_VM_SSH_KEY optional private key + KIT_WINDOWS_VM_SSH_OPTS optional extra ssh options + KIT_WINDOWS_VM_KEEP keep uploaded temp dirs when set non-empty + +Example: + KIT_WINDOWS_VM_X64=kit@127.0.0.1 KIT_WINDOWS_VM_X64_PORT=2225 \\ + scripts/windows_vm.sh run x64 build/probe.exe +EOF +} + +die() { + printf 'windows-vm: %s\n' "$*" >&2 + exit 1 +} + +canon_arch() { + case "${1:-}" in + x64|x86_64|amd64) echo x64 ;; + aarch64|arm64|aa64) echo aarch64 ;; + *) die "unknown arch '${1:-}'" ;; + esac +} + +env_get() { + local name=$1 + printf '%s\n' "${!name:-}" +} + +vm_dest() { + case "$(canon_arch "$1")" in + x64) + if [ -n "${KIT_WINDOWS_VM_X64:-}" ]; then + printf '%s\n' "$KIT_WINDOWS_VM_X64" + else + printf '%s\n' "${KIT_WINDOWS_VM_AMD64:-}" + fi + ;; + aarch64) + if [ -n "${KIT_WINDOWS_VM_AARCH64:-}" ]; then + printf '%s\n' "$KIT_WINDOWS_VM_AARCH64" + else + printf '%s\n' "${KIT_WINDOWS_VM_ARM64:-}" + fi + ;; + esac +} + +vm_port() { + case "$(canon_arch "$1")" in + x64) + if [ -n "${KIT_WINDOWS_VM_X64_PORT:-}" ]; then + printf '%s\n' "$KIT_WINDOWS_VM_X64_PORT" + else + printf '%s\n' "${KIT_WINDOWS_VM_AMD64_PORT:-}" + fi + ;; + aarch64) + if [ -n "${KIT_WINDOWS_VM_AARCH64_PORT:-}" ]; then + printf '%s\n' "$KIT_WINDOWS_VM_AARCH64_PORT" + else + printf '%s\n' "${KIT_WINDOWS_VM_ARM64_PORT:-}" + fi + ;; + esac +} + +ssh_setup() { + local arch="$1" port key opts + SSH_DEST="$(vm_dest "$arch")" + [ -n "$SSH_DEST" ] || die "no VM configured for $(canon_arch "$arch")" + SSH_ARGS=() + port="$(vm_port "$arch")" + key="${KIT_WINDOWS_VM_SSH_KEY:-}" + opts="${KIT_WINDOWS_VM_SSH_OPTS:-}" + if [ -n "$opts" ]; then + # Intentional word-splitting: this is a user-provided ssh option string. + # shellcheck disable=SC2206 + SSH_ARGS=($opts) + fi + if [ -n "$key" ]; then + SSH_ARGS=("${SSH_ARGS[@]}" -i "$key") + fi + if [ -n "$port" ]; then + SSH_ARGS=("${SSH_ARGS[@]}" -p "$port") + fi + SSH_ARGS=("${SSH_ARGS[@]}" -o BatchMode=yes -o StrictHostKeyChecking=accept-new) +} + +ps_sq() { + printf '%s' "$1" | sed "s/'/''/g" +} + +b64_one_line() { + base64 "$1" | tr -d '\n' +} + +b64_arg() { + printf '%s' "$1" | base64 | tr -d '\n' +} + +ps_arg_array() { + local first=1 arg enc + printf '@(' + for arg in "$@"; do + enc="$(b64_arg "$arg")" + if [ "$first" -eq 0 ]; then printf ','; fi + first=0 + printf "'%s'" "$enc" + done + printf ')' +} + +ps_env_assignments() { + local names name val + names="${KIT_WINDOWS_VM_ENV_VARS:-KIT_WIN_PROBE}" + for name in $names; do + val="$(env_get "$name")" + if [ -n "${!name+x}" ]; then + printf '$env:%s = '\''%s'\''; ' "$name" "$(ps_sq "$val")" + fi + done +} + +remote_ps() { + ssh "${SSH_ARGS[@]}" "$SSH_DEST" "$POWERSHELL" -NoProfile -ExecutionPolicy Bypass -Command "$1" +} + +remote_ps_stdin() { + ssh "${SSH_ARGS[@]}" "$SSH_DEST" "$POWERSHELL" -NoProfile -ExecutionPolicy Bypass -Command "$1" +} + +remote_mkdir() { + local token ps + token="kit-vm-$(date +%Y%m%d%H%M%S)-$$-$RANDOM" + ps="\$ErrorActionPreference='Stop'; \$d=Join-Path \$env:TEMP '$(ps_sq "$token")'; New-Item -ItemType Directory -Force -Path \$d | Out-Null; [Console]::Out.Write(\$d)" + remote_ps "$ps" +} + +remote_cleanup() { + local dir=$1 ps + [ -n "${KIT_WINDOWS_VM_KEEP:-}" ] && return 0 + ps="\$d='$(ps_sq "$dir")'; if (Test-Path -LiteralPath \$d) { Remove-Item -LiteralPath \$d -Recurse -Force }" + remote_ps "$ps" >/dev/null 2>&1 || true +} + +run_exe() { + local arch="$1" exe="$2" destdir base upload_ps run_ps args_ps env_ps rc + shift 2 + [ -f "$exe" ] || die "exe not found: $exe" + command -v ssh >/dev/null 2>&1 || die "ssh not found" + command -v base64 >/dev/null 2>&1 || die "base64 not found" + ssh_setup "$arch" + destdir="$(remote_mkdir)" + base="$(basename "$exe")" + upload_ps="\$ErrorActionPreference='Stop'; \$p=Join-Path '$(ps_sq "$destdir")' '$(ps_sq "$base")'; \$b=[Console]::In.ReadToEnd(); [IO.File]::WriteAllBytes(\$p, [Convert]::FromBase64String(\$b))" + if ! b64_one_line "$exe" | remote_ps_stdin "$upload_ps"; then + remote_cleanup "$destdir" + return 1 + fi + args_ps="$(ps_arg_array "$@")" + env_ps="$(ps_env_assignments)" + run_ps="\$ErrorActionPreference='Stop'; ${env_ps}\$exe=Join-Path '$(ps_sq "$destdir")' '$(ps_sq "$base")'; \$argv_b64=$args_ps; \$argv=@(); foreach (\$a in \$argv_b64) { \$argv += [Text.Encoding]::UTF8.GetString([Convert]::FromBase64String(\$a)) }; & \$exe @argv; \$code=\$LASTEXITCODE; if (\$null -eq \$code) { \$code=0 }; exit \$code" + set +e + remote_ps "$run_ps" + rc=$? + set -e + remote_cleanup "$destdir" + return "$rc" +} + +smoke_arch() { + local arch="$1" ps + ssh_setup "$arch" + ps="\$ErrorActionPreference='Stop'; cmd.exe /c ver; [Console]::WriteLine('PROCESSOR_ARCHITECTURE=' + \$env:PROCESSOR_ARCHITECTURE); [Console]::WriteLine('PROCESSOR_ARCHITEW6432=' + \$env:PROCESSOR_ARCHITEW6432)" + remote_ps "$ps" +} + +doctor() { + printf 'host: %s/%s\n' "$(uname -s 2>/dev/null)" "$(uname -m 2>/dev/null)" + printf 'repo: %s\n' "$ROOT" + for tool in ssh base64 sed; do + if command -v "$tool" >/dev/null 2>&1; then + printf ' OK %s (%s)\n' "$tool" "$(command -v "$tool")" + else + printf ' MISSING %s\n' "$tool" + fi + done + printf ' x64 dest=%s port=%s\n' "$(vm_dest x64)" "$(vm_port x64)" + printf ' aarch64 dest=%s port=%s\n' "$(vm_dest aarch64)" "$(vm_port aarch64)" +} + +cmd="${1:-}" +case "$cmd" in + doctor) doctor ;; + smoke) [ $# -eq 2 ] || { usage; exit 2; }; smoke_arch "$2" ;; + run) [ $# -ge 3 ] || { usage; exit 2; }; arch="$2"; exe="$3"; shift 3; run_exe "$arch" "$exe" "$@" ;; + -h|--help|help|"") usage ;; + *) usage; exit 2 ;; +esac diff --git a/test/coff/README.md b/test/coff/README.md @@ -25,7 +25,8 @@ make test-coff This builds `build/test/kit-roundtrip-coff` and runs the embedded unit cases. It also runs `windows-ucrt-hosted-smoke.sh`, which self-skips unless an llvm-mingw UCRT sysroot is available via -`KIT_SYSROOT` or under `/tmp/llvm-mingw*`. Wine is not needed. +`KIT_SYSROOT`, under `build/llvm-mingw/*/ucrt`, or under `/tmp/llvm-mingw*`. +Windows VM execution is optional; see `doc/WINDOWS.md`. ## Layers @@ -36,8 +37,8 @@ self-skips unless an llvm-mingw UCRT sysroot is available via - **C** (cases) — mingw-cross-built `.obj` fixtures. Layer B. The hosted UCRT smoke now covers one aarch64 llvm-mingw sysroot path; broader fixture coverage remains pending. -- **E** (exec) — link + exec via Wine. Layer C/D, gated on Wine - availability (`doc/WINDOWS.md` Phase 3). +- **E** (exec) — link + exec via configured Windows VMs, with Wine as a + fallback when available. Layer A is sufficient to gate the wire encoder / decoder against each other. Layers B/C/D will catch cross-tool agreement and @@ -46,5 +47,5 @@ lands. ## Pointer -See `doc/WINDOWS.md` for the full PE/COFF support plan, including -the Phase-by-Phase task list, ABI notes, and corpus stratification. +See `doc/WINDOWS.md` for the UCRT sysroot provisioning and VM execution +commands. diff --git a/test/coff/windows-system-dlls-smoke.sh b/test/coff/windows-system-dlls-smoke.sh @@ -39,6 +39,7 @@ find_sdk() { local arch=$1 local d for d in \ + "$ROOT"/build/llvm-mingw/*/ucrt/"$arch"-w64-mingw32 \ /tmp/llvm-mingw*/llvm-mingw-*-ucrt-*/"$arch"-w64-mingw32 \ /tmp/llvm-mingw*/"$arch"-w64-mingw32 \ /private/tmp/llvm-mingw*/llvm-mingw-*-ucrt-*/"$arch"-w64-mingw32 \ @@ -267,6 +268,30 @@ no_legacy_crt_imports() { run_wine_if_available() { local label=$1 image=$2 pod_arch=$3 exe=$4 shift 4 + case "$pod_arch" in + amd64) + if [ -n "${KIT_WINDOWS_VM_X64:-${KIT_WINDOWS_VM_AMD64:-}}" ]; then + if "$ROOT/scripts/windows_vm.sh" run x64 "$exe" "$@" \ + > "$work/$label-vm.out" 2> "$work/$label-vm.err"; then + ok "$label-vm" + else + not_ok "$label-vm" "$work/$label-vm.err" + fi + return 0 + fi + ;; + arm64) + if [ -n "${KIT_WINDOWS_VM_AARCH64:-${KIT_WINDOWS_VM_ARM64:-}}" ]; then + if "$ROOT/scripts/windows_vm.sh" run aarch64 "$exe" "$@" \ + > "$work/$label-vm.out" 2> "$work/$label-vm.err"; then + ok "$label-vm" + else + not_ok "$label-vm" "$work/$label-vm.err" + fi + return 0 + fi + ;; + esac if ! command -v podman >/dev/null 2>&1; then skip_test "$label-wine" "podman unavailable" return 0 @@ -294,7 +319,7 @@ run_wine_if_available() { # # link-mode is "console" or "windows" (drives -mconsole vs -mwindows). libs is # a space-separated list of `-l<name>` archives to add (e.g. "gdi32 ws2_32") -# beyond the driver-auto-linked set (kernel32/user32/advapi32/shell32/msvcrt/ +# beyond the driver-auto-linked set (kernel32/user32/advapi32/shell32/ucrt/ # mingwex/mingw32/moldname). Each expected DLL/symbol is one mode-P assert. build_and_check() { local label=$1 csrc=$2 exe=$3 dump=$4 mode=$5 libs=$6 diff --git a/test/coff/windows-ucrt-hosted-smoke.sh b/test/coff/windows-ucrt-hosted-smoke.sh @@ -28,6 +28,7 @@ find_sdk() { local arch=$1 local d for d in \ + "$ROOT"/build/llvm-mingw/*/ucrt/"$arch"-w64-mingw32 \ /tmp/llvm-mingw*/llvm-mingw-*-ucrt-*/"$arch"-w64-mingw32 \ /tmp/llvm-mingw*/"$arch"-w64-mingw32 \ /private/tmp/llvm-mingw*/llvm-mingw-*-ucrt-*/"$arch"-w64-mingw32 \ @@ -355,6 +356,30 @@ matches() { run_wine_if_available() { local label=$1 image=$2 pod_arch=$3 exe=$4 shift 4 + case "$pod_arch" in + amd64) + if [ -n "${KIT_WINDOWS_VM_X64:-${KIT_WINDOWS_VM_AMD64:-}}" ]; then + if KIT_WIN_PROBE=present "$ROOT/scripts/windows_vm.sh" run x64 "$exe" "$@" \ + > "$work/$label-vm.out" 2> "$work/$label-vm.err"; then + ok "$label-vm" + else + not_ok "$label-vm" "$work/$label-vm.err" + fi + return 0 + fi + ;; + arm64) + if [ -n "${KIT_WINDOWS_VM_AARCH64:-${KIT_WINDOWS_VM_ARM64:-}}" ]; then + if KIT_WIN_PROBE=present "$ROOT/scripts/windows_vm.sh" run aarch64 "$exe" "$@" \ + > "$work/$label-vm.out" 2> "$work/$label-vm.err"; then + ok "$label-vm" + else + not_ok "$label-vm" "$work/$label-vm.err" + fi + return 0 + fi + ;; + esac if ! command -v podman >/dev/null 2>&1; then skip_test "$label-wine" "podman unavailable" return 0 @@ -400,7 +425,7 @@ for arch in x86_64 aarch64; do fi # Discovered-but-invalid sysroot was a hard FAIL+exit 1 originally. if [ ! -r "$ARCH_SDK/include/windows.h" ] || - [ ! -r "$ARCH_SDK/lib/libmsvcrt.a" ]; then + [ ! -r "$ARCH_SDK/lib/libucrt.a" ]; then echo "invalid UCRT llvm-mingw sysroot: $ARCH_SDK" > "$work/$label-sysroot.diag" not_ok "$LABEL_SUITE/$label-sysroot" "$work/$label-sysroot.diag" kit_summary "$LABEL_SUITE"