Architecture Decision Records

These ADRs record the load-bearing decisions for Scarab, captured in a design grilling session on 2026-07-12. Each is written in a light MADR style: Context → Decision → Consequences → Alternatives considered.

They are the why. The what (ubiquitous language, system overview) lives in ../../CONTEXT.md.

Status lifecycle

Proposed → Accepted → (Superseded by NNNN | Deprecated). All ADRs below are Accepted unless noted. A decision changes by writing a new ADR that supersedes the old one — never by editing history (mirrors our version-tolerance principle).

Index

ADR	Title	Status
0001	CI as durable execution (the wedge) + non-goals	Accepted
0002	Durability model: durable DAG orchestrator + at-least-once steps	Accepted
0003	Durability substrate: own it — DBOS pattern on Postgres (Rust)	Accepted
0004	Execution topology: pod-per-step + content-addressed workspace	Accepted (data path superseded by 0061)
0005	Tenancy & deployment; Kubernetes as the only backend	Accepted
0006	Pipeline ontology: flat recursive DAG	Accepted
0007	Data-passing model: Workspace / Result / Artifact / Cache	Accepted
0008	Step contract: OCI image + convention + built-in kinds	Accepted (amended by 0058)
0009	DSL: typed Pipeline IR + YAML frontend + CEL	Accepted
0010	Forge integration: agnostic core, OIDC identity, in-repo config	Accepted
0011	Scheduling: durable admission control	Accepted
0012	API surface: REST/OpenAPI + SSE + internal gRPC	Accepted
0013	History & observability: state tables + append-only event log	Accepted
0014	Secrets: envelope-encrypted PG + pluggable providers	Accepted
0015	Supply-chain: Scarab as OIDC issuer + provenance-by-construction	Accepted
0016	Code architecture: hexagonal + adapter crates + converged binary	Accepted
0017	Correctness & testing strategy	Accepted
0018	Container image building: rootless BuildKit	Accepted
0019	Local execution: executor-local behind the Executor port	Accepted
0020	Retry & failure taxonomy	Accepted
0021	Double-effect hazard: fencing tokens + idempotency contract	Accepted
0022	Upgrades & schema evolution: version-tolerant from day one	Accepted
0023	DAG shape: static matrix v1, dynamic reserved	Accepted
0024	Environments: first-class with protection rules	Accepted
0025	Cross-pipeline orchestration: nest vs trigger	Accepted (amended by 0038)
0026	Resource & placement model	Accepted (refined by 0055)
0027	Restart semantics: content-addressed smart invalidation	Accepted
0028	UI stack: SolidJS + TS + generated client	Accepted
0029	Workspace content-addressing: per-file merkle CAS	Accepted
0030	Operational defaults (isolation, retention, notifications, install, DR, limits)	Accepted
0031	Purity = no I/O, not no deps: pure-computation crates allowed (amends 0016)	Accepted
0032	Slice 3–5 implementation decisions (forge/identity, scheduler/gates, secrets/OIDC/BuildKit)	Accepted
0033	Transitive skip: when:false step kept-but-skipped, descendants cascade	Accepted
0034	External gate release via HMAC token	Accepted
0035	Explicit workspace outputs:	Accepted
0036	Local execution dev backend	Accepted
0037	Environment governance: approval-as-gate, scoped secrets, admission at point-of-use	Accepted
0038	invoke = compile-time inlining; local .scarab/lib, third-party via OCI images (amends 0025)	Accepted
0039	Privileged step images: hardened baseline + governed capability grants per Environment	Accepted
0041	Named step results + launch-time interpolation (${{ outputs.<id>.<name> }}); makes 0038 outputs live	Accepted
0042	Trusted per-Pod egress sidecar → fence-scoped results API → Postgres (how k8s captures results)	Accepted
0043	Launch parameters (typed interface.inputs) + manual dispatch as a repo-aware trigger	Accepted
0044	Branch protection matches a symbolic ref, not the commit SHA (fixes allowed_refs)	Accepted
0045	Source provisioning: clone step kind — SHA-pinned git clone in a scarab-clone Pod, .git into CAS, read-only fork token	Accepted
0046	Forge auth is adapter-internal; GitHub + Forgejo adapters in v1; ForgeConnection registry; Project = governed repo (amends 0010)	Accepted
0047	Retry classification (FailureClass), never-started vs post-start retry, new-fence-per-retry, dead-letter/timeout model (implements/amends 0020)	Accepted
0048	Fail-closed startup: validated config, boot refusal on unsafe/auth-off, mandatory Postgres (no API-only mode), opt-in SCARAB_DEV_INSECURE	Accepted
0049	Identity & access: forge-agnostic OAuth/OIDC authn + PG session; Scarab-native RBAC (Org/Project scope, forge-import bootstrap); tenancy scoping	Accepted
0050	Retention & GC: mark-sweep workspace-CAS GC, eligibility keyed on run lifecycle (suspended runs never collected), per-class TTLs (implements 0030)	Accepted
0051	Multi-replica: per-step tail lease, replica-agnostic live-SSE (durable index), shared persistent OIDC key	Accepted
0052	Artifacts: dedicated per-run store (not CAS), convention-emitted (/scarab/artifacts/ + globs), presigned download, own TTL	Accepted
0053	Observability & lifecycle: Prometheus /metrics, JSON logs + request-ids, /readyz vs /healthz, graceful shutdown (implements 0030)	Accepted
0054	Product surface: embed UI in the binary, run-cancel API, OpenAPI drift gate, CLI truthfulness (stubs exit non-zero)	Accepted
0055	Placement profiles: named placement_profiles + control-plane baseline + governed k8s_overlay (refines 0026)	Accepted
0056	Run Takes (derived, human-boundary version lens) + attempt-grain evidence (results/workspace/artifacts/consumption keyed by attempt; restart & re-adoption events)	Accepted
0058	Runtime service containers: co-located Sidecar (default, fenced-by-inheritance) + Run-scoped Shared (opt-in uses:, per-Take instance, unfenced) — not DAG nodes (amends 0008)	Accepted
0059	Per-run scheduler-tick fault isolation (one poison Run can’t stall the fleet) + bounded per-Run failures → dead-letter (generalizes 0058 Fix A/B)	Accepted
0060	Global Settings surface: connection lifecycle (Forgejo onboarding + GitHub observe/re-sync), org/env secret editing via the effective-status matrix, IaC-or-UI connections with single-owner precedence	Accepted
0061	Workspace data path: per-AZ workspace service + Scarab node driver with lazy materialisation; control plane leaves the data path (supersedes 0004’s data path, not its topology)	Accepted (part 2, the node driver, superseded by 0062; part 4 amended by 0064)
0062	Workspace Export: lazy materialisation with no DaemonSet — overlayfs over a reflink-or-copy Snapshot Farm on the Data Depot (hardlinks withdrawn on measurement), mounted by kubelet as a per-Step PVC; the change set is the upper layer; optional node-side write accelerator	Accepted
0063	Step logs live on the Data Depot, not the object store and not the CAS: object storage stops being a silent prerequisite, external log systems are additional sinks only, eviction is loud and pinned until a sink acks; renames the workspace service → Data Depot	Accepted
0064	Durability tiering + write path: warm-first with one batched flush that gates Succeeded, a cold tier proven by st_dev (a second PVC counts, a LocalDir beside the CAS does not), graceful warm-only deployments, durability stamped per Attempt (amends 0061 part 4)	Accepted
0065	Retention, Cache (explicit keyed directory cache — implements 0007’s concept), RetentionProfile (operator-defined, author-selectable), re-derivation is human-triggered via the widened rerun with full-cascade disclosure; no automatic Cache detection; checkout is not a special case	Accepted

For features deliberately deferred until demand appears (as opposed to ADRs to be written), see docs/followups.md.

Open / deferred (to be written when their slice begins)

Workspace CAS internals (chunking) · multi-cluster remote-agent protocol · install Operator · OIDC login/session details · API rate-limit specifics · SLSA/cosign/SBOM export · notification providers.