Skip to content
oto
Menu
Overview
IntroductionConceptsArchitectureContributingReleasing
Design
oto — System Design ProposalDomain Research: Prometheus Alertmanager + Slackoto — Red-Team Memooto — SCOPE BOUNDARY (the doctrine)oto — BINDING SPECIFICATION v1
ADRs
0001 — PostgreSQL is the sole datastore; River is the job queue0002 — `internal/<domain>/{api,service,repository,domain}` with three model sets0003 — Alert, AlertOccurrence and AlertEvent are three distinct entities0004 — oto owns alert identity; Alertmanager's fingerprint is recomputed, not trusted0005 — A durable group key, and the AlertGroup generation owns the Slack thread0006 — The Alertmanager API v2 reconciler is mandatory, not optional0007 — Webhook response contract: 202 or 503. Never 4xx, never 429.0008 — Slack: `chat.update` in place is primary; threads are secondary; one colour attachment0009 — Rule definitions are snapshotted and versioned at fire time0010 — Server-Sent Events with durable resume, not WebSocket0011 — Validation is seven layers with two opposite trust models0012 — Pastel product chrome, saturated colour reserved exclusively for state0013 — oto is alert-first: the Flight Recorder Test defines the scope boundary0014 — Postgres is the only store; no TSDB or column store in v10015 — No stream processor; River in Postgres is the pipeline0016 — Cluster context arrives via MCP, on demand; never the Kubernetes event firehose0017 — Alertmanager-style matchers, not CEL; and a filter AST that keeps the door open0018 — Slack distribution: a customer-built internal app, credentials in config0019 — MIT licence0020 — Broadcast the transitions that must be seen0021 — Correctness and testing strategy0022 — Length-prefixed identity pre-images0023 — Terminal thread states are decided first, and the send is three phases0024 — Retention defaults, what each boundary destroys, and cold storage0025 — The alert list resolves rule snapshots in a batch, not per row0026 — `refire_grace` and the flap thresholds, derived from a real rule corpus0027 — Delivery drills: a synthetic alert through the real pipeline, and the mark that keeps it out of the numbers0028 — Non-urgency motion: one-shot events, honest activity loops, and nothing else0029 — Type and radius scales, read off the product rather than drawn for it0030 — Tier A repainted as Washi & Ink / Konshi Sutra; accent moves to beni0031 — Sharp corners and higher-contrast grid lines within Washi & Ink; colours unchanged0032 — Dark-theme accent moves from beni to plum-blossom blue-and-white; light untouched0033 — The alert filters are a toolbar above the table, not a panel in the rail0034 — Notifications is a destination, and the record it keeps is readable0035 — Decorative ink as a masked wash: one primitive, four motifs, and the placements it is refused0036 — AlertOccurrence becomes AlertCase, and `case` is argued against FR-1 by name0037 — A Wording is Liquid, and the structure around it stays oto's0038 — The group key is derived from the alert's own labels0039 — Group close stays activity-driven0040 — A Case is `open` or `closed`, and it is never reopened0041 — The Alert/Case allocation, and the one rule that decides it0042 — Storm damping is removed, because the defence was built before its object0043 — The Slack action row renders five elements, and the fifth is the snooze pair0044 — A count condition is a silence the operator asked for, and `below_threshold` is how oto says so0045 — A Case is a conversation, and one Slack thread per alert is the accepted cost0046 — Corners round again: ADR 0031's radius override is reversed0047 — A channel answers to a connection, and Slack gets two scopes back0048 — A Wording is spelled by its channel, and the marks in between are oto's0049 — A Wording is selected by its own clause, not by a route0050 — A NotificationTemplate is one whole message, and its policy says which one0051 — A `raw` template owns the whole message, and a template may carry its replies0052 — An Incident is drawn over Cases, and its response is handed off0053 — An Investigator reads, proposes, and never decides delivery0054 — A Remedy earns the write path0055 — An incident-tool integration is data or a bridge, never code inside oto0056 — A Case the upstream stopped speaking about expires, and says why
Setup
Configuration referenceReaching an incident tool: the envelope, a mapping, or a bridgeInvestigatorsRemediesConnecting oto to SlackThe live Slack verification runTuning otoConnecting oto to VictoriaMetrics (vmalert)The generic webhook and `oto.notification.v1`
Runbooks
oto runbooksAlert search — optional partial/substring matching on alert namesoto_clock_skew_secondsoto_delivery_claim_lost_totaloto_ingest_accepted_totaloto_ingest_alerts_totaloto_ingest_duplicate_totaloto_ingest_duration_secondsoto_ingest_fingerprint_mismatch_totaloto_ingest_process_duration_secondsoto_ingest_rejected_totaloto_ingest_shed_totaloto_job_duration_secondsoto_job_queue_depthoto_jobs_dead_totaloto_jobs_enqueued_totaloto_jobs_failed_totaloto_jobs_panics_totaloto_jobs_snoozed_totaloto_jobs_started_totaloto_jobs_succeeded_totaloto_jobs_unknown_version_totaloto_render_invalid_totaloto_slack_unknown_action_totaloto_stream_connectionsoto_stream_events_coalesced_totaloto_stream_events_dropped_totaloto_stream_events_fetched_totaloto_stream_events_published_totaloto_stream_fetch_errors_totaloto_stream_listener_reconnects_totaloto_stream_notify_malformed_totaloto_stream_notify_received_totaloto_stream_poll_recovered_totaloto_stream_resync_totaloto_thread_gap_recovered_totaloto_thread_head_wait_secondsoto_thread_order_decisions_total

0030 — Tier A repainted as Washi & Ink / Konshi Sutra; accent moves to beni

Status: Accepted · 2026-08-16 · supersedes the accent clause of ADR 0012 · --oto-border/ --oto-border-strong values amended by ADR 0031 (2026-08-16, same day) · dark-theme accent (--oto-accent and siblings) further amended by ADR 0032 (2026-08-16, same day) — light theme’s beni accent and both themes’ neutrals are unchanged

Context

Section titled “Context”

ADR 0012 fixed the two-tier system — pastel chrome, saturated colour reserved exclusively for state — and picked a periwinkle (#5B54D6 light / #A6A0FF dark) for the Tier A brand accent, chosen because it sat far from every Tier B state hue.

Separately, the product’s marketing surfaces adopted a Japanese-Zen visual language: Washi & Ink for light (sumi ink on cream washi paper) and Konshi Sutra for dark (indigo manuscript paper written on in gold and silver ink), with sakura-branch and kumo-cloud motifs. Bringing the product UI’s chrome into that language means repainting Tier A’s neutrals (bg/surface/border/text) from a cool periwinkle-tinted grey to a warm washi/konshi one, and choosing a new accent to replace the periwinkle.

The obvious move — reuse the marketing palette’s gold (kincha #B8935A light / kindei #C9A668 dark) as the product accent — does not survive contact with Tier B. Gold sits at ~36-38° hue, which is the same hue lane as --oto-state-acked-solid (~36°) and close to --oto-state-expired-solid (~32°). ADR 0012’s whole argument for the periwinkle was “not adjacent to any state hue”; an accent that reads as amber in an alerting tool risks being misread as the acked state itself. Gold works for a marketing hero; it fails Tier A’s actual job.

Decision

Section titled “Decision”

Two changes, kept separate:

1. Tier A neutrals repaint warm. --oto-bg, --oto-surface, --oto-surface-raised, --oto-surface-sunken, --oto-border, --oto-border-strong, --oto-text, --oto-text-muted, --oto-text-subtle move from a cool periwinkle-grey cast to a warm washi-paper cast (light) / a konshi-indigo cast (dark). --oto-chart-grid moves with --oto-border, as before (it has always shared that value). Tier B is untouched — state fills/borders/text/solids keep their exact hex values in both themes; only the chrome around them changed colour temperature.

2. The accent becomes beni, not gold. --oto-accent (and -hover/-fill/-border/-focus) move to a beni crimson-pink — #B5305C light, #EC8CA6 dark — sitting at ~340° hue. That is the widest open lane on the hue wheel relative to all six state hues (nearest neighbour is firing red at ~3°, a ~25° gap; every other state sits 60-300° away). It keeps the sakura/beni family that carries the rest of the Zen language, without landing in acked’s or expired’s territory the way gold would have. Gold stays exactly where it already worked: the marketing site’s hero, wordmark, and spotlight glow, which have no Tier B system to collide with.

Every replacement value clears its §M.4/§M.5 contrast obligation with the same or better margin than the token it replaced — see the updated tables. --oto-state-*-solid vs --oto-bg rows were recomputed (foreground unchanged, background’s luminance shifted) and all still clear ≥ 3:1; the tightest, acked-solid on the new light --oto-bg, holds the same 3.2:1 margin the periwinkle palette had.

Consequences

Section titled “Consequences”
  • The product UI and the marketing site now share a coherent visual language (washi/konshi neutrals, sakura-family accent) without sharing literal token values — the product’s beni is more saturated and hue-shifted from the marketing gold specifically because Tier A here has a job (staying clear of Tier B) that the marketing site does not.
  • Every Tier A hex in tokens.css changed in both themes; every measured pair in SPEC §M.4/§M.5 that touches a Tier A token was re-measured, not just re-stated. contrast.test.ts and tokens.test.ts are the gates that would have caught a mismatch, and both were run green before this landed.
  • --oto-state-* (Tier B), --oto-chart-1…6, --oto-row-h, and the §M.8 type/radius scales are unchanged — this is a chrome-only repaint.
  • ADR 0012’s two-tier architecture and its “chrome never uses a state hue” rule are unchanged and, if anything, are the reason gold was rejected for Tier A in the first place.

Alternatives rejected

Section titled “Alternatives rejected”
  • Reuse the marketing gold (kincha/kindei) as the product accent. Rejected: ~36° hue lands on top of acked (~36°) and near expired (~32°). An accent that can be mistaken for a state colour defeats the entire premise of ADR 0012.
  • Keep the periwinkle accent, repaint only the neutrals. Considered, but leaves the product visually disconnected from the sakura/beni family that carries the rest of the Zen language on every other surface, for no reason stronger than “it already worked.”
  • Invert Washi & Ink’s light values to build the dark neutrals. Rejected on the marketing side for the same reason it’s rejected here: ink-on-paper depends on the paper being light, and a brightness-flip reads as chalk-on-slate, not indigo manuscript paper. Konshi Sutra’s dark neutrals were tuned independently, the same way the marketing palette was.