Helluvatime Release Control Plane
Multi-surface release control plane (Caddy·Node·React·MySQL·Docker) orchestrating build → stage → promote → rollback → canary deployments across product and internal services.
A production release control plane routing builds through stage and promote phases across multiple surfaces (web UI, API server, desktop, services, schema) with a Releases dashboard, canary + live build resolution at the edge, and autonomous triage-to-fix→PR→merge pipelines driven by headless Claude CLI sessions.
- TypeScript
- Node 24
- React 19
- Vite
- Tailwind CSS 4
- Express
- MySQL 8
- Docker
- Caddy
- Discord.js
- Octokit
- WebSocket
Helluvatime is the orchestration backbone: every push to main lands as a build in the admin Releases panel. The control plane (Express + MySQL) oversees the release spine (build → stage → promote → rollback), maintains the source of truth in release.surfaces.json, and triggers helper scripts for each surface kind. A Tier-3 dispatcher at the edge (apex helluvatime.com) per-request decides which build tree each visitor sees (live vs. admin's canary pin), probes health through scheduled tasks, and fallback pages on maintenance/outage.
Six surfaces feed through the pipeline: database (schema + seed), server (API/WS container), web (static marketing), admin (SPA Releases UI), desktop (Electron installer), and two first-class service surfaces. The discord-bot syncs GitHub issues with a Discord feedback forum and hosts an admin override panel; the issue-fixer polls issues, triages them with Claude, spins a headless Claude CLI session in a bind-mounted worktree, and opens (or auto-merges) fix PRs. Both own their own databases and are promoted/rolled back by the orchestrator like any other service.
Live deployment runs in Docker on a VPS — control + dispatcher + caddy + mysql in compose, with `promote-surface.mjs` building service images and health-polling on promote. The admin Releases UI is a React SPA hot-linked from the file system so promotions (symlink swap) are live without restart. Error budgets feed auto-rollback; canary assignments are signed cookies pinning test admins to unstaged builds; and a full audit trail of every promote/rollback/config action sits in the control database.
- Edge-driven multi-surface orchestration: one release.surfaces.json config feeds control API, helper scripts, dispatcher, admin UI, and CI matrix — add a surface, everything auto-discovers it.
- Canary + live build resolution at the apex: per-request cookie verification, signed table-map forwarding for Tier-4 sandbox schemas, kill switches (maintenance sentinel, outage flag), and SPA fallback with asset-pool.
- Autonomous triage-to-deploy loop: discord-bot ↔ GitHub issue ↔ issue-fixer (headless Claude in bind-mounted worktree), direct Discord thread posts, admin override panel, and control-plane auto-promotion.
- Idempotent per-surface deployment: static (asset-pool merge + symlink swap), service (Docker build/recreate/health-poll), schema (idempotent SQL with Tier-4 destructive-schema sandbox snapshots), artifact (upload publish).
- Minimal surface: zero nginx, zero PHP — Caddy handles HTTPS + edge routing; Express hosts control API + SPA; dispatcher is ~300 LOC TypeScript proxying to server surface.