Skip to content

The overnight stack: defaults that change behaviour

Part of upgrading from before the overnight stack (#48–#69).

Each of these is on, or different, as soon as the new version starts. To opt out, use hangar config set NAME VALUE, which keeps the value in hangar's database across redeploys, or the hangar-site config map, which wins over the database. Settings marked restart apply after kubectl -n hangar rollout restart deploy/hangar; hangar config set says so. A change to hangar-site always needs that restart: the pod reads it as environment variables when it starts.

What Before → now Opt out
hangar-envd (#60) an exec through the Kubernetes API server (pods/exec) for every command, file and terminal → HANGAR_ENVD=auto: on a sandbox's first use, hangar streams a ~3 MB static daemon into its /tmp over one exec, then reaches it at the pod IP on port 49983. That needs deploy/k8s/envd-network.yaml, which up.sh now applies. If the daemon can't be reached, hangar falls back to exec and tries again after 5 minutes, so nothing breaks; it is just slower. The image build is slower too: the Dockerfile compiles the daemon for x86_64 and aarch64 in a rust:1.97-alpine stage. On a 2-vCPU machine, with the crates already downloaded, one architecture took 2 min 1 s (release, LTO). Expect Cloud Build to take several minutes longer than before. up.sh allows 30. HANGAR_ENVD=off (restart). The build stage stays.
Live turns (#59) Claude Code and Codex turns ran one-shot → they run under hangar's driver, which can steer, interrupt and ask for approvals. That needs python3 ≥ 3.8 in the sandbox; without it a turn falls back to one-shot. Approvals stay auto, meaning no prompts and permission checks off, as before. HANGAR_LIVE_TURNS=false
The uid split (#66) n/a → HANGAR_UID_SPLIT=auto. On GKE it falls back to a single user: GKE Sandbox runs a pod as one non-root user with no capabilities, so hangar logs "turns run … as the agent's own user" and carries on. With required, turns that use ask or deny-risky approvals fail on GKE (#69 added deny-risky). The split only happens on Docker. HANGAR_UID_SPLIT=off
Hardened turns (#69) Docker only; on GKE turns run in one user, as above. An ask or deny-risky turn in a split sandbox starts its harness hangar's way: the binary from the pack or bake, hangar's copy of the environment, and settings hangar writes. Such a turn no longer reads the repository's .claude/settings*.json or .mcp.json, the agent's user settings, or skills, and plugins the environment enables don't load. The environment's own .claude/settings.json (agent_config.files), its mcp: servers and instructions, and the repository's CLAUDE.md still apply. Codex turns run with --disable hooks. Every harness in a split sandbox, auto turns' too, runs with no_new_privs: setuid programs and sudo inside a turn don't gain root. Commands hangar runs as the agent outside a turn (exec, streamed execs, background processes) don't get it. On GKE, ask and deny-risky turns now carry a warning naming what isn't protected there. Move what those turns need from the repository into the environment (steering.md); HANGAR_UID_SPLIT=off turns the split, and all of this, off
Streaming (#68) turns' events were read every session_poll_seconds (5 s) → each running turn's log is followed through hangar-envd (tail -F, one more process per running turn in its sandbox), so events reach readers as the agent writes them; without the daemon, polling as before. New for clients: exec(stream=true), the process MCP tool, SSE routes for exec, background processes, watches and session and job events, and MCP watch resources (streaming.md). Caps per token: 16 open streams, 16 background processes, 8 directory watches; 512 streams in all; per stream 1 MiB/s (a command printing faster to a streamed exec is slowed down) and 64 MiB; event and watch streams close after 12 h. Over a cap: an MCP error or HTTP 429. On GKE the load balancer ends an HTTP response after the BackendConfig's timeoutSec (3600 s), so an SSE stream lasts at most an hour there; clients reconnect with Last-Event-ID. HANGAR_STREAM_TURN_LOGS=false; the caps are HANGAR_STREAM_MAX_PER_TOKEN, HANGAR_STREAM_MAX_TOTAL, HANGAR_STREAM_BYTES_PER_SECOND, HANGAR_STREAM_MAX_BYTES, HANGAR_STREAM_MAX_HOURS, HANGAR_BACKGROUND_MAX_PER_TOKEN, HANGAR_WATCH_MAX_PER_TOKEN
Prompt-cache TTL (#63) the harness default (5 min) → HANGAR_CACHE=auto: a Claude Code turn asks for the 1-hour cache when its session can wait more than 5 minutes for the next message. With the default HANGAR_SESSION_IDLE_MINUTES=30, that is most sessions. 1-hour cache writes cost 2× the input price, against 1.25× for 5-minute writes; reads cost the same. It pays off when a session's next message comes after 5 minutes and before an hour. With a Claude subscription login rather than an API key, nothing is billed per token. pi gets the same policy; Codex and Antigravity cache by themselves. Best-of-N attempts and forks started together now wait up to HANGAR_CACHE_STAGGER_SECONDS (15) for the first one to write the shared prefix. HANGAR_CACHE=5m (or off), HANGAR_CACHE_STAGGER_SECONDS=0
Verify inside the turn (#65) a job's verify ran after the turn → it also runs inside the turn, and a failure goes back to the agent, up to HANGAR_MAX_VERIFY_ROUNDS (3) times per turn. Turns can take longer and use more tokens. While the gate is on, a job whose diff changes the check's own test files is delivered as a draft PR, with the files listed in result.touched_tests. HANGAR_VERIFY_IN_LOOP=false, or verify_in_loop: false per job
PR auto-fix (#65) n/a → on for every repo (ci,reviews). A failing check on a job's own PR, or a trusted reviewer's "changes requested" or inline comments, starts a follow-up turn on that job's session (which costs money). There are at most HANGAR_AUTOFIX_MAX_ROUNDS (3) rounds per PR, 120 s apart. It only fires once the webhook sends the new events (§4). hangar config set HANGAR_GITHUB_AUTOFIX '{"owner/repo": "off"}' per repo, or "autofix": "off" per job. HANGAR_AUTOFIX_MAX_ROUNDS=0 also stops it, but posts an "Auto-fix stopped" comment the first time.
/metrics (#55) none → GET /metrics on the public URL, for admin tokens and new metrics-scoped tokens only. Traces stay off unless OTEL_* variables name an exporter. HANGAR_METRICS_ENABLED=false (restart)
The E2B-compatible API (#61) none → mounted at /e2b on the public URL, with the sandbox router in front of the app. It is for admin tokens and new sandboxes-scoped tokens, rate-limited per sandboxes token. HANGAR_E2B_API_PATH="" (restart)
Database backups (#57) none → with HANGAR_SESSION_STORE on a gs:// bucket: one ~5 minutes after each start when none is recent, then every 24 h, into gs://<bucket>/_backups/, keeping the newest 14. They are gzipped and encrypted with the vault key. Until the first one exists, hangar doctor warns "none yet". Without a bucket nothing is scheduled, and hangar doctor warns "no scheduled backups of hangar's database": hangar config set HANGAR_BACKUP_HOURS 24 writes them to the PVC, which a PD snapshot then covers too. HANGAR_BACKUP_HOURS=0; HANGAR_BACKUP_KEEP sets the count
Rate limits and caps on the front doors (#50, #56) none → per caller (a jobs or mcp token, a GitHub or Slack user, a webhook source): 30 job submissions or follow-ups a minute (bursts of 10) and 10 unfinished jobs. Each front door (github, slack, webhook, a2a, api, mcp) gets 120 a minute (bursts of 60). Request bodies are capped at 100 MB for a valid token and 64 KB without one. Admin tokens and schedules are exempt. A refused request gets a 429 and is audited. 0 turns one off: HANGAR_JOB_RATE_PER_MINUTE, HANGAR_JOB_SOURCE_RATE_PER_MINUTE, HANGAR_JOB_MAX_ACTIVE_PER_OWNER; size with HANGAR_MAX_REQUEST_MB
Replays (#50) a webhook redelivered with a new delivery id ran again → a signed body seen in the last HANGAR_WEBHOOK_REPLAY_HOURS (168) is a duplicate and does nothing; so is a GitHub redelivery of an issues or issue_comment event that was already handled. HANGAR_WEBHOOK_REPLAY_HOURS
OAuth for MCP clients (#50, #56) any https redirect on the allowed hosts → when a client registers, its redirect must be https, or http to loopback only (localhost, 127.0.0.1, ::1). A custom scheme needs an exact scheme://host entry; cursor://anysphere.cursor-mcp is now allowed by default, so Cursor signs in. An empty HANGAR_OAUTH_REDIRECT_HOSTS now allows no hosts (it used to allow any). /authorize refuses a resource other than HANGAR_PUBLIC_URL or <HANGAR_PUBLIC_URL>/mcp (RFC 8707), so clients must use the same URL as HANGAR_PUBLIC_URL. up.sh sets both from the same domain. Clients registered before the upgrade keep their registration. add entries to HANGAR_OAUTH_REDIRECT_HOSTS (restart)
MCP sessions (#52) stateless → pre-2026 clients get an Mcp-Session-Id, so hangar can ask them questions and send them notifications. Sessions live in the pod's memory: after this upgrade's restart, and after every one since, a client's old session id gets a 404, and the MCP spec has the client start a new session. HANGAR_MCP_STATELESS_HTTP=true (restart)
Slack Connect (#50) anyone in a shared channel → a member of another organization counts only when listed in HANGAR_SLACK_USERS (as <team>/<user> or by user id); Enterprise Grid workspaces go in HANGAR_SLACK_TEAMS. list them
The git gateway (#48) still off unless HANGAR_GATEWAY_URL is set. When it's on: branches under hangar/session/ are reserved for their sessions, a grant gets at most 8 requests at once, and a source address with 30 failed grant checks in a minute gets 429s. —
The Docker network (#62) laptop and single VM only, not GKE: sandboxes go on hangar's own hangar-sandboxes network (bridge hangar0), where they can't reach each other. Running sandboxes stay on their network until they're recreated. HANGAR_DOCKER_NETWORK=bridge (restart)
Shutdown (#57) SIGTERM ended everything at once → long-polls answer straight away, streams and terminals get HANGAR_SHUTDOWN_GRACE_SECONDS (15), then hangar shuts down. The Deployment's terminationGracePeriodSeconds is now 45 (§4). —
I/O threads (#53) Python's default (CPUs + 4) → 32 per CPU, between 64 and 128, for blocking provider calls. HANGAR_IO_THREADS (restart)

Not changed for GKE: the default provider, the runtime class, warm pools, and approvals (auto). Harness packs are the same except Codex's, whose tag changes (§4). Tiers and hot resume (#64) apply only to environments that ask for tier: fast and to the e2b provider, which isn't enabled on the GKE install.