0012 Cutting edge: what to adopt next¶

Status: partly built (2026-10-15): items 1, 3–7 and 15 shipped; the rest isn't scheduled. Was: proposal. · Date: 2026-10-13 · Builds on: 0003 (gateway, egress), 0004 (performance, routing), 0006 (environments), 0007 (policy, experiments, learnings), 0008 (forks, swarms), 0010 (sessions), 0011 (factory roadmap)

The goal is for hangar to be the best way to run coding agents on your own infrastructure, and to give MCP clients cloud sandboxes, with easy setup and current methods. This spec compares hangar with the cloud coding agents and sandbox platforms as of October 2026. It names what to adopt, in what order, and how to tell whether each change worked.

Every external claim carries a source tag such as [S12], resolved in §10. Claims I could not check against a primary source are marked unverified, and §9 lists them. Numbers marked measured come from experiments on this machine (§4.3), whose scripts are in docs/research/0012/.

1. Summary: the decisions¶

1. Vendors now run their agent loops on customer compute, so hangar should be the compute. By October 2026 the major vendors all ship this:
2. Claude Code cloud sessions run on self-hosted runners [S3];
3. Cursor runs cloud agents on self-hosted workers [S19];
4. Devin runs on Outposts [S22];
5. Ona runs on runners in your VPC [S25];
6. Warp Oz offers self-hosted environments [S26].

Each vendor supplies a runner binary that dials out. It needs a machine with the repo's toolchain, and a provisioner to create one per session. That provisioner is exactly what hangar's environments, warm pools, gateway and egress policy already are. Item 2 makes hangar the provisioner, starting with Claude Code's spawn-runner hook. Engineers then get claude.ai, the mobile app, --teleport and auto-fix on hangar's sandboxes, without hangar rebuilding any of that UX. 2. hangar's protocol surface is current; its gaps are in the agent loop. hangar already serves MCP 2026-07-28 with tasks and MRTR elicitation, and A2A 1.0 with 0.3 compatibility (docs/mcp.md, docs/a2a.md). That puts it ahead of most clients. What it lacks is what the hosted products do inside a session: - steering a running turn; - environments that rebuild themselves; - a verify gate inside the agent's loop; - a PR loop that fixes CI failures and review comments; - egress levels, and credentials injected from outside the sandbox; - cache-aware session handling. 3. Prompt caching is the cheapest cost and latency lever, and today hangar works against it. hangar suspends idle sessions after 30 minutes, and agents pause on needs_input for human-scale gaps. Claude Code with an API key uses a 5-minute TTL [S6], and GPT-5.6+ caches live for 30 minutes [S15]. On Opus 5.5, re-reading a 150k-token conversation after the cache expires costs $0.75; reading it from cache costs $0.03 [S11]. Item 1 sets a TTL policy, never changes model mid-session, and staggers best-of-N forks. 4. Memory fork is now cheap on the stack hangar already uses. In a local test, a gVisor checkpoint restored into two independent sandboxes from one image. A 256 MB process restored in 0.3–1.25 s, against a 1.4–2.2 s cold start (measured, §4.3). GKE Pod Snapshots went GA in May 2026 [S41]. The docs show a snapshot restored into a new Pod named by annotation [S40], which may allow forks on GKE, not just pause and resume. Item 8 verifies this live before anything is built on it. 5. Providers. Keep Docker, GKE and E2B. Add the vendor-runner pools from decision 1. Don't add Daytona, Modal, Vercel, Cloudflare or Morph as providers now: each is an E2B-like hosted VM platform, and hangar's value is the owner's own cluster (§4.2). Revisit this if an owner asks for one, through the OpenAI Agents SDK's sandbox interface (item 14), which already speaks to seven of them [S17].

2. Method¶

• Primary sources only: vendor docs, specs, release notes and repos, fetched on 2026-10-13. Secondary sources are used only where noted.
• Local CLIs were checked for flags this spec relies on: Claude Code 2.1.296, Codex 0.162.1 and pi 1.1.0, all in the overnight scratchpad. The Claude Code checks covered self-hosted-runner, orchestrator, --input-format stream-json, --exclude-dynamic-system-prompt-sections and CLAUDE_CODE_PROMPT_CACHE_TTL; the Codex checks covered codex queue and codex app-server.
• Two experiments on this machine: gVisor checkpoint/restore and fork, and dependency install with and without a warm cache (§4.3). This is a shared 2-vCPU, 8 GB VM with other agents running, so treat the timings as relative, not absolute.
• What hangar has today was read from docs/*.md, specs 0001–0011, and src/hangar at origin/overnight/perf.

3. Cloud coding agents¶

3.1 What each does well¶

3.2 What hangar lacks, in priority order¶

1. Hosting other vendors' runners. Claude, Cursor, Devin, Ona and Warp all ask customers for a provisioner. hangar is one but doesn't speak their hooks (item 2).
2. Steering a running turn. Claude queues messages, Codex has turn/steer [S14], Cursor allows follow-ups. hangar refuses a send while the session is busy (item 3).
3. Self-maintaining environments. Every product rebuilds its cached environment by itself: Claude every ~7 days or on change, Devin every ~24 h, Codex caches for 12 h and runs a maintenance script. hangar needs hangar env publish, and has no maintenance step on claim (item 4).
4. PR loop. Claude's auto-fix reacts to CI failures and review comments. hangar acts only on /hangar comments (item 6).
5. Network levels and credentials kept outside the sandbox for everything except git (item 7).
6. Verification inside the agent's loop. hangar verifies after the turn and runs a repair turn. Claude Code's own guidance is a Stop hook that blocks the end of a turn until the check passes [S8] (item 5).
7. Code intelligence. Claude Code cloud sessions don't start language-server plugins [S9]. hangar can, which makes this a differentiator, not just a gap (item 9).
8. Review UX inside the client. The diff, inline comments and an answer form exist on claude.ai and Cursor. MCP Apps can put them in Claude, VS Code and Goose [S47] (item 10).

4. Sandbox infrastructure¶

4.1 Comparison¶

"1 vCPU + 4 GiB for an hour" is the list price for that shape. Hosts that bill CPU only while it is active are noted. GKE's figure is hangar's own estimate: an on-demand n2-standard-8 at about $280/month (docs/sessions.md), split per vCPU with 4 GB each. Spot capacity is cheaper (unverified).

4.2 What to support, and what to borrow¶

• Providers: keep Docker (dev, plus Kata microVMs), GKE (production) and E2B (hosted burst, or self-hosted Firecracker).
• Add pools for vendor runners (item 2). These are not new sandbox backends: they run inside hangar's existing providers.
• Don't add Daytona, Modal, Vercel, Cloudflare or Morph now. Each needs a vendor account and keeps code off the owner's infrastructure, which is the opposite of hangar's purpose. Daytona's public code has also stopped [S33].
• An owner who wants one of these can reach most of them through item 14's Agents SDK interface. The SDK ships Blaxel, Cloudflare, Daytona, E2B, Modal, Runloop and Vercel [S17].
• Borrow for GKE:
• Memory fork through Pod Snapshots (item 8). The how-to restores a named snapshot into a new Pod with a matching spec [S40]. hangar's docs say GKE "restores a sandbox only from its own snapshots", which looks too conservative. It needs a live test. The local gVisor proof in §4.3 shows the mechanism works on the same runtime.
• Lazy restore. Both GKE and Firecracker let the app run before memory is fully loaded [S40][S43]. Measure "first useful event", not "pod Running" (GKE's own advice [S40]).
• An in-sandbox daemon like E2B's envd [S31][S42]: a process, filesystem, PTY and port API inside the sandbox, so hangar doesn't go through the Kubernetes API server for every exec. Agent Sandbox already ships an envd example. Build it only if GKE exec latency turns out to be the bottleneck (item 15). Docker exec p50 today is 75 ms (bench/results/2026-10-13-perf-docker-ab-after.json), with no GKE number yet.
• Uniqueness after restore. Any memory fork must re-randomize what has to stay unique: session ids, HANGAR_* grant tokens, RNG-derived state. GKE says to re-create secrets and read the changed environment from /proc/gvisor/spec_environ [S40]; Firecracker says user-space state is not reseeded [S43]. hangar's per-sandbox gateway grant must be minted after a fork, never inherited.

4.3 Experiments¶

gVisor checkpoint/restore and fork (runsc release-20261005.0, systrap, --network=none).

The workload is a Python process that imports modules and builds N MB of heap, standing in for a warmed language server or test runner. It writes a heartbeat file every 200 ms. Each run checkpoints the process, then restores the same image into two new sandbox IDs, f1 then f2.

Findings:

1. One image can be restored into several independent sandboxes, and each continues on its own. So the memory-fork mechanism under GKE Pod Snapshots works on gVisor itself.
2. Restore time grows with image size, about 1 s per 0.5 GB here. Checkpoint is cheap.
3. Each fork holds its own copy of memory. At 512 MB, two forks plus the host load exhausted this 8 GB VM: f2 took 80 s and f1 died, probably killed for memory (unverified; the kernel log was not checked). Memory forks need admission control on node memory. Their real payoff is state that is expensive to rebuild, such as a language-server index or a warmed dev server. A synthetic heap is cheap to rebuild.

Dependency install with and without a cache. Two runs each. Python used hangar's own uv.lock (187 packages, 210 MB venv). Node used a TypeScript/Vite/Vitest/ESLint/React manifest (76 MB node_modules).

A warm package cache removes 96–99% of install time. With one, an incremental "maintenance" step on claim (item 4) costs well under 3 s, so the baked layer may be days old without hurting start time.

5. Protocols¶

6. Agent effectiveness on hangar¶

6.1 Environment and context engineering¶

• Instructions: keep CLAUDE.md/AGENTS.md short. Claude Code's guidance is that bloated files get ignored, and anything needed only sometimes belongs in a skill [S8]. 0008's evidence points the same way: curated skills gave +16.2 pp, self-generated ones −1.3 pp, and 2–3 focused skills beat comprehensive ones (0008 §2, SkillsBench). hangar's skills: and agent_config.instructions are the right shape. What's missing is measuring them (item 13).
• Cached dependency layers: §4.3. Item 4 makes them keep themselves fresh.
• Language servers: item 9. Install the server binary and the code-intelligence plugin in the image, so the agent gets diagnostics after every edit [S9]. Whether plugins start under claude -p was not verified (no API key here). That is the first test.
• Stable paths: Claude Code's cache is "scoped to one machine and directory". The system prompt carries the working directory, platform, shell and OS [S6]. hangar's fixed /workspace/<repo> checkout and pinned images already give forks the same prefix. Keep it that way.

6.2 Verification loops¶

hangar verifies after the turn (0002, repo.verify) and gives the agent one repair turn. Claude Code's guidance is to make the check a gate the agent can't stop past [S8]: a Stop hook that runs the check and blocks the turn from ending until it passes (with a cap on consecutive blocks), or a /goal condition. Item 5 injects a Stop hook that runs repo.verify in Claude sandboxes. hangar's post-turn verification stays as the referee, because a gate inside the sandbox can be edited by the agent. For Codex and pi, the repair turn remains the mechanism. Ask for evidence (test output) in RESULT.md, as [S8] recommends.

6.3 Learning from outcomes (0007/0008)¶

hangar already records verified outcomes, repair turns, needs_input questions and PR review comments (once item 6 reads them). The cheapest learning loop is ACE-style small deltas to a repo skill, proposed as a PR and A/B-tested with 0007's experiments (0008 §2, §5.3). Item 13 limits this to one source: review comments and repair causes on merged PRs.

6.4 Model routing, cost and latency¶

Current list prices per MTok (input / output) [S11][S16]:

• Anthropic: Opus 5.5 $4 / $20, Sonnet 5.5 $2 / $10, Haiku 5.5 $0.10 / $0.50 (≤100k prompt).
• OpenAI: gpt-6.1-sol $2 / $10, gpt-6-luna $0.10 / $0.50, gpt-5.3-codex $1.75 / $14.

Route per session or job, at creation, never per turn: a model switch makes the next request re-read the whole conversation uncached [S6]. Sensible defaults:

• the strongest model for planning, best-of-N judging and anything with needs_input;
• Sonnet-class models for well-specified issue fixes with a verify command;
• Haiku- or luna-class models for map shards and labeling.

Choose and confirm these with 0007 experiments, on cost per verified task (0004 §5.8). That is item 12.

6.5 Prompt caching and long sessions¶

Facts:

• Anthropic: 5-minute writes cost 1.25× input, 1-hour writes 2×. Reads cost 0.1× generally, and 0.05× on Opus 5.5 and Sonnet 5.5 [S10].
• Claude Code: the main conversation uses a 1-hour TTL only on a subscription within plan usage. API keys and cloud providers get 5 minutes. CLAUDE_CODE_PROMPT_CACHE_TTL=1h or the promptCacheTtl setting overrides this [S6].
• Claude Code: a resumed conversation keeps the system prompt it started with, so a resume in a fresh sandbox can still hit the cache [S6].
• Claude Code: a fan-out holds all but the first agent for up to 5 s so the rest read the first one's cache [S6].
• OpenAI: GPT-5.6+ reads cost 0.1× and writes 1.25× (0.05× reads for gpt-6.1-sol), with a TTL of 30 m [S15].

What a session that returns after a gap costs, at 150k tokens of context (Anthropic prices [S11]):

At about 400k tokens written in a session, the 1-hour TTL pays for itself once there are two gaps of 5–60 minutes. A needs_input round trip with a human, or a follow-up after review, is usually such a gap. Gaps longer than an hour lose the cache either way.

Best-of-N: four forks launched together from a 100k-token prefix each write it: 4 × $0.50 = $2.00 on Opus 5.5. Staggered, one writes and three read: $0.50 + 3 × $0.02 = $0.56. That assumes simultaneous identical requests don't share a write in flight, which is inferred from Claude Code's own fan-out hold [S6] and not measured. Item 1 implements both policies.

7. Prioritised items¶

Effort: S ≤ 1 day, M ≈ 2–3 days, L ≈ 4–6 days (as in the specs README).

8. Recommended next three waves¶

Each wave is a set of parallel workstreams, one PR each, like 0011 §3.

Wave 3: the agent loop. Items 1, 3, 5, 6, and 4(c–e). These are small to medium and need no new infrastructure or owner decision; most are harness flags plus front-door triggers.

• Exit:
• a steered turn;
• a PR that went red and was fixed by the agent without a human;
• a multi-turn session with ≥ 85% cache reads;
• an env whose baked layer rebuilt itself after a lockfile change.

Wave 4: the owner's compute as the platform. Items 2 (Claude first), 7, 8, 9, and 4(a–b).

• Owner decision first: an Anthropic Team or Enterprise org with self-hosted environments, for item 2.
• Exit:
• a claude.ai session running on a hangar GKE sandbox behind the egress proxy;
• no secrets readable in sandboxes;
• a measured yes or no on GKE memory forks;
• language-server diagnostics in Claude sandboxes.

Wave 5: meet users in their tools; learn. Items 10, 11, 12, 13, then 14 and 15 as needed.

• Exit:
• review and answer a session from Claude Desktop (MCP App) and from Zed (ACP);
• routing on by default with a measured cost drop;
• a first learned-skill PR with its experiment.

9. What I couldn't verify¶

• E2B: start and restore latency, and snapshot pricing. Not stated on the pages fetched [S29][S30].
• Morph: latency, pricing, memory branching details. Only the docs index was reachable [S35].
• Cloud start latency for Modal, Vercel and Cloudflare: none stated on the fetched pages.
• Northflank and Daytona: the latency figures are vendor claims.
• Fly Sprites: figures come from a secondary source [S38].
• GKE Pod Snapshots:
• whether one snapshot can seed several new Pods at once;
• real restore times for coding workloads. The 89% startup reduction and "8 s" figures are for model serving [S41].
• Codex Cloud: how the new version reaches private services, and whether it has a self-hosted runner [S13].
• Amp: whether orbs can run on customer compute [S27].
• Self-hosted Claude Code sessions: whether Claude Code's LSP plugins start in claude -p mode, and in self-hosted cloud sessions. The doc only says Anthropic-hosted cloud sessions don't start them [S9].
• gVisor fork at 512 MB: that f1 was OOM-killed is inferred from memory pressure; the kernel log was not checked.
• Best-of-N write sharing: that concurrent identical requests each pay a cache write is inferred, not measured.
• hangar's OAuth server: whether it accepts Client ID Metadata Documents was not checked.
• Pricing: every price was list price on 2026-10-13; regional and committed-use prices differ.

10. Sources¶

Fetched 2026-10-13.

Cloud coding agents

• S1 Claude Code: use Claude Code in the cloud
• S2 Claude Code: cloud environments
• S3 Claude Code: self-hosted environments
• S4 Self-hosted environments reference
• S5 Customize self-hosted sessions (hooks, spawn-runner)
• S12 Codex Cloud (legacy) environments, redirected from developers.openai.com/codex/cloud/environments
• S13 Using Codex Cloud
• S18 Cursor cloud agents
• S19 Cursor self-hosted cloud agents; changelog 2026-03-25
• S20 Jules environment setup
• S21 Devin blueprints
• S22 Devin Outposts
• S23 Copilot cloud agent environment
• S24 Factory Droid Computers
• S25 How Ona works
• S26 Warp launches Oz
• S27 Amp manual
• S28 OpenHands runtimes

Claude Code behavior and models

• S6 How Claude Code uses prompt caching
• S7 Agent SDK: modifying system prompts
• S8 Claude Code best practices
• S9 Code intelligence plugins
• S10 Anthropic prompt caching
• S11 Anthropic pricing
• S15 OpenAI prompt caching
• S16 OpenAI pricing

Protocols and SDKs

• S14 Codex app-server README
• S17 The next evolution of the Agents SDK (2026-04-15)
• S45 MCP 2026-07-28 changelog
• S46 MCP roadmap
• S47 MCP Apps
• S48 MCP Registry
• S49 A2A at one year (Linux Foundation, 2026-04-09)
• S50 ACP:
• introduction
• agents
• v2 draft
• remote transport RFD
• S51 The ACP Registry is live (Zed)

Sandbox platforms and runtimes

• S29 E2B: billing, pricing
• S30 E2B snapshots
• S31 e2b-dev/infra
• S32 Daytona: VM pause and fork, pricing
• S33 Nightona README
• S34 Modal: sandbox snapshots, pricing
• S35 Morph Cloud docs
• S36 Cloudflare: Containers and Sandboxes GA, Containers pricing
• S37 Vercel Sandbox: pricing, snapshots
• S38 Simon Willison on Sprites (secondary)
• S39 Northflank sandboxes
• S40 GKE Pod snapshots: concepts, restore
• S41 InfoQ: GKE Pod Snapshots benchmarks (2026-09-27)
• S42 Agent Sandbox: README, envd example
• S43 Firecracker snapshot support
• S44 KEP-2008 forensic container checkpointing

Local verification

• gVisor runsc release-20261005.0, from gs://gvisor/releases/release/20261005/x86_64/gvisor.tar.zstd (sha512 checked).
• claude 2.1.296, codex 0.162.1 and pi 1.1.0 --help output.
• hangar's own bench/results/2026-10-13-perf-docker-ab-after.json.