The agent's own computer, and an environment it can diagnose¶

Environment quality decides what a cloud agent produces, and environment failures are silent: a missing secret or a blocked registry doesn't stop the run, it just makes the output worse. hangar gives every agent two things to counter that:

• Its own computer (opt-in per environment): a virtual display, Chromium through Playwright, screenshots, recorded demos, and a screen people can watch or take over from hangar.
• Self-diagnosis (every sandbox): hangar-agent env doctor and an MCP server in the sandbox that say what in the environment works and what doesn't, and hangar-agent env request to ask the owner for a secret or a host. Requests are never granted automatically, and the agent keeps working on everything they don't block.

Turning the computer on¶

# envs/web.yaml
name: web
extends: node
computer: true            # or a map:
# computer:
#   resolution: 1440x900  # the display, the browser window and the recording frame (default 1280x800)
#   autostart: true       # start the display and VNC at claim and every turn (default)
#   browser_mcp: false    # Playwright's MCP server for every harness (adds Node 22 to the image)
#   record_max_seconds: 300

The image gets xvfb, x11vnc, openbox, ffmpeg, xdotool, fonts, and Playwright for Python pinned to playwright==1.63.0 with its Chromium in /opt/hangar/computer/browsers (PLAYWRIGHT_BROWSERS_PATH). That is about 400 MB, which is why it is opt-in. It needs a Debian or Ubuntu base (apt). browser_mcp: true also installs @playwright/mcp@0.0.83 and the Chromium build its own Playwright wants.

At claim, and again at the start of every turn (a restore or resume brings no processes back), hangar starts the display :99, the window manager and x11vnc on 127.0.0.1:5900, and sets DISPLAY=:99 for every shell. Starting is idempotent and takes about a second.

What the agent runs¶

Inside a job or session turn, screenshots land in the turn's artifacts as screenshots/<name>.png and recordings as demos/<name>.mp4, through the existing context exchange (hangar-agent artifact). Recordings are capped below HANGAR_EXCHANGE_MAX_ARTIFACT_MB and at record_max_seconds. Outside a turn they stay in ~/.hangar/computer/.

Sandboxes with a computer also get the hangar-verify-ui skill in the harness's skills directory. It walks the agent through running the app, checking the page loads cleanly, driving the changed flow with a Playwright script, and recording a short demo. Job prompts in such environments ask for exactly that when a change affects the UI.

Where the media goes¶

• Session events: a progress event per screenshot or recording, with its hangar:// artifact URI.
• The job's PR body: a "Screenshots and demos" section. By default (HANGAR_PR_MEDIA_LINKS=off) it lists the files and how to fetch them through hangar, signed in: hangar session artifacts <session> --get <path>, MCP get_artifact, or GET /api/jobs/<id>/artifacts/<turn>/<path>. Those are owner-scoped like every record, so only the job's owner and admins can open them.

HANGAR_PR_MEDIA_LINKS=signed embeds images and links recordings instead, through signed links (/api/media/<token>) that name one artifact and expire after HANGAR_PR_MEDIA_LINK_DAYS (7). The link is the credential: anyone who can read the PR (or its notification emails) and reach hangar can open the file, and GitHub's image proxy fetches and caches embedded images. A screenshot of a private repo's app can show its code, data or admin pages, so turn this on only where everyone who reads the PRs may see the work (a public repository, say). Turning it back off revokes every link already posted: /api/media/ answers only while it is on.

Why not embedded and authenticated: GitHub renders PR images through its own proxy, which can't carry a hangar login, and the dashboard signs in with a bearer token rather than a cookie, so an authenticated link can't be an image in a PR or a plain click. Names plus the signed-in ways to fetch them are the safe default.

Watching and taking over¶

hangar screen job-1a2b3c4d            # opens a view-only screen in your browser
hangar screen my-session --control    # your keyboard and mouse reach the sandbox

REST: POST /api/screen/tickets {"ref", "control"}. Each returns a link to /ui/vnc.html with a one-time ticket in the URL fragment (browsers never send fragments to servers), valid for 60 seconds. The page is a noVNC 1.7.0 client. There is no MCP tool for it: claude.ai caches tool lists indefinitely and hangar's listing is at its size cap, and a person opens a screen in a browser anyway.

How it stays private:

• No port is exposed. x11vnc listens on 127.0.0.1 inside the sandbox. hangar opens the provider's terminal stream into the sandbox (Docker exec, the Kubernetes exec websocket, or envd) and runs hangar-computer bridge, which relays the VNC connection as base64 lines. Lines survive every transport, including Kubernetes' exec stream, which decodes output as UTF-8.
• Owner-scoped. A ticket is minted only for the session's, job's or sandbox's owner, or an admin (a teammate's gets "not found", as everywhere else, #88). Each ticket is single-use (redeeming removes it) and only its hash is kept, so it can't be replayed, and a ticket that reaches a log (the websocket URL carries it) is already spent.
• A ticket is a bearer for its 60 seconds. Whoever opens the link first gets the screen, with the control the ticket was minted with; the websocket can't carry a hangar login, so the ticket can't be bound to the person who asked for it. Sharing one shares the screen once. The audit records where each was used from.
• View-only is enforced by hangar. Without control, hangar parses the viewer's side of the protocol and drops key, pointer, clipboard, resize, xvp and QEMU key messages. It ends the stream on anything it can't parse. The browser's view-only flag is not trusted.
• Audited. screen.ticket (who asked, view or control), screen.view / screen.control (with the client address) and screen.end (with how long) go to the audit log, and the session gets an event ("ops is watching the screen", "... took control of the screen"). At most 4 viewers per sandbox.
• Inside the sandbox, VNC is open to everything. x11vnc runs with -nopw on 127.0.0.1, and the X display has no access control, so any process in the sandbox can watch and drive the screen: the agent, its tools, a test server, and on Kubernetes the pod's sidecars too (they share its network namespace). That's accepted: they are all the agent's own processes, which already own the display. Nothing outside the sandbox can reach it.

How each harness drives the browser¶

Every harness can run shell commands, so hangar-agent screenshot|record|browse and Playwright scripts (/opt/hangar/computer/venv/bin/python script.py) work with all of them. Beyond that:

There is no model-routing subagent: the agent in the sandbox drives the browser itself.

Requests: asking the owner, never granted automatically¶

hangar-agent env request secret STRIPE_KEY --why "the payment tests need a test-mode key"
hangar-agent env request egress registry.npmjs.org --why "npm ci fails: the registry is blocked"
hangar-agent env requests                   # what was filed and what the owner said

The request is a file under $HANGAR_IO/out/requests/. While the turn runs, hangar looks for new ones every HANGAR_ENV_REQUEST_POLL_SECONDS (30; 0 means only when the turn ends) and again when it ends. Each one is validated (a variable name or a host, and a reason, which is redacted) and recorded in the env_requests table. Only the kind, the name or host and the reason are taken from the file: its state, scope and owner are hangar's.

An agent can't flood its owner. Per session, at most 5 new requests a turn and 5 waiting for an answer; per owner, 30 an hour across all their sessions and jobs; and a host or secret the owner denied isn't asked for again in that session. Files over a limit are deleted without being read, and the turn gets a warning (at most 3). hangar-agent env request says so before filing when 5 already wait.

The owner is told:

• an MCP elicitation, when someone is following the work with wait_for_turn: "Job job-1a2b (fix checkout): the agent asks for the secret STRIPE_KEY ... Its reason, in its own words: "..."", answered approve or deny with a note;
• a session event and a job event, and an audit entry (env.request.secret, env.request.egress);
• GET /api/env-requests and hangar requests ls; in MCP, pending_approvals on get_session, get_job and wait_for_turn, next to the tool calls the agent waits on (the id starts with rq-).

Answer with hangar requests approve|deny <id> [--note ...], POST /api/env-requests/<id>, MCP answer_approval(ref, approval=<rq- id>, decision=allow|deny, message=<note>) or the elicitation. Widening an egress approval to the whole environment is REST and CLI only (--scope environment, admins). Only the work's owner or an admin can decide. A decision is recorded once, audited, written into the sandbox's status file for the agent (on a task of its own, so the answer doesn't wait on the sandbox), and added to the session's events.

Nothing is ever granted automatically. No path from what a sandbox writes leads to decide: it is called only by a person's REST or CLI call, their MCP call, or their answer to the elicitation.

• Secret: approving means "handled". Store the credential and map it in the environment (secrets:) first. A request never carries a value, and the form says not to paste one.
• Egress: the decision goes to egressapi.on_decided, which allows the host for the session, or for the whole environment when an admin widens scope to environment.

egressapi.py is the interface between this and egress enforcement: the EgressRequest record, policy, check, may_widen and on_decided. The version here is a stub that reports open and enforces nothing; the egress allowlist replaces its bodies with the same signatures.

Prompting for autonomy¶

Job prompts tell the agent that nobody is watching the run live and a blocked agent sits idle for hours. When the environment blocks something, the agent runs hangar-agent env doctor, files the blocker with env request, keeps working on everything else, and says in its summary what is still blocked. hangar-agent ask stays for decisions only a person can make. In environments with a computer, the prompt also asks for end-to-end verification and a recorded demo when the change affects the UI.

Integration points¶

• Supervised services (envbuilds). The doctor reads sb.info["env_build"], ["services"] and ["setup_failures"], and calls hangar-agent svc status --json when the sandbox has the supervisor. Xvfb, the window manager and x11vnc are started by hangar-computer computer start. The supervisor can own them as supervised terminals instead; nothing else changes.
• Egress enforcement. See egressapi.py above. envrequests.decide is the single decision path, so an egress approval API calls it rather than writing rows.
• An inbox. EnvRequests.listeners is called with every new request; an attention inbox subscribes there.

Moved from this page¶