Browser Agent

What it is

A portfolio demo of a read-only browser agent over a synthetic records website, with a Chrome MV3 side-panel UI and a hosted live view of its browser. It shows a sourced result, a blocked write, a blocked off-site navigation, a sign-in pause and a metadata-only audit log. Not a client deployment.

What it does

• Five preset tasks on the live demo: a lookup, a blocked write, a blocked off-site navigation, a sign-in pause and conflicting sources.
• Demo video (2 min 30 s) recorded with the real model claude-sonnet-5-5.
• README states 185 automated tests, including redirect chains, WebSocket/WebRTC egress and prompt-injection cases.
• Two rounds of independent adversarial review, each finding fixed with a red-to-green regression test (per ACCEPTANCE_REPORT.md as cited in README).
• Live limits: one run at a time, 5 runs per visitor per hour, $3 of model spend per day, then a recorded real run plays instead.

The problem

A browser agent that can click anything can also delete records, leave the approved site or read a login form. Telling the model not to is not enough. This demo shows an agent that is read-only because the code below the model enforces it.

Design decisions

• Give the model five narrow tools and no clicking, typing, selectors or script evaluation. The tradeoff is that it can only do the read tasks the tools allow.
• Never let the browser follow a redirect by itself. Every hop is checked as a new navigation, so a long redirect chain cannot reach a write route. The cost is more code than a simple first-URL check.
• Set the source URL and retrieval time in the coordinator, and re-check the model's answer against the evidence it saw. Conflicting sources give an uncertain result instead of a guess.
• Run the public demo with limits: one run at a time, 5 runs per visitor per hour and a daily spend cap, after which a recorded real run plays. The tradeoff is that most visitors watch a limited demo.

How it works

1. Task parserAccepts exactly one record ID in the form SYN-####, and rejects malformed or multiple IDs before any browser work.
2. Model adapterCalls an approved model with the five tools and marks tool results as untrusted; an unapproved model is rejected before any network call.
3. Policy engineChecks every tool argument and every URL against exact host, port and route rules, and treats any POST or delete-style route as a write.
4. Playwright bridgeDrives a dedicated Chromium with a request route that blocks redirects, WebSockets and popups, and reads pages in an isolated world.
5. CoordinatorChecks the final answer against the extracted evidence and ends the run as blocked, uncertain or human-action-required when needed.
6. Audit store and panelKeeps a metadata-only audit log in memory and streams steps and the result to the web page or side panel.

Security: enforced, simulated and known gaps

• Enforced in code: URL and route allowlist, write detection, no redirect following, WebSocket and popup blocking, credential-field detection in every frame, a spend ledger that fails closed, and a per-launch check that the backend only attaches to the browser it launched.
• Simulated: all records are synthetic. In the two blocked-action presets the real model refuses, so one scripted tool call is substituted to show the policy blocking it, and the page labels this.
• Known gaps from the architecture note: egress containment uses browser flags and is not guaranteed, text hidden by colour or tiny font can still be read, and the audit store is in memory. Production would need an OS-level egress firewall and isolated browser workers.

What broke and how it was fixed

What broke

The URL policy checked only the first hop of a redirect, so a chain of allowed pages could end at a delete route and change data.

How it was found

An external adversarial review of the code found it. The tests showed the site's mutation counter at 1 instead of 0.

The fix

Chromium is no longer allowed to follow redirects. The route handler checks each Location target, aborts the request and re-issues it as a new checked navigation, with a hop cap of 5.

What guards it now

tests/review-fixes.test.ts, finding 1, covers a 2-hop chain, an off-site chain and a loop, and asserts the mutation counter stays at 0.

Reference

5d905f7

Where it runs

A container behind Traefik serves the live demo at shawndemos.com/browser-agent/. A recorded demo video is attached to the GitHub release.

Who did what

The owner conceived the product, architecture, specs and acceptance tests and directed the work. Coding agents (Claude, Codex, Gemini) did much of the implementation and review. The acceptance report records two rounds of independent adversarial review, and the live-demo task graph was written to be built with the agent swarm.

Limits and honest status

• Synthetic data only; not a client deployment and not built for real patient data.
• In the video's blocked-write and off-site beats the real model refused, so one scripted tool call is substituted to show the policy blocking it; the same simulation applies to the two blocked presets on the live page, and both are labelled.
• Tested only on Ubuntu with Playwright's bundled Chromium 153; other OSes and Chrome versions are untested. The OpenAI provider is a stub.

Connected to

Agent SwarmCareOps