Skip to content

feat(phase-1): Chrome relay sidecar — drive the user's real Chrome via an opt-in CDP relay - #199

Open
domidex01 wants to merge 2 commits into
fitchmultz:mainfrom
domidex01:chrome-relay-sidecar
Open

domidex01 wants to merge 2 commits into
fitchmultz:mainfrom
domidex01:chrome-relay-sidecar

Conversation

@domidex01

Copy link
Copy Markdown

Summary

Phase 1 of the Chrome adoption plan: vendor oh-my-pi's browser-relay (MIT) as an opt-in sidecar so agent-browser connect ws://127.0.0.1:9224/cdp can drive the user's real, headed Chrome (signed-in sessions, extensions) through an MV3 chrome.debugger extension — no --remote-debugging-port, which Chrome 136+ blocks on default profiles anyway.

Changes

  • chrome-relay/: bridge.ts/protocol.ts vendored verbatim (single documented import-specifier delta), server.ts = Node port of the Bun original (same routes, 503→200 discovery impersonation, 256 MiB max payload, 30 s pings, loopback-only bind), promises.d.ts (ES2024 decl), VENDOR.md + LICENSE (attribution verified against upstream package.json: MIT © Stencil Labs, Inc.)
  • chrome-relay/extension/: manifest/options verbatim; background.ts gains a configurable bare-host relay address (WSL2 NAT) — built background.js committed
  • scripts/chrome-relay.mjs → pi-agent-browser-chrome-relay bin: start/status/stop/token/extension-path, temp-file state, token never logged
  • package.json: new bin + files entries; ws runtime dep, @types/ws dev dep
  • Docs lockstep: docs/CHROME_RELAY.md (setup incl. Windows/WSL2 NAT recipes + security model) + cross-links in README, COMMAND_REFERENCE (human region — baseline check green), ARCHITECTURE, SUPPORT_MATRIX RQ-0153
  • test/chrome-relay.sidecar.test.ts: 10 offline cases through the real vendored bridge with fake extension/CDP peers — discovery 503→200, minted PAGE ids, chrome:// hidden, Target.* emulation, forwarded-command round-trip via extension rpc, Browser.close refusal, Origin-403, token-401, loopback bind, targetCreated fan-out

Safety invariants (pinned by test)

loopback-only bind · Origin rejection on /cdp · chrome-extension:// origin + optional shared token on /ext · Browser.close acknowledged but never forwarded · chrome:// pages ineligible · 256 MiB max payload

Test plan

  • npx tsx --test test/chrome-relay.sidecar.test.ts — 10/10 pass
  • Full npm test (builds dist, 1220 tests): zero regressions — the 6 failures reproduce identically on pristine HEAD (verified in a clean worktree: Electron path checks, HOME pinning, startup-arg env leakage on this box, plus the parser.deref keep-alive flake in extension-ref-guards which fails on baseline runs too)
  • npm run docs -- command-reference check — in sync
  • CLI smoke: start (prints ws URL + extension path) → status (running/pid/port/extensionConnected) → SIGTERM → state cleared
  • Live dogfood vs real headed Chrome (follow-up; needs manual chrome://extensions load — steps in docs/CHROME_RELAY.md)

…a an opt-in CDP relay

Vendors oh-my-pi's browser-relay (MIT) as chrome-relay/: bridge.ts/protocol.ts
byte-identical (one import-specifier delta, documented), server.ts is the Node
port of the Bun original, and the MV3 extension gains a configurable host field
(WSL2 NAT) with its esbuild bundle committed. scripts/chrome-relay.mjs adds the
pi-agent-browser-chrome-relay bin (start/status/stop/token/extension-path,
temp-file state). The wrapper tool itself is unchanged: connect
ws://127.0.0.1:<port>/cdp is an ordinary upstream connect against the relay's
Chrome discovery façade.

Safety invariants inherited and pinned by test: loopback-only bind, Origin
rejection on /cdp, chrome-extension:// origin + optional token on /ext,
Browser.close never forwarded, chrome:// pages ineligible, 256 MiB max payload.

Docs: docs/CHROME_RELAY.md (setup, Windows/WSL2 NAT recipes, security model)
plus cross-links in README, COMMAND_REFERENCE, ARCHITECTURE, SUPPORT_MATRIX
(RQ-0153). Offline coverage: test/chrome-relay.sidecar.test.ts (10 cases, fake
extension + fake CDP peers, no Chrome in the default gate).
@domidex01

Copy link
Copy Markdown
Author

**Phase 2 dogfood evidence (Tasks 2.1–2.2)

Rebase (2.1) — rebased onto 40e1c2a (v0.7.0); one conflict resolved (docs/ARCHITECTURE.md: re-slotted the sidecar section around the renamed Product priorities heading).

  • Targeted suite: 10/10 (npx tsx --test test/chrome-relay.sidecar.test.ts)
  • Full gate: 1255 tests, 1205 pass, 6 fail / 4 cancelled. Failure-set comparison vs a pristine origin/main worktree: branch failures = the 4 known env failures on this box (cold-start budget, startup-args precedence, Electron profile path, HOME pinning). The baseline roll additionally flaked test/agent-browser.execution-lock.test.ts (new in v0.7.0 — different subtests fail on different rolls: multi-browser operations... on baseline, namespace close drains... on an isolated re-roll, absent from the branch gate): timing-flaky, independent of this diff. Zero regressions.
  • npm run docs -- command-reference check green; PR MERGEABLE.

Linux gauntlet (2.2) — packaged sidecar (scripts/chrome-relay.mjs start --port 9224, rebased dist) + Chrome for Testing 153 (--headless=new --load-extension):

  • connect ws://127.0.0.1:9224/cdp (fresh) → OK; minted target PAGE147327175
  • tab list → 1 tab; open https://example.com → title + URL correct
  • snapshot -i → 2 refs (e1 heading, e2 link); get url → https://example.com/
  • eval 1+1 → 2; screenshot → 16,601-byte PNG (real render)
  • click @e2 → real navigation to https://www.iana.org/help/example-domains (verified via get url)
  • close --all → 1 session closed, 0 failed

Observed: the sidecar CLI passes no logger into the bridge, so connection events go unseen (status JSON is the observable) — acceptable, noted for a possible follow-up. Tab-strip grouping is not observable in headless (no tab strip UI); grouping validation moves to the operator's headed Windows Chrome.

Windows leg (2.3) in progress — operator Load-unpacked pending; NAT docs fix (2.4) and merge (2.5) follow its result.

…tab guidance

Dogfood findings from the live Windows-Chrome validation:
- docs/CHROME_RELAY.md: the WSL2 section now documents the VALIDATED recipe —
  127.0.0.1 + localhostForwarding works as-is; socat-in-WSL is the fallback;
  direct WSL-IP dialing is refused by the loopback bind (previous advice was
  wrong). Adds adopted-browser tab semantics (open navigates the session's
  current tab; close tabs by explicit tN; close --all ends only the session)
  and notes status/stop track the most recent start.
- scripts/chrome-relay.mjs: --token-gen now prints the token once so the
  documented options-page flow is actually usable.
@domidex01

Copy link
Copy Markdown
Author

**Phase 2 dogfood evidence (Tasks 2.3–2.4) — Windows leg, operator's real Chrome

No-token connect: extension Loaded-unpacked from the staged folder → extensionConnected: true within seconds, Host left at 127.0.0.1 — WSL2 localhostForwarding delivered it to the loopback-bound relay with no portproxy, no socat, no host change. /json/version → 200 with the real Windows UA (Windows NT 10.0 … Chrome/153.0.0.0).

Real-tab adoption: tab list through the relay returned the operator's 30 live tabs (sanitized here on purpose). Drive validation: snapshot -i → 2 refs, eval 1+1 → 2; close --all ended only the wrapper session — browser and tabs untouched.

Incident → fix → docs: upstream wrapper semantics bit on an adopted browser: open <url> navigates the session's current tab and no-arg tab close closes it, which closed one of the operator's real tabs (restored via tab new). f99ea0a documents the safe pattern (tab new + explicit tab close <tN>) in docs/CHROME_RELAY.md.

Token round: relay restarted with --token → extension 401-rejected until the token was pasted in options, then extensionConnected: true. Wrong-token rejection stays pinned by the offline suite.

Grouping: tab-strip visual confirmation still pending from the operator (headless can't show it); noted as the one open observation.

Task 2.4 shipped in f99ea0a: NAT section rewritten to the validated recipe (localhostForwarding primary; socat-in-WSL fallback; why direct WSL-IP dialing is refused by the loopback bind — the previous advice was wrong), adopted-browser tab-semantics section, --token-gen now prints the token once (documented flow was unusable without it), status/stop most-recent-start note.

Both connect modes validated. Ready to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant