Chrome Browser Relay

Chrome Browser Relay lets the AI control your real Chrome browser — with your existing sessions, cookies, and logins intact. Unlike the built-in Browser tools (which open a clean Electron window), Chrome Relay operates on the Chrome you use every day.

How It Works

AI Chat → ChromeRelay tools → Alma Server → WebSocket → Chrome Extension → chrome.debugger → Your Tab

Alma ships with a Chrome extension that bridges your browser to the AI via the Chrome DevTools Protocol (CDP). The extension connects to Alma over a local WebSocket, and the AI sends commands like navigate, click, type, and screenshot through this channel.

Setup

Option 1: One-Click Launch (Recommended)

1. Open Alma → Settings → Chrome Relay
2. Copy the Auth Token displayed on the page
3. Click Launch Chrome with Extension
4. Wait for the Connection Status to turn green

WARNING

For best results, close all Chrome windows before clicking "Launch Chrome with Extension". The --load-extension flag works most reliably when Chrome is not already running.

Option 2: Manual Install

If one-click launch doesn't work (e.g., Chrome is already running):

1. Open Alma Settings → Chrome Relay
2. Expand the Manual Install section and copy the Extension Path
3. Open Chrome and navigate to chrome://extensions
4. Enable Developer mode (toggle in the top-right corner)
5. Click Load unpacked and paste the extension path
6. Click the extension icon in the Chrome toolbar to open the popup
7. If not auto-connected, click Settings in the popup footer
8. Enter the Port (23001 by default) and Auth Token from Alma
9. Click Save — the extension will reconnect automatically

Verifying the Connection

• The extension badge shows ON (green) when connected
• Alma Settings → Chrome Relay shows a green Connected status
• Attached tabs are listed in both the extension popup and Alma Settings

Available Tools

Tool	Description
ChromeRelayListTabs	List all open Chrome tabs with ID, URL, title, and active state
ChromeRelayNavigate	Navigate a tab to a URL, or create a new tab — returns the final URL and title
ChromeRelayClick	Click an element by ref or CSS selector, and report what was clicked
ChromeRelayType	Type text into a field, optionally press Enter — reports the value that landed
ChromeRelayScreenshot	Capture a screenshot for visual analysis
ChromeRelayRead	Read visible page content as Markdown (via Readability)
ChromeRelayReadDom	Snapshot every interactive element with a ref, role, name, and a ready-made selector
ChromeRelayEval	Execute JavaScript in the page context, waiting for promises to settle
ChromeRelayScroll	Scroll the page up or down
ChromeRelayBack	Navigate back in history
ChromeRelayForward	Navigate forward in history

Pointing at Elements

ChromeRelayReadDom is the map the AI works from. Every line carries enough to identify an element and act on it — the ref, its accessibility role and name, the underlying tag, plus whatever is useful for that kind of element: id, href for links, name and type for form controls, and a sel= selector that can be used as-is.

e11 link "Pull requests" <a#pr-nav> href=/pulls sel=#pr-nav
e12 textbox "Search or jump to…" <input> type=search name=q sel=input[name="q"]
e13 button "Sign in" <button> type=submit sel=button[type="submit"]

A ref such as e12 can be passed straight to the selector parameter of ChromeRelayClick or ChromeRelayType — there is no need to hand-assemble CSS from the description. The sel= hint is there for the cases where a stable, reusable selector is preferable.

Anything the user cannot reach is listed but flagged, using the same rule Click enforces, so the map and the action never disagree:

e21 button "Subscribe" <button#sub> sel=#sub [hidden: covered at its click point by <div.modal-overlay>]

A flagged element still exists in the page — it may become clickable once a dialog closes or a menu opens — but clicking it now is refused unless allowHidden is passed. Elements that are merely scrolled out of view carry no flag.

WARNING

Refs belong to the snapshot they came from. Once the page navigates or re-renders, the old refs are stale — the tools say so explicitly rather than clicking the wrong thing, and the fix is always to run ChromeRelayReadDom again.

What Actions Report Back

Every action returns what it actually did, not just whether it returned without throwing. This matters because "success" on the wrong element looks identical to success on the right one.

Click

ChromeRelayClick reports how many elements the selector matched, which one it clicked (matchedIndex, the first by default), what that element is, whether the click caused a navigation, and whether it spawned new tabs:

{
  "success": true,
  "matchCount": 3,
  "matchedIndex": 0,
  "element": { "tag": "a", "id": "repo-link", "text": "yetone/alma", "href": "/yetone/alma" },
  "navigated": { "from": "https://github.com/search?q=alma", "to": "https://github.com/yetone/alma" },
  "openedTabs": []
}

A high matchCount is a signal that the selector is too loose — the click landed on the first match, which may not be the intended one.

When a click opens a new tab, it appears under openedTabs with its id, url, and title. That tab is not attached to the debugger yet: Chrome requires your approval per tab, so the AI can see the new tab exists but cannot act on it until you grant access.

Hidden elements

By default, Click refuses elements a human could not click, and it judges that by reachability rather than by the element's own CSS alone: display: none, visibility: hidden, opacity: 0, transform: scale(0), a zero-size box, a box covered at its click point by something painted on top, and a box parked outside the document (the left: -9999px screen-reader trick, which no amount of scrolling brings into view).

Elements that are merely scrolled away — below the fold, or scrolled out of their own overflow container — are not refused: clicking scrolls to them.

It returns success: false and names the reason instead of silently dispatching an event into a void:

{
  "success": false,
  "matchCount": 1,
  "element": { "tag": "button", "id": "hidden-submit", "visible": false, "hiddenReason": "display:none" },
  "error": "Element is not visible (display:none) — a real user could not click it. Pass allowHidden: true to click it anyway."
}

Some pages legitimately drive hidden controls (offscreen file inputs, custom widgets). Pass allowHidden: true to go ahead — the receipt then records that the element clicked was a hidden one.

Type

ChromeRelayType distinguishes its failure modes instead of reporting a generic error: the element does not exist, the element is not typable (not an input, textarea, or contenteditable), it is disabled, it is readonly, or the write did not stick. On success it echoes the value that actually ended up in the field, which is how you confirm a controlled React component accepted the input:

{ "success": true, "element": { "tag": "input", "name": "q", "type": "search" }, "value": "TypeScript best practices" }

Eval

ChromeRelayEval waits for promises to resolve. await fetch(...), new Promise(...), and anything else async return their real value rather than an empty {}. The result comes back with its JavaScript type so a "null" string can be told apart from an actual null:

{ "result": "{\"login\":\"yetone\",\"public_repos\":42}", "type": "object" }

Read and Navigate

ChromeRelayRead strips display: none and visibility: hidden content before extracting the article, so it agrees with what ChromeRelayReadDom considers visible — no more links in the output that do not exist on the screen.

ChromeRelayNavigate returns the real final URL and title of the tab, including when it creates a new tab.

Usage Examples

Browse with your login sessions

"Check my GitHub notifications"
"Open my Gmail and summarize the latest emails"
"Go to my Jira board and list the open tickets"

The AI uses your existing Chrome sessions, so you don't need to log in again.

Interact with pages

"Search for 'TypeScript best practices' on Google"
"Click the first search result"
"Fill in the contact form with my name and email"

Read and extract content

"Read the article in my current tab and summarize it"
"List all the links on this page"
"Take a screenshot of the page"

Chrome Relay vs Built-in Browser

	Chrome Relay	Built-in Browser
Browser	Your real Chrome	Electron window
Sessions/Cookies	Your existing logins	Clean session
Extension required	Yes	No
Use case	Tasks needing your accounts	General browsing

TIP

If you need to interact with a site where you're already logged in (Gmail, GitHub, Jira, etc.), use Chrome Relay. For general web browsing or research that doesn't need authentication, the built-in Browser tools work fine.

Settings

All Chrome Relay settings are in Settings → Chrome Relay:

Setting	Description
Connection Status	Shows whether the extension is connected
Auth Token	Token for authenticating the extension (copy, regenerate)
Launch Chrome	One-click button to start Chrome with the extension
Manual Install	Step-by-step instructions for manual setup
Attached Tabs	List of tabs with an active debugger session

Troubleshooting

Extension shows OFF (red badge)

• Verify Alma is running and port 23001 is available
• Check the Auth Token matches in both Alma Settings and extension Options
• Click Reconnect in the extension popup

"Launch Chrome with Extension" has no effect

• Close Chrome completely (including background processes) first
• On macOS, check Activity Monitor for lingering Chrome processes
• Fall back to the Manual Install method

AI doesn't use ChromeRelay tools

• Mention "my Chrome browser" or "my real browser" explicitly in your prompt
• The AI distinguishes between ChromeRelay (real Chrome) and Browser (Electron) based on context

A click reports success: false for an element you can see

The element is probably visible somewhere else on the page — check matchCount in the result. If the selector matched several elements, the first match may be a hidden duplicate (menus often render both a desktop and a mobile copy). Re-run ChromeRelayReadDom and use the ref of the specific element instead.

"Stale ref" errors

The page changed after the snapshot was taken. Run ChromeRelayReadDom again to get fresh refs; refs are never valid across a navigation.

A click opened a tab the AI can't use

New tabs start unattached. Approve the tab for debugging (via the extension popup or Alma Settings → Chrome Relay → Attached Tabs) and the AI can then act on it.

Extension disconnects frequently

• Chrome may suspend the service worker; the extension uses keep-alive alarms to prevent this
• If the issue persists, check your network/firewall settings for localhost connections
• Try regenerating the Auth Token in Alma Settings

Token regenerated, extension disconnected

This is expected. After regenerating the token:

1. Open the extension popup → Settings
2. Enter the new token
3. Click Save