Browser Automation
Browser Automation lets a deployed agent capture a webpage or operate a stateful cloud browser. Agent steps call the typed ctx.sapiom.browserAutomation capability; Sapiom supplies the authenticated cloud connection when the deployed agent runs.
Choose the lightest browser mode
Section titled “Choose the lightest browser mode”| What the agent needs | Recommended operation |
|---|---|
| An image of a public webpage | browserAutomation.screenshot(...) |
| A check that expected content is visible before the capture | browserAutomation.withSession(...) plus session.screenshot(...) |
| Navigation, clicks, forms, page evaluation, or shared cookies | browserAutomation.withSession(...) plus a CDP-compatible client |
| A browser that starts with a previously provisioned login identity | browserAutomation.withSession(..., { identityId }) |
| A step only a person should do (sign-in, one-time code, payment) | browserAutomation.withSession(...) plus session.liveViewUrl sent to them |
| The session settlement or manual control over when the session is closed | browserAutomation.sessions.create() and sessions.close(sessionId) |
| A browser task that follows plain-language instructions and can ask a person for input | Managed sessions with browserAutomation.tasks.start(...) |
| Retry-safe session creation, saved profiles, or recordings | Managed sessions with browserAutomation.sessions.createManaged(...) |
Do not open a session just to capture a public URL. A one-shot screenshot has no session to manage. Use a session when the browser must preserve state across multiple actions.
Capture a webpage
Section titled “Capture a webpage”Call screenshot for a one-shot capture:
const screenshot = await ctx.sapiom.browserAutomation.screenshot({ url: "https://example.com", fullPage: true,});
return { screenshotUrl: screenshot.url, expiresAt: screenshot.expiresAt,};The URL must be non-empty. Optional fields control viewport width and height, waitMs, full-page capture, and "png" or "jpeg" output. imageQuality applies to JPEG output.
The result contains an absolute hosted url and its ISO 8601 expiresAt. Treat the URL as temporary rather than storing it as a durable artifact.
Let the page settle before capturing
Section titled “Let the page settle before capturing”A one-shot screenshot has no added delay after page load unless you set waitMs. The image can show a loading screen or an anti-bot challenge even when the screenshot call succeeds.
To give the page time, pass waitMs, the number of milliseconds to pause after load and before the capture:
const screenshot = await ctx.sapiom.browserAutomation.screenshot({ url: "https://example.com", waitMs: 10_000,});From a coding agent connected to the hosted capability MCP (the sapiom-direct alias), the same option is wait on sapiom_screenshot: an optional integer number of milliseconds from 0 to 30000. Omitting it adds no delay. For anything longer than 30000, use a session and check the rendered page before capturing, as described below.
{ "url": "https://example.com", "wait": 10000}A fixed delay does not check page readiness. The time needed can change with each load. Use a session when the agent must check that the expected content is visible before capture.
Treat an initial 403 as a possible challenge
Section titled “Treat an initial 403 as a possible challenge”An initial HTTP 403 can contain an anti-bot challenge instead of a permanent block. For example, Cloudflare identifies a challenge response with cf-mitigated: challenge. If the browser completes the challenge, the target page can load after it. Some challenges need human action or do not clear.
Check for these two cases:
- A screenshot can show the challenge page even when the call succeeds.
- The response returned by
page.goto()can have status403. A later page change can occur after that response, so the status alone does not show the current page state.
Check the DOM, title, or current URL for the content the task needs. Use a bounded wait. If the expected content does not appear, inspect the page before deciding whether the cause is a challenge, an access restriction, or a loading failure.
Each one-shot screenshot opens a new browser session. A retry can meet the same challenge again because it does not retain cookies from the previous call. To retain cookies and check content, navigate over CDP in a session, wait for the expected page content, then call session.screenshot(). Choose content that is specific to the target page; a generic heading or title can also appear on a challenge page.
This example assumes the agent project includes playwright-core. It waits for text that example.com shows. Change the URL and the text together for the target site.
import { chromium } from "playwright-core";
const result = await ctx.sapiom.browserAutomation.withSession( async (session) => { const browser = await chromium.connectOverCDP(session.cdpUrl);
try { const context = browser.contexts()[0]; if (!context) throw new Error("Browser session has no default context"); const page = context.pages()[0] ?? (await context.newPage());
// A response status alone does not confirm that the expected content is visible. await page.goto("https://example.com", { waitUntil: "domcontentloaded" });
// Wait for text that is specific to this page, then capture. await page .getByText("This domain is for use in documentation examples") .waitFor({ state: "visible", timeout: 45_000 }); const screenshot = await session.screenshot({ fullPage: true });
return { title: await page.title(), screenshotUrl: screenshot.url }; } finally { await browser.close(); } },);Operate an interactive browser
Section titled “Operate an interactive browser”A session exposes a Chrome DevTools Protocol WebSocket at session.cdpUrl. Connect a compatible library such as Playwright or Puppeteer to navigate, click, fill forms, evaluate page content, and work with cookies.
This example assumes the agent project includes playwright-core:
import { chromium } from "playwright-core";
const result = await ctx.sapiom.browserAutomation.withSession( async (session) => { const browser = await chromium.connectOverCDP(session.cdpUrl);
try { const context = browser.contexts()[0]; if (!context) throw new Error("Browser session has no default context");
const page = context.pages()[0] ?? (await context.newPage()); await page.goto("https://example.com", { waitUntil: "domcontentloaded", });
const screenshot = await session.screenshot({ fullPage: true }); return { title: await page.title(), finalUrl: page.url(), screenshotUrl: screenshot.url, screenshotExpiresAt: screenshot.expiresAt, }; } finally { await browser.close(); } },);session.screenshot() injects the current session ID, so url is optional after the browser has navigated. Playwright’s connectOverCDP supports Chromium-based browsers and provides lower-fidelity control than Playwright’s own protocol; keep the automation to behavior the target browser actually supports.
There are two cleanup layers in this example. browser.close() disconnects the Playwright client. After the callback finishes or throws, withSession attempts to close the Sapiom browser session in a finally block.
Start with a login identity
Section titled “Start with a login identity”Pass an existing identity ID to open the browser with its stored login state:
const result = await ctx.sapiom.browserAutomation.withSession( async (session) => { // Connect a CDP client and operate the authenticated browser here. return { expiresAt: session.expiresAt }; }, { identityId: "<identity-id>" },);browserAutomation.identities.create(...) provisions an identity from a login-page URL and credentials. Treat that as a deliberate provisioning operation: read credentials from agent secrets, never place them in source code, runtime defaults, caller input, or logs, and do not create a new identity during every ordinary run.
Hand a step to a human
Section titled “Hand a step to a human”A session also exposes session.liveViewUrl: an interactive live view of the same browser that opens in any web browser, on any device. Use it when the run reaches something only a person should do — a sign-in, a one-time code, a payment confirmation — and the agent must not hold that person’s credentials.
The pattern is: drive the browser over CDP up to the blocking step, send liveViewUrl to the person, wait for the page to move on, then continue over the same CDP connection. Cookies and page state carry across because the person acted inside the agent’s session.
import { chromium } from "playwright-core";
const result = await ctx.sapiom.browserAutomation.withSession( async (session) => { const browser = await chromium.connectOverCDP(session.cdpUrl);
try { const context = browser.contexts()[0]; if (!context) throw new Error("Browser session has no default context"); const page = context.pages()[0] ?? (await context.newPage());
await page.goto("https://app.example.com/login", { waitUntil: "domcontentloaded", });
// Hand the browser to the person however this agent reaches them // (email, chat, a notification). They sign in inside this session. await sendToUser(`Finish signing in here: ${session.liveViewUrl}`); await page.waitForURL("**/dashboard", { timeout: 5 * 60_000 });
// The session now carries the signed-in cookies; keep going over CDP. return { title: await page.title(), finalUrl: page.url() }; } finally { await browser.close(); } },);The link works for as long as the session does, and anyone holding it can act in the browser. Treat it like a credential: send it to one person, over a channel you trust, and close the session when the step is done. Do not post it where a second reader could open it. session.liveViewMode reports the lifetime the capability applied — today that is always "persistent".
The session’s own timers still apply. The person has to finish inside the idle timeout and maxDurationSec; the live view does not extend the maximum duration. Tell them what to do before you send the link, and keep the wait in your code bounded, as waitForURL is above.
To embed the live view in your own page instead of sending a link, put liveViewUrl in an iframe (the src below stands in for the session’s value):
<iframe src="https://live.example/session" sandbox="allow-same-origin allow-scripts" allow="clipboard-read; clipboard-write" style="border: 0; width: 100%; height: 100%"></iframe>Add pointer-events: none to the iframe’s style to show the browser without letting the viewer act on it. Append &show_cursor=1 to the URL to draw the remote browser’s own pointer in the stream.
Local Run’s stub session has no liveViewUrl — the field is absent, not fake — so code that sends the link must tolerate undefined there. Verify a real handoff with a small production run, the same way as CDP connectivity.
Test the behavior locally
Section titled “Test the behavior locally”Local Run replaces ctx.sapiom.browserAutomation calls with deterministic stubs. Built-in defaults are enough to exercise one-shot screenshots, session callbacks, session-bound screenshots, and the withSession close path without creating a cloud browser.
Override the exact capability path when a step branches on a particular result:
{ "version": 1, "steps": { "capture-page": { "browserAutomation.screenshot": { "url": "https://fixtures.example/screenshot.png", "expiresAt": "2099-01-01T00:00:00.000Z" } } }}After the run, require both unusedStubs and stubWarnings to be empty. A misspelled path or implausible response shape means the override did not prove the intended branch.
Manage session lifecycle in production
Section titled “Manage session lifecycle in production”Prefer withSession for ordinary automation. It attempts session closure even when the callback throws, while preserving the callback’s original result or error if closure also fails.
Use the manual lifecycle only when the agent needs the returned settlement or cannot fit the work inside one callback:
const session = await ctx.sapiom.browserAutomation.sessions.create();
try { // Connect through session.cdpUrl and do the browser work.} finally { const settlement = await ctx.sapiom.browserAutomation.sessions.close(session.sessionId);
console.log(settlement.settled, settlement.creditsUsed);}Read expiresAt and maxDurationSec from the created session rather than assuming a fixed lifetime. maxDurationSec is the browser’s maximum duration from creation. expiresAt includes an additional settlement period; it does not guarantee that the browser remains available until that time. The idle timeout can end the browser sooner.
Set the session lifetime
Section titled “Set the session lifetime”A session ends when either of two timers fires. The idle timeout counts from the moment the last client disconnects from the browser, whether that was a CDP connection or a live-view viewer, and reconnecting resets it. The maximum duration counts from creation and is never extended; reconnecting does not add time. Both default to values that suit a single automated task: 5 minutes idle and 20 minutes total.
When the agent needs the same browser to survive across several user messages, for example to keep a signed-in session between conversational turns, pass longer values when opening the session. sessions.create(), sessions.createWithIdentity(), and withSession() all accept the same two options, which are optional integer minutes:
| Option | Default | Range |
|---|---|---|
idleTimeoutMinutes | 5 | 1–60 |
maxDurationMinutes | 20 | 1–240 |
const session = await ctx.sapiom.browserAutomation.sessions.create({ idleTimeoutMinutes: 30, maxDurationMinutes: 180,});
// Hand session.sessionId and session.cdpUrl to later turns; reconnect to// session.cdpUrl on each one, and close the session when the conversation ends.console.log(session.maxDurationSec);To open the long-lived session with an identity instead, pass the same options alongside identityId to sessions.createWithIdentity() or to the second argument of withSession().
Omit an option to keep its default. A value outside the range, a non-integer, or a string is rejected before the session is created. The two timers are independent: an idle timeout longer than the maximum duration is accepted, and the session simply ends at the maximum. Coding agents on the hosted MCP pass the same two options when they open a session, with or without an identity. The created session reports the applied idleTimeoutMinutes and maxDurationMinutes alongside expiresAt.
Session creation authorizes $1 for each started hour of the requested maximum duration. The 180-minute example above authorizes $3; the default 20-minute maximum authorizes $1. Changing only idleTimeoutMinutes does not change this amount.
The authorization is a payment hold. Explicit close requests settlement from the provider’s reported usage. If usage cannot be retrieved, settlement can capture the full authorization. The hold does not cap provider usage; settlement captures the reported usage up to the authorized amount.
A longer lifetime keeps the session, and its usage, open for longer. Close the session explicitly as soon as the conversation is done rather than waiting for the maximum duration to expire.
A CDP connection error comes from the browser client; a capability request failure comes from Browser Automation. Inspect the failed production step to distinguish the two.
Use managed sessions
Section titled “Use managed sessions”Managed sessions add retry-safe creation, browser tasks that can ask a person for input, saved profiles, and recordings. Your own model can drive the browser over CDP, or a browser task can, one at a time. You create the session with your Sapiom API key; there is no separate browser account, provider key, or model to configure. Managed sessions require @sapiom/tools 0.44.0 or later.
| Standard sessions | Managed sessions | |
|---|---|---|
| Open and close | withSession(), or sessions.create() and sessions.close() | withManagedSession(), or sessions.createManaged() and sessions.closeManaged() |
| Control | Your CDP client | Your CDP client or a browser task, one at a time |
| Lost response | A retry opens another session | A retry returns the same session |
| Saved state | Identities | Profiles |
| Recording | No recording controls | Chosen at creation |
Standard sessions keep their current API. Close each kind with its own method.
Open a managed session
Section titled “Open a managed session”browserAutomation.withManagedSession(input, fn, options) creates a session, runs fn, and closes the session in a finally block. input requires an explicit recording boolean. idleTimeoutMinutes (default 5), maxDurationMinutes (default 20), and profileId are optional; the timers work as described in Set the session lifetime. The session passed to fn has a sessionId, a cdpUrl, and, when available, a liveViewUrl.
The helper generates an idempotency key, resends an uncertain creation with the same key and input, and closes any browser that a lost creation left behind. It retries the close until settlement completes, and calls options.onPendingClose(sessionId) if a close still has not completed, so you can call sessions.closeManaged again later. Settlement must complete within 2 minutes after the maximum duration; stop retrying after that. Expiry alone never settles a session.
To reconcile a creation later with sessions.recover, pass your own idempotencyKey and store it first. Recovery accepts only the credential that sent the creation, and inside a deployed agent each step run can get a new one. The lower-level createManaged, recover, and closeManaged methods are described in the SDK README.
Control the browser with your model or a task
Section titled “Control the browser with your model or a task”Connect a CDP client such as Playwright to cdpUrl and let your own model choose each action, or call tasks.start with plain-language instructions and let Sapiom’s browser agent act. Sapiom does not block CDP commands while a task runs, so hand over control explicitly:
- Stop CDP actions before
tasks.startortasks.resume. - To take over, call
tasks.pauseand wait for its success receipt. Then polltasks.getuntil the status ispaused,waiting_for_input, or final. Only then act over CDP, or let a person act throughliveViewUrl. - Never act while the task is
queuedorrunning. If a pause stays unconfirmed, close the session.
Inside a step, this drives the browser itself. It assumes the agent project includes playwright-core:
import { chromium } from "playwright-core";
const title = await ctx.sapiom.browserAutomation.withManagedSession( { recording: false }, async (session) => { const driver = await chromium.connectOverCDP(session.cdpUrl); try { const context = driver.contexts()[0]; if (!context) throw new Error("Browser session has no default context"); const page = context.pages()[0] ?? (await context.newPage()); await page.goto("https://example.com", { timeout: 45_000 }); // Your own model chooses each next action on this driver. return await page.title(); } finally { await driver.close(); } }, { onPendingClose: retryCloseLater }, // Application-specific cleanup job.);Run a task and answer its questions
Section titled “Run a task and answer its questions”tasks.start takes an idempotencyKey, a sessionId, instructions of up to 64,000 characters, an optional start url, maxSteps from 1 to 80, and an outputSchema (a JSON Schema object). Optional protectedValues are named values for the task to enter into forms; they require an unrecorded session without a profile. A session runs one task at a time.
The start result is a receipt, not the task’s state, and so is the status: "success" returned by tasks.pause, tasks.resume, and tasks.respond. Poll tasks.get(taskId) with a deadline:
| Status | Meaning |
|---|---|
queued, running | The task controls the browser. |
waiting_for_input | The task needs answers; read tasks.interventions(taskId). |
paused | A confirmed pause is in effect. |
completed, failed, canceled | Final. Closing the session cancels an unfinished task. |
Sapiom passes result and error through unchecked, so validate the result before using it. Answer each open request with tasks.respond({ taskId, requestId, response, idempotencyKey }); a response is a string, a boolean, or an object, and an object counts as protected input. Ask the person once, keep the request ID, answer, and key together, and resend all three unchanged after an uncertain outcome.
Task, profile, and recording calls take an idempotency key but are not retried for you. The examples resend them with this helper:
import { BrowserAutomationHttpError } from "@sapiom/tools";
/** Resend one call, with its original key and input, while the outcome is uncertain. */export async function resend<T>(send: () => Promise<T>): Promise<T> { for (let attempt = 1; ; attempt++) { try { return await send(); } catch (error) { const uncertain = !(error instanceof BrowserAutomationHttpError) || error.status === 429 || error.status >= 500 || error.code === "browser_outcome_unknown"; if (!uncertain || attempt === 6) throw error; await new Promise((resolve) => setTimeout(resolve, 5_000)); } }}This read-only task asks the user to choose a book, then returns validated details:
import { randomUUID } from "node:crypto";import { setTimeout as sleep } from "node:timers/promises";import { z } from "zod/v4";import { resend } from "./resend";
const browser = ctx.sapiom.browserAutomation;const Book = z.object({ title: z.string(), author: z.string() });
const book = await browser.withManagedSession( { recording: false }, async ({ sessionId }) => { const start = { idempotencyKey: randomUUID(), sessionId, url: "https://openlibrary.org/search?q=the+left+hand+of+darkness", instructions: "Only read pages. Ask the user which matching book they mean, " + "then return its title and author.", maxSteps: 25, outputSchema: { type: "object", properties: { title: { type: "string" }, author: { type: "string" } }, required: ["title", "author"], }, }; const { taskId } = await resend(() => browser.tasks.start(start)); const answered = new Set<string>();
for (const end = Date.now() + 10 * 60_000; Date.now() < end; ) { await sleep(2_000); // This example retries all read errors until the deadline. // In production, stop on permanent errors such as HTTP 401 or 403. const task = await browser.tasks.get(taskId).catch(() => null); // Results pass through unchecked: validate before use. if (task?.status === "completed") return Book.parse(task.result); if (task?.status === "failed" || task?.status === "canceled") throw new Error(`Browser task ${task.status}`); if (task?.status !== "waiting_for_input") continue;
const { requests } = await browser.tasks.interventions(taskId); for (const { requestId, message } of requests) { if (answered.has(requestId)) continue; // askUser is application-specific. Keep the request ID, answer, // and key together so every resend is identical. const response = await askUser(message ?? ""); const reply = { taskId, requestId, response, idempotencyKey: randomUUID(), }; await resend(() => browser.tasks.respond(reply)); answered.add(requestId); } } throw new Error("Browser task did not finish in 10 minutes"); },);Save and restore a profile
Section titled “Save and restore a profile”A profile keeps a session’s cookies, local storage, and cache, for example a signed-in account. Call profiles.save({ sessionId, idempotencyKey }) while the session is active; closing the session completes the save. Poll profiles.get(profileId) until its status is ready, then pass the profileId to a new session with recording: false. Profile IDs are generated, so store them; profiles.delete({ profileId, idempotencyKey }) removes one. Only an unrecorded session that has not received protected input can save a profile, and a session that saved or restored one cannot accept protected input.
import { randomUUID } from "node:crypto";import { setTimeout as sleep } from "node:timers/promises";import { resend } from "./resend";
const browser = ctx.sapiom.browserAutomation;const profileId = await browser.withManagedSession( { recording: false }, async ({ sessionId }) => { // Sign the person in here, for example through the live view. const save = { sessionId, idempotencyKey: randomUUID() }; const { profileId } = await resend(() => browser.profiles.save(save)); await storeProfileId(profileId); // Application-specific secret storage. return profileId; },);
// Closing the session completed the save. Wait until it is ready.for (const end = Date.now() + 5 * 60_000; ; await sleep(5_000)) { const profile = await browser.profiles.get(profileId).catch(() => null); if (profile?.status.toLowerCase() === "ready") break; if (Date.now() >= end) throw new Error("Profile not ready; check later");}
// Later: a new session that starts with the saved state.await browser.withManagedSession( { recording: false, profileId }, async (session) => { // Connect to session.cdpUrl as in the CDP example. },);Record a session
Section titled “Record a session”Pass recording: true when you create the session; recording cannot be turned on later, and recorded sessions cannot use protected input or profiles. While the session is active, recordings.pause and recordings.resume leave parts out of the video. After the close, recordings.list shows the primary recording and recordings.fetch({ sessionId, range, signal }) streams it as video/mp4; a byte range such as bytes=0-1023 returns HTTP 206 with a Content-Range header. Each fetch must finish within 120 seconds, and Sapiom permits read access for 7 days; this is not a provider retention guarantee. recordings.delete acknowledges a deletion, but the receipt does not prove the file is already gone.
import { randomUUID } from "node:crypto";import { createWriteStream } from "node:fs";import { Readable } from "node:stream";import { pipeline } from "node:stream/promises";import type { ReadableStream } from "node:stream/web";import { setTimeout as sleep } from "node:timers/promises";import { resend } from "./resend";
const browser = ctx.sapiom.browserAutomation;const sessionId = await browser.withManagedSession( { recording: true }, async (session) => { // Drive session.cdpUrl here, then leave one part out of the video. const pause = { sessionId: session.sessionId, idempotencyKey: randomUUID(), }; await resend(() => browser.recordings.pause(pause)); // Actions here are not recorded. const resume = { sessionId: session.sessionId, idempotencyKey: randomUUID(), }; await resend(() => browser.recordings.resume(resume)); return session.sessionId; },);
// The video can take a moment to appear after the close.for (const end = Date.now() + 2 * 60_000; ; await sleep(5_000)) { const { items } = await browser.recordings.list(sessionId); if (items.some((item) => item.isPrimary)) break; if (Date.now() >= end) throw new Error("Recording not listed yet");}
const video = await browser.recordings.fetch({ sessionId, signal: AbortSignal.timeout(110_000),});if (!video.body) throw new Error("Recording response has no body");await pipeline( Readable.fromWeb(video.body as ReadableStream<Uint8Array>), createWriteStream(file),);await browser.recordings.delete({ sessionId, recordingId: "primary" });Handle errors and uncertain outcomes
Section titled “Handle errors and uncertain outcomes”Failed calls throw BrowserAutomationHttpError with the HTTP status, a Sapiom code when the response supplies one, and the parsed body; do not log the body. Treat a network error, HTTP 429 or 5xx, or browser_outcome_unknown as uncertain: resend the original key and input, and never substitute a new key. A task start or control that stays unconfirmed never resolves, so stop controlling the session and close it. The SDK README lists what a resend can settle for each call and the HTTP endpoints.
Keep managed handles private
Section titled “Keep managed handles private”After creation, possession of a session, task, or profile ID authorizes operations on it, whichever API key sends them. Treat these IDs, creation keys, cdpUrl, and liveViewUrl as credentials: keep them and protected input out of logs, analytics, URLs, and step output, and check your own per-user access before using them.
Test managed sessions locally
Section titled “Test managed sessions locally”Local Run stubs every managed method, and withManagedSession runs over the stubbed createManaged and closeManaged. Stub sessions have a fake cdpUrl and no liveViewUrl; tasks complete at once with no result, so the book example’s validation fails until you override browserAutomation.tasks.get with a valid result. Verify CDP control, profiles, and video with a small production run.
Usage and limits
Section titled “Usage and limits”One-shot screenshots and browser sessions have different usage lifecycles. A one-shot capture is accounted for on that call. Call sessions.close(sessionId) to close the browser and request settlement of its usage. Automatic session expiry does not perform settlement. A screenshot attached to an existing session participates in that session’s usage rather than creating a separate one-shot session.
A managed session authorizes $1 for each started hour of maxDurationMinutes plus a $5 task allowance: $6 by default and $9 at 240 minutes. Tasks have no authorization of their own; their steps count toward the session’s usage. A creation retry that returns the existing session does not authorize again. The authorization is a payment hold, not the charge: closeManaged settles the reported usage up to that amount, and if usage cannot be read yet, settlement stays pending instead of capturing the whole hold.
Use the signed-in capability catalog for currently available pricing. Close sessions promptly, avoid idle waits, and use one-shot screenshots when no interaction or shared state is required.
© 2026 Sapiom, Inc.