Browser Automation

Browser Automation lets a deployed agent capture a webpage or operate a stateful cloud browser. Agent steps call the typed ctx.sapiom.browserAutomation capability; Sapiom supplies the authenticated cloud connection when the deployed agent runs.

Choose the lightest browser mode

What the agent needs	Recommended operation
An image of a public webpage	`browserAutomation.screenshot(...)`
Navigation, clicks, forms, page evaluation, or shared cookies	`browserAutomation.withSession(...)` plus a CDP-compatible client
A browser that starts with a previously provisioned login identity	`browserAutomation.withSession(..., { identityId })`
The session settlement or manual control over when the session is closed	`browserAutomation.sessions.create()` and `sessions.close(sessionId)`

Do not open a session just to capture a public URL. A one-shot screenshot has no session to manage. Use a session when the browser must preserve state across multiple actions.

Capture a webpage

Call screenshot for a one-shot capture:

const screenshot = await ctx.sapiom.browserAutomation.screenshot({
  url: "https://example.com",
  fullPage: true,
});

return {
  screenshotUrl: screenshot.url,
  expiresAt: screenshot.expiresAt,
};

The URL must be non-empty. Optional fields control viewport width and height, waitMs, full-page capture, and "png" or "jpeg" output. imageQuality applies to JPEG output.

The result contains an absolute hosted url and its ISO 8601 expiresAt. Treat the URL as temporary rather than storing it as a durable artifact.

Operate an interactive browser

A session exposes a Chrome DevTools Protocol WebSocket at session.cdpUrl. Connect a compatible library such as Playwright or Puppeteer to navigate, click, fill forms, evaluate page content, and work with cookies.

This example assumes the agent project includes playwright-core:

import { chromium } from "playwright-core";

const result = await ctx.sapiom.browserAutomation.withSession(
  async (session) => {
    const browser = await chromium.connectOverCDP(session.cdpUrl);

    try {
      const context = browser.contexts()[0];
      if (!context) throw new Error("Browser session has no default context");

      const page = context.pages()[0] ?? (await context.newPage());
      await page.goto("https://example.com", {
        waitUntil: "domcontentloaded",
      });

      const screenshot = await session.screenshot({ fullPage: true });
      return {
        title: await page.title(),
        finalUrl: page.url(),
        screenshotUrl: screenshot.url,
        screenshotExpiresAt: screenshot.expiresAt,
      };
    } finally {
      await browser.close();
    }
  },
);

session.screenshot() injects the current session ID, so url is optional after the browser has navigated. Playwright’s connectOverCDP supports Chromium-based browsers and provides lower-fidelity control than Playwright’s own protocol; keep the automation to behavior the target browser actually supports.

There are two cleanup layers in this example. browser.close() disconnects the Playwright client. After the callback finishes or throws, withSession attempts to close the Sapiom browser session in a finally block.

Pass an existing identity ID to open the browser with its stored login state:

const result = await ctx.sapiom.browserAutomation.withSession(
  async (session) => {
    // Connect a CDP client and operate the authenticated browser here.
    return { expiresAt: session.expiresAt };
  },
  { identityId: "<identity-id>" },
);

browserAutomation.identities.create(...) provisions an identity from a login-page URL and credentials. Treat that as a deliberate provisioning operation: read credentials from agent secrets, never place them in source code, runtime defaults, caller input, or logs, and do not create a new identity during every ordinary run.

Test the behavior locally

Local Run replaces ctx.sapiom.browserAutomation calls with deterministic stubs. Built-in defaults are enough to exercise one-shot screenshots, session callbacks, session-bound screenshots, and the withSession close path without creating a cloud browser.

Override the exact capability path when a step branches on a particular result:

{
  "version": 1,
  "steps": {
    "capture-page": {
      "browserAutomation.screenshot": {
        "url": "https://fixtures.example/screenshot.png",
        "expiresAt": "2099-01-01T00:00:00.000Z"
      }
    }
  }
}

After the run, require both unusedStubs and stubWarnings to be empty. A misspelled path or implausible response shape means the override did not prove the intended branch.

Manage session lifecycle in production

Prefer withSession for ordinary automation. It attempts session closure even when the callback throws, while preserving the callback’s original result or error if closure also fails.

Use the manual lifecycle only when the agent needs the returned settlement or cannot fit the work inside one callback:

const session = await ctx.sapiom.browserAutomation.sessions.create();

try {
  // Connect through session.cdpUrl and do the browser work.
} finally {
  const settlement =
    await ctx.sapiom.browserAutomation.sessions.close(session.sessionId);

  console.log(settlement.settled, settlement.creditsUsed);
}

Read expiresAt and maxDurationSec from the created session rather than assuming a fixed lifetime. A CDP connection error comes from the browser client; a capability request failure comes from Browser Automation. Inspect the failed production step to distinguish the two.

Usage and limits

One-shot screenshots and browser sessions have different usage lifecycles. A one-shot capture is accounted for on that call. Session activity is settled when the session closes or expires; a screenshot attached to an existing session participates in the session lifecycle rather than creating a separate one-shot session.

Use the signed-in capability catalog for currently available pricing. Close sessions promptly, avoid idle waits, and use one-shot screenshots when no interaction or shared state is required.

Use capabilities in a step Work from the installed ctx.sapiom TypeScript contract.

Test locally Run real step code with deterministic capability responses.

Deploy an agent Build an immutable cloud version before testing a real browser session.

Inspect a run Separate capability failures from browser-client and target-site failures.