Haystack + Anchor: Browser Agents in Python

Haystack is deepset's open-source AI orchestration framework for building production-ready LLM applications. Its core abstraction—modular pipelines of retrievers, generators, routers, and tools—gives you explicit control over exactly how information moves through your system. What the framework doesn't include out of the box is a browser.

Anchor gives it one. This guide walks through connecting a Haystack ChatAgent to real, cloud-hosted browser sessions so your pipelines can navigate pages, fill forms, and extract structured data from the live web.

How they fit together

Haystack's ChatAgent accepts a list of Tool objects. Each tool wraps a Python function with a name, description, and parameter schema—enough for the model to decide when and how to call it. Anchor provides the browser infrastructure: persistent sessions with stable fingerprints, Playwright-compatible CDP endpoints, and IPs that pass bot filters. You wire Anchor into a tool, hand the tool to the agent, and Haystack handles the rest.

Setup

pip install haystack-ai openai playwright requests
playwright install chromium

export ANCHOR_API_KEY="your-anchor-api-key"
export OPENAI_API_KEY="your-openai-api-key"

Defining the browser tool

import os
import requests
from playwright.sync_api import sync_playwright
from haystack.components.agents import ChatAgent
from haystack.tools import Tool

ANCHOR_API_KEY = os.environ["ANCHOR_API_KEY"]

def browse(url: str, goal: str) -> str:
    '''Open a URL in an Anchor session and return visible page text.'''
    session = requests.post(
        "https://api.anchorbrowser.io/v1/sessions",
        headers={"anchor-api-key": ANCHOR_API_KEY, "Content-Type": "application/json"},
        json={"headless": True},
        timeout=30,
    ).json()["data"]

    try:
        with sync_playwright() as p:
            browser = p.chromium.connect_over_cdp(session["cdp_url"])
            page = browser.contexts[0].pages[0]
            page.goto(url, wait_until="networkidle", timeout=60_000)
            text = page.evaluate("() => document.body.innerText")
            browser.close()
        return text[:6000]
    finally:
        requests.delete(
            f"https://api.anchorbrowser.io/v1/sessions/{session['id']}",
            headers={"anchor-api-key": ANCHOR_API_KEY},
        )

browse_tool = Tool(
    name="browse",
    function=browse,
    description="Open a URL and return the page text. Use this to research any website.",
    parameters={
        "type": "object",
        "properties": {
            "url": {"type": "string", "description": "The URL to visit"},
            "goal": {"type": "string", "description": "What to look for on the page"},
        },
        "required": ["url", "goal"],
    },
)

Running the agent

from haystack.components.generators.chat import OpenAIChatGenerator

agent = ChatAgent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o"),
    tools=[browse_tool],
    max_agent_steps=10,
)

result = agent.run(
    messages=[{
        "role": "user",
        "content": (
            "Compare the pricing pages of stripe.com and paddle.com "
            "and summarize the key differences in a markdown table."
        ),
    }]
)

print(result["messages"][-1].text)

Haystack's ChatAgent handles the full reasoning loop automatically—it calls browse as many times as needed, reads the results, and returns a final answer. You write zero loop logic.

Persistent profiles for authenticated sessions

Most production tasks require a login. Pass a profile_id when creating the session: Anchor saves browser state—cookies, local storage, fingerprints—under that ID. Every subsequent run rehydrates the authenticated state without re-entering credentials.

def browse_authenticated(url: str, goal: str) -> str:
    session = requests.post(
        "https://api.anchorbrowser.io/v1/sessions",
        headers={"anchor-api-key": ANCHOR_API_KEY, "Content-Type": "application/json"},
        json={
            "headless": True,
            "profile": {"id": "my-saas-account", "persistent": True},
        },
        timeout=30,
    ).json()["data"]
    # ... rest of function unchanged

auth_browse_tool = Tool(
    name="browse_authenticated",
    function=browse_authenticated,
    description="Open a URL as a logged-in user using a saved browser profile.",
    parameters={
        "type": "object",
        "properties": {
            "url": {"type": "string"},
            "goal": {"type": "string"},
        },
        "required": ["url", "goal"],
    },
)

Production tips

• Truncate tool output — Page text can exceed 100k characters. The [:6000] cap above is a safe starting point; narrow it further for token-sensitive pipelines.
• Always clean up sessions — Use a finally block to call DELETE /v1/sessions/{id}. Leaked sessions count against your concurrent limit.
• Use networkidle for SPAs — JavaScript-rendered content isn't visible on load. networkidle waits until all XHR activity settles.
• Geo-route with Anchor VPN — Add "vpn": {"type": "datacenter", "country_code": "DE"} to the session payload for region-specific price monitoring or compliance checks.
• Swap models freely — Replace OpenAIChatGenerator with Haystack's Anthropic or Gemini generators without changing any tool code. The browser layer stays independent of the model layer.

What's next

From here, add a fill_form tool using page.fill() and page.click() for interactive workflows, or chain multiple browser tools in a Haystack Pipeline for multi-step research agents. The modular design means you can swap, extend, or compose tools without touching the reasoning loop.

Try Anchor free →