Introducing Action Index: From Inference to Indexing

Today we're introducing Action Index, execution memory for web agents. On Online-Mind2Web¹, gpt-5.6-luna with Action Index completes 93.3% of tasks, matching claude-opus-5 (93.0%) at $0.016 per task instead of $1.57, a 98x reduction in cost.

Web agents today re-derive the same workflows on every run. They screenshot, reason, click and reason again, and they pay for inference on each step, even when the same task on the same site was solved an hour earlier. Action Index stores workflows that have already succeeded and replays them as deterministic code. The model is called only at steps where the page has changed. Because the index is keyed by website, every successful run on a site makes the next run on that site cheaper.

Online-Mind2Web, 300 tasks on 136 live websites, graded with WebJudge. Cost per completed task is cost per task divided by success rate. Methodology is in the appendix.

The effect is not specific to one model or one benchmark. In paired runs on 120 tasks from the Skyvern task set, Action Index reduces p90 token usage by 74%, p90 steps by 75% and p90 wall-clock time by 61%. It also lifts open-weight models by 21 to 26 points: qwen3.8-27b rises from 67.7% to 88.7%, and GLM 5.3 from 58.7% to 84.7%.

The rest of this post covers how Action Index works and what we measured:

• How it works. Successful runs are captured as parameterized actions, retrieved by domain, and replayed as code with model fallback at the step level.
• Matching a frontier model. On Online-Mind2Web, a small model with Action Index reaches frontier success rates while costing less than the same small model on its own.
• Where the savings come from. Median runs improve modestly. The expensive tail of runs, where agents get lost, improves sharply.
• Open-weight models. Memory closes most of the gap between open models and the frontier on known workflows.
• Limitations. What Action Index does not do yet, and what our benchmarks do not show.

Action Index is available today to all Anchor accounts, including the free tier, inside web tasks, as tools on the Anchor MCP server and through the REST API. Any MCP-compatible agent can use it with a configuration change. See the Action Index documentation to get started.

How Action Index works

The standard agent loop treats every run as a new problem, so its cost scales with the number of steps rather than with how novel the task is. Most production agent traffic is not novel: it is the same portals, the same forms, the same lookup and checkout flows. Action Index separates the two cases. Novel work goes to the model. Known work is replayed.

A run with Action Index proceeds as follows:

1. The agent receives a task and queries the index for the target domain.
2. If a verified action matches, Anchor replays it as deterministic code in a browser session.
3. If the page has changed at a given step, the model handles that step only, and replay resumes.
4. If no action matches, the agent runs its normal loop. When that run succeeds, its path is verified and added to the index.

Three components implement this loop.

• Capture. When a run succeeds, Anchor records the workflow as a parameterized, deterministic action keyed by domain. A typical action is 1 to 15 KB.
• Retrieve. Agents query the index through MCP tools (search_action_index, get_action_index, execute_action_index) or the equivalent REST endpoints. A per-site MCP server exposes every active action for a single site as a typed tool.
• Replay with fallback. Actions execute as code inside an Anchor browser session. When a selector or page state no longer matches, the model takes over for that step and the run continues, so a small change to a page costs one inference call rather than a full re-derivation.

Matching a frontier model at a fraction of the cost

Action Index changes the small model on both axes at once. Compared with gpt-5.6-luna alone, the same model with Action Index completes 30.6 more points of tasks (62.7% to 93.3%) while costing 43% less per task ($0.028 to $0.016). Both effects have the same cause: steps that are replayed as code are steps that do not fail through reasoning errors and do not consume tokens.

Against claude-opus-5, we read the result as parity rather than a win. The 0.3-point difference comes from a single run per configuration and should not be interpreted as an improvement. The cost difference, however, is two orders of magnitude: $4.87 for the full 300-task run with Action Index, compared with $472. Our preliminary review attributes most of the remaining failures to proxy blocking and CAPTCHAs rather than to agent reasoning.

This is why we report cost per completed task alongside success rate. Leaderboards that rank by success rate alone favor the most expensive configuration. For an agent running the same workflow thousands of times a day, the relevant figure is what each completed task costs: $1.69 with claude-opus-5 and $0.017 with gpt-5.6-luna plus Action Index.

The tail is where the savings are

To isolate the effect of the index, we ran each of the first 120 tasks of the Skyvern task set twice, once with Action Index and once without. Both runs launched at the same time so that they shared site state, proxy and environment conditions.

Token, step and time figures cover the 58 tasks on which both runs completed without infrastructure errors. Success rate is measured across all 120 tasks.

Median runs improve modestly. The p90 improves sharply: total tokens fall by 74%, steps by 75% and wall-clock time by 61%. The expensive tail of runs is where agents get lost, revisiting pages, retrying interactions and re-planning. That tail accounts for a disproportionate share of an agent's bill, and it is the part a known path removes. Success rate also rose, from 55.8% to 63.3%, or 9 additional tasks out of 120.

The medians understate the effect for two reasons. These runs used a staging environment with infrastructure noise, including proxy failures, 502 errors and stalled browsers. Some domains in the set also had no entries in the index yet, so those runs could not benefit from it.

Small and open models benefit most

Online-Mind2Web success rate, graded with WebJudge.

The weaker the model, the larger the gain. GLM 5.3 improves by 26.0 points and qwen3.8-27b by 21.0 points. With Action Index, qwen3.8-27b, a 27B-parameter open-weight model, comes within 4.3 points of claude-opus-5 on the same benchmark. On workflows the index has already seen, memory substitutes for model scale. For teams that need to run agents on their own infrastructure, this narrows the gap between what they can deploy and what the frontier can do.

Limitations

• Coverage. Action Index helps only on sites and workflows that have already been solved. The first run on a new site pays full price. The benefit of the index grows with usage, which also means it is smallest on rarely visited sites.
• Benchmark scope. Online-Mind2Web and the Skyvern task set are public benchmarks, not customer workloads. Results on production traffic will depend on how repetitive that traffic is.
• Single runs. We report one run per configuration. Live-web benchmarks vary between runs, and we will publish repeated-run results as they complete.
• Efficiency on Online-Mind2Web. For the 300-task evaluation we report success rate and cost. Token, step and latency reductions are measured in the paired study above.
• Freshness. Sites change. Step-level fallback handles small changes, but an action that fails repeatedly must be re-verified or retired.

What's next

We are growing the index from production traffic and free-tier usage, extending Action Index to more inference providers using our remote MCP integration with Groq as the template, and preparing an open cost-to-completion evaluation so that others can reproduce these results and measure their own agents on the same terms.

Getting started

The quickest way to try Action Index is inside a web task: set use_action_index_tools: true on Perform Web Task, and the agent uses the cached actions for the site it is on and browses normally everywhere else. You can also look up a site's actions and run one directly through the REST API, with no agent involved.

From an MCP client such as Cursor, Claude or ChatGPT, connect to the Anchor MCP server at https://api.anchorbrowser.io/v1/mcp and call search_action_index, get_action_index and execute_action_index. To work with a single site, use https://api.anchorbrowser.io/v1/mcp/actions/{domain}, which exposes every verified action for that site as a typed tool.

The full guide, with code samples for each option, is in the Action Index documentation. MCP setup is covered at docs.anchorbrowser.io/advanced/mcp.

References

1. T. Xue, W. Qi, T. Shi, C. H. Song, B. Gou, et al., "An Illusion of Progress? Assessing the Current State of Web Agents," Conference on Language Modeling (COLM), 2025. arxiv.org/abs/2504.01382

Appendix: Evaluation methodology

Online-Mind2Web. The benchmark consists of 300 tasks on 136 live websites¹. We graded completions with WebJudge, the benchmark's automatic LLM-as-a-judge evaluator, which its authors report agrees with human judgment roughly 85% of the time. Each configuration was run once over all 300 tasks. Cost per task is the average cost recorded per task across the 300 tasks, and cost per completed task is cost per task divided by success rate.

Paired study. We used the first 120 tasks of the Skyvern task set in a staging environment. Each task ran twice, with and without Action Index, with both runs launched simultaneously so that they shared site state, proxy and environment conditions. Success rate is reported across all 120 tasks. Token, step and wall-clock figures are reported for the 58 tasks on which both runs completed without infrastructure errors.

Open-weight models. qwen3.8-27b and GLM 5.3 were evaluated on Online-Mind2Web with WebJudge, with and without Action Index, under the same protocol as the main results.