Product pages look uniform from a distance—title, price, image, add to cart—but underneath they are some of the least standardized pages on the web. Every storefront renders its price block differently, hydrates inventory status after the initial load, and reshuffles its DOM on every redesign. Scraping them well is less about grabbing HTML and more about getting a small set of decisions right before you write any extraction code.
Why Product Pages Are Harder Than They Look
Three things break naive scrapers on e-commerce sites specifically. First, pricing and stock are frequently injected by client-side JavaScript after the initial response, so a raw HTTP fetch returns a shell with no price at all. Second, the same field shows up in wildly inconsistent shapes across sites—"$49.99", "49.99 USD", and "USD 49.99/mo" can all represent the same underlying value. Third, catalog sites actively defend high-traffic pages against automated access, so requests that look like a script rather than a browser get degraded responses or blocks.
Start With a Schema, Not a Selector
The fix for inconsistent output is to define the shape you want before you extract anything, then let extraction fill it in and validation reject what doesn't fit. With Anchor, you can hand a page and a Pydantic model directly to an agent task instead of hand-rolling selectors for every layout:
from pydantic import BaseModel
from typing import Optional
import ast
class ProductSchema(BaseModel):
title: str
description: str
price: Optional[str] = None
structured_result = anchor_client.agent.task(
"Extract the product title, description, and price from this product page",
task_options={
"output_schema": ProductSchema.model_json_schema(),
"url": "https://www.example.com/dp/B0D7D9N7X3"
}
)
product = ProductSchema.model_validate(ast.literal_eval(structured_result))
print(product.title, product.price)Because the schema travels with the request, the same code handles a redesign or an entirely different storefront—the model adapts to the layout, and Pydantic rejects any result that doesn't match the contract.
Match the Tool to the Job
Not every scrape needs a full browser session. For one-shot fetches—pulling the rendered HTML of a single product or category page—Anchor's Web Unlocker is the leaner option: no session to start, no cold start, and it's already routed through stealth browsers with residential proxies and fingerprint handling built in.
curl -X POST "https://api.anchorbrowser.io/v1/tools/fetch/webpage" \
-H "anchor-api-key: $ANCHOR_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "url": "https://www.example.com/products/example-item" }'Reach for a full browser session instead when the task is multi-step—paging through search results, applying filters, or following "load more" before you can see the full catalog. A session keeps state across those steps the way a one-shot fetch can't.
Treat Scale as a Data Quality Problem
Once extraction works on one page, the failure modes shift from "did it parse" to "is the dataset trustworthy." A few habits pay off quickly: key every record on a stable identifier (SKU, ASIN, or the product URL) so re-scraping the same listing updates a row instead of duplicating it; flag records missing a required field like price or availability instead of silently writing nulls downstream; and stay inside each site's published rate limits and terms of service—sustainable scraping is a long-term data source, aggressive scraping gets your access revoked.
Getting product data right is mostly about front-loading these decisions—schema, tool, and identity strategy—before scale makes mistakes expensive. Anchor handles the rendering, proxying, and validation plumbing so your extraction logic can stay focused on the data itself. Get an API key and try it against your own product catalog.
