How to Automate CoStar / LoopNet (Vacancy/Listing Extraction, Comps, Analytics Pipelines — No API Required)

Introduction

CoStar and LoopNet are leading commercial real estate data and listing platforms. While CoStar offers APIs and data products for enterprise clients, browser automation provides a practical way to extract vacancy and listing data, pull comps, and feed analytics pipelines when API access is restricted or when you need data directly from the CoStar or LoopNet web experience.

Why Use Browser Automation for CoStar / LoopNet?

• Limited API Access: CoStar API and bulk data are often restricted to specific contracts or products
• Vacancy/Listing Extraction: Extract property listings, vacancy data, and availability from search and property pages
• Comps: Pull comparable sales, leases, and market comps from property or market views
• Analytics Pipelines: Feed listing and comp data into internal analytics, BI, or reporting pipelines
• UI-Only Data: Many market and comp views are available primarily in the web interface
• Custom Queries: Run geography, property type, and filter combinations that may not map to available APIs
• Audit and Research: Export data for internal analysis and underwriting

Setting Up CoStar / LoopNet Automation

Here's how to automate vacancy/listing extraction, comps, and analytics pipelines using CoStar or LoopNet with browser automation:

import { chromium } from 'playwright';

const response = await fetch("https://api.anchorbrowser.io/api/sessions", {
  method: "POST",
  headers: {
    "anchor-api-key": "YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    'headless': false,
    'proxy': { 'type': 'residential', 'country': 'US' }
  }),
});

const { id } = await response.json();
const connectionString = `wss://connect.anchorbrowser.io?apiKey=YOUR_API_KEY&sessionId=${id}`;

const browser = await chromium.connectOverCDP(connectionString);
const context = browser.contexts()[0];
const ai = context.serviceWorkers()[0];
const page = context.pages()[0];

await page.goto("https://www.loopnet.com");

await ai.evaluate(JSON.stringify({
  prompt: 'Log in to LoopNet or CoStar if required. Accept cookies or dismiss modal if shown. Navigate to search or homepage.'
}));

Use Case 1: Vacancy/Listing Extraction

Extract vacancy and listing data from search results and property pages:

const runVacancyListingExtraction = async (page, ai, criteria) => {
  await ai.evaluate(JSON.stringify({
    prompt: 'Navigate to search. Set location, property type (office, retail, industrial, etc.), and filters. Run search.'
  }));
  
  await page.waitForLoadState('networkidle');
  
  await ai.evaluate(JSON.stringify({
    prompt: 'From search results: extract listing ID, address, property type, size (SF), asking rent/price, vacancy or availability. Return as JSON array. Respect pagination limit. Do not include broker PII.'
  }));
  
  if (criteria.detailLevel === 'full') {
    await ai.evaluate(JSON.stringify({
      prompt: 'Open first N property detail pages. Extract: description, cap rate, NOI if visible, tenant info (high-level only). Return as JSON. No PII. Limit rate of requests.'
    }));
  }
  
  const result = await ai.evaluate(JSON.stringify({
    prompt: 'Return collected listings as JSON array. No credentials or PII.'
  }));
  
  return { result: typeof result === 'string' ? JSON.parse(result) : result, completedAt: new Date().toISOString() };
};

Use Case 2: Comps

Pull comparable sales, leases, and market comps from property or market views:

const runComps = async (page, ai, criteria) => {
  await ai.evaluate(JSON.stringify({
    prompt: criteria.address || criteria.market
      ? 'Navigate to property or market view for the given address or market. Open the property or market page.'
      : 'Navigate to CoStar or LoopNet. Open the comp or market view needed.'
  }));
  
  await page.waitForLoadState('networkidle');
  
  await ai.evaluate(JSON.stringify({
    prompt: 'From the page: extract comparable sales, leases, or market comps (address, size, price/rent, date). Open comparables or market data section if available. Return as JSON. No broker or tenant PII.'
  }));
  
  await page.waitForLoadState('networkidle');
  
  const result = await ai.evaluate(JSON.stringify({
    prompt: 'Return comp data as JSON. No credentials or PII.'
  }));
  
  return { result: typeof result === 'string' ? JSON.parse(result) : result, completedAt: new Date().toISOString() };
};

Use Case 3: Analytics Pipelines

Feed extracted data into analytics, BI, or reporting pipelines:

const runAnalyticsPipeline = async (page, ai, criteria) => {
  const listingData = await runVacancyListingExtraction(page, ai, { ...criteria, detailLevel: criteria.detailLevel || 'summary' });
  const compData = criteria.includeComps ? await runComps(page, ai, criteria) : null;
  
  const payload = {
    source: 'costar-loopnet',
    extractedAt: new Date().toISOString(),
    listings: listingData.result,
    comps: compData?.result || null,
    geography: criteria.location || null,
    propertyType: criteria.propertyType || null
  };
  
  if (criteria.webhookUrl) {
    await fetch(criteria.webhookUrl, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(payload)
    });
  }
  
  return { payload, completedAt: new Date().toISOString() };
};

Exporting and Storing Results

Save extracted data for analytics while avoiding PII:

const exportCoStarData = async (data) => {
  const sanitized = JSON.stringify(data, (k, v) => {
    if (['email', 'phone', 'brokerName', 'tenantName'].includes(k)) return undefined;
    return v;
  });
  return { json: sanitized, completedAt: new Date().toISOString() };
};

Best Practices for CoStar / LoopNet Automation

• Terms of Service: Review CoStar and LoopNet ToS and license terms; use automation only where permitted under your agreement
• Rate Limiting: Add delays between requests; avoid burst traffic that could impact the site or trigger blocks
• No PII: Do not collect or store broker, tenant, or owner PII beyond what is necessary and legally permitted
• Vacancy/Listing Extraction: Prefer official CoStar data feeds where available; use browser automation for gaps or internal pipelines within policy
• Comps and Analytics: Document source and extraction date for compliance and audit; treat comps as estimates where applicable
• Error Handling: Handle login expiry, captchas, and session timeouts gracefully; back off on errors
• Compliance: Align with data licensing and commercial real estate data use policies

Handling Authentication and Access

CoStar and LoopNet often require login for full data access:

const handleCoStarAuth = async (page, ai, credentials) => {
  await page.goto("https://www.loopnet.com");
  
  await ai.evaluate(JSON.stringify({
    prompt: 'Sign in with the provided credentials. Complete SSO or MFA if required. Wait for the portal to load.'
  }));
  
  await page.waitForLoadState('networkidle');
};

Resources

• Anchor Browser Documentation - API reference and guides
• Anchor Browser Playground - Try browser automation in your browser

Conclusion

Browser automation provides a flexible way to extract vacancy and listing data, pull comps, and feed analytics pipelines from CoStar and LoopNet when API or data feed access is limited. By using intelligent browser agents with respectful rate limits and ToS awareness, you can automate vacancy/listing extraction, comp pulls, and pipeline feeds for internal analytics and underwriting. Whether you need listing and vacancy data by market, comparable sales and leases, or structured data for BI and reporting, browser automation can support your commercial real estate workflow while you stay within license terms and applicable law.

Start automating your CoStar / LoopNet vacancy extraction, comps, and analytics pipelines today.