Crawlbot Chrome Extension: Real-Time Shelf Intelligence on Every Product Page

Brand managers spend hours each week browsing competitor product pages, manually copying prices into spreadsheets, and toggling between retailer sites to compare content quality. Crawlbot Market Intel v2.0 eliminates that workflow entirely. It is a Chrome extension that opens a side panel on any supported product page and instantly surfaces competitor pricing, content health scores, price history charts, and actionable recommendations - all without leaving the page you are already on.

The extension connects directly to Crawlbot's database of daily product scrapes across 36 retailers in five countries. Every data point you see in the side panel is the same data that powers our automated reports and Brand Checker tool. The difference is context: instead of navigating to a dashboard, the intelligence comes to you while you are already browsing the shelf.

What the extension does

Crawlbot Market Intel runs as a Chrome side panel. When you click the extension icon, a panel slides open on the right side of your browser. If you are on a supported retailer's product detail page (PDP), the extension's content script automatically detects the product - extracting the MPN, title, brand, and price from the page DOM using JSON-LD, meta tags, or retailer-specific parsing logic. It then queries the Crawlbot API to pull back everything we know about that product across every retailer we monitor.

The side panel is divided into seven sections, each designed to answer a specific question a brand or category manager would ask while looking at a product page:

• Product Header - Shows the detected retailer, MPN, product title, current price, and brand. This confirms the extension has correctly identified the product before you look at the competitive data.
• Price Comparison - A ranked table of every retailer that stocks this exact product (matched by MPN), sorted from lowest to highest price. Each row shows the retailer name, price in local currency, and the percentage difference from the best price. Colour-coded status indicators flag whether your listing is competitively priced.
• Content Health - For each retailer listing, shows photo count, video presence, and A+ content status as colour-coded indicators (green, amber, red). A listing with 8 photos, a video, and A+ content shows three green dots. A listing with 2 photos, no video, and no enhanced content shows three red dots. This is the same data our Content Inspection system captures daily.
• Reviews - Review score (out of 5) and total review count for each retailer, with status indicators that highlight where social proof is weak compared to competitors.
• Price History (90 days) - An interactive line chart showing price movements over the last 90 days, with separate lines for each retailer. Built on Chart.js with time-series axes, this makes it trivial to spot promotional cycles, permanent price drops, or retailers that have quietly increased their price.
• Top Competitors - The five closest competing products in the same category and price range, showing their name, brand, price range across retailers, number of stockists, and average review score.
• Recommendations - Prioritised action items generated from the data. Examples: "Price is 12% above cheapest retailer," "Missing video on Currys," "Only 3 reviews on AO - consider review generation campaign." Each recommendation is tagged by priority (high, medium, low).

How product detection works

The extension uses a content script that runs on every supported retailer URL. When the page finishes loading, the script attempts to extract product identifiers using a three-stage process. First, it checks for structured data in JSON-LD format - many retailers embed a @type: "Product" schema with MPN, brand, and price. Second, it falls back to OpenGraph meta tags and retailer-specific DOM selectors. Third, for retailers that use non-standard structures (Magento stores, custom frameworks), it applies retailer-specific parsing logic to extract the MPN from spec tables, description text, or data attributes.

The extracted product data is sent to the background service worker, which calls the Crawlbot API's product search and diagnostic endpoints. The API matches the product using our four-stage matching waterfall: MPN exact match, then EAN match, then spec hash, then creation of a new canonical record. If a match is found, the API returns the full diagnostic payload including cross-retailer pricing, content health, price history, and competitor analysis.

Manual search when auto-detection misses

Not every product page includes structured data that the content script can parse. Some retailers render product details client-side, use heavy JavaScript frameworks, or simply do not include MPNs on their pages. When the extension cannot auto-detect a product, it shows a "No Product Detected" screen with a manual search box. You can type any MPN, product name, or keyword and the extension will search Crawlbot's database directly.

If a product genuinely is not in the database - perhaps it is a new launch that has not been scraped yet - you can submit a scrape request directly from the extension. The "Request this product to be scraped" button opens a form pre-filled with the current page URL, where you can add the MPN and any notes. The request goes to the Crawlbot team for review and the product is typically added to the scraping schedule within 24 hours.

Supported retailers across five countries

The extension activates its content script on product pages from 36 retailers across the UK, South Africa, France, Poland, and Ireland. The full list:

How to install the extension

Crawlbot Market Intel v2.0 is a Manifest V3 Chrome extension distributed directly to Crawlbot subscribers. Installation takes under a minute:

1. Download the extension package from your Crawlbot dashboard or request it from your account manager. You will receive a .zip file containing the extension files.
2. Open Chrome extensions by navigating to chrome://extensions/ in your browser.
3. Enable Developer mode using the toggle in the top-right corner of the extensions page.
4. Click "Load unpacked" and select the unzipped extension folder. Chrome will install the extension and show the Crawlbot icon in your toolbar.
5. Pin the extension by clicking the puzzle piece icon in Chrome's toolbar and pinning Crawlbot Market Intel for easy access.
6. Log in by clicking the extension icon on any page. The side panel will open with a login form. Use your Crawlbot account credentials - the same ones you use for the web dashboard. Your session persists until the token expires (24 hours) or you clear browser storage.

Once logged in, simply navigate to any product page on a supported retailer. Click the Crawlbot icon to open the side panel and the data loads automatically. No configuration, no API keys, no setup beyond the initial login.

Use case: competitor pricing research

The most immediate use case is price benchmarking while browsing. Suppose you are a brand manager for a laptop manufacturer and you are reviewing your product pages on Currys. You land on one of your SKUs and open the Crawlbot side panel. The Price Comparison section instantly shows you that the same laptop is listed at five other UK retailers, with Box and Ebuyer both undercutting Currys by 4%. The 90-day price chart reveals that Argos dropped their price two weeks ago and has not reverted, suggesting a permanent repositioning rather than a temporary promotion.

Without the extension, gathering this information would require opening five separate tabs, finding the same product on each retailer (not always straightforward when URLs and product names differ), and manually recording prices. With the extension, it takes one click and three seconds.

For more detailed pricing analysis across your full catalog, see our pricing plans which include automated price alerts and daily price tracking reports.

Use case: content quality audit on the go

The Content Health section turns every browsing session into a mini content audit. As you click through your product pages on any retailer, the side panel shows you at a glance which listings have full photo sets, which have video, and which have A+ enhanced content. Red indicators stand out immediately, so you can flag content gaps in real time rather than waiting for a quarterly audit report.

This is particularly powerful when combined with physical store visits or trade show conversations. If a retail buyer mentions that your product page is underperforming, you can open the Crawlbot extension on your phone (via Chrome mobile) or laptop right there and show them exactly what content is live versus what is missing. The data backs up the conversation with specifics rather than generalities.

For a deeper look at how content quality affects conversion, read our analysis on how product content quality impacts sales on Currys, Amazon, and Argos.

Use case: monitoring competitor shelves

The extension is not limited to your own products. When you browse a competitor's product page, the same intelligence is available. Visit a competitor laptop listing on Amazon and the side panel shows you their pricing across all retailers, their content quality, their review strength, and which other products compete in the same category slot.

The Top Competitors section is especially useful here. It shows the five closest competing products by category and price, along with their price range, stockist count, and average review score. This gives you a quick read on the competitive landscape around any product without running a separate query in the main dashboard.

Combine this with Crawlbot's Share of Voice tracking for a complete picture. The extension shows you the product-level detail; the SoV dashboard shows you the category-level trends over time.

How it fits into the Crawlbot platform

The Chrome extension is one interface into the broader Crawlbot intelligence platform. Think of it as the contextual layer - the data that surfaces exactly when and where you need it, without requiring you to log in to a separate dashboard or wait for an email report.

The full platform includes:

• Share of Voice tracking - Hourly category page scraping across 26 retailers to track brand visibility, sponsored vs organic positions, and competitive movements.
• Content Inspection - Daily PDP scraping capturing 62 fields per product including photos, video, A+ content, specs, reviews, and pricing.
• Automated reports - Daily and weekly email reports with visibility maps and content scorecards.
• Price alerts - Real-time notifications when competitor prices change or when your products drop below or above defined thresholds.
• Brand Checker - A free self-service tool for a quick snapshot of your brand's digital shelf presence.

The extension uses the same API and the same JWT authentication as the web dashboard. Your Crawlbot login works everywhere. Data is always live - when you open the side panel, it queries the API in real time, so you are always seeing the latest scrape results rather than cached or stale data.

Technical details for IT teams

For IT and security teams evaluating the extension, here are the key technical details:

• Manifest V3 - Built on Chrome's latest extension platform with a service worker background script, not the deprecated Manifest V2 background page.
• Permissions - Requires sidePanel, storage, activeTab, and tabs. No access to browsing history, bookmarks, or downloads. Host permissions are limited to https://crawlbot.pl/* for API calls.
• Content scripts - Run only on matched retailer PDP URLs (the 36 retailers listed above). They read product data from the page DOM but do not modify the page or inject any visible elements.
• Authentication - JWT tokens stored in chrome.storage.local with 24-hour expiry. Credentials are sent over HTTPS to crawlbot.pl only.
• No third-party tracking - The extension does not include analytics SDKs, ad trackers, or any third-party scripts. The only external communication is with the Crawlbot API.
• Session storage - Tab-specific data is held in chrome.storage.session which is cleared when Chrome closes.

Getting started

If you are already a Crawlbot subscriber, request the extension package from your account manager or download it from the dashboard. Install it, log in, and start browsing. Every product page you visit on a supported retailer will have competitive intelligence one click away.

If you are not yet using Crawlbot, the extension is included with every plan. Our pricing page has the full breakdown, or you can start a free trial to experience the extension alongside the full platform. The extension alone replaces hours of manual competitor research each week. Combined with automated SoV tracking and Content Inspection, it gives brand teams a continuous, real-time view of the digital shelf that no amount of manual browsing can match.

For brands competing across multiple UK and South African retailers, the question is not whether you need this level of visibility. The question is how long you can afford to operate without it.