Bot Traffic From Price Comparison and Reseller Scrapers
Comparison engines and unauthorized resellers live on your prices and stock data, refreshed by the minute. How to tell helpful indexing from extraction, and what levers you have.
Somewhere right now, a scraper is reading your prices. It might belong to a comparison service your customers genuinely use, a marketplace seller undercutting you with your own product data, or a competitor's pricing tool adjusting against you hourly. The requests look similar. The value to your store ranges from real referral revenue to pure extraction, and treating all of it identically gets both ends wrong.
Price scraping is not going away, because prices on the open web are a resource everyone builds on. The useful skill is separating the scrapers you benefit from, the ones you tolerate, and the ones worth actively stopping, using evidence rather than a block everything instinct that also blocks your customers' tools.
What the traffic looks like when prices are the target
Price scrapers have a diet: product pages and feed like endpoints, visited in catalog order, refreshed on suspiciously regular intervals, with little interest in images, reviews, or anything a shopper reads. They often arrive from hosting provider networks rather than consumer connections, and they care intensely about variants and stock states. A nightly pass that walks your catalog alphabetically at machine pace is not a devoted customer. It is a refresh cycle with a database behind it.
The three categories, and why the middle one is hard
Known comparison and shopping services that send real buyers sit in the first category: name them, verify them, let them work. Competitor and reseller extraction sits in the last: no referral value, pure cost, sometimes arming the exact listings that undercut you. The hard middle is unknown but polite scrapers, slow, identified, possibly a future channel, possibly a competitor's vendor. Judge the middle by behavior over weeks and by whether any value ever flows back, and keep them rate limited while they are being judged.
Levers that work better than a blanket block
Rate shaping by behavior, so aggressive refresh cycles get slowed to uselessness while ordinary browsing never notices. Feed control, publishing the prices you want syndicated through official product feeds and keeping everything else less convenient to harvest. And watermarking at the edges: slight, legitimate variations in how data is presented per channel can reveal where leaked pricing is being consumed, turning an anonymous extraction problem into an identified conversation.
The data you publish decides what you defend
Every public data point has a defense cost, so choose which ones earn it. Final prices must be public, that is commerce. Live stock counts by variant, cost revealing discount structures, and pre launch product data usually do not need the same exposure, and each one you close removes a scraper incentive without touching a single shopper's experience. The strongest scraper strategy is boring: publish what selling requires, protect what it does not, and spend enforcement effort only where extraction demonstrably costs you sales.
Related reading
If this topic is useful, these earlier guides go deeper on connected questions: How Bots Distort Your Conversion Rate Without Buying Anything, Bot Traffic on Gift Card Balance Check Pages: The December Fraud Rehearsal, and How Bots Distort Black Friday Waitlists and Early Access Lists.
Frequently asked questions
Is price scraping illegal?
It depends on jurisdiction, method, and what is taken, and it is rarely the practical first lever. Stores get further, faster, with technical controls, feed strategy, and terms enforcement against identified parties than with legal threats aimed at anonymous traffic. Specific cases with named parties are a counsel conversation.
How do I know if a comparison site is sending real value?
Match their scrape pattern against referral reality: sessions arriving from the service, converting at something like normal rates. A scraper with heavy extraction and no measurable referrals over a full quarter is not a channel, whatever it calls itself, and it can be treated as extraction.
Can rate limiting hurt real shoppers?
Done by behavior, rarely: shoppers do not request four hundred product pages in catalog order at fixed intervals. Rate limits aimed at machine patterns, with generous room for fast human browsing, slow extraction below usefulness while leaving customers untouched. Start with observation mode if you are unsure.
Should I hide stock levels to stop scrapers?
Exact live counts are rarely needed by shoppers. Showing availability states, in stock, low, out of stock, serves buyers fully while removing one of the most valuable scraper data points, especially for resellers timing their own listings against your inventory.