Scrapers That Hit Size Guides, Stock Feeds, and the Quiet Corners
Not all scraping aims at product pages. Size guides, availability endpoints, and feed files tell a competitor about conversion weak spots and inventory position. Here is how to watch the quiet URLs.
Defensive attention goes to product and checkout pages, so extraction moves to the edges: the size guide a rival uses to map your fit problems, the availability endpoint the product page calls, the feed file left readable at a guessable path. None of these trigger a checkout alarm. All of them are in the logs, being read by someone, at a volume no human audience explains.
Inventory data is the sensitive part. Repeated, systematic checks of stock status on scarce or high value items are not casual browsing. They feed resale decisions, undercutting, and launch timing. If a human had to refresh a page that often, they would set an alert. A script has no such tell, except in the pattern itself.
Related reading on this site: Bots Inflating Add to Cart Metrics and How to Spot Them, Bots Scraping Prices and Inventory: How to Spot the Pattern, Spotting Fake Traffic in Social Referral Spikes.
Inventory the quiet endpoints
List what your storefront exposes beyond pages: availability and price endpoints, search APIs, feed files for shopping channels, sizing and fit content, store stock lookup by postcode. For each, note who is meant to call it, at what rate, and whether it needs to be public at all. Feeds intended for one channel partner are often readable by anyone who finds the URL, which is a publishing decision made by accident.
Then read access logs for those paths alone. Volume by source network, regularity of interval, and breadth across products separate the channel partner syncing hourly from the unknown network enumerating your stock position every few minutes.
Set a different bar for stock data
Price is public by nature, a shop window. Stock depth is operational information. Where the platform allows, return availability as a state, in stock, low, out, rather than exact counts to anonymous callers, rate limit the lookup endpoints harder than content pages, and require the signed or tokenised feed URLs your channels actually support.
Watch for the hybrid pattern: low rate, residential network, perfectly ordinary browser fingerprint, checking a short list of high value products very frequently. That is monitoring built to stay under thresholds, aimed at exactly the products where stock knowledge is worth money. Behaviour over time is the signal, and it only appears if someone keeps the history.
Do not break the legitimate quiet users
Price comparison services, accessibility tools, researchers, and your own monitoring all read quiet corners too, and search engines legitimately fetch feeds and sizing content. Verification and allow listing for named partners, behaviour based limits for everyone else, and a contact path for blocked researchers keep the control honest.
State the line in your review: extraction that mirrors the catalogue and its stock, at scale, covertly, is the target. Reading a public page is not. A control that cannot explain that distinction will eventually block a partner, a crawler, or a customer, and lose the support it needs to stay in place.
Frequently asked questions
Why would a scraper target a size guide?
Size guides reveal fit information, return prone categories, and product dimensions competitors can use to position against you. More often, the guide is one stop in a full catalogue mirror, where every content type is copied, not just prices and product names.
Is stock information really sensitive if the product page shows it?
A shopper seeing in stock is normal. A script recording stock status across the catalogue, at short intervals, builds a sales and inventory picture no single page view reveals. Sensitivity comes from scale and system, not from any one request.
How can exact stock counts be protected?
Show anonymous visitors a stock state rather than a count, rate limit availability endpoints, use tokenised feed URLs for partners, and monitor endpoint access by source and interval. Product pages rarely need to expose precise counts to sell effectively.
What distinguishes monitoring from extraction in the logs?
Monitoring revisits a defined set of items on a schedule, usually prices. Extraction works across the catalogue or an endpoint systematically, building a dataset, often including stock, attributes, and content. Intent is inferred from breadth, rate, and what data is taken together.