The Uptime Monitor in Your Bot Report: The False Positives You Keep Counting
The bot report says 43 percent of the site's traffic is bots, and the number has been climbing for a month. The team starts a bot mitigation project. Then someone checks the source, and the bots are the site's own uptime monitor, pinging the homepage every minute from five locations, and the bot share was never a bot problem at all.
The monitor that counts as a bot
An uptime monitor is a service that checks a site is alive by requesting a page at a regular interval, usually every minute or every five minutes. The request is a bot by definition, because no human made it, and it lands in the traffic report unless it is filtered out. Every uptime service publishes its user agent and its source IP ranges, and the filter that removes it belongs in the analytics tool, not in the report, so the raw data stays intact and the filtered view is the one the team reads.
The scale is what surprises people. A monitor that checks from five locations every minute makes 7,200 requests a day. A site with 10,000 human visits a day would see its bot share jump by more than 40 percentage points, and a site with 1,000 human visits a day would see the monitor dominate the report entirely. The monitor is not malicious. It is just counted, and the counting is the problem, because the bot report is supposed to measure the traffic that might be fraudulent, and the monitor is traffic that is neither human nor fraudulent, but it is counted as both.
The GIVT vs SIVT post draws the MRC line between the traffic that gets refunded and the traffic that does not. The point here is the source: the uptime monitor is a known bot, and known bots are the category the filter exists for. The MRC definition of General Invalid Traffic, GIVT, covers known bots, crawlers, and data center traffic, but a monitor is a legitimate service, and the correct treatment is to filter it, not to count it as fraud. The distinction matters, because a bot report that counts the monitor as fraud is a report that overstates the fraud, and a fraud number that is overstated is a number nobody trusts.
The other monitors hiding in the report
The uptime monitor is not the only legitimate bot in the report. Search engine crawlers, the Googlebot that indexes the site, the Bingbot that does the same, and the social crawlers that fetch link previews are all bots, and they are all supposed to be there. The crawler post draws the line: a crawler that identifies itself, respects robots.txt, and requests at a reasonable rate is a legitimate bot, and a scraper that hides its identity, ignores robots.txt, and requests at a rate no human could match is not.
The performance monitors are the same category. A synthetic monitoring service that loads the page from a dozen locations every few minutes, a RUM beacon that fires on every page load, a status page that checks the site every minute, all of them make requests that land in the traffic report, and all of them are legitimate. The filter has to cover them all, and the filter has to be maintained, because the services change their IP ranges and their user agents, and a filter that was correct last year is not correct this year.
The check is the user agent list. Export the top user agents from the bot report, and read them. The ones that belong to a service the site pays for, the uptime monitor, the synthetic monitor, the status page, are the ones to filter. The ones that belong to no service the site recognizes, the headless browser with a generic user agent, the datacenter IP with no reverse DNS, are the ones to investigate. The list is the diagnosis, and it takes an afternoon to read.
The filter that fixes the number
Filter by the monitor's user agent and its source IP ranges, in the analytics tool, so the raw data stays intact and the filtered view is the one the team reads. Do not delete the raw data. The raw data is the evidence, and the filtered view is the report. A team that deletes the raw data cannot audit the filter, and a filter that cannot be audited is a filter that cannot be trusted.
Then re-run the bot share. The number will drop, and the drop is the measure of how much of the bot problem was the site's own monitoring. The remaining bots are the ones worth investigating, and the investigation is worth doing, because the report now measures the traffic that might be fraudulent, instead of the traffic the site asked for. The monitor is still there, still pinging every minute, and still keeping the site alive. It is just no longer counted as a bot problem, because it never was one.
Frequently asked questions
What is an uptime monitor, and why does it appear in bot reports?
An uptime monitor is a service that checks your site is alive by requesting a page at a regular interval, usually every minute or every five minutes. The request is a bot by definition, because no human made it, and it lands in the traffic report unless it is filtered out.
Can a monitor really distort the bot share that much?
Yes, and that is the point. A monitor that checks from five locations every minute makes 7,200 requests a day, and a site with 10,000 human visits a day would see its bot share jump by more than 40 percentage points. The monitor is not malicious. It is just counted.
How do I filter an uptime monitor out of the report?
Filter by the monitor's user agent and its source IP ranges. Every uptime service publishes both, and the filter belongs in the analytics tool, not in the report, so the raw data stays intact and the filtered view is the one the team reads.
Is an uptime monitor invalid traffic?
No. A monitor is a known bot, and known bots are a different category from invalid traffic. The MRC definition of General Invalid Traffic covers known bots, crawlers, and data center traffic, but a monitor is a legitimate service, and the correct treatment is to filter it, not to count it as fraud.