Google appears to have gotten much better at blocking the tools that scrape its search results, and rank tracking data across the industry took a hit because of it. One provider reported losing roughly 80 percent of the data it could pull. If you looked at a ranking report in the last week and something seemed off, this is probably why. Here is what happened and how to read your data until it settles.
What Happened
Barry Schwartz reported on September 18 that Google seems to have put new blocking mechanisms in place around September 13. The timing sits alongside a run of ranking movement, with reversals noted on September 4 and September 12 and further movement on September 15.
The blocking is aimed at AI scrapers, search scrapers, and third-party tracking tools. In other words, the tools most SEOs open every morning.
It also lines up with Google testing human-verification prompts and CAPTCHAs on search results. Bing has been doing something similar. Both search engines appear to be tightening the same door at the same time.
Which Tools Were Affected
The reports came from the providers themselves, which is what makes this more than speculation.
- Nozzle. Derek Perkins reported approximately "an 80% drop in data" he could retrieve from Google Search results.
- DataForSEO. Hit with similar restrictions over the same period.
- Sistrix. Their status page on September 16 said: "Google has made further changes to the way search results are delivered. As a result, our data collection is currently running at a reduced rate."
- Ahrefs and Semrush. Glenn Gabe's analysis found Ahrefs recovered its data fastest, Semrush recovered with a lag, and Sistrix lagged significantly behind both.
Steve Paine from Sistrix put the longer pattern into context, saying that "Google has been continuously changing how search results are delivered for about a year now, currently at a very high pace." That is the part worth holding onto. This is not a one-week outage. It is an acceleration of something that has been building for a year.
Why This Matters More Than It Sounds
One of the biggest mistakes I see is treating rank tracking data as a measurement of reality rather than a sample of it. It has always been a sample. Your tool checks a keyword from a particular location at a particular time and reports what it saw.
When collection rates drop, the sample gets smaller and noisier. A keyword that looks like it fell six positions might have fallen six positions, or your tool might have gotten a partial result on a bad day.
Here is where this gets expensive. The blocking happened in the same window as real ranking volatility, so you have actual movement and degraded measurement landing on top of each other. Separating the two is close to impossible from inside a dashboard. If you report to a client or a boss on rankings, this is the month you could explain a drop that did not happen, or miss one that did.
How To Handle Your Data Right Now
You do not need to stop using your tools. You need to stop treating their output as precise for the next few weeks. Here is the process.
- Check your tool's status page before you check your rankings. Sistrix posted about this publicly. Others did too. Thirty seconds there saves you an hour of panic.
- Verify anything alarming by hand. If a money keyword shows a big drop, search it yourself in an incognito window from the right location. Confirm before you act.
- Lean on Search Console. Google's own data does not depend on scraping Google. For the next few weeks, impressions and average position in Search Console are more trustworthy than a third-party tracker.
- Widen your reporting window. Compare a rolling 28 days instead of week over week. A collection gap shows up loudly in a seven day view and washes out in a longer one.
- Add a note to your reports. If you send ranking reports to anybody, say plainly that industry-wide collection issues affected the period. Do it now, not after somebody asks.
If you are rethinking which tools you rely on, it is worth looking at what else is in the market and how each one collects its data, because collection method is exactly what got tested here. Our rundown of keyword research tools covers the main options and what each is actually good at.
The Longer Term Read
I do not think this reverses. Google has spent a year making search results harder to collect at scale, and the pressure from AI scrapers gives it every reason to keep going. Rank tracking is going to keep getting less reliable and more expensive to do well. That is not a reason to abandon it. It is a reason to stop building your whole reporting story on it.
Traffic, conversions, and revenue do not depend on anybody scraping Google. Rankings do. The more weight you put on metrics that come from your own site, the less exposed you are next time. That is also a better way to think about your ranking factors and overall strategy, because the goal was never the position. It was the business the position brings in.
Question to Answer:
If your rank tracking tool went dark for a month, could you still tell whether your SEO was working?
In Summary
Google appears to have improved its blocking of scrapers and tracking tools around September 13, and providers including Nozzle, DataForSEO, and Sistrix reported reduced data collection, with one citing roughly an 80 percent drop. It landed in the same window as real ranking volatility, so your reports may be showing you a mix of genuine movement and collection gaps.
Check your tool's status page, verify big swings by hand, and lean on Search Console until this settles. Longer term, treat this as a nudge to build your reporting around traffic and conversions rather than position, because collecting Google's results is only going to get harder from here.
0 comments