SERP tools like SEMRush affected by anti-scraping measures by Google’s latest update
Google might have just rolled out a startling new development, turning the way SEO insights are obtained right on its head. And it could be serious.
Many heavy-hitting SERP tools, like SEMrush and SE Ranking, have faced a huge outage of data, stalling many SEO marketer’s attempts at harvesting data for their work. With these global outages of information, it begs the question— is this type of data blocking by Google going to become permanent?
If so, then the way we approach search engine optimization is about to undergo a massive transformation. With the rise of AI-driven tools like Ahrefs and ChatGPT, marketers have begun to get used to unlimited access to unprecedented levels of data and automation. A lot have even started to rely on them. But while these tools are game-changing, they don’t replace the nuanced insights and strategic thinking that human expertise brings to the table.
The secret to successful SEO lies in the artful combination of AI’s capabilities with human ingenuity. That’s why, in our personal opinion, a good SEO marketing team doesn’t have anything to worry about if Google has pulled the rug out from under everyone.
What Happened?
On January 15th, Barry Schwartz reports, Google implemented some measures “far beyond traditional captchas” that prevent these SERP tools from being able to scrape data it’s usually able to obtain. AI Overview also seems to have been a big feature that’s been affected, however, all the information is still being reported on to see how wide-spread this goes.
One user reports, “Let’s be honest, Google is definitely tunning an update. The fluctuations are just too extreme.”
Some Context On How These Tools Are Used
AI tools like Ahrefs and ChatGPT have sped up our approach to keyword research. Ahrefs, for instance, excels at identifying keyword opportunities by analyzing search volume, competition, and backlink profiles. With just a few clicks, it can provide a comprehensive list of keywords tailored to a specific industry or niche. Really, really helpful! On the other hand, ChatGPT offers creative and contextual insights, generating keyword ideas based on conversational prompts and even predicting how users might phrase their queries.
These tools thrive on speed and scale, sifting through massive datasets in seconds. An invaluable asset for SEO Marketers wanting to focus their energies on utilizing this data in tangible ways instead of gathering it. They can uncover trends and patterns that might take humans a lot longer to identify.
However, while AI tools are undeniably powerful, they are not infallible. Algorithms are only as good as the data they analyze, and they lack the context and critical thinking required to turn raw data into actionable strategies. And when the powers-that-be decide to change the rules… well. Those tools are rendered useless.
So… If This Is Permanent, What Do We Do?
Firstly, Don’t panic. We don’t have all the information yet, as this is a developing story. As startling as it is, it’s important to remember that AI isn’t what makes SEO successful— Smart marketers are.
AI tools operate on algorithms and historical data, which means they are already inherently limited by the quality and scope of the information they process. For instance, an AI tool might suggest high-volume keywords, but it cannot always discern whether those keywords align with your brand voice or target audience’s intent. Similarly, AI can’t understand the cultural nuances, emotional resonance, or industry-specific jargon that often make or break a successful campaign. But a human who’s spent countless hours getting to know your company can.
This is where human expertise becomes indispensable. Marketers bring an intrinsically deep understanding of a brand’s identity, customer personas, and competitive landscape—elements that these tools simply cannot replicate. Humans can assess whether a suggested keyword aligns with a brand’s messaging or whether it addresses a specific pain point of the target audience. They can also adapt strategies based on real-time feedback and evolving market conditions, something AI struggles to do effectively.
So, if our SERP tools are down indefinitely, it may throw a wrench into the speed of current plans or projects. But it won’t affect the quality of your work, or an SEO professional’s ability to properly do their jobs.
Shay Ahrel from Similarweb reports that “…These updates temporarily disrupted rank tracking and SERP data solutions across the industry […] as many existing configurations were rendered nonfunctional.”
Action Steps
Wait it out as news rolls in. And in the meantime, fall back on your tried-and-true SEO tactics that have been garnering great results since the beginning.
Focus on User Intent and Behavior
- Understand Your Audience: Keep conducting your own thorough research to know what your target audience is searching for, their pain points, and how they phrase their queries.
- Leverage First-Party Data: Use tools like Google Analytics, CRM systems, and social media insights to understand what content resonates with users.
Develop High-Quality Content
- Prioritize Content Depth: Create comprehensive, well-researched, and useful content that addresses user needs.
- Use Natural Language: Write conversationally to align with voice search trends and user expectations. This is an opportunity to write some more casual pieces of content without relying on your usual tools!
Manual Search and Competitor Analysis
- Run Manual Queries: Search relevant keywords on search engines to understand rankings and trends the old-school way. Look at the type of content, tone, and format that performs well.
- Competitor Benchmarking: Analyze top-performing competitors directly. Review their websites for structure, content, backlinks, and other SEO signals.
Stay Informed
- Join Industry Communities: Participate in forums, Slack groups, or webinars to share insights and learn from peers. This story could change within hours as new information becomes available, so keep your ears to the ground!
- Read SEO Blogs: Stay updated with insights from Moz, Search Engine Journal, or Google’s official channels to adapt to changes effectively. Hopefully, Google even makes a statement in the near future…
Remember, SEO marketers can thrive even without SERP tools. So don’t be dismayed.
The Takeaway
SEO success is rarely the result of a single tool or tactic. So, if your tools go down, it may feel like you’ve had your legs cut out from under you, but in reality, you just need to shift your focus until more information becomes available. After all, this change could very well be temporary.
Good SEO is the outcome of a well-balanced approach that combines the computational power of AI with the critical thinking and creativity of human marketers. It’s nice to be able to leverage AI for its speed and data insights, but it won’t break your process if your tools are unavailable.
Instead, let this be a hopeful reminder that relying on human expertise for strategic refinement is the crux of what makes a good SEO team stand out from one that just clicks and spits out data without much thought. Your gathered knowledge is more valuable than a single tool that, even without this bombshell, is already ever-changing.