Iain Harper's Blog

The deal that sent organic traffic to your website for free has almost entirely collapsed. It is gone. It is not coming back. And the sooner we come to our senses and stop wasting time and effort on futile optimisation, the sooner we can focus on more productive things.

In a famous scene from Monty Python and the Holy Grail, King Arthur comes upon a bridge guarded by the Black Knight. In the ensuing fight, the Black Knight steadfastly refuses to yield, even after being dismembered, insisting “‘tis but a scratch!”

I recalled this as I received the latest of many similar emails. This one invited me to a webinar called “From Backlinks to Frontlinks”. Apparently, I would learn “what organisations should actually measure in the AI era”, and that “the brands that win over the next few years won't simply rank higher. They'll become the brands AI trusts enough to recommend.” All of which is, to put it politely, complete horseshit.

This latest missive crystallised my ongoing misgivings about this new phase of SEO (GEO?, AEO?, pick an acronym), something I've spent my whole career working on in various capacities. The scales finally fell from my eyes, and I realised that the entire premise of organic search optimisation is not changing; it is broken beyond repair.

One admission before I start. Most of the data I am about to cite comes from companies that sell search or AI visibility tooling. Ahrefs, Semrush, SE Ranking and BrightEdge all have a commercial interest in how large and how tractable this channel appears. I use their numbers because, at this scale, there is nothing else, and I have tried to indicate where the commercial incentive lies in each case. Apply the same suspicion to my argument that I am asking you to apply to theirs.

The Faustian pact

For a quarter of a century, organic search ran as a kind of implicit barter. You produced content at your own expense, let the search engine crawler index it, dutifully marked it up as specified, and in return, mostly Google (other search engines were available!) sent some number of visitors back. The only costs were producing the content, maintaining the site, and any additional efforts you made to optimise it.

Much time and effort were expended on the optimisation part, trying to move up the organic search rankings through fair means and, occasionally, foul. Because even a single place improvement could deliver a meaningful increase in traffic, some portion of which would convert or buy or do whatever the thing you wanted them to do was.

Less sophisticated organisations saw this traffic as free. Others saw it for what it was and mapped it to things like human effort and agency expenditure, so any benefit could be assessed alongside other traffic acquisition channels with more directly measurable costs. But this world has now fundamentally changed with the migration of search to AI agents.

Pew Research tracked 68,879 searches from 900 US adults in March 2025 and found that 18% triggered an AI summary. When the summary appeared, users clicked a traditional result 8% of the time, versus 15% without one.

SparkToro and Similarweb put the US zero-click rate at 68% for the first four months of 2026, up from 60.45% in 2024. Bain's February 2025 consumer research put the consequence at a 15% to 25% fall in organic traffic across many sectors. Chartbeat data covering more than 2,500 news sites, shared with Axios, found Google Search referrals down 34% year-on-year, with small publishers losing 60% of search referrals over two years, versus 22% for large ones.

So your side of the organic search bargain has not changed at all. You still write the content, still ship the schema, still let the crawler do its prosaic work. Only now, the other half of the bargain, the traffic received in return, has collapsed dramatically. The natural reaction is to panic and look for a quick fix, which is what has brought the usual shysters out of the woodwork, claiming easy solutions and techniques.

Google’s AI Overviews and AI Mode are now default features of Search. They run on Googlebot, and the Search index, and the only levers that affect them are nosnippet and max-snippet, which also remove your ordinary search snippets and destroy your click-through rate. You cannot leave the AI layer without leaving Search altogether (you can exclude your content from Gemini training without a search penalty, however).

If this is not your first rodeo, you may recall E-A-T, a previous era where we spent a great deal of time and effort convincing ourselves it was a dial we could turn, a score we could improve by demonstrating Experience, Expertise, Authoritativeness and Trustworthiness so the algorithm would bestow additional blessings. You'd be forgiven for noticing the eerie similarity to current “optimise for citation” advice.

Such was the noise, Google was forced into a rare comment in December 2022, that these kinds of quality rater guidelines “don't directly influence ranking”. Gary Illyes, then a Webmaster Trends Analyst at Google, had already explicitly told PubCon Las Vegas in October 2019 that there is no internal E-A-T score and no E-A-T element correlates meaningfully to the algorithm. Yet much of the SEO field kept its fingers in its ears and pushed on regardless, desperate for some variable, any variable, to optimise.

The tiny little prize

For argument's sake, let's pretend the current AI search optimisation industry is right and you can successfully manipulate your way into being consistently cited in AI answers. Before you break out the champagne, look at what that citation is worth. In the same Pew dataset linked above, among visits in which a Google AI summary appeared, the share of visits in which a user clicked a source cited in the summary was 1%. That’s one in a hundred, and Pew only counted the first three cited links.

True, that number is specific to Google AI Overviews in March 2025, and other assistants do behave differently. ChatGPT referrals jumped sharply after clickable brand links shipped in May 2026. But the overall direction of travel is not in dispute. Just put it beside the channel it is supposed to replace. SE Ranking's analysis of 101,574 sites measures AI referral traffic at 0.32% of all website visits for January to April 2026, compared with 42.75% for organic search of all kinds. Around 134 times more visitors from search than from every AI platform combined.

Two important caveats: SE Ranking's number sits at the low end of the published range. Conductor, analysing 13,770 enterprise domains, put AI referral traffic at 1.08%, roughly three times higher. And referrer-based measurement and attribution are structurally blind to most of what an AI answer does, because a visitor who reads about you in ChatGPT and then types your name into Google arrives tagged as branded organic or direct, breaking the attribution. But even if you take the highest credible figure and grant the measurement problem in full, it is still a rounding error compared to search.

Despite this, Rand Fishkin estimated in January 2026 that over $100m a year was already going into AI visibility tracking alone. It is just an estimate, but it sits alongside agency retainers, consultants and the usual conference circuit suspects and LinkedIn Lunatics, all telling you this problem is both traceable and solvable.

Let's look at a known positive for a moment: AI referral visitors do genuinely seem to convert well. Semrush's June 2025 study across more than 500 digital marketing and SEO topics found that the average AI search visitor was worth 4.4 times that of a traditional organic visitor, measured by conversion rates. Note the base: one industry vertical, chosen by a company that sells AI visibility tooling and owns Search Engine Land.

But you do not need a Fields Medal to calculate that a 4-to-5x conversion multiplier on a channel delivering around 1% of visits is still a minuscule absolute number. What often follows is the hand-waving argument that it's all very new, and that low volume with strong growth and high conversion quality is the profile of a promising early channel.

And here is where we get to the nub of the issue: every user an AI answer engine sends off-platform is a missed opportunity for an ad, a follow-up prompt, or a subscription upsell. For Google, AI Overviews are designed and optimised specifically to create a broader advertising surface rather than send traffic out to your site.

[With AI Overviews] free organic traffic from search fell by 40%. Paid traffic held steady. If your free visitors from Google are disappearing, the route Google left open is the one you pay for, and that was clearly not an accident.

For OpenAI, which is building an increasingly sophisticated ad business of its own and burns money on every query, the incentive is even starker. These companies are not distribution platforms that happen to summarise. In search terms, they are answer engines whose economics depend on not referring a user away. Most other platforms work the same way. Including an external link on X or LinkedIn posts is a sure-fire way to get buried.

I find the hardest part of all of this is accepting that the Google of 2026 is a fundamentally different beast from the one we have lived with for a generation. It’s like we’ve been in a long-term relationship with a partner who, despite their flaws, we’ve generally coexisted with quite happily.

Now that partner has left us, it’s easier to see the patterns of their behaviour that should have been obvious for years. Featured snippets, knowledge panels, “People Also Ask” boxes, shopping units, and four stacked ads. Each iteration squeezed the truly organic links. AI Overviews are not so much a departure from this trajectory as its zenith.

Even the remaining organic share is rapidly eroding as Google expands its own AI layers. Every study measuring click-through rate when an AI Overview appears has found a steep decline, though the magnitude depends heavily on the method and the type of search. Ahrefs measured a 34.5% drop for top-ranking pages in April 2025 and 58% by February 2026. Seer Interactive, tracking 25.1 million impressions across 42 organisations, found that organic CTR on AI Overview queries fell by 61%, from 1.76% to 0.61%. Amsive, across 700,000 keywords, found around 15%. Pick your methodology, and the answer moves by a factor of four, which tells you something about the validity of any platform claiming to be able to reliably monitor this space.

The damage to publishers is easier to see. Growtika's analysis of ten major tech titles found combined US organic traffic falling from 112 million monthly visits to 47 million, with Digital Trends dropping from 8.5 million monthly Google clicks in March 2024 to 264,861 in January 2026. The figures are Ahrefs traffic estimates rather than publisher analytics, and Google called the analysis fundamentally flawed. Digital Trends also laid off most of its full-time staff in early 2025, so its 97% is not a clean read on AI Overviews. The Verge, How-To Geek, and ZDNet each losing more than 85% over the same period is harder to explain away as flawed methodology.

So any growth curve for organic AI referrals (if there is one at all) appears to have an inbuilt ceiling set by the platform owners. It will only be allowed to exist at whatever level they decide, based on their commercial goals, and nothing you can do will change that. How many entities on the open web survive this transition remains to be seen, but based on current data, it does not look promising.

Say what you like about Meta, but when they decided to dramatically throttle brands' organic reach on Facebook between 2014 and 2016, at least the direction was unmistakable (which is not the same as transparent). Others are now belatedly following suit.

It is particularly awkward for Google, a company that built itself on the open web and harvested great riches, to be playing such an active role in destroying it. Small wonder they hide behind complexity, weasel words and methodological quibbling. It’s not a good look so they just hope it’s all too confusing or boring for most people to care until it’s too late.

Nobody knows how AI sources get chosen

Suppose you accept the bad maths. Suppose you argue that aggregators like Google will let organic referrals grow over time, even though that runs directly counter to their commercial interests. Suppose, then, that there is some value in spending your time trying to influence your appearance in this mythical future organic AI search segment.

You still need a mechanism — something you can do that reliably changes where, or whether, you appear in the answers. SEO had more than its fair share of snake oil, tricks, and techniques, but it eventually boiled down to “have a well-structured, performant site with helpful, unique content that solves problems for users.” That still applies. When it comes to appearing in AI search results, there is no other evidence-based method that moves the needle.

Worse, the waters between optimising for traditional search engines and AI search agents have been muddied, but they work in fundamentally different ways. Ahrefs sketches the retrieval process in its own research. An AI assistant rarely searches for the question you typed. It expands your prompt into a spray of related sub-queries — the fan-out — retrieves documents for each, then merges the resulting lists. Ahrefs offers Reciprocal Rank Fusion as one plausible mechanism for that merge, a formula that rewards pages turning up consistently across several retrievals. Google has confirmed the fan-out; nobody has confirmed the fusion, which is at best educated guesswork.

Surfer found that when an AI Overview cites something that does rank, it is ranking for a fan-out and not the main query 29.2% of the time, against 19.6% the other way round. So fan-outs appear to matter more than the head term.

But that still does not give you a target list. The expansion is not stable, which is why Fishkin’s volunteers received the same set of recommendations less than once in a hundred. Reading it once gives you a sample of one from a distribution nobody has published. The queries are long-tail enough that most have no measurable volume, so they do not exist as rows in a keyword tool. And each engine runs a different index, so “rank for these” needs a WHERE clause. Ahrefs found Perplexity overlapping Google’s top ten around 29% of the time, compared with 7 to 8% for ChatGPT, Gemini and Copilot.

What you can get is a map of the topic space an answer drew on. That is a content gap diagnostic, and it is worth something. But it is not a ranking strategy, even though it is being sold as one.

There is a deeper problem with what follows from this. The merge step itself doesn't assess your credibility. But the candidate documents it merges arrive from ranking systems saturated with quality and authority signals, so the credibility assessment has not disappeared; it has moved upstream and out of sight.

That is worse for you, not better. The judgement you are being sold tools to influence is happening in a layer nobody will describe, at a distance from anything you can observe. So any talk of “becoming a brand AI trusts enough to recommend” is utter bollocks, because there is no trust register to influence or enter.

Alongside the theoretical fragility, the measurement evidence is worse than incomplete. In March 2026, Ahrefs analysed 863,000 keyword SERPs and 4 million URLs, reporting that 37.9% of AI Overview citations came from pages ranking in Google's top 10. In February 2026, BrightEdge measured the same surface and the same relationship, but put it at around 17% and said it had been flat for months. Same quarter, same question, a factor of two apart, and neither publishes a confidence interval. Basically: pick a number, any number.

Cross-engine, it is worse still. Kevin Indig's analysis of 20,000 prompts found that only 2.37% of cited URLs appeared in ChatGPT, Perplexity and Google AI Overviews for the same prompt, while 91% appeared in only one. These are not three rankings of one pool. They are three largely disjointed pools, which makes any single composite visibility score meaningless by construction.

Then there is the question of whether one answer tells you anything at all. A variance-components study published in July 2026 broke down 12,933 LLM responses about 20 brands across eight languages and three models. When measuring sentiment polarity, brand identity accounted for 1.5% of the variance in a single response. The language of the query accounted for 26.5%. Resampling noise accounted for 34.8%. Brand-ranking reliability from a single answer was near 0.01, meaning a single answer carries almost no brand-discriminating signal.

Even read generously, the conclusion is still awkward. There is a real distinction between tracking your position in an answer, which the evidence says is close to pure noise, and heavily sampled aggregate brand measurement across many prompt variations, models and languages, which has a kernel of rigour. The second is what Żatuchin's decision-study allocation describes, which is defensible in principle. Żatuchin's decision-study allocation sets out how to do it, though on sentiment rather than on the visibility metrics that the tools typically report.

It is also expensive; it tops out at a reliability of 0.36, and it is not the product most people are buying. Almost every dashboard on the market reports a position, and even Avinash Kaushik, who is about as thoughtful an advocate for this discipline as exists, names Average Position as one of his three headline KPIs and treats it as the priority opportunity. Fishkin's research says a position in an AI recommendation list is a statistically meaningless snapshot of a probabilistic output. They cannot both be right, and the primary data does not support the dashboards.

So you are trying to optimise for a moving target whose selection rules are unknown, whose outputs are non-deterministic, whose measurements do not replicate across engines, and whose vendor studies disagree by a factor of two within a single quarter. Trying to optimise against those structural realities is panicked madness.

But you do not have to take my word for the last part. Kaushik, in the middle of a detailed methodology walkthrough for a tool he has invested in, reaches the point where the numbers move and he writes: “Coach went down by 3, probably not significant. Kate Spade went up by 11; that is massive. Why? No idea. We have such little visibility into LLM black boxes, we can only pray to Krishna and Jesus that the next set of weights being used favours our brands.” He also warns his readers that there are lots of magic beans around, and that two tools can both report a Visibility Score while measuring entirely different things.

I wrote about the measurement problem in detail, and here is the short version. The category has raised roughly $300m since mid-2025. The leading platform's own research shows that most cited sources change over time; nobody publishes confidence intervals; and the tools usually measure through an API that does not match the consumer product. What you are paying for is at best a tiny signal in a sea of noise.

The decision that was already made

Maybe you’re a hardened marketer who rode o’the choppy waters of the Penguin and Panda algorithm updates. Maybe, like me, you spent 18 months of your life recovering from a Google de-listing because your predecessor thought it was a good idea to buy £30,000 of backlinks with some spare end-of-year budget. That makes all of this very hard to accept as anything other than another hump in the long road, another challenge to work around.

We become hyper-focused on fixing the agentic search “problem”, because that’s what we’ve always done. And of course, there are no shortage of people trying desperately to flog services and tools that are mostly thinly rebadged SEO approaches desperate to stay relevant. But I sense that perhaps, at some level, even the LinkedIn Lunatics realise their days are numbered.

It’s a lot to process. So step back and return to the fundamentals. Bain's September 2025 work found 85% of B2B buyers purchase from their “day one” list, the vendors they had in mind before searching anything. 6sense's 2025 Buyer Experience Report, based on more than 4,000 buyers, found that the vendor preferred before any seller contact still wins around 80% of deals, and that buyers now purchase from their day one shortlist 95% of the time.

So if we’re scrupulously honest, the search query was never the moment of persuasion. It was the point at which a decision already taken was executed through a search bar, because that was the easiest way to reach a site whose name you already knew. The entire attribution model that credited search with the conversion was misattributing a navigation step as a discovery event. Now that the navigation step has effectively been removed, it is plain to see.

There is a counterargument that if day-one lists determine the outcome, and those lists are formed before anyone searches, then a citation that produces no clicks could still be doing work. Appearing in the answer might seed the list. Clicks would therefore be the wrong yardstick. I think this is partly true, and it is the strongest thing the search optimisation industry currently has.

It is also what makes any product unsellable. If the value of a citation is brand memory formed off-platform, months before a purchase, then you cannot attribute it, you cannot measure it on a dashboard, and, based on the evidence above, you cannot reliably steer your position in it. You are being sold a whizzy dial, but there is no wiring behind it.

There may well be an effect, and if there is, the sensible response to an unattributable, unsteerable brand-memory effect is not to buy a subscription. It is to do the things that build brand memory, which we have understood for about seventy years.

People were always getting ready for tomorrow. I didn't believe in that. Tomorrow wasn't getting ready for them. It didn't even know they were there.” Cormac McCarthy, The Road

The only sane reaction is not to cling to the hope that you can somehow master this new world through clever techniques and expensive tools that will staunch the bleed of organic referrals.

Build an audience you can reach without an intermediary, because direct is the only distribution channel where nobody can pull the rug from under you. Publish for humans who will remember your name in a procurement conversation eleven months from now, not for a crawler that keeps the click for itself. Show up in the places those day-one lists get formed: peer networks, communities, review sites, and other people's podcasts. That is much easier said than done. it requires really hard work and focus over a long time. It requires being interesting and different and providing something unarguably excellent.

Or you could spend your time learning the ways that AI is transforming paid advertising with technical tools of a power that was previously unimaginable. In fact, do anything other than try to preserve the flow of organic referrals because you’ve got used to seeing them. Those are permanently gone.

The dismembered torso of Monty Python's Black Knight shouts “I'll bite yer legs off!” defiantly after King Arthur. It is an amusing and enduring emblem of the search optimisation industry steadfastly refusing to give up when the battle is very clearly lost.

I am a partner in Better than Good. We help smaller companies build tools and processes using machine learning and artificial intelligence that make lasting improvements to their operations. Talk to us today: https://betterthangood.xyz/#contact