Do you remember the game of telephone?
You sit in a circle. The first kid whispers a phrase to the next. “The cat sat on the mat.” By the time it travels around the room, through twelve kids and twelve whispered handoffs, it emerges as something else entirely. “The bat stabbed a rat.” Everyone laughs. That’s the joke. The distortion is the point.
Now scale that game to the entire internet. Replace the kids with AI models. Replace the phrase with the accumulated knowledge, argument, and nuance of twenty years of human publishing. Replace the laughter at the end with the quiet, confident delivery of a wrong answer to someone who needed a right one.
Nobody’s laughing.
That’s the loop we’ve already entered. And the strange part isn’t that it’s happening. The strange part is how few people are willing to say it out loud.
How we got here
Content marketing was always a volume game, long before AI entered the picture.
Search engines rewarded publishing cadence. The companies that published more, more often, on more topics, tended to rank higher. Quality mattered at the margins. Volume mattered at the core. The incentive structure was baked in from the beginning.
Then AI dropped the cost of producing content to somewhere near zero.
The natural response was exactly what you’d expect: flood the zone. If publishing fifty posts a month was good, publishing five hundred was better. If an agency could deliver ten pieces of content for $5,000, an AI workflow could deliver a thousand for $500. The math was irresistible. The business case was airtight.
What nobody paused to account for was where all that content goes.
It doesn’t vanish after it’s published. It gets indexed. It gets crawled. It circulates. And eventually, through the unglamorous, invisible plumbing of the modern web, it gets ingested as training data for the next generation of models.
We didn’t just automate content creation. We automated the feedstock.
The loop
Here is the mechanism, stated plainly: large language models learn from what exists on the web. The web is increasingly composed of what large language models wrote. The next generation of models will learn from that. And the generation after that will learn from the generation before it.
This is not a hypothetical. It is already happening.
Researchers at the University of Oxford and other institutions have documented a phenomenon they call model collapse: the degradation that occurs when AI models are trained on data generated by prior AI models. The findings are consistent and sobering. Each successive generation of model trained on synthetic data performs worse than the one before it. Errors propagate. Nuance disappears. The distribution of outputs narrows. The model becomes, over time, a confident, fluent, increasingly unreliable echo of itself.
The telephone game, formalized.
What makes this particularly insidious is the mechanism of distortion. In the children’s game, errors happen because human memory is imperfect and whispers are hard to hear. In the AI loop, errors happen for a different reason: assertive wrong answers outcompete cautious right ones.
Models are rewarded, in training and in user preference, for sounding clear and confident. Hedged language registers as less useful than a clean declarative sentence. Phrases like “the evidence is mixed” or “I’m not certain, but” get smoothed away in favor of something that sounds like an answer. So the hedges get dropped. The uncertainty disappears. The caveats that represent the actual texture of human knowledge get optimized out.
What survives is plausible. What survives sounds authoritative. What survives is not necessarily true.
What gets lost in the loop
The first casualty is the caveat.
Human knowledge advances through qualification. “This is what we observed, under these conditions, with these limitations.” Science runs on error bars. Expertise runs on knowing what you don’t know. The phrase “it depends” is not a failure of intelligence. It is often the most intelligent thing you can say.
AI systems, trained on AI systems, trained on AI systems, lose the error bars. Each generation smooths the rough edges. Each generation delivers the answer with slightly more confidence and slightly less warrant. The photocopy of a photocopy of a photocopy doesn’t announce its own degradation. It just gets a little blurrier each time.
The second casualty is the original source.
Primary research is expensive. First-person observation takes time. Expert interviews require finding experts, convincing them to talk, and doing the unglamorous work of transcription and synthesis. None of that is necessary when an AI can generate a plausible account of what those sources probably said.
But plausible is not the same as true. And when the plausible account gets published, indexed, and fed back into the next training run, the distinction between “what someone actually found” and “what a model predicted someone would have found” begins to collapse.
We don’t lose facts all at once. We lose the thread back to where the facts came from.
The part nobody wants to say out loud
We are doing this deliberately.
Not accidentally, not as an unforeseen side effect, not as the unintended consequence of a well-meaning technology. The production of AI-generated content for the explicit purpose of ranking in search engines and AI answer engines is a deliberate business decision, made by marketers, executives, and founders who understand exactly what they are doing.
The incentive to publish has not weakened. It has intensified. Every business that publishes competes with every other business that publishes. Dropping out of the content game means ceding ground to competitors who haven’t. The logic is individually rational and collectively catastrophic. It is the classic shape of a tragedy of the commons.
So everyone keeps whispering into the ear of the next kid in line.
And the phrase keeps changing.
And nobody is keeping careful track of what it was supposed to be.
The web is not becoming noisier in a uniform way. It is becoming noisier in a specific way: confident, fluent, superficially coherent, and progressively detached from the primary sources, lived observations, and genuine uncertainty that made knowledge worth having in the first place.
Is there a way out?
Honest answer: unclear.
There are things that could help. Models could be designed to weight primary sources more heavily, to prefer the peer-reviewed paper over the blog post that summarized it, to prefer the firsthand account over the AI article that paraphrased the account. Some researchers are working on this. It is technically difficult and commercially inconvenient.
Original human voice could become a quality signal precisely because it becomes scarce. If the web floods with content that sounds a certain way — competent, complete, oddly similar — then the thing that sounds different might start to stand out. Idiosyncratic. Specific. Uncertain in the right places. There is a version of the future where genuine human perspective becomes the premium product it arguably always should have been.
Or the loop just keeps going.
The output becomes progressively more homogeneous. The confident wrong answers multiply. The thread back to primary sources gets longer and harder to follow. The model trained on the model trained on the model delivers its answer with perfect fluency and decreasing accuracy, and the person on the receiving end has no way of knowing which generation of the telephone game they’re standing at.
These are not rhetorical questions designed to lead somewhere comfortable. They are open problems. The people who should be most worried about them are also the ones with the most to lose from slowing down: the ones building the systems, publishing the content, and designing the incentives.
That tension does not resolve neatly. It just sits there.
A confession
This post was written with AI assistance.
Which means it is, at least in part, exactly what it’s describing. A human thought carefully about the argument. A language model helped arrange the words. The ideas originated somewhere in the conversation between the two.
It will be published on the web.
Where it will be indexed. Where it will be crawled. Where it may one day be ingested as training data for the next model, which will learn, among other things, that there is a thing called the AI telephone problem, and that someone once wrote about it in these terms, and that this is what the argument sounds like.
The telephone game continues.
We just added one more kid to the chain.
Work with us
Ready to get out of the complex?
We work with lean SMB tech teams that are done optimizing the old model. Let's find out if there's a fit.
Let's talk →