Nobody understands AI. I said it out loud this week to people I meet regularly, and I included myself in the sentence.
Where and who doesn't matter, and I'm going to keep it that way. A group of people around a long table at the end of a long day. Everyone there had already given their day to their own work and turned up anyway, because the reason we meet is bigger than any one of us. There was no glory in it. There never is. Just an hour and a half of the plain, unglamorous work that keeps shared things alive.
I want to be honest about my own state that evening. My brain has been running at redline for weeks. I've pushed it to the edge on work, on products, on building things that didn't exist six months ago. By rights I should have had nothing left. And yet I sat there and felt energy coming back in, not draining out. Doing something that isn't for yourself, alongside people you've chosen to spend time with, turns out to be a different kind of work. It doesn't spend you. It refuels you. I keep relearning that and I keep being surprised by it.
That was the first lesson of the evening. The second one only showed up because of the first. Being properly present with other people, instead of staring at a screen, is the only place you can actually watch how understanding works. And it doesn't work the way we think it does.
What understanding each other actually is
For an hour and a half I watched a group of capable adults do what humans do whenever they sit together: try to understand each other. We predicted who'd push back on an idea, who'd go quiet, who'd step up and who'd need a nudge. We read tone, we read pauses, we read history. And we were mostly right. It felt like understanding.
But look closely at how that works. We don't have access to anyone else's inner workings. What we actually do is hold up a mirror. We assume the person opposite is wired like us, we run their situation through our own machinery, and we call the output "understanding them." It works surprisingly well, because we share the same hardware. Same evolutionary history, same fears, same hunger for recognition, same tiredness at seven in the evening. The reflection is close enough to pass.
Here's the uncomfortable part. We don't understand how we work either. Decades of research on confabulation say the same thing from different angles: people act first and explain afterwards, and the explanation is often invented on the spot without the person knowing it. The cleanest version is a Swedish experiment from 2005. People chose the more attractive of two faces, the researchers swapped the card by sleight of hand, and most participants then explained, fluently and in detail, why they'd picked the face they hadn't picked. Ask anyone why they made a decision and you'll get a story, not a mechanism. So our understanding of other people rests on a reflection of something we haven't understood in the first place.
Neither do I
Hold that thought, because near the end of the evening the conversation drifted to AI, and someone asked the question straight: do people actually understand this stuff?
No, I said. They don't.
Then I added the part that caused the stir: neither do I.
The reaction was genuine surprise, followed by a half-joking "you probably shouldn't say that in public." I understand the instinct. I run an AI company. I did doctoral research in this field in another country, in another decade, before it was a topic anyone outside a lab cared about. If anyone at that table was supposed to understand AI, it was me.
I'm not unusual in this, just unusually willing to say it. The largest survey on the subject, 48,000 people across 47 countries, found that two thirds use AI regularly, three in five say they use it effectively, and about half admit they don't understand it. Australia sits below the global line on every measure except use. A quarter of us have had any AI training. More than 60 per cent report low knowledge. We rank near the bottom of the world for interest in learning more. A country that uses the thing fluently and understands it least.
Mileage is not understanding
So let me be precise about what I do have. I have mileage. Thousands of hours with these systems, building with them, breaking them, deploying them for businesses that need them to work on Monday morning. That is experience. Experience is real, it's valuable, and it's exactly what I sell. But experience is not understanding, and the two get confused constantly.
Experience is not understanding, and the two get confused constantly.
I know how these models are built. I can describe the architecture, the training, the mathematics underneath. So can a few thousand other people. And none of us, including the teams that built the largest of them, can tell you why a specific answer came out the way it did. Interpretability is an active research field, not a solved problem. We built something whose operation we can fully specify and whose behaviour we can't fully explain. That is a genuinely new kind of object in the world.
Memory is something you hand it
Someone at the table then told a story that made the point better than I could. They'd written a document with AI that morning. It drafted, they did the human pass, they fed it back and asked for improvements. It took the draft apart line by line, as if a stranger had written it. Their reaction: you wrote the thing.
Nothing went wrong there, and neither did the person. That's what makes it worth explaining. These systems can remember. Give them a project, a running thread, the earlier drafts, and they'll pick up exactly where they left off and treat the earlier draft as their own. Open a fresh conversation and paste the text in, and they meet it as a stranger, because for them it is one. Memory isn't something the model carries around. It's something you hand it, every time, by deciding what sits in front of it. Most of us never think about that, because for a human, memory and self arrive as a package. Here they come apart. The model's continuity is a setting, and that single fact tells you more about what kind of thing you're dealing with than any benchmark. We keep reaching for a human-shaped explanation because that's the only mirror we own.
The outputs rhyme. The insides don't.
And here the trap closes. You might say: wait, you just told me human thinking is prediction based on experience with a story attached. That's a fair description of a language model too. So maybe it's not so alien. Maybe the mirror works.
It doesn't, and the resemblance is why. The outputs rhyme. The insides don't. We predict from a body, a childhood, a fear of dying, a need to be liked by the people at the table. It predicts from text, with no body, no stakes, a memory that exists only when someone supplies it, and no reason to care whether the next sentence is true. Two systems can produce similar-looking behaviour from completely different machinery. A bird and a drone both fly. Study the bird all you like and you'll still know nothing about the drone. The familiarity of the output is precisely what makes our reflection feel accurate when it's wrong.
Which brings the two halves of the evening together. We understand other people by reflection, and the reflection works because we share wiring. We don't share wiring with this. Whatever is happening inside a large language model, it's a different way of building a thought, if thought is even the word. So the real question isn't whether people understand AI yet. It's whether a mind can ever understand a way of thinking that isn't built like itself. I don't know the answer. I'm not sure anyone does. I'm fairly sure the honest answer starts with admitting the question.
Calibration, not understanding
So what do you do with something you can't understand? The same thing we've always done. Sailors didn't understand the ocean. They crossed it anyway, with instruments, with respect, with rules paid for in lost ships. You treat it as capable and foreign at the same time. You verify what it gives you instead of trusting the fluency. You learn what it needs in front of it before it can do the job, because that's the part you control. You stop asking whether it "really" thinks and start asking where it's reliable and where it isn't, which is an engineering question and one we can actually answer. Understanding would be nice. Calibration is what you can have today.
The median trap
Now let me bring this down to the level of a business, because that's where the cost of not understanding actually lands.
Here's the one piece of mechanism every business owner should know, and almost none do. A language model is trained to produce the most probable continuation of whatever you give it, and then tuned so that its answers please the average evaluator. Read that twice. Left to itself, the default output of these systems is the statistical middle of everything humans have written, polished to be agreeable. It's not the best answer. It's the most expected one.
The newest models can reason, and that changes the game, but not in the way most people assume. A reasoning model can work a problem, check itself, discard its first idea and go looking for a better one. That's real, and I use it every day. But reasoning needs something to reason about. Hand it an empty page and it will reason its way to the median faster and more convincingly than the old models ever did. The variable that moves the answer off the middle isn't the model. It's what sits in front of it. Your context. Your history. The memory you hand it, which is the same mechanism I described earlier, now with money attached.
So there are two ways to use the same tool, and they lead to opposite places. Start with a blank page and let the model write, and you default yourself: your business becomes the most probable version of a business like yours. Start with your own material, your decisions, your way of seeing a client, and let the model extend that, and you climb, because now it's reasoning about you instead of about the average. Same model. Same subscription. The difference is whether it's a replacement for your thinking or an extension of it.
Now look at what people point it at. In Anthropic's own usage data, content creation and copywriting is the single biggest category of requests, close to a quarter of everything, ahead of coding and research. The task where a blank page does the most damage is the task we hand it most.
That's fine for a first draft. It's lethal as a strategy. Picture twenty businesses on the same coast, in the same sector, each opening a fresh chat and asking the same tool to write their positioning, their website, their client email, their plan for next year. They won't get twenty answers. They'll get one answer, twenty times, with the names changed. None of them will lose to AI. They'll lose to each other, by becoming indistinguishable. I call this the median trap, and I'm watching it close around businesses that believe they're ahead because they adopted early.
None of them will lose to AI. They'll lose to each other, by becoming indistinguishable.
The mechanism connects straight back to the mirror. The model's default answer is itself a reflection, not of you but of the median human. Accept it without a fight and you don't just use AI. You normalise to everyone else who used it. Your voice, your judgement, your particular way of seeing a client's problem, the things that made people choose you, get sanded down to the most probable version of a business like yours.
This isn't a hunch. In 2024 two researchers at UCL and Exeter gave 300 writers the same task, some with AI story ideas and some without, and had 600 people judge the results. The AI-assisted stories scored higher on novelty and usefulness. They were also 10.7 per cent more similar to each other. Individual gain, collective loss, measured. Later studies found the same shape in essays and in brainstorming: each person does better, the group converges. And every one of those experiments started the writers with the same blank page.
So the skill that matters in 2026 isn't prompting. Everyone can prompt. The skill is understanding what you're pointed at and refusing the middle. In practice that means three things. Bring your own material before you ask for anything: your cases, your numbers, your earlier work, your way of doing things, because the model can only be specific with what you put in front of it, and a reasoning model with your context is a different animal from the same model without it. Make it argue against itself, every time, and ask for the answer it would give if the obvious one were wrong. And keep the decision human. Let it inform the call, never make it. The businesses that will stand out next year aren't the ones using AI most. They're the ones that treat its first answer as the floor, not the finish, and never let it see a blank page.
Attention is one of the three currencies of this economy, and I've written before that trust is another. Both are earned by being distinct. A business that hands its thinking to the median has spent both without noticing.
I know that I know nothing
This is why, on podcasts and on stages, I keep declining the "AI expert" label. If expert means someone with more hours than most, fine, I'll take it. If it means someone who knows what this is, no. The most accurate thing I can say about my position in 2026 is one of the oldest lines in philosophy: I know that I know nothing. The difference is I've done enough work to know precisely where the nothing starts.
Knowing is not understanding. Using is not understanding. Building is not even understanding. We're sailing into waters that aren't on anyone's chart, and we don't yet know which direction to look. Every answer this technology hands us opens a thousand questions we didn't know how to ask last year. It feels like we're learning fast. We are. It also feels like we're closing in on something. We're not. The horizon is moving with us.
Two things from one tired evening. Do something that isn't for yourself when you have nothing left; it gives more than it takes. And be suspicious of anyone who tells you they understand AI, including anyone who's happy with its first answer. They've either stopped looking, or they're selling you something.
The people who worry me least are the ones still asking the question.
For the record, since this issue is about exactly that: I write these with AI. The thinking, the facts, the evening at the table and the argument are mine, and I steer every paragraph. The model gets my material first and my disagreement after. What it does last is polish the English, and for someone who thinks in Polish and writes in English, that part isn't a shortcut. It's a relief.
Blank page, never. Cognitive extension, every time.
Sources for the numbers
- Choice blindness: Johansson, Hall, Sikström and Olsson, "Failure to detect mismatches between intention and outcome in a simple decision task", Science 310, 7 October 2005, doi:10.1126/science.1111709. Immediate detection under 10 to 13 per cent, total detection about a fifth to a quarter depending on the count; the 26 per cent figure follows the authors' later summaries. Jam and tea: Hall, Johansson, Tärning, Sikström and Deutgen, Cognition, 2010, roughly a third detected. Objects by touch: Steenfeldt-Kristensen and Thornton, i-Perception, 2013, about 22 per cent for similar pairs, PMC3690411.
- Use versus understanding: Gillespie, Lockey et al., Trust, attitudes and use of artificial intelligence: A global study 2025, University of Melbourne and KPMG, 48,340 respondents, 47 countries, fieldwork November 2024 to January 2025. Global: 66 per cent use AI regularly, 60 per cent say they use it effectively, 39 per cent have had training, about half report low knowledge, 46 per cent willing to trust. Australia: 48, 24, over 60 and 36 per cent respectively. KPMG Australia summary; Melbourne Business School key findings; full report.
- Request topics: Anthropic Economic Index, global usage profile, period to May 2026, shares of sampled conversations by top-level request topic. Augmentation 51.4 per cent, automation 48.6 per cent. Observed usage, not evidence about employment. anthropic.com/economic-index.
- Individual gain, collective loss: Doshi and Hauser, "Generative AI enhances individual creativity but reduces the collective diversity of novel content", Science Advances 10(28), 12 July 2024, doi:10.1126/sciadv.adn5290. With five AI ideas: novelty up 8.1 per cent, usefulness up 9 per cent; similarity between stories up 10.7 per cent with one AI idea versus none. Summary with the percentages: ScienceDaily, 16 July 2024. Same pattern in essays: Moon, Green and Kushlev, 2024 (human essays contributing two to eight times more to collective semantic diversity than GPT-4 essays); in ideation: Anderson, Shah and Kreminski, Creativity and Cognition, 2024.
- Earlier Artificial Ignition pieces referenced: The New Currency of the AI Economy (July 2026, the three currencies); The Brain Got a Body (October 2026).
🖖
Artificial Ignition is written from the Sunshine Coast, Queensland, Australia.
#ArtificialIgnition #AI #ExponentialShift #AICompass #SunshineCoastBusiness
