Iain Harper's Blog

The technology industry has spent the past three years debating artificial intelligence with the zeal of medieval theologians disputing angels on pinheads. Boardrooms have AI strategies, and governments have AI safety frameworks. LinkedIn has AI thought leaders, which is arguably the strongest case yet for existential risk. But somewhere beneath the acronym and the data centres in space, one central question remains unanswered. What, exactly, is intelligence?

We are developing systems we call intelligent, regulating systems we call intelligent, and worrying about systems we call intelligent, without a shared scientific consensus on what that term means when applied to humans, let alone machines. That is, to say the least, a problem.

The indefinable word

Ask a psychologist what intelligence is, and you’ll step into the epicentre of a fierce debate that has lasted for more than a century. The oldest and most statistically reliable answer comes from Charles Spearman, who in 1904 observed that people who did well on one kind of cognitive test also tended to do well on others. He called this underlying factor g, or general intelligence. The g factor is among the most replicated findings in psychology. It predicts academic performance, job performance, income, health outcomes, and even longevity, with a consistency that makes most social-science results look like coin flips.

And yet g tells you almost nothing about what intelligence really is. It is a statistical regularity, not a mechanism. Saying someone has high g is a bit like saying a car is fast. The measurement works, but the explanation is missing.

Howard Gardner tried to blow the whole thing up in 1983 with his theory of multiple intelligences, arguing that intelligence is not one thing but at least eight distinct varieties, from linguistic and logical-mathematical to musical, bodily-kinaesthetic, spatial, interpersonal, intrapersonal, and naturalistic. Teachers lapped this up. It confirmed their intuition that the kid who struggles with algebra but plays the cello like a prodigy is smart in ways traditional testing misses.

The problem is that decades of factor analysis have stubbornly refused to confirm Gardner’s categories as truly independent. Musical ability and spatial reasoning correlate, as do linguistic and interpersonal skills, and, in fact, everything correlates, which is more or less Spearman’s original point. Multiple intelligences is a useful pedagogical framework but a weak empirical theory, which is a polite way of saying it works better in classrooms than in laboratories.

Then there is François Chollet’s definition, which originates from the AI community and is arguably the most rigorous recent attempt to clarify the concept. In his 2019 paper “On the Measure of Intelligence,” Chollet defined intelligence not as the ability to perform any specific task, but as the efficiency with which a system acquires new skills, especially when confronted with tasks it has never encountered before.

This led him to develop the Abstraction and Reasoning Corpus (ARC), a benchmark of visual puzzles designed specifically to assess this ability. Humans usually solve most ARC tasks within minutes. The latest version, ARC-AGI-3, published in March 2026, makes the gap even clearer by placing agents in interactive environments where they must infer goals and plan action sequences without explicit instructions. Humans solve 100% of these tasks. At the time of the paper’s publication, frontier AI systems scored less than 1%. This gap cannot be closed simply by better prompt engineering. Whether this means current AI lacks intelligence, or only a particular kind of adaptive reasoning that humans excel at remains an open question.

The definitional problem is not just academic. Every claim about AI being intelligent, not intelligent, or dangerously intelligent depends on an implicit definition. Call a model intelligent, and you usually mean it produces outputs that would require human intelligence. Deny it, and you mean it lacks the comprehension, consciousness, or intentionality you consider necessary for genuine intelligence. Both claims are unfalsifiable without a shared definition, which explains why the debate generates much heat but little clarity.

What neuroscience understands (less than many think)

If psychology cannot agree on what intelligence is, perhaps neuroscience can explain how it works. The short answer is that it can, at least partly, though large gaps remain. We know a great deal about the brain’s individual components. We can map neural circuits, measure neurotransmitter activity, image blood-oxygen levels as proxies for activity, and trace connectivity patterns across cortical regions.

We know that the prefrontal cortex plays a key role in planning and abstract reasoning, that the hippocampus is central to memory consolidation, and that the cerebellum (once thought to be merely a motor coordination device) participates in cognitive processes that are not yet fully understood. We also observe that, within a species, larger brains tend to correlate weakly with cognitive ability, and that connection density and efficiency matter more than overall volume.

What we cannot do is explain how any of this produces thought. We have a parts list and some wiring diagrams, but no operating manual. The situation is roughly equivalent to having an inventory of components for a Boeing 787 without understanding aerodynamics. You could describe the wings, the engines, the control surfaces, and the hydraulic systems, and still have no theoretical framework for why the thing flies.

Two research programmes have made the most ambitious attempts to close this gap, and both illustrate how far there is to go.

Predictive processing

The first concept is predictive processing, most closely associated with philosophers Andy Clark and Karl Friston. They suggest that the brain is not a passive receiver of sensory data. It functions as a prediction machine that constantly builds models of what it expects to perceive, then updates them when reality differs from those expectations. Perception, in this view, is not bottom-up (data in, interpretation out) but top-down (expectation generated, error signal compared, model revised). You do not see the world as it is. You see your best guess about the world, corrected at the edges by incoming data.

Friston formalised this idea in the free energy principle, a mathematical framework suggesting that all adaptive behaviour can be understood as the minimisation of “free energy,” which roughly measures the gap between an organism’s internal model and the sensory evidence it receives. The framework is mathematically coherent and broadly applicable, and that is precisely the problem. If every possible behaviour of any living system can be reinterpreted as free-energy minimisation, then the theory rules nothing out, raising serious questions about its scientific credibility.

Integrated information theory

The second programme is Integrated Information Theory (IIT), developed by the neuroscientist Giulio Tononi. IIT takes on the even harder problem of consciousness rather than intelligence per se, but the two are tangled enough that progress on one would likely tell us something about the other. The theory proposes that consciousness corresponds to a quantity called phi (Φ), which measures the amount of information a system generates “above and beyond” its individual parts. A system with high phi is one whose behaviour cannot be reduced to its components acting independently. The whole, in a precise mathematical sense, is more than the sum of its parts.

IIT makes some bold predictions. It implies that consciousness is a property of a system’s physical structure, not its function. A digital simulation of a brain that runs the same computations on different hardware might have zero consciousness under IIT, even if it behaves identically to the original. In practice, calculating phi for any system more complex than a handful of nodes is computationally intractable, which limits the theory’s practical utility. You can define consciousness precisely and still be unable to measure it in any real system, which is a bit like having a perfect recipe for a cake you can never bake.

Neither predictive processing nor IIT amounts to a theory of intelligence in the way that general relativity is a theory of gravity. They are frameworks, useful and generative but incomplete, and they throw light on aspects of cognition without explaining the whole. And the gap between “aspects” and “the whole” may be permanent, for reasons we will get to.

Why large language models work

If the science of biological intelligence is patchy, the science of artificial intelligence is in an even stranger position. The engineering works spectacularly well, while the theory lags behind, like a civil engineer who builds bridges that hold up beautifully but cannot fully explain the physics of load distribution.

We understand the mechanics of large language models in fine detail. A transformer architecture processes sequences of tokens through layers of attention mechanisms, and during training the model adjusts billions of parameters to minimise the error between its predicted next token and the actual one. Scaling laws, first characterised by Jared Kaplan and colleagues at OpenAI in 2020, describe a remarkably smooth power-law relationship between compute, dataset size, model parameters, and performance.

These are genuine scientific results. They let engineers predict, with useful accuracy, how a model of a given size trained on a certain amount of data will perform on standard benchmarks. What they do not explain is why training a system to predict the next word in a sequence produces behaviour that appears like reasoning, planning, analogy, and (occasionally) creativity.

The most provocative explanation comes from the compression hypothesis, most forcefully articulated by Ilya Sutskever, then of OpenAI. The argument roughly runs like this. Predicting the next token accurately requires modelling the process that generated the text, and that process is human cognition. To predict well, you must compress the structure of human thought into your parameters. Compression, in this view, is not merely correlated with intelligence but constitutive of it. A model that achieves better compression has, in a meaningful sense, come to understand the world better.

This is philosophically interesting and empirically suggestive, but it is not a complete theory. It does not explain why certain abilities appear discontinuously as models scale. Small models cannot perform multi-step arithmetic. Larger models can suddenly, without anyone having specifically trained them for it. These “emergent capabilities” are predicted by no current theory and explained by no current framework. They simply happen, and then engineers and researchers argue about what they mean.

Mechanistic interpretability, an active research programme at Anthropic among others, is perhaps the most promising attempt to open the black box. The work identifies specific circuits within trained models that correspond to identifiable computations, so that one cluster of neurons detects sentiment and another tracks syntactic dependencies. The results are revealing, but they are roughly at the stage where neuroscience was when it discovered that specific brain regions correspond to specific functions. Knowing where a computation happens is useful. Knowing why the system learned to do it, and why it generalises beyond the patterns in the training data, is the harder question.

The “stochastic parrots” critique, most prominently advanced by Emily Bender, Timnit Gebru, and colleagues in 2021, argued that LLMs are only sophisticated statistical mimics. Noam Chomsky has made similar arguments, insisting that next-token prediction cannot amount to genuine linguistic comprehension. Melanie Mitchell has taken a more cautious position, arguing that current AI systems lack the conceptual abstraction and analogy-making she sees as central to intelligence, while leaving open the possibility that future architectures might achieve it.

The honest answer is that nobody knows who is right. The stochastic-parrot position seemed more defensible in 2021 than it does in 2026, because the systems have kept improving in ways a “mere statistical mimic” would not obviously be expected to. But the lack of a theory means that “would not obviously be expected to” is carrying more weight in that sentence than it should. We do not have the theoretical tools to distinguish genuine comprehension from a sufficiently convincing imitation of it, and those tools are not arriving quickly.

Can intelligence be formalised at all?

Here we reach the question beneath the question, and the answer is uncomfortable for anyone who prefers their science tidy. A hidden hope in much AI research is that intelligence resembles thermodynamics: messy and chaotic at the micro level, but governed by clean, discoverable laws at the macro level. Individual gas molecules move unpredictably, yet aggregate behaviour follows the ideal gas law with almost miraculous precision. Perhaps intelligence works the same way, messy at the level of individual neurons or attention heads, but obeying some elegant principle at a higher level of description.

The problem is that thermodynamics works because you can ignore which specific molecule is where. It is far from clear that cognition has this property. The specific structure of a person’s knowledge, the particular history of their experiences, and the exact wiring of their neural connections all seem to matter in ways that resist averaging out. A brain is not a gas. Its macro-behaviour may not separate cleanly from its micro-state, and if it does not, no thermodynamics-style theory is possible.

There is a deeper problem. Any formal theory of intelligence needs to specify what intelligence is for, what problem it solves, and what it optimises. A thermostat optimises temperature, a chess engine optimises board position, and both can be fully described by their objective function. But intelligence seems to be precisely the capacity to redefine what counts as the problem. A human can decide whether to play chess at all, invent a new game, or abandon the entire framing and go for a walk. Formalising that kind of open-ended reframing may require a kind of mathematics that does not exist yet, or it may resist formalisation altogether.

This is where the biology analogy becomes revealing. There is no “theory of organisms” in the same sense that there is a theory of electromagnetism. Biology has a powerful organising framework, evolution by natural selection, along with a vast accumulation of mechanisms, trade-offs, and contingent historical facts. You can explain any feature of an organism after the fact. You cannot derive organisms from first principles. The evolutionary biologist Stephen Jay Gould argued that if you replayed the tape of life from the same starting conditions, you would get a completely different set of organisms. The outcomes are historically contingent, not mathematically necessary.

Intelligence may be the same kind of thing: a product of evolutionary tinkering, cultural accumulation, and developmental contingency that allows useful generalisations but not the kind of closed-form theory that would satisfy a physicist. We may end up knowing intelligence the way we know weather: well enough to make useful short-term predictions, poorly enough that long-range forecasting remains unreliable, and never with the exact analytical solution that would let us derive tomorrow’s clouds from first principles.

Why does any of this matter outside of a philosophy seminar?

The temptation is to treat all of this as an abstract debate, the sort of thing academics argue about while engineers get on with building things that work. That temptation should be resisted, because the theoretical vacuum has practical consequences.

AI safety without a theory of intelligence is navigation without a map. The field depends on assumptions about what future systems can achieve and how those abilities will develop. If we do not understand why current systems perform as well as they do, we cannot predict whether the next generation will improve steadily or make a sudden leap, as we have seen with Anthropic’s Mythos. Scaling laws tell us that larger models perform better, but not what “better” means at scales we have not reached. Will a model 100 times larger than current frontier systems merely write more polished prose, or develop something qualitatively different? Nobody knows, and we lack the framework to reason about the question.

Regulation without definitions is theatre. Governments are drafting AI rules around distinctions (general-purpose versus narrow, high-risk versus low-risk) that depend on a theoretical grasp of intelligence we do not possess. The EU’s AI Act defines a “general-purpose AI model” by compute thresholds that are essentially arbitrary, because no theory links compute to ability in a way that would make any threshold principled. The fault is not the regulators’, but the tools’, which are insufficient.

A business strategy built on vibes is expensive. The corporate world is investing hundreds of billions on the assumption that current trends will continue. Perhaps they will. But the history of technology is full of S-curves that plateau earlier than expected, and the lack of a theory makes it harder than it should be to distinguish genuine improvement from benchmark gaming and evaluation contamination. When a model scores 90% on a medical exam, does that mean it has medical knowledge, or that enough medical-exam text was in the training data? The answer is “it depends what you mean by knowledge,” and we are back to square one.

Where this leaves us

This is not a counsel of despair. Science often advances without complete theories. Medicine cured scurvy centuries before vitamin C was discovered, and engineers built steam engines before thermodynamics was formalised. Practical progress does not require a finished theory, though it helps, especially when the stakes are high enough that mistakes carry consequences beyond a failed experiment.

We are roughly where physics was before Newton. We have observations (scaling laws, emergent abilities, benchmark performance), useful heuristics (more compute and data tend to produce better models), and fragments of theory (compression, mechanistic circuits, predictive processing), but no framework that unifies them and makes novel predictions. The “I” in AI is still a placeholder, a trillion dollars of investment balanced on a word we cannot define.

I am a partner in Better than Good. We help companies make sense of technology and build lasting improvements to their operations. Talk to us today: https://betterthangood.xyz/#contact