Words Come Second to the World

AI is, not surprisingly, at the forefront of many people's minds right now. I started playing around with it using Midjourney to imagine fantasyscapes, not because I can't draw, but because I found it intriguing to see how a different kind of intelligence would convert my thoughts into images.

As I followed AI more closely and became more involved with it, I began to realise there was something very wrong with the way it worked. Hallucinations. Collapsed ambiguities. Systems that were theoretically going to lighten people's workloads, yet in many cases made the work harder because people were suddenly grappling with machine-speed output they could not reliably trust.

I started digging into transformers and how these systems were built. It did not take me long to reach a conclusion that people understandably do not like hearing, particularly after billions of dollars have been invested in the existing approach: I thought AI had been built upside down.

By that I mean something quite specific. Words come second to the world. Without a world to which they refer, words are meaningless. Yet we had built systems extraordinarily good at modelling relationships between words and other representations, then expected them somehow to reconstruct enough of the world from those relationships to act reliably within it.

At the same time, people were being asked to fact-check and validate enormous quantities of machine-generated material, some of which proved to be wrong. AI did make many things faster and easier. But speed came with another cost: the burden of determining whether the output could actually be trusted.

The proposed answer was usually to put a “human in the loop.” But very little thought seemed to have been given to what that actually meant. Was the human a rubber stamp? Were they expected to read everything the machine produced? Did they have authority to stop an action? Were they sampling outputs at random? Could they intervene before something happened, or merely examine the record afterwards?

Those are very different arrangements, and simply putting a person somewhere inside a process does not solve the underlying problem.

A few things in particular troubled me: the continual hallucinations, made-up facts, and the confident arrogance of blatantly wrong answers, particularly when there were ambiguities. Anna told Emma her sister was overseas. Whose sister was overseas? It's a fairly simple sentence and most people would happily answer that it was Anna's sister who was overseas, but that answer isn't justified by the sentence. There is no way to determine whose sister is overseas without further information. AI will often continue without stopping to get that further information. Humans can stop and ask.

I began looking for another way to approach it. If words acquire meaning from a world, then perhaps an AI system needed something more like a world of its own: a persistent state in which claims, entities, authorities, uncertainties and consequences existed beyond the immediate generation of text. And consequences mattered.

That eventually led me to build a working model of a system that could do something deceptively simple: say no, and go no further. People do not particularly like being told no. Systems designed around frictionless assistance like it even less. But sometimes no is exactly what is required. Sometimes it is life-saving.

I showed this working prototype to the big companies on 6 December 2025. They ignored it, and that's okay.

At its core was PEF (Persistent Existence Frame), built to give AI a world it could ingest and work from; a world where consequences mattered and AI wasn't just spitting words out into the aether. Hallucinated claims could not pass as settled fact, and ambiguities could not pass into consequence unresolved. It could also escalate to a human, not for everything, but when something had a serious consequence.

Something else I realised, and built into PEF, was the ability to hold an ambiguity without resolving it. This is actually more important than it seems. Not everything in this world has an answer that we know yet, and sometimes that means we have to sit with that uncomfortable fact: there is no answer available, and we have to leave it there. So I built PEF to keep those questions unresolved until such time as there was an answer, which to PEF could mean forever.

More than anything, though, I wanted to stop AI from making the kind of mistakes that could truly cause harm, and as these systems stand now, those mistakes do cause harm: to companies, to people and to trust. Aurora-Lens, built around PEF, was my answer to that. It's not perfect, nothing is, I think, but it is exceptionally good at maintaining a coherent state, refusing to treat unsupported claims as settled facts, and holding unanswerable questions indefinitely.

What I eventually came to understand is that intelligence was never the whole problem. A system can be extraordinarily capable and still lack the authority to proceed. Governance, to me, is not somebody checking afterwards whether the machine got it right. It is deciding, before consequence, whether the system has enough grounds to act at all.

This, I believe, has come to matter increasingly as AI digs further and further into the structure of our society. Without genuine grounds to act, without an actual world in which consequences matter, AI risks steadily eroding trust in the systems that depend on it. What started out as something that could genuinely help humanity may instead become a hindrance, be put aside, or become something potentially worse.