Bits of “Intelligence”
The AI engines tap into some powerful mechanisms, as does the Internet - -the idea that meaning comes from context and patterns and is not intrinsic -- a concept that is at odds with most people's reality. It is more revolutionary than Copernicus, who shifted a reference frame.
25-Apr-2023Version 2: 2023-05-28 17:09:26


This is a first draft, so beware of typos or words that the AI engines (Grammarly and Word) thought I meant.

These comments are inspired by a post on a mailing list that recommended viewing this talk by Tristan Harris and Aza Raskin. There are two key points:

  • Everything is reduced to tokens.
  • One of the big dangers is spoofing people and wreaking havoc.

These engines are described as language models. That is an appropriate term, but we need to think about what we mean by “language”. Words in a language don’t have intrinsic meaning. That meaning comes from context. The word you say and the word I hear may be different. Alas, the very language we use to talk about this is hobbled by the same ambiguities.

Take the word “hear” – it can mean to simply detect the soundwaves, as in a typical hearing test. It can also mean hearing in the sense of understanding, as in hearing what you’re saying. An interesting example is the difference between a pause and a pregnant pause. An observer doesn’t even know that meaning is being conveyed, let alone whether the listener (is that the appropriate word) is anxiously awaiting the end of the silence.

A key point is that the relationship between the endpoints of the conversation is independent of whatever is between them. And in the case of silence, there may be nothing.

I want to keep this essay short, so I won’t go into the ability of those along the path to distort or block the exchange or to listen in. What meddlers can’t do is know for sure what is happening in people’s heads though they might guess or have a theory of mind.

The idea that we are communicating between two endpoints or even within one endpoint (as in thinking to ourselves) by the exchange of symbols is the same concept that has made the Internet so transformative and so fundamentally different from traditional telecommunications, which purported to transport meaning.

As I wrote in the Age of Software, the idea of decoupling meaning from symbols, bits as representation, is powerful and fundamental. The idea that we can use a common packet infrastructure for all exchanges is an idea that is still difficult for people to fully appreciate. Hence, we persist in public policies, oblivious to this new landscape.

It's also why I’ve compared the conceptual shift to the shift from geocentrism to heliocentrism. We were no longer at the center of the universe, and now, meaning is no longer intrinsic. There is no intrinsic good or evil. So much of our public policy and social constructs are based on the presumption of intrinsic meaning.

It's telling that so much of our view of these AI engines has been informed by looking at how it writes prose or generates images which are soft targets which makes it easy to miss the limitations, though, in Why AI art struggles with hands, you can get a good sense of the limitations. There’s a reason that DaVinci did dissections – he needed to find the hidden mechanisms and not just guess from the surface manifestations.

I use the tools of a programmer assistant, which brings these limitations into sharp relief. It’s like working with an enthusiastic assistant who knows tropes but doesn’t really understand what is going on. In a similar vein, when I do ask for an essay about a technical topic, I see an echo of conventional wisdom that resists innovative thinking. Kind of ironic.

For that matter, such assistance is very frustrating when I’m writing – I use the tools to catch many errors, but I spend just as much time fighting back as it fights any attempt to use modern concepts where words have a different nuance, if not meaning.

Alas, as Raskin points out, these limitations aren’t an issue when we use the technology in service of misinformation. That’s a deep issue in its own right.

Stepping back, there is a deep philosophical issue about where meaning comes from. One way I think of it is as patterns. Heliocentrism is a good example. Ptolemy and Copernicus were observers seeing patterns. There was no difference in the underlying reality but all the difference in the world (and out of it) in how they saw it.

The approach of finding patterns is fundamental to how systems evolve. More to the point, coevolve in multiple contexts – there is no simple measure of fitness. The observer is not anthropomorphic, but rather we just use such terminology because that’s also how language works – repurposing with new meanings. And it’s also the problem with language as the meaning of the words change and drift apart. It’s amazing we can communicate at all! And often we fail to even when using the same words!

At this point, we need an articulate understanding of the new concepts in which meaning is not intrinsic and is fundamentally ambiguous (in multiple contexts).

I tried to simplify the model by ignoring distortions in the “between”. That’s fine for a model, but when we take these models further, issues like relative timing come to the fore as another kind of context. And properties we attribute to intent or intelligence might be implicit. We don’t need to learn to walk if our bodies are built to walk – we just tweak the mechanism.

This is why I worry about those who think we can just represent our minds as an array of bits. It’s reminiscent of the early days of mechanical systems and the idea that we could think of brains as simple machines.

The other related idea is the assumption that we can predict the future and make wise choices. Perhaps I’m overly influenced by time in the world of finance in which generations of mavens have hit upon the way to predict the future – until, as Nassim Taleb noted in The Black Swan, they couldn’t. One reason is that if everyone can predict the future, then no one can. (Similar to the halting problem in computing).

The other reason is that all these mechanisms rely on a useful heuristic of locality. This is what has enabled us to share computers when each program seemed to need the entire memory of the computer. We didn’t normally reference the computer’s memory hither and yon, but rather, references stayed relatively local.

Tools like neural nets had a similar problem – being able to extrapolate from local samples. It’s a great heuristic, but it can easily fail, as when people know the earth is flat because that’s all they see.

The value of heliocentrism wasn’t just that it made navigation easier. It gave us insights into the universe at large by finding commonalities among mechanisms. I have the same approach to learning – I don’t feel I’ve learned something until I have an insightful understanding and don’t have to memorize the details. If I know the subject, I don’t need to study – I can figure it out on the test. If I have to memorize, it means I’m not being taught a pattern. Alas, sometimes, as in language, there may not be a simple pattern, and in the AI engines, the seeming correlations might be spurious.

Recently Sam Altman observed that the current technique of blindly searching for patterns may be played out. While the technique has worked far better than I would’ve imagined, it is not magic.

Humans have a compelling advantage, for now, in that we’ve co-evolved with our environment. We can glance at a traffic intersection and see if it’s safe without elaborate computation because we’re in a dance with the other drivers. This is not a new idea – machines may do arithmetic faster than people, but we have the advantage of billions of years of evolution. We didn’t have to invent hearing from scratch – we just benefited from the accidental property of a jawbone that could react to vibration and ran with it. An AI engine won’t have a reason to stumble upon this concept.

We’re still at the earliest stages of these engines. Few people even understand the way similar principles made the Internet happen. And just as the Internet has created new opportunities for bad actors, these engines have created far more.

I remember Marvin Minsky commenting that if we had unlimited short-term memory, we wouldn’t need intelligence. We now have unlimited memory.

But, perhaps, the real worry is those who want to do us good just as the eugenics of yore wanted to do good for humanity. The real danger may be hubris that scales faster than we can understand it.

A Word from our Overlords

I am amused by the challenge of the nuance of writing. One example is

“Similarly” by itself doesn’t have the same nuance as “in a similar vein”. Not all writers would agree with my choice, but, as with hands, the engines don’t have the full context.

Another example is:

“I am amused” and “amuses me” aren’t really the same. Alas, not all humans hear the nuance either.

Of course, I had to ask ChatGPT 4 to rewrite this essay for me:

Title: The Puzzle of Artificial Intelligence and Language


Artificial intelligence and language models, such as those developed by AI engines, have made considerable strides in recent years. However, it's essential to remember that language itself is a complex and context-dependent system. This essay explores the challenges and limitations of AI in understanding and generating language, as well as the broader implications of these issues for society.

The Power and Limitations of AI Language Models:

AI language models can perform impressive feats, like generating coherent prose and images. However, they also reveal their limitations when used as programming assistants or tasked with writing technical essays. These models often struggle to grasp the nuances of modern concepts or engage in innovative thinking. Furthermore, their performance in detecting misinformation or understanding the deeper meaning of language is still far from perfect.

The Challenge of Intrinsic Meaning:

One of the key challenges for AI language models is that meaning is not intrinsic to words, but rather derived from context. This can lead to misunderstandings and miscommunications, even among humans. To better utilize AI engines, we need a more profound understanding of the role of context and the inherent ambiguity of language.

The Role of Patterns and Coevolution:

Understanding meaning often involves recognizing patterns and adapting to new contexts. The value of this approach extends beyond language – it can also help us gain insights into various fields, from navigation to learning. However, AI engines' reliance on heuristics like locality and extrapolation from local samples can limit their ability to identify more complex patterns and gain deeper understanding.

The Human Advantage:

Despite the progress made in AI language models, humans still have a significant edge due to our coevolution with our environment. This allows us to intuitively understand complex situations and navigate them efficiently. For example, humans can quickly assess the safety of a traffic intersection, whereas AI engines would struggle to do the same without extensive computation. Our ability to draw from our evolutionary history gives us an advantage in understanding and using language that AI engines may struggle to match.

The Future of AI and Its Challenges:

As AI engines continue to develop, they will inevitably create new opportunities and challenges. These technologies have the potential to be used for nefarious purposes or to amplify human hubris. We must be aware of these risks and work to address them, while also striving to improve our understanding of language, context, and meaning to unlock the full potential of AI language models.

I then asked it to regenerate the essay again:

Title: The Power and Limitations of Artificial Intelligence


Have you ever wondered about the capabilities of artificial intelligence (AI) and its impact on our lives? AI engines, like language models, have come a long way, but they also have limitations. In this essay, we will explore the power of AI, its potential dangers, and the philosophical issues surrounding meaning and context.

Main Points:

  • AI and language models: Language models are powerful AI engines that can understand and generate text. However, the meaning of words comes from their context, and the same word can have different meanings for different people. Language models still struggle with understanding context and the true meaning of words.
  • The power of symbols: AI engines have been successful at generating text and images, but they often miss the nuances of meaning in context. This is because they treat everything as tokens or symbols, which can lead to misunderstandings.
  • Limitations in understanding: AI engines can be helpful in detecting patterns and providing assistance, but they sometimes struggle with innovative thinking and understanding modern concepts. They might be great at generating content that conforms to conventional wisdom, but they may not be able to come up with truly groundbreaking ideas.
  • The dangers of AI: AI engines can be used to spread misinformation, which is a significant concern. It's essential to be aware of the potential risks and take steps to mitigate them.
  • Meaning and context: Meaning is not intrinsic; it comes from context and patterns. Understanding this concept is crucial for appreciating AI's capabilities and limitations. The observer's role in interpreting patterns is vital, and understanding meaning can be highly subjective.
  • The future of AI: While AI has made significant strides, it is not infallible. Humans have the advantage of billions of years of evolution, which has allowed us to co-evolve with our environment. AI engines may not have the same advantage, and their development may be limited by their lack of contextual understanding.


AI engines have immense potential, but they also have limitations. Understanding the importance of context and meaning is crucial to appreciate their capabilities and drawbacks. As AI technology continues to advance, it's essential to be aware of its potential dangers and work together to ensure its responsible use for the betterment of society.

And I asked Google’s Bard:

The Dangers of AI Engines

AI engines are powerful tools that can be used for good or evil. They are based on the idea of reducing everything to tokens, which are small units of meaning. This allows AI engines to process information very quickly and efficiently. However, it also means that they are vulnerable to being manipulated by those who understand how the tokens work.

One of the biggest dangers of AI engines is that they can be used to spoof people. This means that they can be used to create fake news articles, social media posts, and even videos that look and sound like they are from real people. This can be used to spread misinformation, sow discord, and even incite violence.

Another danger of AI engines is that they can be used to wreak havoc on our infrastructure. For example, they could be used to hack into power grids, transportation systems, and financial markets. This could cause widespread disruption and chaos.

It is important to be aware of the dangers of AI engines so that we can take steps to protect ourselves from them. We need to develop new security measures to prevent AI engines from being used for malicious purposes. We also need to educate the public about the dangers of AI engines so that they can be more critical of the information they see online.

The Future of AI Engines

AI engines are still in their early stages of development, but they have the potential to revolutionize many aspects of our lives. They could be used to improve our healthcare, education, and transportation systems. They could also be used to create new forms of art, music, and literature.

However, it is important to remember that AI engines are tools. They can be used for good or evil. It is up to us to decide how we will use them. We need to be careful not to let AI engines control our lives. We need to use them to make our lives better, not worse.