Language, AI and what we can learn from Wittgenstein

As Generative AI through Large Language Models (LLMs) started to roll out, I had several déjà vu moments. The philosophy of language has had lots to say on the nature of language, meaning, its relationship to the world and the limits of thought. Wittgenstein is perhaps the most famous of these philosophers and there seemed to be real parallels between what AI was being implemented through AI and his views on language. I kept being drawn back to the Tractatus and Philosophical Investigations.

Many mistakenly see AI, as Wittgenstein saw language in his first book, the Tractatus, as a direct correspondence between propositions and the real world of truth. The early pioneers of AI, with their focus symbolic AI and logic, had limited success as they too had this strictly representational view of language, which is flawed. Wittgenstein famously rejected his earlier idea in his later philosophy for a more complex view of language as a tool for communication not direct representation, as laid out in his ‘Philosophical Investigations’.

 

Family resemblance

He comes close to describing what LLMs do when describing words as having ‘family resemblances’, related to each by various degrees of resemblance in meaning or multifarious relationships. The mistake is to think that universals, like the word ‘game’ can be defined separately from these relationships. We have a tendency to see these universals, or abstract terms, as having some underlying definition, quality or essence that lies beneath all instances of games, but there is no Platonic game or even definition of ‘game’.  Dictionary definitions come from use, not the other way round. There is a nexus of game activities with different qualities. This is the problem with identifying key terms such as AI, intelligence, bias, learning, hallucinations and alignment. We treat these as essential things when there is, in fact, no single definition of any of them.

 

Wittgenstein’s Family resemblance is similar to the way language is represented in LLMs, as relationships between words or tokens. Family resemblance is actually too tight a metaphor, as it has lineage and a timeline, whereas LLMs are mostly timeless and far more complex and multidimensional. Where Wittgenstein is right is in seeing language as the sum total of probabilistic relationships, not defined by the relationships of words to the real world, but in relationships with each other. However, would be a mistake to take this too far, as language also embodies relationships with the real world. It is not as if LLMs have no context and are totally de-anchored from the real word, only that they store and generate outputs using complex relationships of meaning captured in the maths. 

 

Another way of looking at this, is to see meaning as use and LLMs as optimised for use. They respond to prompts by people and continue to respond, in real dialogue. Language has indeed developed for dialogue, not the block representation of the world in written prose, so LLMs play to our natural use of language, which brings us to another of Wittgenstein’s ideas – language games.

 

Language games

In his ‘Philosophical Investigations’ Wittgenstein uses the term ‘language games’ to describe the various ways in which language is used by humans. He saw us engaging is different types of communication, ‘language games’, where language is used in a relatively tight context. Each game represents a specific use of language defined by grammatical rules, and they are embedded in a broader cultural and social context.

 

Here are a few examples of language games that Wittgenstein mentions in his work:

Giving orders, and obeying them, with its direct, purposeful communication, such as to a soldier, at work or in school. Describing an object's appearance, or its measurements describing the physical properties or dimensions of things in the world by architects, artists, engineers, and others. Constructing an object from a description (a drawing) involves not just a description of an object, but one that is intended to help someone else recreate the original object, in areas like engineering and carpentry. Reporting an event communicates what has happened to someone who wasn't there. This is a fundamental to storytelling, journalism, and many forms of social interaction or speculating about an event involves making hypotheses or guesses about what might happen in the future, based on current information. Playing a game with rules, even playing a physical game involves a language game, in which the rules of the game are discussed and agreed upon. There’s also making a joke; telling it, laughing, the complex interaction of telling a joke and understanding the humour, which often requires a shared cultural and linguistic context. These are all examples of the diverse ways in which language is used, each with its own specific rules and contexts. The meaning of words and sentences is closely tied to these activities and cannot be separated from them.

 

As LLMs contain so much language, it would take 22,000 years to read ChatGPT3s training data, they contain many, many different species of ‘language games’, and can reproduce their styles – from academic lectures to chatty dialogue, even formal prose as in stories and poetry, in any genre or style. It is not a homogeneous lump of language but language in all of its forms and stylistic complexity. So when you ask ChatGPT to give you text in the form of a teacher, engineer or journalist, it will do so, as it recognises the language patterns and games within that domain. In one sense, it remarkably uses the closer family resemblance type relationships between words to reproduce languages games. You do get this strong sense when using ChatGPT that it is engaging, not just in the presentation of language but the presentation of language in a style or context.

 

Forms of life

The wider context for using a ‘language game’ for Wittgenstein is what he calls a ‘form of life’, not biological life, but a shared cultural or social background that makes mutual understanding and communication possible. This is the complex, shared background of customs, practices, beliefs, and languages that individuals in a community inhabit. It is the shared human activity that constitutes the background against which our language and our concepts acquire meaning.

 

To imagine a language is to imagine a form of life. In other words, the way we understand language and the meaning of words is deeply influenced by our form of life, our way of living, language is a social practice shaped by these forms of life, and it only has meaning within these specific contexts.

 

This is the problem of assumed context in AI. LLMs both capture these forms of life as embodied in language, but the capture is not complete, as time, action and a world view can be missed. We may be dealing with a limited set of de-anchored or impoverished language games. As AI develops it will include these contexts, sense of time, action and world views.

 

Ethics

Another fascinating area of congruence is in ethics. Wittgenstein was always careful when discussing ethics, as seeing it as beyond language, as something that cannot be discussed meaningfully using the facts of the world, which is the proper domain of language. He categorizes these statements as ‘unsayable’ not in the sense of trivial but because they transcend what can be expressed in language. Ethical statements, in the context of a language game, can be seen as expressions of a way of life, or as part of a particular cultural or social practice. They are not descriptions of facts about the world, but rather they are part of a human activity. They make sense within the practices and forms of life where these language games are played. So ethical statements as deeply embedded within our specific cultural and social practices. They gain their meaning and importance not from corresponding to some objective moral fact, but from the role they play within our lives. In other words, he approaches ethics not as a matter of knowledge, but of practice and living.

 

One could speculate and see him as being highly suspicious of the definite ethical stances taken by the doomsayers and fearmongers, as that is likely to be overreach by language which leads us astray into thinking it has deep certainty and meaning when it does not.

 

Conclusion

For Wittgenstein language is the limitation of thought. He explored this limit by being honest about the very nature of language as generated by the human mind in a cultural and social context. Wittgenstein was a philosopher, mathematician, logician, teacher and architect. His exploration of the nature and limits of thought and language have much to teach us, especially about being careful when thinking about these new forms of AI. His critique of essentialism led him to believe that words do not have common underlying definitions but rather get their meaning from use in relation to other words in language games. They may constitute something we have never encountered before, a new language game or games, a new form of life. His insights into how language works on resemblance between words seems prescient as is his theory of language games as do his views on ethics and language. I think what we can learn here is that language is promiscuous. It tends ti take us forward in a way that makes us think we know what we are talking about but is actually just 'language'. This reification, essentialism or faith in universals is what he criticises and what many in AI move towards, not only in describing its qualities but also its ethical consequences. It is not as good as we think and not as bad as we fear.