Language is perhaps the most wondrous and dangerous of our discoveries and technologies, a magic that must be respected and used very carefully. This magic is a central focus of our creation myths including the ancient Egyptian creator God Ptah’s divine utterance (Memphite Theology), Genesis (God said “Let there be light” and there was light) and the Greek concept of Logos where speech and language is a generative concept (Logos spermatikos) and gives us universal reason and order. The magical or divine breath of language (logos) is the force or father of reason (logic).
Words are more or less useful tools and language can give us an illusion of representation of some aspect of reality. Perhaps the better the linguistic performance, the closer the approximation, but we should not mistake this game for the deeper ‘unspeakable’ reality – to the extent we do make that mistake we move further away from approximating the ineffable.
AI generated image
As Bruce Lee noted “It’s like a finger pointing away to the moon. Don’t concentrate on the finger or you will miss all that heavenly glory.”
“Words bend our thinking to infinite paths of self-delusion, and the fact that we spend most of our mental lives in brain mansions built of words means that we lack the objectivity necessary to see the terrible distortion of reality which language brings.”
(Dan Simmons, Hyperion)
Our use of language must start with our understanding of its risks of delusion and confusion – appreciation of its irreducible limitations. Language use is a game of context, and a wonderful one when used well, but our problems with language and our wider issues with beliefs about the reality of abstract concepts can descend into a collective mental illness. Logic also has some irreducible incompleteness that shows that truth cannot be determined entirely within a self-referential framework. Recursive logic is a language that can be useful, but that does not mean it is necessarily truthful.
In this Part, we will consider some wider philosophical issues with language and logic.
Wittgenstein’s Language Games
“Without philosophy thoughts are … cloudy and indistinct: its task is to make them clear and to give them sharp boundaries.” 2
(Wittgenstein)
Like DNA, languages carry the remnants of our evolution. Words and language are extraordinary compressions of history and infectious carriers of culture. For example, every time I say hello and goodbye, I give praise to God and wish that he is with you (even if not consciously and despite that I am not religious).
Words can be beautiful, lyrical, assonant or dissonant. They might sound like the thing they are referring to (unlike onomatopoeia!).
“Philosophy is a battle against the bewitchment of our intelligence by means of language. The philosopher’s treatment of a question is like the treatment of an illness.”3 (Wittgenstein)
AI generated image
For such little things, words do a tremendous amount of work. These superhero sherpas can carry whole worlds on their backs.
“Could mortal lip divine, / The undeveloped Freight, / Of a delivered syllable
/ Twould crumble with the weight.”
(Emily Dickinson)
Each word is a sign and a seed, ready to grow into a many-branched tree. Their strength is in how much we can compact into them and how gregarious they are, flocking together in great murmurations. Words, when working well together, are a wonderful example of algorithmic efficiency.
Ludwig Wittgenstein can be a challenging philosopher to read. He started his philosophical career at the analytic school which focused on the philosophy of language and logic. His view on the meaning of language changed radically throughout his life.
His earlier position can be summarised as follows:
Language is a picture of reality.
The meaning of a proposition is only related to its truth conditions.
Only propositions that can be verified or falsified have meaning.
Propositions that cannot be verified or falsified are therefore meaningless.
“Whereof one cannot speak, thereof one must be silent.”
(Wittgenstein)
To get an understanding of some of his earlier views, consider the following sentence:
The giraffe is lying on the couch.
This proposition has meaning because it can be verified or falsified. We can verify this proposition by looking at the giraffe and seeing if it is sitting on the couch. The meaning of each term is well understood (albeit note an alternative interpretation of the sentence is that it might mean the Giraffe speaks falsehoods from the couch). Now consider the following:
God is love.
This statement cannot be verified or falsified. The statement does not refer to something that we can find in the world and point to or illustrate. In Wittgenstein’s earlier view, this sentence literally has no sensible meaning.
Thankfully, Wittgenstein relented and repentedin his later years and saw sense. The analytic tradition was unhealthy for many reasons, including its overly technical obsession with language and its attempts to make all uses of language equivalent to logical statements, axioms or propositions.
Wittgenstein argued in Tractatus, that language is a way of representing reality. When we speak or write, we are creating a picture of the world and the meaning of a picture is determined by what it depicts. Similarly, the meaning of a proposition is determined by what it represents. He sought to equate our everyday use of language to logical statements that could be verifiable, falsifiable or indeterminate.
“What’s in a name? That which we call A rose by any other name would smell as sweet” (William Shakespeare), AI generated image
“Language disguises thought. So much so, that from the outward form of the clothing it is impossible to infer the form of the thought beneath it, because the outward form of the clothing is not designed to reveal the form of the body, but for entirely different purposes. The tacit conventions on which the understanding of everyday language depends are enormously complicated”.
(Wittgenstein)
Wittgenstein’s later understanding of language was much more realistic and creative:
Language is a tool that we use to play games
There are many kinds of language games, each with its own rules and conventions
The meaning of a word is only determined by its use in the specific language game being played
Later Wittgenstein believes that language is a tool that we use to play games with each other and compares language to a game like chess. Words have no inherent meaning; the context gives words meaning (this is similar to the Buddhist doctrine of Śūnyatā and also the concept of affordance).
The giraffe is lying on the couch.
Wittgenstein now believes that there is no independent truth proposition in this sentence. ‘Giraffe’ (and indeed any other word) has no meaning other than how it is used in the language games we play with each other.
It is not necessary for ‘giraffe’ to refer to anything that exists in the physical world and, depending on the discussion (the game being played) it could be used as a shorthand for something abstract, comical, artistic etc. That is, it may not refer to an animal at all. It all depends on the players of the language game; the type of language game being played and the context of the word within that game.
God is love.
Likewise ‘God is Love’ is intelligible and has meaning within some language games. For example, the players could understand it to be referring to an abstract entity or power in the universe that has good will towards living things (or at least humans!). They may mean or understand much more (a particular type of god, a particular kind of love) depending on the context and the freight they perceive those words to carry. Alternatively, the players may use the phrase in a parodic or satirical way. It could even be used as an unconventional shorthand within a smaller community of speakers, e.g., for their mutual dislike of a brutal religious dictatorship that forces public utterance of such statements on its people whilst practising hate and violence in the name of that same god.
Wittgenstein’s philosophical shift on language can be summarised as follows:
Early Views (Tractatus Logico-Philosophicus):
Picture Theory: Language mirrors reality, with propositions corresponding to facts in the world.
Truth Conditions: Meaning is determined by verifiability or falsifiability; statements lacking this are meaningless.
Logical Atomism: Complex propositions break down into simpler, verifiable elements.
Later Views (Philosophical Investigations):
Language Games: Language is a tool for various social activities, each with its own rules and context.
Meaning as Use: Words gain meaning through their function in specific language games, not inherent correspondence to reality.
Anti-Essentialism: There’s no fixed, universal essence to language or meaning; it’s fluid and context-dependent.
In essence, Wittgenstein moved from a rigid, logical view of language as mirroring reality to a more flexible, pragmatic view of language as a tool for social interaction and meaning-making. Later Wittgenstein teaches that a language game is any communication between two or more parties whereby the meaning of the language is determined by the rules of the game agreed by the parties playing. This superposition of meaning even allows words to sometimes have two opposing meanings depending on the context. The meaning of words and phrases within any language are often somewhat ambiguous and they evolve over time. Wittgenstein’s focus on language games reminds us that words only have a relative variable meaning that can be ascertained contextually. This also makes understanding language particularly challenging for computer programming and AI.
“Human language is built on a foundation of symbolic reference, and the cognitive resources required for symbolic thought are unparalleled in other species.”4
(Terrence W. Deacon)
Wittgenstein’s insights into the shared relative meaning of language have been profoundly influential and can be seen in seminal works such as by Terrence Deacon, which argue that language is not just a complex form of communication but a symbolic system, meaning that words do not directly correlate to objects or actions but stand for them through shared conventions.
Deacon’s work provides a rich framework for understanding the symbolic complexity of human language. This stands in stark contrast to how, for example, AI systems currently handle language generation. While AI can produce impressive simulations of language, to date it lacks the cognitive architecture that gave rise to human symbolic thought. That said, evolutionary pressures on AI systems could give rise to deeper symbolic dexterity.
Turing’s Imitation Game – Can Machines Think?
“I PROPOSE to consider the question, ‘Can machines think?’ This should begin with definitions of the meaning of the terms ‘machine’ and ‘think’.”5
(TURING)
Alan Turing went on to state “The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous. If the meaning of the words ‘machine’ and ‘think’ are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, ‘Can machines think?’ is to be sought in a statistical survey such as a Gallup poll. But this is absurd.”
Language did not arrive in humans fully formed, it is a technology that evolved from simpler beginnings and became more complex as it increased the competitive advantages to the language users.
AI generated
The Turing Test can be simplified and reformulated as follows:
A human interrogator is placed in a room separate from two other participants: a human and a machine.
The interrogator can communicate with the other two participants through a text-based interface (it is a ‘behind the veil’ test).
The interrogator’s goal is to determine which of the other two participants is the human and which is the machine.
The machine’s goal is to convince the interrogator that they are human.
The test is conducted over a period of time, and the interrogator is allowed to ask any questions they want.
Turing believed that by the year 2000, it would be possible to have a machine that would give an interrogator no more than a 70% chance of determining it was a machine within 5 minutes of questioning:
“The original question, ‘Can machines think!’ I believe to be too meaningless to deserve discussion. Nevertheless I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.”
Turing argued that the imitation game is a good way to measure machine intelligence because it is based on the ability of a machine to carry on a natural language conversation with a human. He also argued that the test is objective because it does not rely on any subjective criteria, such as what the interrogator thinks about the participant’s personality or appearance. He gave short shrift to theological objections about man’s immortal soul. He quite rightly asked, if that was the case why elephants should not also have immortal souls and indeed any animate life form. Turing then turned his thoughts to the ‘head in the sand’ objection:
“We like to believe that Man is in some subtle way superior to the rest of creation. It is best if he can be shown to be necessarily superior, for then there is no danger of him losing his commanding position. The popularity of the theological argument is clearly connected with this feeling. It is likely to be quite strong in intellectual people, since they value the power of thinking more highly than others, and are more inclined to base their belief in the superiority of Man on this power. I do not think that this argument is sufficiently substantial to require refutation. Consolation would be more appropriate…”
Turing also discussed Gödel’s theorem – and the impact of their being undecidable logical statements – on the ability of a machine to respond to certain questions appropriately or at all (i.e. without going into a potentially infinite loop as per the ‘halting problem’ that we will consider below in the ‘Limits of Logic’):
“The short answer to this argument is that although it is established that there are limitations to the powers of any particular machine, it has only been stated, without any sort of proof, that no such limitations apply to the human intellect… I do not think too much importance should be attached to it. We too often give wrong answers to questions ourselves to be justified in being very pleased at such evidence of fallibility on the part of the machines…In short, then, there might be men cleverer than any given machine, but then again there might be other machines cleverer again, and so on.”
Turing points out (quoting from Professor Jefferson’s ‘Lister Oration for 1949‘) that another main objection is the argument from consciousness:
“Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.”
As Turing states, this objection ultimately leads to solipsism and extreme prejudice, since the only way to know if and how a machine, or indeed another human, thinks is to be that a machine or human. Such an objection must be abandoned because, whilst it may still be a logically valid view (in some respects), such a view makes communication and dialogue all but impossible and so defeats the purpose of the enterprise – which is to test whether a computer program can play the imitation game effectively.
To take up Turing’s point on solipsism, as Turing noted, his test works just as well for humans. The assumption in the Turing test (which I will use as it is the modern name for the imitation game) is that at a sufficiently fine-grained level of certainty, there is a point at which we can not know whether another agent is similarly human or not and all we have to determine the question is evidence based on the agent’s actions. This is the same as the inability of humans to know what is happening in other human’s minds (the wider problem of ‘other minds’).
Our knowledge of other human minds is inferred indirectly from their behaviour, our assumptions and our understanding of how we ourselves function. This problem becomes most apparent with humans that have sociopathic tendencies which allow them to simulate normality (think of the charming Jeffrey Dahmer) whilst secretly torturing or murdering other people for their own pleasure (though often with limited self-control). It is usually justified by, or gives rise to, a lack of belief in the validity of the other people to be meaningful agents themselves i.e. other people are just objects.
In Turing’s paper, he pre-empted some other objections such as it is just a parrot that repeats its inputs (readers will note that the parrot line keeps rearing its ugly beak) or that it cannot do something unusual or surprising. All these arguments are defeated by experience and have little theoretical power. Interestingly, Turing also considered, as a near-future possibility, that a machine and program might observe its own operations and outputs and learn from them to modify its operations to better achieve a targeted objective.
“…it is not altogether unreasonable to describe digital computers as brains..If it is accepted that real brains, as found in animals, and in particular in men, are a sort of machine it will follow that our digital computer suitably programmed, will behave like a brain.”
Turing’s non-discriminatory approach – which seeks to avoid unjustified human exceptionalism – still remains valid today as a starting point for assessing the intelligence of computer programs. Its non-discriminatory principle also has much wider relevance for assessing the intelligence of different species and life forms.
Searle’s Chinese Room
“Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a database) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output).“6
(John Searle)
Searle wished to counter Turing and his test for machine intelligence. In Searle’s view, his Chinese Room Argument (CRA) shows that mere proficiency at a language game is not equal to understanding:
“The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese”.
He states that if a man does not understand Chinese despite using such tools, then neither does a computer “because no computer, qua computer, has anything the man does not have.” With the CRA, Searle hopes to have demonstrated the following:
Simulation of understanding something is not equivalent to understanding something
Intentionality in human beings (and animals) is a product of causal features of the brain
Algorithmic operations and processing using programs are not sufficient in themselves to create an agent with intentionality
Algorithm: a process involving the use of discrete steps or rules to solve problems in a manner that can be replicated without deviation in the output. In simpler terms, algorithms can be considered recipes to make something which if followed will result in the same output each time (e.g. a cake well baked).
Hanzi: the Chinese characters used for writing. They are logograms meaning that each character represents a word or a morpheme. A morpheme is a meaningful word or language-building block that does not stand alone in the context (unlike a word) but is used to create words e.g., if you look at the etymology of many English words you see they are made up of a number of conjoined root words (e.g. Kaleidoscope). In the case of English often roots from different languages are joined.
“strong AI has little to tell us about thinking, since it is not about machines but about programs, and no program by itself is sufficient for thinking.”
Searle further states that, if he is the person in the room who responds in Chinese (using an algorithmic process) to questions and statements written in Chinese, then it proves that the Turing Test is inadequate – since his responses would lead an observer outside of the room to conclude that he knows Chinese when Searle knows that he does not.
Searle’s wider claim is that strong AI can not exist because computer programs are not like human minds and any formal syntactic skill (parsing the structure and ordering of words and skill with grammar) can not of itself equate to semantic reasoning (understanding and communication with meaning). It allows for a form of weak AI to exist since this is simply the ability to simulate understanding (in the eyes and ears of an observer) without any real understanding:
“Computation is defined purely formally or syntactically, whereas minds have actual mental or semantic contents, and we cannot get from syntactical to the semantic just by having the syntactical operations and nothing else….A system, me, for example, would not acquire an understanding of Chinese just by going through the steps of a computer program that simulated the behavior of a Chinese speaker” 7
First, let’s give Searle some credit. The fact that his argument has annoyed so many philosophers for decades must be considered a good sign. If it was so easy to refute or dismiss, we would not be discussing it here 43 years later. Also, I think the CRA is helpful to consider what might be needed to get closer to artificial general intelligence. There have been many objections over the years to his CRA including the systems reply (the room understands Chinese), the robot reply, the brain simulator reply, the other minds reply, and the many mansions reply. Searle does a good job of defending against these counter-arguments.
To simplify, in the CRA, the person using the tools does not have any understanding of what the logograms mean. This is similar to my understanding of the following translation of the previous sentence:
在他的中文房間爭論中，出發點是使用工具的人對語標的含義沒有任何理解，這類似於我對以下內容的理解
These logograms were created by knowing that Google Translate can take any English sentence, parse and decode it into Chinese and return the output. Searle’s argument is that neither Google Translate (nor myself using it) can thereby be said to understand Chinese. I can agree as regards my understanding. The issue of what, if anything, is ‘understanding’ Chinese within Google Translate is obviously part of the issue raised by Searle. Searle’s CRA hints at a dualistic position that if there is no “ghost in the machine” there can be no real understanding:
“Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them, those who send in the input and those who interpret the output”
An obvious objection to Searle’s thought experiment is that it is not empirical – after all Turing created a thought experiment for digital intelligence that could one day be tested – whereas Searle’s thought experiment is not really testable as it was constructed. However, this is only a very limited objection since thought experiments are very useful even if not immediately testable (Einstein famously used them). Indeed, Turing himself also used a thought experiment of a magical computer that you could imagine but that logically could not exist to prove a conjecture using contradiction (see the halt program in the Limits of Logic below).
Now let’s deal with some stronger objections to the CRA:
Searle does not create a setup that would enable an observer to test whether an agent in one room really understands Chinese and one in another room does not, so his argument really has little to do with Turing’s test. His argument is against an entirely algorithmic or computational view of ‘understanding’ which he somehow equates to intentionality.
The Turing test does not test for intention or understanding nor does Turing equate passing the test with having either of those qualities or functionalities.
Searle’s CRA merely asserts a conceivable truth condition – there is an agent that does not understand Chinese whilst the interactions involved in manipulating Chinese symbols could lead an observer to believe they did understand Chinese. That said, I think Searle is correct that we must agree that this is possible whether the agent uses a book or a program and whether the agent is a human or a computer.
Not only does the CRA not test for understanding, but Searle does not successfully define what understanding Chinese (or any language) means. He equates it variously with having “intentionality”, “internal causal powers equivalent to those of brains” “mental states” and “its ability to produce intentional states”. It is true that Turing also did not define intelligence. However, he gave us a test for it that seems intuitively useful given the complexity of human behaviour and contextual use of language (success in the language game being much more than just the amount of horsepower available for complex calculations).
Unlike Turing – who knew a thing or two about computers and code – Searle created a theoretical operational system (the human with the instruction manual and Chinese symbols) in a room that could not actually pass Turing’s test. Searle does not create a test for mental states or intentionality which is the very thing he assumes computer programs do not have.
Let’s delve a little deeper into some of these points.
The way Searle set up the CRA thought experiment does not really deal with the subtlety of the Turing test. Remember, the Turing test is not a test of whether some hidden agent ‘understands’ English or Chinese (whatever ‘understands’ means) but whether a computer could pass as a human using the language game (or whether it is undecidable – and therefore a win for the computer). The contention is that this is functionally equivalent to intelligence and so observable and non-discriminatory. The Turing Test requires that we do not know whether we are interacting with a program or person (since the responder is hidden) though if we consider an expertly crafted humanoid robot the responder can be in plain sight and the Turing Test works just as well since it continues to beg the apposite question: how would we know which agent is digital and which is human based solely on a language game?
What is intelligence?
The deep question that is intentionally avoided by Turing is: what is intelligence and what is thinking? He avoided it precisely because it risks being indeterminate or metaphysical (given that we do not really understand what it is in humans) and therefore not capable of sensible discussion and verification. The deep question raised by Searle is: what is understanding? Unfortunately, Searle neither defines it properly nor creates a test to verify it.
Searle’s CRA boils down to a rejection in principle of the idea that syntactic or even semantic manipulation skill is equivalent to a semantic understanding of language. He is right to point out that the process of answering something cannot be the reason or motive for answering something. Likewise, a cooking recipe is not the cake, nor is it the reason for baking or eating the cake. Searle suggests that this may have been what some people were asserting with descriptions of strong AI. However, Turing was not seriously suggesting that an algorithm is by itself both the process to reach a conclusion and the consciousness, desire or agency to do the same.
Searle’s argument is deceptively clever; he created a thought experiment of human ignorance to prove his argument against digital intelligence. However, the CRA is also a form of circular reasoning where its premise assumes the truth of its conclusion rather than setting up an experiment to prove the argument or conclusion. The circularity arises from how Searle frames the concept of “understanding.” He presupposes that true understanding requires something intrinsic to the human mind (like consciousness or intentionality) that a computer program inherently lacks. This is the conclusion he wants to reach. However, in his thought experiment, he defines understanding as something the person in the room doesn’t have, simply because they are following a set of rules. This becomes the premise of his argument.
In essence, the Searle argument goes as follows:
The person in the room doesn’t understand Chinese (premise 1).
A computer program is like the person in the room (premise 2).
Therefore, a computer program cannot understand Chinese (conclusion).
The problem is that the first premise already assumes the conclusion to be true (and so the argument is a form of logical fallacy). Searle hasn’t independently established what “understanding” means or how to measure it, he has simply defined it in a way that excludes the possibility of a computer program ever achieving it.
To avoid circularity, Searle would need to:
Provide a clear, objective definition of “understanding” that is not inherently biased against computer programs.
Propose a way to test for this understanding, independent of the process used to generate responses (whether it’s a human with a rule book or a computer program).
Searle creates a straw man that he can then more easily set on fire. Searle might be accused of a sort of intellectual laziness and appeal to prejudice given many of these objections:
“As long as the program is defined in terms of computational operations on purely formally defined elements, what the example suggests is that these by themselves have no interesting connection with understanding.”
In short, Searle argues that any computer program that passes the Turing test does not thereby pass the Searle test. How does a computer program pass the Searle test?
We have no idea, and Searle does little to assist us be the wiser. No observations are suggested that would allow us to infer intentionality, mental states or causal powers. His CRA keeps our, as yet poorly understood and undefined, mammalian minds in a black box and gives us no way to test for equivalence in digital minds (even if such things can exist).
“Could a machine think?” On the argument advanced here only a machine could think, and only very special kinds of machines, namely brains and machines with internal causal powers equivalent to those of brains. And that is why strong AI has little to tell us about thinking, since it is not about machines but about programs, and no program by itself is sufficient for thinking.”
We could also pose a few further challenges to Searle’s own definition of intelligence and understanding. For example, Searle does not consider learning as a process that happens over time and which, interestingly, often starts with mimicry. For example, he is surely not claiming that when Young and Champollion started to uncover the rudimentary structure of ancient Egyptian hieroglyphic language, that they did not understand ancient Egyptian because only a person with real understanding of the meaning of specific hieroglyphs can pass his test of real linguistic understanding? In fact, if I understand his challenge properly, syntactic skill is no test of understanding language at all since he is not interested in tests of skill with symbols and signs as that could be synthetic intelligence lacking understanding.
Hieroglyphs are largely a mixture of logograms (that record words or morphemes as elements of the language) and phonograms (that replicate the sounds made when speaking the words the symbols represent) e.g., an animal hieroglyph may be used to represent the sound or first letter vocalised when speaking that animals name in oral Egyptian. Although sometimes referred to as an ideogrammatic language—where abstract concepts are directly represented through visual images, such as mathematical symbols—the Egyptian hieroglyphic system primarily relied on logograms and phonograms, with ideograms being comparatively rarer.
Young and Champollion first started to understand hieroglyphs using their semantic and, more importantly, their syntactic language skills for other languages. Over time this ability to parse and decode elements of Egyptian hieroglyphs (relying heavily on their knowledge of other languages) led to greater understanding of the now-extinct language. One of the key breakthroughs was the decipherment of the Rosetta Stone which originally contained the same meaningful sentences in Greek, hieroglyphs and demotic (although only fragments of each remained in modern times).
en_el_Papiro_de_Hunefer.jpg, Wikipedia
It is true that understanding ancient Egyptian culture and the meaning of language in that culture is more than just a syntactic exercise. However, using the Rosetta stone to parse and decode hieroglyphics was primarily an algorithmic operation and one that a computer could do with sufficient syntactic skill in other languages and data to work with. Likewise, any child learning an expressed language starts by getting a feel for the grammar (largely subconsciously) and words (largely consciously) used for entities and emotions.
“Imitate, Assimilate, Innovate”
(Clark Terry)
Neither intelligence nor understanding are discrete binary states or absolute positions. Understanding takes place within a spectrum (to some extent Searle acknowledges this in the CRA paper). Understanding also takes place on different levels, e.g., we can get a feeling from a poem that may well be the feeling the poet hoped to invoke or elicit though we may not know why or be able to explain it. Likewise, we may not understand all of the words used in an essay but can still get the gist of what is being communicated (though less so for academic papers on linguistics!).
Anyone examining the issue could likely agree that for an AI program or a human to understand Chinese requires that their cognitive abilities are more than just the sum of available algorithmic processes. Turing would also no doubt agree that understanding and processing are not equivalent.
What is intentionality?
Intentionality is a fuzzy concept. It is suggested generally to mean doing something (including thinking and playing a language game) with purpose or intent. A more technical definition is by the German psychologist and philosopher Franz Brentano, who defined it as the property of (internal) mental experiences that refer to external objects or entities. This seems similar to what I surmise is Searle’s working definition. Searle also believes that intentionality is representation, meaning that every intentional state or event has an intentional content that represents its conditions of satisfaction (my abstract thoughts and desire for a baked cheesecake can be satisfied by eating a baked cheesecake). We will explore these issues further in ‘Come Now Mr Searle, That’s Just Semantics!’ below.
The only meaningful tests we have for understanding a language involve observable measurable tests. The CRA is therefore an argument that a computer program could pass all these tests and still fail the understanding/intentionality test. It is very difficult to see how the poor computer program can ever prove its intelligence through language games if intelligence means intentionality. Searle subsequently enjoyed considering and dismissing the idea of functionally equivalent intelligence involving other substrates, such as toilet paper and beer cans.
“John Searle… has gotten a lot of mileage out of the fact that a Turing machine is an abstract machine, and therefore could, in principle, be built out of any materials whatsoever…he pokes merciless fun at the idea that thinking could ever be implemented in a system made of such far-fetched physical substrates as toilet paper and pebbles… or a vast assemblage of beer cans and ping-pong balls bashing together. In his vivid writings, Searle gives the appearance of tossing off these humorous images lightheartedly and spontaneously, but in fact he is carefully and premeditatedly instilling in his readers a profound prejudice, or perhaps merely profiting from a preexistent prejudice.”8
(Douglas R. Hofstadter)
To my mind, it is fair to say that Turing was much more careful in his imitation game test than Searle was in his Chinese Room argumentation. Turing quite rightly focused on the limits of what we can know and how we can objectively verify what we know, which is why he sought a non-discriminatory test of intelligence.
After all this wordplay, we are left with great difficulty in knowing whether a human or inhuman AI entity really understands languages unless we are more definite in what we mean when we use terms such as ‘understand’. To give Searle some benefit of the doubt, the language game is to some extent predicated on us agreeing that when we use the term ‘think’ we may also imply ‘understand’ (words flock together after all) – by which we mean something like having a general framework to contextualise the meaning of what is communicated and a reason or motive for communicating. To say that this must also imply a new concept of intentionality is however an unjustified step too far.
“‘It is no secret. All power is one in source and end, I think. Years and distances, stars and candles, water and wind and wizardry, the craft in a man’s hand and the wisdom in a tree’s root: they all arise together. My name, and yours, and the true name of the sun, or a spring of water, or an unborn child, all are syllables of the great word that is very slowly spoken by the shining of the stars.”9
(Ursula Le Guin)
Come Now Mr Searle, That’s Just Semantics!
“A problem that proponents of AI regularly face is this: When we know how a machine does something ‘intelligent,’ it ceases to be regarded as intelligent. If I beat the world’s chess champion, I’d be regarded as highly bright.”10 (Fred Reed)
Before assuming that AI is not capable of intelligence with language, we first need to take another look at what ‘understanding’ means. Given the relative contextual meaning of any words, let us start with a word map of the more semantically related words to ‘understanding’.
Semantic word cloud using 50 tokens that are most closely related to ‘understanding’ obtained from OpenAI ChatGPT. I then put these into the University of Arizona’s word cloud software (I used the ‘cosine coefficient’, ‘lexical centrality’ and ‘star forest’ settings).
You will note that the concept of ‘intentionality’ – introduced by John Searle in his Chinese Room Argument – is not showing as being very closely related to the meaning of ‘understanding’, though it lives in the same neighbourhood. I asked Bard for the approximate semantic delta between the word pair, it suggested they are quite closely related – somewhere between 0.2 – 0.4 on a scale of 0 to 1. With ‘0’ meaning semantically synonymous and ‘1’ meaning a word pair are diametrically different or maximally dissimilar in meaning. LLM’s suggest that ‘understanding’ and ‘intentionality’ are approximately as closely related as: Knowing vs. Believing, Perception vs. Interpretation; Thought vs. Idea; Memory vs. Imagination; and Consciousness vs. Awareness.
Semantic Shell Games
The CRA was quite a neat trick by Searle. He started by seeking to dispute the concept of intelligence and functional equivalence used by Turing. To do this he changed the test of intelligence to one of understanding (undefined) and he then equated understanding to the concept of intentionality (which he also leaves semantically amorphous).
Does this word game really take us closer to the heart of the matter?
We are in danger of creating semantic shell games, where each time we find a certain quality or feature of understanding exists then anyone can just move the argument to the other related meanings (and further derivatives) until we either agree that the test of understanding is met or we argue that is not what we actually meant or not what matters anyway. This has been called the ‘AI effect’ and is precisely what Turing sought to avoid.
2D representation of some other main concepts implied by ‘understanding’.
This problem also goes to highlight that the Buddhist doctrine of dependent origination applies equally to these abstract concepts. Individual terms that we use to explain complex things are inherently empty of meaning. Concepts like ‘intelligence’ and ‘understanding’ are contextual, multivariate and multi-modal aggregates.
It does not assist us in trying to make sense of (or if feeling brave, disprove or prove) the existence of an abstract concept by introducing a slightly different but related abstract concept (as Searle does with his CRA). An argument or appeal against computer or AI ‘intelligence’ relying on other compound concepts (such as understanding or intentionality) does not help us move forward, instead, it sets up an inherently recursive loop. It breeds an indeterminate and defective language game.
Searle suggests in his CRA paper that he is willing to grant a machine the ability to understand a language in principle – though, one suspects, not in practice for digital computational machines. However, if we follow Searle’s suggestion, intentionality likely requires identity and so knowledge of other things that do not share the same identity. Under this definition, adult and young animals, like children, have intentionality but perhaps not the very young or some animals such as some species of insects (or perhaps some insects would likely not be able to evidence individual intentionality given their swarm like intelligence and collective identity, however, the group might have collective intentionality). This likely brings the need for evidence of acting with self-awareness and desire and so on.
Searle’s argument therefore leads us ever further away from the question and test put forward by Turing, but like an errant prophet, he does not deliver us to the promised land of understanding.
Understanding a language requires comprehension and not just linguistic competence. It encompasses cultural awareness, emotional intelligence, and the ability to use language creatively and authentically. Any test must therefore focus on deep and nuanced understandings of specific languages within their cultural contexts. Language mastery cannot be divorced from understanding specific histories of peoples and their unique cultures.
If LLMs can solve new language problems and play new language games that go beyond their training data – which are themselves governed by partly unsupervised learning – and which would persuade a language expert that they are fluent and sophisticated in a specific language, then it is difficult to see how we can exclude AI LLMs from having the capacity of ‘understanding language’. In this respect, Searle’s CRA may be helpful to consider some of the issues involved in what we infer from information available, but it is logically fallacious as an in-principle refutation of understanding language by computer programs. Imagining a human or machine that can simulate understanding of a language whilst not understanding it does not invalidate the ability for human or digital understanding of language. Indeed, without a suitable test of linguistic comprehension or understanding the CRA is specious. with no predictive power.
[Philosophers and linguistic experts should pay much more attention to poets, they frequently convey emotions and meaning by bending and sometimes breaking the commonly accepted language rules.]
Noam Chomsky famously distinguished between universal grammar (see e.g. his essays in Language and Mind, 1968 (reissued and updated) and Knowledge of Language: Its Nature, Origin, and Use, 1986) and unconscious internal language (I-language) – which is not directly accessible to our conscious understanding – from E-language, which is the expression of that unconscious I-language. In other words, in his view linguistics is the study of expressions of language and not of language itself. His view is that humans, uniquely amongst lifeforms, make use of Universal Grammar (UG).
UG is Noam Chomsky’s theory that all humans are born with an innate ability to learn language. According to Chomsky, this biological predisposition consists of a set of grammatical principles and structures shared by all languages, which he calls the “universal” aspect of grammar. Chomsky argues that while languages differ in their vocabulary and specific rules, they all follow certain underlying principles, such as sentence structure, word order, and recursion. This shared structure allows children to learn language quickly and efficiently, even with limited exposure, suggesting that language acquisition is not solely dependent on environmental factors.
The concept of UG addresses a fundamental question: How do children acquire language so rapidly across so many disparate cultures? Chomsky’s answer is that the human brain comes pre-equipped with a faculty and structure. He argues that this also sets limits on the extent to which languages can vary, and this makes it possible for children to internalise the rules of any language before they are even exposed to it.
In Chomsky’s view, UG is genetic and so something which all Homo sapiens have inherited (and which perhaps separated us as a subspecies sometime in the past).
Human language is unique among all forms of animal communication. It is unlikely that any other species, perhaps even including our close genetic cousins the Neanderthals, ever had complex language, and so-called sign ‘language’ in Great Apes is nothing like human language.12
Initially, his view was that UG is a conceptual layer of innate in principle language capacity that has built-in parameters that limit language in certain ways. More recently, and perhaps due to many exceptions being found to his universal language rules, he has limited his assertions to UG being simply computational recursion as a capability innate to all humans. UG is therefore central to Chomsky’s broader theory of generative grammar, which seeks to describe the implicit rules that govern the structure and generation of all possible human sentences.
By computational recursion, I believe he means language’s generative ability (due to recursion) to allow theoretically ‘infinite’ expression from finite rules, concepts and words. Recursion enables us to combine words and phrases in creative ways to create semantically and syntactically sensible sentences and utterances that have never been said before.
“the most elementary property of the language faculty is the property of discrete infinity… this property is virtually unknown in the biological world”
However, compare ‘Language as a discrete combinatorial system, rather than a recursive-embedding one‘. This article13 argues that language cannot be a recursive-embedding system in the terms of Chomsky (1965 et seq.) but must simply be a discrete combinatorial system , basically in the sense of dependency grammar.
Whilst we have not found conclusive evidence of the same use by animals of this generative recursive power of language we must bear in mind, as Chomsky has noted, that all life forms – including humans – are built from the same language of virtually limitless (or discrete infinities) found everywhere in nature.
We and everything about us, including our use of languages, have evolved from this older language. The language of life found in DNA – and represented by theoretically infinite recursions of GATC – has enabled a wondrous and endless diversity of life forms. That is not to say the human language for communication and understanding is the same as the language of DNA for information transfer – it is to say that our use of language is a natural and explicable extension of deeper natural laws governing the evolution of all life forms. The human capacity with language is another order higher than that found in the animal kingdom and may even be unique. However, we should always see language as primality an evolutionary product.
“Thus, from the war of nature, from famine and death, the most exalted object which we are capable of conceiving, namely, the production of the higher animals, directly follows. There is grandeur in this view of life, with its several powers, having been originally breathed into a few forms or into one; and that, whilst this planet has gone cycling on according to the fixed law of gravity, from so simple a beginning endless forms most beautiful and most wonderful have been, and are being, evolved.”14
(Charles Darwin)
In Chomsky’s view, Universal Grammar is the foundation that allows for a human under the right stimulus to develop an internalised understanding of language (I-Language) which is general language capability for any actual expression of language. I-Language in turn enables the human to develop externalised language abilities, for example, to speak English or German and so the human moves to language performance. It appears, though it is not explicitly stated, that Chomsky believes that UG is signless, i.e., it is formless and entirely without pictures or words.
Chomsky’s views on language can be difficult to read and summarise. For example, previously Chomsky claimed a theory of generative grammar, derived from universal grammar, that every sentence has two levels of representation: surface structure and deep structure. The surface structure is the form of written or spoken language (what we would usually call the syntactic level that includes the order of words) whilst the deep structure provides meaning (the semantic level) and helps to generate the surface structure. He stated that the ability to understand any language is therefore a result of the deep structural knowledge which is not conscious.
This is not a criticism per se but it does make it difficult to know whether Chomsky has managed to develop a coherent testable general theory of language. These are what I believe to be the common features of his theories – language is:
A cognitive phenomenon: language is a product of the human mind and is not simply a product of our social environment.
Universal: all human languages share some common properties due to our innate capacity for language.
Generative and recursive: we can generate an ‘infinite’ number of sentences from a finite set of words and rules. Sentences can embed other sentences within them.
Rule-governed: The deeper rules of language are a product of our biological makeup and not conscious learning.
Minimalist: in his later work he deprecated the strong focus on rules in exchange for a strong minimalist theory, i.e., that language optimises efficiency and has minimal rules and structures.
An evolutionary discontinuity: It is qualitatively different to other functions arising through evolutionary forces. He has variously claimed unique properties for language including that it arose through single gene mutation but perhaps did not confer an obvious immediate advantage. However he seems to have modified his stance on this since:
“Suppose that some ancestor, perhaps about 60,000 years ago, underwent a slight mutation rewiring the brain, yielding unbounded Merge. Then he or she would at once have had available an infinite array of structured expressions for use in thought (planning, interpretation, etc.), gaining selectional advantages transmitted to offspring, capacities that came to dominate, yielding the dramatic and rather sudden changes found in the archeological record.”
(‘On Phases‘, 2008).
Chomsky is supportive of David Marr’s three levels of system analysis and believes that current developments in AI (and indeed much of behaviourist approaches to science) are misguided, in that they focus too much on ever-increasing data sets and not enough on the implementation layer of language.
“McClamrock: Marr’s Three Levels”, 1991.
“David Marr presents his…summary of “the three levels at which any machine carrying out an information-processing task must be understood“:
Computational theory: What is the goal of the computation, why is it appropriate, and what is the logic of the strategy…?
Representation and algorithm: How can this computational theory be implemented? In particular, what is the representation for the input and output, and what is the algorithm for the transformation?
Hardware implementation: How can the representation and algorithm be realized physically?”
In Chomsky’s view there is no algorithm for language, so it cannot be reduced to algorithmic processes (echoing Searle’s criticisms of Turing’s functional equivalence approach).15
To my mind, rather than trying to constrain our ability with language within an elaborate definition of grammar, as Chomsky does, it is much simpler and easier to allow grammar and language to have their everyday meanings. After all language is a communication game between people. We should also move away from an overly formalistic approach. In addition, any claims about language, and our ability with it, must be firmly rooted in wider frameworks that connect us to all lifeforms on Earth going back billions of years. It should branch from existing sound scientific frameworks.
We don’t know that much about flea education and communication, however we do know that bees waggle to tell their sisters about the direction and distance of food, whilst crows are exceptionally talented communicators and may even understand the concept of recursion (though Chomsky is not convinced).17
Much of our thoughts and research on language have been human-centric.18 However, recently we have seen efforts to situate humans within theories of language that take account of our brothers, sisters, cousins and even more distant relatives on the Tree of Life.
Indeed, who knows, AI may even help us start decoding some aspects of dolphin language and others. 19
“Chimpanzees, among other primate species communicate different kinds of information in the wild and in captivity. In the wild, chimpanzees use vocal communication to warn others about the presence of potential threats, modulate social dynamics, communicate about food, and to greet and indicate social status.”20
(Voinov, P. V., Call, J., Knoblich, G., Oshkina, M., & Allritz, M.)
Theories of language have only recently started to shake off some of the more overly formalistic shackles of linguistic theory (in that sense, mirroring Wittgenstein’s philosophical journey from perceiving language as an expression of some truth proposition or thing in the world to language as communal play with words having no inherent meaning in themselves). By situating linguistics within a broader evolutionary framework, we avoid the difficulties involved in claiming that language is universal in humans – and yet not underpinning that claim within the evolution of humans and other animals and evidencing it within that wider living landscape.
I think Chomsky’s later position on language and grammar encourages a multidisciplinary approach that more strongly incorporates evolutionary theory to try to make sense of the apparent uniqueness of the human use of language.
Notwithstanding his stature as the ‘father of modern linguistics’, Chomsky’s views have, more rarely, come in for some stern criticism.
“But Chomsky is no Einstein. And linguistics is not physics. Unlike Einstein, for example, Chomsky has been forced to retract at one time or another just about every major proposal he has made up to his current research, which he calls ‘Minimalism’. …And unlike physics, there is no significant mathematics or clear way to disprove Chomsky’s broader claims…”21
(Daniel Everett)
Much as there is much to admire in Chomsky and his work, he has at times made unnecessary claims for overly specific views of language and grammar (insisting on quite specific definitions). Many of his older theories seem unnecessary to explain the human faculty with language. Using Ockham’s razor, we should look to remove any superfluous interpretations or theories that do not add to our understanding. Likewise, we can usually ignore theoretical claims that are not capable of falsification. On this basis, modern linguistic theory and practice should be strongly focused on AI, as an almost unimaginably fertile playground to test linguistic theories and hypotheses.
“The minimal meaning-bearing elements of human languages…are radically different from anything known in animal communication systems. Their origin is entirely obscure, posing a serious problem for the evolution of human cognitive capacities, particularly language.”22
(Berwick, R. C., & Chomsky, N)
Chomsky’s core concept of universal grammar is however powerful, simple and useful. It echoes the idea of pre-existing shared landscapes for concepts and meaning (not unlike the Jungian concept of the collective unconscious – the notion that humans are born with certain instincts, inherited fears such as of snakes, symbolic identities and drives) and it helps to focus our attention on inmate inherited shared abilities for abstract thought, complex conceptualisation and use of symbols, signs and sounds (this ultimately leads to the ability to perform in any specific language).
It also seems to be the case that other animals have not been able to incorporate the generative recursive power of language within their expressed communications, even if they can make use of internal symbolic representations.
However, we need to remain open to the claim by Daniel Everett that little more than general intelligence is innate.
“Language does not seem to be innate. There seems to be no narrow faculty of language nor any universal grammar. Language is ancient and emerges from general human intelligence, the need to build communities and cultures.”
Chomsky has also made the somewhat radical and very interesting suggestion that language may not even have evolved primarily for communication purposes. Key behavioural drivers of language can also be aesthetic and emotional and not purely functional. Language also evolved due to the benefit of communicating feelings and compound ideas, and not just to ask things or tell someone where or what something is or how much of it there is.
Whatever the complex drivers of our ability with language, it must have conferred a great benefit to modern humans in the battle to survive and thrive. Our intense universally widespread love of songs, stories and legends suggests that, from an early stage, we made use of the creative freedom of language for mirth, myths, mischief and well-meaning lies (or fiction if you prefer the polite term!).
“What I think is a primary ‘fact’ about my work, that it is all of a piece, and fundamentally linguistic in inspiration. […] The invention of languages is the foundation. The ‘stories’ were made rather to provide a world for the languages than the reverse. To me a name comes first and the story follows”23
Compossible - that which can live together 
Leave a comment