Stevan Harnad

Almost all words are the names of categories. We can learn most of our words (and hence our categories) from dictionary definitions, but not all of them. Some have to be learned from direct experience. To understand a word from its definition we need to already understand the words used in the definition. This is the “Symbol Grounding Problem” [1]. How many words (and which ones) do we need to ground directly in sensorimotor experience in order to be able to learn all other words via definition alone? The answer may shed some light both on the developmental origin of word meanings and on the evolutionary origin and adaptive value of language. We used an algorithm to reduce each of our dictionaries (Longmans LDOCE, Cambridge CIDE and WordNet) to its “grounding kernel” (“Kernel”) (which turned out to be about 10% of the dictionary) by systematically eliminating..

This is a paperback reissue of a 1988 special issue of Cognition - dated but still of interest. The book consists of three chapters, each making one major negative point about connectionism. Fodor & Pylyshyn (F&P) argue that connectionist networks (henceforth 'nets') are not good models for cognition because they lack 'systematicity', Pinker & Price (P&P) argue that nets are not good substitutes for rule-based models of linguistic ability, and Lachter & Bever (L&B) argue that nets can only model the associative relations between cognitive structures, not the structures themselves.

Behavioral scientists studied behavior; cognitive scientists study what generates behavior. Cognitive science is hence theoretical behaviorism (or behaviorism is experimental cognitivism). Behavior is data for a cognitive theorist. What counts as a theory of behavior? In this paper, a methodological constraint on theory construction -- "neoconstructivism" -- will be proposed (by analogy with constructivism in mathematics): Cognitive theory must be computable; given an encoding of the input to a behaving system, a theory must be able to compute (an encoding of) its outputs. It is a mistake to conclude, however, that this constraint requires cognitive theory to be computational, or that it follows from this that cognition is computation

I am going to attempt to argue, given certain premises, there are reasons, not only empirical, but also logical, for expecting a certain division of labor in the processing of information by the human brain. This division of labor consists specifically of a functional bifurcation into what may be called, to a first approximation, "verbal" and "nonverbal" modes of information- processing. That this dichotomy is not quite satisfactory, however, will be one of the principal conclusions of this chapter, for I shall attempt to show that metaphor, which in its most common guise is a literary, and hence a fortiori a "verbal" phenomenon, may in fact be more a function of the "nonverbal" than the "verbal" mode. (For alternative attempts to account for cognitive lateralization, see e.g. Bever, 1975; Wickelgren, 1975; Pendse, 1978.).

"Symbol Grounding" is beginning to mean too many things to too many people. My own construal has always been simple: Cognition cannot be just computation, because computation is just the systematically interpretable manipulation of meaningless symbols, whereas the meanings of my thoughts don't depend on their interpretability or interpretation by someone else. On pain of infinite regress, then, symbol meanings must be grounded in something other than just their interpretability if they are to be candidates for what is going on in our heads. Neural nets may be one way to ground the names of concrete objects and events in the capacity to categorize them (by learning the invariants in their sensorimotor projections). These grounded elementary symbols could then be combined into symbol strings expressing propositions about more abstract categories. Grounding does not equal meaning, however, and does not solve any philosophical problems

Explaining the mind by building machines with minds runs into the other-minds problem: How can we tell whether any body other than our own has a mind when the only way to know is by being the other body? In practice we all use some form of Turing Test: If it can do everything a body with a mind can do such that we can't tell them apart, we have no basis for doubting it has a mind. But what is "everything" a body with a mind can do? Turing's original "pen-pal" version (the TT) only tested linguistic capacity, but Searle has shown that a mindless symbol-manipulator could pass the TT undetected. The Total Turing Test (TTT) calls for all of our linguistic and robotic capacities; immune to Searle's argument, it suggests how to ground a symbol manipulating system in the capacity to pick out the objects its symbols refer to. No Turing Test, however, can guarantee that a body has a mind. Worse, nothing in the explanation of its successful performance requires a model to have a mind at all. Minds are hence very different from the unobservables of physics (e.g., superstrings); and Turing Testing, though essential for machine-modeling the mind, can really only yield an explanation of the body.

Turing's celebrated 1950 paper proposes a very general methodological criterion for modelling mental function: total functional equivalence and indistinguishability. His criterion gives rise to a hierarchy of Turing Tests, from subtotal ("toy") fragments of our functions (t1), to total symbolic (pen-pal) function (T2 -- the standard Turing Test), to total external sensorimotor (robotic) function (T3), to total internal microfunction (T4), to total indistinguishability in every empirically discernible respect (T5). This is a "reverse-engineering" hierarchy of (decreasing) empirical underdetermination of the theory by the data. Level t1 is clearly too underdetermined, T2 is vulnerable to a counterexample (Searle's Chinese Room Argument), and T4 and T5 are arbitrarily overdetermined. Hence T3 is the appropriate target level for cognitive science. When it is reached, however, there will still remain more unanswerable questions than when Physics reaches its Grand Unified Theory of Everything (GUTE), because of the mind/body problem and the other-minds problem, both of which are inherent in this empirical domain, even though Turing hardly mentions them.

Turing set the agenda for (what would eventually be called) the cognitive sciences. He said, essentially, that cognition is as cognition does (or, more accurately, as cognition is capable of doing): Explain the causal basis of cognitive capacity and you’ve explained cognition. Test your explanation by designing a machine that can do everything a normal human cognizer can do – and do it so veridically that human cognizers cannot tell its performance apart from a real human cognizer’s – and you really cannot ask for anything more. Or can you? Neither Turing modelling nor any other kind of computational r dynamical modelling will explain how or why cognizers feel

The ethical case for Open Access (OA) (free online access) to research findings is especially salient when it is public health that is being compromised by needless access restrictions. But the ethical imperative for OA is far more general: It applies to all scientific and scholarly research findings published in peer-reviewed journals. And peer-to-peer access is far more important than direct public access. Most research is funded so as to be conducted and published, by researchers, in order to be taken up, used, and built upon in further research and applications, again by researchers (pure and applied, including practitioners), for the benefit of the public that funded it – not in order to generate revenue for the peer-reviewed journal publishing industry (nor even because there is a burning public desire to read much of it). Hence OA needs to be mandated, by researchers' institutions and funders, for all research

Jerry Fodor argues that Darwin was wrong about "natural selection" because (1) it is only a tautology rather than a scientific law that can support counterfactuals ("If X had happened, Y would have happened") and because (2) only minds can select. Hence Darwin's analogy with "artificial selection" by animal breeders was misleading and evolutionary explanation is nothing but post-hoc historical narrative. I argue that Darwin was right on all counts. Until Darwin's "tautology," it had been believed that either (a) God had created all organisms as they are, or (b) organisms had always been as they are. Darwin revealed instead that (c) organisms have heritable traits that evolved across time through random variation, with survival and reproduction in (changing) environments determining (mindlessly) which variants were successfully transmitted to the next generation. This not only provided the (true) alternative (c), but also the methodology for investigating which traits had been adaptive, how and why; it also led to the discovery of the genetic mechanism of the encoding, variation and evolution of heritable traits. Fodor also draws erroneous conclusions from the analogy between Darwinian evolution and Skinnerian reinforcement learning. Fodor’s skepticism about both evolution and learning may be motivated by an overgeneralization of Chomsky’s “poverty of the stimulus argument” -- from the origin of Universal Grammar (UG) to the origin of the “concepts” underlying word meaning, which, Fodor thinks, must be “endogenous,” rather than evolved or learned.

Turing's celebrated 1950 paper proposes a very general methodological criterion for modelling mental function: total functional equivalence and indistinguishability. His criterion gives rise to a hierarchy of Turing Tests, from subtotal ("toy") fragments of our functions (t1), to total symbolic (pen-pal) function (T2 -- the standard Turing Test), to total external sensorimotor (robotic) function (T3), to total internal microfunction (T4), to total indistinguishability in every empirically discernible respect (T5). This is a "reverse-engineering" hierarchy of (decreasing) empirical underdetermination of the theory by the data. Level t1 is clearly too underdetermined, T2 is vulnerable to a counterexample (Searle's Chinese Room Argument), and T4 and T5 are arbitrarily overdetermined. Hence T3 is the appropriate target level for cognitive science. When it is reached, however, there will still remain more unanswerable questions than when Physics reaches its Grand Unified Theory of Everything (GUTE), because of the mind/body problem and the other-minds problem, both of which are inherent in this empirical domain, even though Turing hardly mentions them.

What language allows us to do is to "steal" categories quickly and effortlessly through hearsay instead of having to earn them the hard way, through risky and time-consuming sensorimotor "toil" (trial-and-error learning, guided by corrective feedback from the consequences of miscategorisation). To make such linguistic "theft" possible, however, some, at least, of the denoting symbols of language must first be grounded in categories that have been earned through sensorimotor toil (or else in categories that have already been "prepared" for us through Darwinian theft by the genes of our ancestors); it cannot be linguistic theft all the way down. The symbols that denote categories must be grounded in the capacity to sort, label and interact with the proximal sensorimotor projections of their distal category-members in a way that coheres systematically with their semantic interpretations, both for individual symbols, and for symbols strung together to express truth-value-bearing propositions

Suppose Boeing 747s grew on trees. They would first sprout as embryonic planes, the size of an acorn. Then they would grow until they reached full size, when they would plop off the trees, ready to fly. Suppose also that we knew how to feed and care for them, how to make minor repairs, and of course how to fly them. But let us suppose that all of this transpired at a very early stage in our scientific history, when we did not yet understand the physics or the engineering of flight: Hence the phenomenon was a complete mystery to us. (To keep things simple, let us suppose that no other entity on earth could fly, only 747s.) And for the last ingredient in this fantasy world, suppose that computers likewise grew on trees, and we knew how to use and fix them too.

1.1 The predominant approach to cognitive modeling is still what has come to be called "computationalism" (Dietrich 1990, Harnad 1990b), the hypothesis that cognition is computation. The more recent rival approach is "connectionism" (Hanson & Burr 1990, McClelland & Rumelhart 1986), the hypothesis that cognition is a dynamic pattern of connections and activations in a "neural net." Are computationalism and connectionism really deeply different from one another, and if so, should they compete for cognitive hegemony, or should they collaborate? These questions will be addressed here, in the context of an obstacle that is faced by computationalism (as well as by connectionism if it is either computational or seeks cognitive hegemony on its own): The symbol grounding problem (Harnad 1990).

The problem seems apparent even in Glasgow's term ``depict'', which is used by way of contrast with ``describe''. Now ``describe'' refers relatively unproblematically to strings of symbols, such as those in this written sentence, that are systematically interpretable as propositions describing objects, events, or states of affairs. But what does ``depict'' mean? In the case of a picture -- whether a photo or a diagram -- it is clear what depict means. A picture is an object (I will argue below that it is an analog object, relative to what it is a picture of) and it DEPICTS yet another object: the object it is a picture OF. But in the case of an array, whether described formally, with numerical coordinates, or stored in a machine, or ``depicted'' diagrammatically by way of a secondary illustration, it is not at all clear whether the entity in question is indeed a picture, or merely yet another set of symbols that is INTERPRETABLE as referring to a picture, which picture in turn depicts an object! It is clear that we are dealing with many layers of interpretation here already, and so far we are still talking only about external objects (such as pictures, symbols and objects simpliciter). We still have not gotten to MENTAL objects, such as mental ``images''

Le modele d'ancrage propose ici est simple a recapituler. Les projections sensorielles analogiques sont les intrants des reseaux neuronaux qui doivent apprendre a connecter certaines des projections avec certains symboles (le nom de leur categorie) et certaines autres projections avec d'autres symboles (les noms d'autres categories pouvant se confondre les unes aux autres), en trouvant et en utilisant les invariants qui les representent de facon a favoriser l'accomplissement d'une categorisation juste. Les symboles ancres sont alors enfiles dans des combinaisons d'ordre superieur (descriptions symboliques ancrees) par un deuxieme processus combinatoire qui presente une difference critique a l'egard de la manipulation symbolique classique. Dans la manipulation symbolique standard (non ancree), la syntaxe est la seule contrainte a laquelle les combinaisons de symboles sont soumises et elle s'applique a la configuration (arbitraire) des symboles. Dans un systeme symbolique ancre, on doit tenir compte d'une deuxieme contrainte, celle de la forme non arbitraire des invariants sensoriels qui connectent le symbole a la projection sensorielle analogique de l'objet auquel il se rapporte. Je ne peux m'etendre sur la nature de ces systemes symboliques ancres a double contrainte , si ce n'est que pour indiquer que la perception categorielle humaine peut apporter quelques indices quant a la nature de cette interaction entre les contraintes analogiques et syntaxiques.

According to "computationalism" (Newell, 1980; Pylyshyn 1984; Dietrich 1990), mental states are computational states, so if one wishes to build a mind, one is actually looking for the right program to run on a digital computer. A computer program is a semantically interpretable formal symbol system consisting of rules for manipulating symbols on the basis of their shapes, which are arbitrary in relation to what they can be systematically interpreted as meaning. According to computationalism, every physical implementation of the right symbol system will have mental states.

Brian Rotman argues that (one) “mind” and (one) “god” are only conceivable, literally, because of (alphabetic) literacy, which allowed us to designate each of these ghosts as an incorporeal, speaker-independent “I” (or, in the case of infinity, a notional agent that goes on counting forever). I argue that to have a mind is to have the capacity to feel. No one can be sure which organisms feel, hence have minds, but it seems likely that one-celled organisms and plants do not, whereas animals do. So minds originated before humans and before language --hence, a fortiori, before writing, whether alphabetic or ideographic.

Harnad's main argument can be roughly summarised as follows: due to Searle's Chinese Room argument, symbol systems by themselves are insufficient to exhibit cognition, because the symbols are not grounded in the real world, hence without meaning. However, a symbol system that is connected to the real world through transducers receiving sensory data, with neural nets translating these data into sensory categories, would not be subject to the Chinese Room argument. Harnad's article is not only the starting point for the present debate, but is also a contribution to a longlasting discussion about such questions as: Can a computer think? If yes, would this be solely by virtue of its program? Is the Turing Test appropriate for deciding whether a computer thinks?

What lies on the two sides of the linguistic divide is fairly clear: On one side, you have organisms buffeted about to varying degrees, depending on their degree of autonomy and plasticity, by the states of affairs in the world they live in. On the other side, you have organisms capable of describing and explaining the states of affairs in the world they live in. Language is what distinguishes one side from the other. How did we get here from there? In principle, one can tell a seamless story about how inborn, involuntary communicative signals and voluntary instrumental praxis could have been shaped gradually, through feedback from their consequences, first into analog pantomime with communicative intent, and then into arbitrary category names combined into all powerful, truth value bearing propositions, freed from the iconic "shape" of their referents and able to tell all.

Why, oh why do we keep conflating this question, which is about the uncertainty of sensory information, with the much more profound and pertinent one, which is about the functional explicability and causal role of feeling? " _Kant: How is it possible for something even to be a thought? What are the conditions for the_ _possibility of experience at all?_ " That's not the right question either. The right question is not even an epistemic one, about "thought" or "knowledge" but an "aesthesiogenic" one: How and why are there any feelings at all?

Steels & Belpaeme's (S&B's) simulations contain all the right components, but they are put together wrongly. Color categories are unrepresentative of categories in general and language is not merely naming. Language evolved because it provided a powerful new way to acquire categories (through instruction, rather than just the old way of other species, through trial-and-error experience). It did not evolve so that multiple agents looking at the same objects could let one another know which of the objects they had in mind, co-coining names for them on the fly.