Stevan Harnad

There has been much discussion recently about the scope and limits of purely symbolic models of the mind and about the proper role of connectionism in cognitive modeling. This paper describes the symbol grounding problem : How can the semantic interpretation of a formal symbol system be made intrinsic to the system, rather than just parasitic on the meanings in our heads? How can the meanings of the meaningless symbol tokens, manipulated solely on the basis of their shapes, be grounded in anything but other meaningless symbols? The problem is analogous to trying to learn Chinese from a Chinese/Chinese dictionary alone. A candidate solution is sketched: Symbolic representations must be grounded bottom-up in nonsymbolic representations of two kinds: iconic representations, which are analogs of the proximal sensory projections of distal objects and events, and categorical representations, which are learned and innate feature-detectors that pick out the invariant features of object and event categories from their sensory projections. Elementary symbols are the names of these object and event categories, assigned on the basis of their categorical representations. Higher-order symbolic representations, grounded in these elementary symbols, consist of symbol strings describing category membership relations. Connectionism is one natural candidate for the mechanism that learns the invariant features underlying categorical representations, thereby connecting names to the proximal projections of the distal objects they stand for. In this way connectionism can be seen as a complementary component in a hybrid nonsymbolic/symbolic model of the mind, rather than a rival to purely symbolic modeling. Such a hybrid model would not have an autonomous symbolic module, however; the symbolic functions would emerge as an intrinsically dedicated symbol system as a consequence of the bottom-up grounding of categories ' names in their sensory representations. Symbol manipulation would be governed not just by the arbitrary shapes of the symbol tokens, but by the nonarbitrary shapes of the icons and category invariants in which they are grounded

Libet, Gleason, Wright, & Pearl (1983) asked participants to report the moment at which they freely decided to initiate a pre-specified movement, based on the position of a red marker on a clock. Using event-related potentials (ERPs), Libet found that the subjective feeling of deciding to perform a voluntary action came after the onset of the motor “readiness potential,” RP). This counterintuitive conclusion poses a challenge for the philosophical notion of free will. Faced with these findings, Libet (1985) proposed that conscious volitional control might operate as a selector and a controller of volitional processes rather than as an initiator of them.

Many special problems crop up when evolutionary theory turns, quite naturally, to the question of the adaptive value and causal role of consciousness in human and nonhuman organisms. One problem is that -- unless we are to be dualists, treating it as an independent nonphysical force -- consciousness could not have had an independent adaptive function of its own, over and above whatever behavioral and physiological functions it "supervenes" on, because evolution is completely blind to the difference between a conscious organism and a functionally equivalent (Turing Indistinguishable) nonconscious "Zombie" organism: In other words, the Blind Watchmaker, a functionalist if ever there was one, is no more a mind reader than we are. Hence Turing-Indistinguishability = Darwin-Indistinguishability. It by no means follows from this, however, that human behavior is therefore to be explained only by the push-pull dynamics of Zombie determinism, as dictated by calculations of "inclusive fitness" and "evolutionarily stable strategies." We are conscious, and, more important, that consciousness is piggy-backing somehow on the vast complex of unobservable internal activity -- call it "cognition" -- that is really responsible for generating all of our behavioral capacities. Hence, except in the palpable presence of the irrational (e.g., our sexual urges) where distal Darwinian factors still have some proximal sway, it is as sensible to seek a Darwinian rather than a cognitive explanation for most of our current behavior as it is to seek a cosmological rather than an engineering explanation of an automobile's behavior. Let evolutionary theory explain what shaped our cognitive capacity (Steklis & Harnad 1976; Harnad 1996, but let cognitive theory explain our resulting behavior

Churchland underestimates the power and purpose of the Turing Test, dismissing it as the trivial game to which the Loebner Prize (offered for the computer program that can fool judges into thinking it's human) has reduced it, whereas it is really an exacting empirical criterion: It requires that the candidate model for the mind have our full behavioral capacities -- so fully that it is indistinguishable from any of us, to any of us (not just for one Contest night, but for a lifetime). Scaling up to such a model is (or ought to be) the programme of that branch of reverse bioengineering called cognitive science. It's harmless enough to do the hermeneutics after the research has been successfully completed, but self-deluding and question-begging to do it before.

It is hypothesized that words originated as the names of perceptual categories and that two forms of representation underlying perceptual categorization -- iconic and categorical representations -- served to ground a third, symbolic, form of representation. The third form of representation made it possible to name and describe our environment, chiefly in terms of categories, their memberships, and their invariant features. Symbolic representations can be shared because they are intertranslatable. Both categorization and translation are approximate rather than exact, but the approximation can be made as close as we wish. This is the central property of that universal mechanism for sharing descriptions that we call natural language.

My purpose is to explain, first, that there is an alternative to Harnad's version of the symbol grounding problem, which is known as the problem of primitives; second, that there is an alternative to his solution (which is externalist) in the form of a dispositional conception (which is internalist); and, third, that, while the TTT, properly understood, may provide partial and fallible evidence for the presence of similar mental powers, it cannot supply conclusive proof, because more than observable symbolic manipuation and robotic behavior is involved here, as he admits (Harnad 1991). Carrying the problem further appears to require inference to the best explanation.

2. Invariant Sensorimotor Features ("Affordances"). To say this is not to declare oneself a Gibsonian, whatever that means. It is merely to point out that what a sensorimotor system can do is determined by what can be extracted from its motor interactions with its sensory input. If you lack sonar sensors, then your sensorimotor system cannot do what a bat's can do, at least not without the help of instruments. Light stimulation affords color vision for those of us with the right sensory apparatus, but not for those of us who are color-blind. The geometric fact that, when we move, the "shadows" cast on our retina by nearby objects move faster than the shadows of further objects means that, for those of us with normal vision, our visual input affords depth perception. From more complicated facts of projective and solid geometry it follows that a 3-dimensional shape, such as, say, a boomerang, can be recognized as being the same shape Ð and the same size Ð even though the size and shape of its shadow on our retinas changes as we move in relation to it or it moves in relation to us. Its shape is said to be invariant under these sensorimotor transformations, and our visual systems can detect and extract that invariance, and translate it into a visual constancy. So we keep seeing a boomerang of the same shape and size even though the shape and size of its retinal shadows keep changing

Turing set the agenda for (what would eventually be called) the cognitive sciences. He said, essentially, that cognition is as cognition does (or, more accurately, as cognition is capable of doing): Explain the causal basis of cognitive capacity and you’ve explained cognition. Test your explanation by designing a machine that can do everything a normal human cognizer can do – and do it so veridically that human cognizers cannot tell its performance apart from a real human cognizer’s – and you really cannot ask for anything more. Or can you? Neither Turing modelling nor any other kind of computational r dynamical modelling will explain how or why cognizers feel

The ethical case for Open Access (OA) (free online access) to research findings is especially salient when it is public health that is being compromised by needless access restrictions. But the ethical imperative for OA is far more general: It applies to all scientific and scholarly research findings published in peer-reviewed journals. And peer-to-peer access is far more important than direct public access. Most research is funded so as to be conducted and published, by researchers, in order to be taken up, used, and built upon in further research and applications, again by researchers (pure and applied, including practitioners), for the benefit of the public that funded it – not in order to generate revenue for the peer-reviewed journal publishing industry (nor even because there is a burning public desire to read much of it). Hence OA needs to be mandated, by researchers' institutions and funders, for all research

Jerry Fodor argues that Darwin was wrong about "natural selection" because (1) it is only a tautology rather than a scientific law that can support counterfactuals ("If X had happened, Y would have happened") and because (2) only minds can select. Hence Darwin's analogy with "artificial selection" by animal breeders was misleading and evolutionary explanation is nothing but post-hoc historical narrative. I argue that Darwin was right on all counts. Until Darwin's "tautology," it had been believed that either (a) God had created all organisms as they are, or (b) organisms had always been as they are. Darwin revealed instead that (c) organisms have heritable traits that evolved across time through random variation, with survival and reproduction in (changing) environments determining (mindlessly) which variants were successfully transmitted to the next generation. This not only provided the (true) alternative (c), but also the methodology for investigating which traits had been adaptive, how and why; it also led to the discovery of the genetic mechanism of the encoding, variation and evolution of heritable traits. Fodor also draws erroneous conclusions from the analogy between Darwinian evolution and Skinnerian reinforcement learning. Fodor’s skepticism about both evolution and learning may be motivated by an overgeneralization of Chomsky’s “poverty of the stimulus argument” -- from the origin of Universal Grammar (UG) to the origin of the “concepts” underlying word meaning, which, Fodor thinks, must be “endogenous,” rather than evolved or learned.

Turing's celebrated 1950 paper proposes a very general methodological criterion for modelling mental function: total functional equivalence and indistinguishability. His criterion gives rise to a hierarchy of Turing Tests, from subtotal ("toy") fragments of our functions (t1), to total symbolic (pen-pal) function (T2 -- the standard Turing Test), to total external sensorimotor (robotic) function (T3), to total internal microfunction (T4), to total indistinguishability in every empirically discernible respect (T5). This is a "reverse-engineering" hierarchy of (decreasing) empirical underdetermination of the theory by the data. Level t1 is clearly too underdetermined, T2 is vulnerable to a counterexample (Searle's Chinese Room Argument), and T4 and T5 are arbitrarily overdetermined. Hence T3 is the appropriate target level for cognitive science. When it is reached, however, there will still remain more unanswerable questions than when Physics reaches its Grand Unified Theory of Everything (GUTE), because of the mind/body problem and the other-minds problem, both of which are inherent in this empirical domain, even though Turing hardly mentions them.

What language allows us to do is to "steal" categories quickly and effortlessly through hearsay instead of having to earn them the hard way, through risky and time-consuming sensorimotor "toil" (trial-and-error learning, guided by corrective feedback from the consequences of miscategorisation). To make such linguistic "theft" possible, however, some, at least, of the denoting symbols of language must first be grounded in categories that have been earned through sensorimotor toil (or else in categories that have already been "prepared" for us through Darwinian theft by the genes of our ancestors); it cannot be linguistic theft all the way down. The symbols that denote categories must be grounded in the capacity to sort, label and interact with the proximal sensorimotor projections of their distal category-members in a way that coheres systematically with their semantic interpretations, both for individual symbols, and for symbols strung together to express truth-value-bearing propositions

Searle's celebrated Chinese Room Argument has shaken the foundations of Artificial Intelligence. Many refutations have been attempted, but none seem convincing. This paper is an attempt to sort out explicitly the assumptions and the logical, methodological and empirical points of disagreement. Searle is shown to have underestimated some features of computer modeling, but the heart of the issue turns out to be an empirical question about the scope and limits of the purely symbolic model of the mind. Nonsymbolic modeling turns out to be immune to the Chinese Room Argument. The issues discussed include the Total Turing Test, modularity, neural modeling, robotics, causality and the symbol-grounding problem

In his chapter titled "Consciousness, Charles Taylor suggests that the traditional mind/body, mental/physical dichotomy is an undesirable legacy of the seventeenth century. Its faults are that it gives rise to a dualism that must then be resolved in various unsatisfactory ways. The most prevalent of these ways is currently "functionalism," which explains cognition in terms of functional states and processes like those of a computer and "marginalizes" (i.e., minimizes or denies completely the causal role of) consciousness. The alternative, "interactionism," gives due weight to consciousness but at the cost of adding an independent domain to the physical one, namely, the mental, and possibly tampering indeterminately with physics thereby.

Computationalism. According to computationalism, to explain how the mind works, cognitive science needs to find out what the right computations are -- the same ones that the brain performs in order to generate the mind and its capacities. Once we know that, then every system that performs those computations will have those mental states: Every computer that runs the mind's program will have a mind, because computation is hardware independent : Any hardware that is running the right program has the right computational states.

After people learn to sort objects into categories they see them differently. Members of the same category look more alike and members of different categories look more different. This phenomenon of within-category compression and between-category separation in similarity space is called categorical perception (CP). It is exhibited by human subjects, animals and neural net models. In backpropagation nets trained first to auto-associate 12 stimuli varying along a onedimensional continuum and then to sort them into 3 categories, CP arises as a natural side-effect because of four factors: (1) Maximal interstimulus separation in hidden-unit space during autoassociation learning, (2) movement toward linear separability during categorization learning, (3) inverse-distance repulsive force exerted by the between-category boundary, and (4) the modulating effects of input iconicity, especially in interpolating CP to untrained regions of the continuum. Once similarity space has been "warped" in this way, the compressed and separated "chunks" have symbolic labels which could then be combined into symbol strings that constitute propositions about objects. The meanings of such symbolic representations would be "grounded" in the system's capacity to pick out from their sensory projections the object categories that the propositions were about.

Research is done (mostly at universities) and funded (publicly and privately) in order to advance scientific and scholarly knowledge as well as to produce public benefits (technological and biomedical applications as well as educational and cultural ones). Research and researchers are accordingly funded not only to conduct their research, but to make their findings public, by publishing them. Their employment, salaries, careers and research funding depend on publishing their findings. This is what is often called "publish or perish.&quot

I want to report a thoroughly (perhaps surreally) modern experience I had recently. First a little context. I've always been a zealous scholarly letter writer (to the point of once being cited in print as "personal communication, pp. 14 - 20"). These days few share my epistolary penchant, which is dismissed as a doomed anachronism. Scholars don't have the time. Inquiry is racing forward much too rapidly for such genteel dawdling -- forward toward, among other things, due credit in print for one's every minute effort. So I too had resigned myself to the slower turnaround but surer rewards of conventional scholarly publication. Until I came upon electronic mail: almost as rapid and direct and spontaneous as a telephone call, but with the added discipline and permanence of the written medium. I quickly became addicted, "logging on" to check my e mail at all hours of the day and night and accumulating files of intellectual exchanges with similarly inclined e epistoleans, files that rapidly approached book length

In innate Categorical Perception (CP) (e.g., colour perception), similarity space is "warped," with regions of increased within-category similarity (compression) and regions of reduced between-category similarity (separation) enh ancing the category boundaries and making categorisation reliable and all-or-none rather than graded. We show that category learning can likewise warp similarity space, resolving uncertainty near category boundaries. Two Hard and two Easy texture learning tasks were compared: As predicted, there were fewer successful Learners with the Hard task, and only the successful Learners of the Hard task exhibited CP. In a second experiment, the Easy task was made Hard by making the corrective feedback during learn ing only 90% reliable; this too generated CP. The results are discussed in relation to supervised, unsupervised and dual-mode models of category learning and representation.The world is full of things that vary in their similarity and interconfusability.O rganisms must somehow resolve this confusion, sorting and acting upon things adaptively. It might be important, for example, to learn which kinds of mushrooms are poisonous and which are safe to eat, minimising the confusion between them (Greco, Cangelosi & Harnad 1997).