Istvan Stephen Norman Berkeley

Connectionism is a style of modeling based upon networks of interconnected simple processing devices. This style of modeling goes by a number of other names too. Connectionist models are also sometimes referred to as 'Parallel Distributed Processing' (or PDP for short) models or networks.1 Connectionist systems are also sometimes referred to as 'neural networks' (abbreviated to NNs) or 'artificial neural networks' (abbreviated to ANNs). Although there may be some rhetorical appeal to this neural nomenclature, it is in fact misleading as connectionist networks are commonly significantly dissimilar to neurological systems. For this reason, I will avoid using this terminology, other than in direct quotations. Instead, I will follow the practice I have adopted above and use 'connectionist' as my primary term for systems of this kind.

In this paper the issue of drawing inferences about biological cognitive systems on the basis of connectionist simulations is addressed. In particular, the justification of inferences based on connectionist models trained using the backpropagation learning algorithm is examined. First it is noted that a justification commonly found in the philosophical literature is inapplicable. Then some general issues are raised about the relationships between models and biological systems. A way of conceiving the role of hidden units in connectionist networks is then introduced. This, in combination with an assumption about the way evolution goes about solving problems, is then used to suggest a means of justifying inferences about biological systems based on connectionist research.

This dissertation opens with a discussion and clarification of the Classical Computational Theory of Mind. An alleged challenge to this theory, which derives from research into connectionist systems, is then briefly outlined and connectionist systems are introduced in some detail. Several claims which have been made on behalf of connectionist systems are then examined, these purport to support the conclusion that connectionist systems provide better models of the mind and cognitive functioning than the CCTM. It is argued that most claims made on behalf of connectionism often amount to little more than myths. A significant difficulty with standard connectionist research methodology is then described. This difficulty derives from the fact that connectionist systems are seldom subject to detailed analysis and interpretation after training and has serious consequences for the plausibility of connectionist research in cognitive science and as the basis for a challenge to the CCTM. A technique for network analysis is described in detail and the results from the analysis of a particular network are then introduced, in order to show how these difficulties can be overcome. The analyzed network exhibited a number of surprising and intriguing properties. These properties provide the basis for a detailed assessment of the network with respect to the CCTM. There then follows a discussion of the results of the network analysis, with respect to two features which are commonly associated with the CCTM, namely systematicity and compositionality. It is argued that the network has some properties similar to those associated with the CCTM and exhibits, in a weak sense, compositionality and systematicity. It is aiso argued that the network amounts to a genuinely cognitive theory. This suggests that there is insufficient evidence at the current time to determine whether or not connectionism presents a genuine challenge to the CCTM. The plausibility of the claim that networks provide the basis of an alternative to the CCTM is then traced, in part, to a revisionist tendency in some contemporary histories of connectionist research. Finally, future research directions in philosophy and cognitive science are suggested.

Since the emergence of what Fodor and Pylyshyn (1988) call 'new connectionism', there can be little doubt that connectionist research has become a significant topic for discussion in the Philosophy of Cognitive Science and the Philosophy of Mind. In addition to the numerous papers on the topic in philosophical journals, almost every recent book in these areas contain at least a brief reference to, or discussion of, the issues raised by connectionist research (see Sterelny 1990, Searle, 1992, and O Nualláin, 1995, for example). Other texts have focused almost exclusively upon connectionist issues (see Clark, 1993, Bechtel and Abrahamsen, 1991 and Lloyd, 1989, for example). Regrettably the discussions of connectionism found in the philosophical literature suffer from a number of deficiencies. My purpose in this paper is to highlight one particular problem and attempt to take a few steps to remedy the situation.

According to the standard (recent) history of connectionism (see for example the accounts offered by Hecht-Nielsen (1990: pp. 14-19) and Dreyfus and Dreyfus (1988), or Papert's (1988: pp. 3-4) somewhat whimsical description), in the early days of Classical Computational Theory of Mind (CCTM) based AI research, there was also another allegedly distinct approach, one based upon network models. The work on network models seems to fall broadly within the scope of the term 'connectionist' (see Aizawa 1992), although the term had yet to be coined at the time. These two approaches were "two daughter sciences" according to Papert (1988: p. 3). The fundamental difference between these two 'daughters', lay (according to Dreyfus and Dreyfus (1988: p. 16)) in what they took to be the paradigm of intelligence. Whereas the early connectionists took learning to be fundamental, the traditional school concentrated upon problem solving.

Berkeley [Minds Machines 10 (2000) 1] described a methodology that showed the subsymbolic nature of an artificial neural network system that had been trained on a logic problem, originally described by Bechtel and Abrahamsen [Connectionism and the mind. Blackwells, Cambridge, MA, 1991]. It was also claimed in the conclusion of this paper that the evidence was suggestive that the network might, in fact, count as a symbolic system. Dawson and Piercey [Minds Machines 11 (2001) 197] took issue with this latter claim. They described some lesioning studies that they argued showed that Berkeley’s (2000) conclusions were premature. In this paper, these lesioning studies are replicated and it is shown that the effects that Dawson and Piercey rely upon for their argument are merely an artifact of a threshold function they chose to employ. When a threshold function much closer to that deployed in the original studies is used, the significant effects disappear.

There is widespread belief that connectionist networks are dramatically different from classical or symbolic models. However, connectionists rarely test this belief by interpreting the internal structure of their nets. A new approach to interpreting networks was recently introduced by Berkeley et al. (1995). The current paper examines two implications of applying this method: (1) that the internal structure of a connectionist network can have a very classical appearance, and (2) that this interpretation can provide a cognitive theory that cannot be dismissed as a mere implementation.

Connectionist research first emerged in the 1940s. The first phase of connectionism attracted a certain amount of media attention, but scant philosophical interest. The phase came to an abrupt halt, due to the efforts of Minsky and Papert (1969), when they argued for the intrinsic limitations of the approach. In the mid-1980s connectionism saw a resurgence. This marked the beginning of the second phase of connectionist research. This phase did attract considerable philosophical attention. It was of philosophical interest, as it offered a way of counteracting the conceptual ties to the philosophical traditions of atomism, rationalism, logic, nativism, rule realism and a concern with the role symbols play in human cognitive functioning, which was prevalent as a consequence of artificial intelligence research. The surge in philosophical interest waned, possibly in part due to the efforts of some traditionalists and the so-called black box problem. Most recently, what may be thought of as a third phase of connectionist research, based on so-called deep learning methods, is beginning to show some signs of again exciting philosophical interest.

In 1988, Smolensky proposed that connectionist processing systems should be understood as operating at what he termed the `subsymbolic' level. Subsymbolic systems should be understood by comparing them to symbolic systems, in Smolensky's view. Up until recently, there have been real problems with analyzing and interpreting the operation of connectionist systems which have undergone training. However, recently published work on a network trained on a set of logic problems originally studied by Bechtel and Abrahamsen (1991) seems to offer the potential to provide a detailed, empirically based answer to questions about the nature of subsymbols. In this paper, a network analysis procedure and the results obtained using it are discussed. This provides the basis for an insight into the nature of subsymbols, which is surprising.

The notion of a ‘symbol’ plays an important role in the disciplines of Philosophy, Psychology, Computer Science, and Cognitive Science. However, there is comparatively little agreement on how this notion is to be understood, either between disciplines, or even within particular disciplines. This paper does not attempt to defend some putatively ‘correct’ version of the concept of a ‘symbol.’ Rather, some terminological conventions are suggested, some constraints are proposed and a taxonomy of the kinds of issue that give rise to disagreement is articulated. The goal here is to provide something like a ‘geography’ of the various notions of ‘symbol’ that have appeared in the various literatures, so as to highlight the key issues and to permit the focusing of attention upon the important dimensions. In particular, the relationship between ‘tokens’ and ‘symbols’ is addressed. The issue of designation is discussed in some detail. The distinction between simple and complex symbols is clarified and an apparently necessary condition for a system to be potentially symbol, or token bearing, is introduced.