David Danks

In this essay, we examine the use of resting state fMRI data for psychological inferences. We argue that resting state studies hold the paired promises of discovering novel functional brain networks, and of avoiding some of the limitations of task-based fMRI. However, we argue that the very features of experimental design that enable resting state fMRI to support exploratory science also generate a novel confound. We argue that seemingly key features of resting state functional connectivity networks may be artefacts resulting from sampling a ‘mixture distribution’ of diverse brain networks active at different times during the scan. We explore the consequences of this ‘mixture view’ for attempts to theorize about the cognitive or psychological functions of resting state networks, as well as the value of exploratory experiments.

Many investigations into the world, including philosophical ones, aim to discover causal knowledge, and many experimental methods have been developed to assist in causal discovery. More recently, algorithms have emerged that can also learn causal structure from purely or mostly observational data, as well as experimental data. These methods have started to be applied in various philosophical contexts, such as debates about our concepts of free will and determinism. This paper provides a “user's guide” to these methods, though not in the sense of specifying exact button presses in a software package. Instead, we explain the larger “pipeline” within which these methods are used and discuss key steps in moving from initial research idea to validated causal structure.

Many approaches to evidence amalgamation focus on relatively static information or evidence: the data to be amalgamated involve different variables, contexts, or experiments, but not measurements over extended periods of time. However, much of scientific inquiry focuses on dynamical systems; the system’s behavior over time is critical. Moreover, novel problems of evidence amalgamation arise in these contexts. First, data can be collected at different measurement timescales, where potentially none of them correspond to the underlying system’s causal timescale. Second, missing variables have a significantly different impact on time series measurements than they do in the traditional static setting; in particular, they make causal and structural inference much more difficult. In this paper, we argue that amalgamation should proceed by integrating causal knowledge, rather than at the level of “raw” evidence. We defend this claim by first outlining both of these problems, and then showing that they can be solved only if we operate on causal structures. We therefore must use causal discovery methods that are reliable given these problems. Such methods do exist, but their successful application requires careful consideration of the problems that we highlight.

We make constant use of causal beliefs in our everyday lives without giving much thought to the source of those beliefs, even for situations about which we have no specific prior causal knowledge. We can ask two distinct types of questions about these causal judgments: descriptive questions and normative questions. The primary goal of this dissertation is to apply normative research on causal judgment to our descriptive theories. ;I begin this dissertation by describing the primary results of research on the normative question: the framework of Bayesian networks, and algorithms that provably extract maximal causal information from observations and interventions in the environment. I then show that we can use the framework of Bayesian networks to represent the three major descriptive theories of causal judgment. The three theories are not directly comparable, however, since one of them is algorithmic, while the other two are computational. Therefore, I provide a fully general characterization of the Rescorla-Wagner model. ;With this computational description of the Rescorla-Wagner model in hand, we can compare all three of the theories against the known experimental data. Unfortunately, the experimental data has all been obtained using one particular experimental methodology, which contains serious problems. In addition to critiquing the currently dominant methodology, I provide a "proof of concept" of a different methodology that does not face these problems. ;More importantly, the psychological theories all focus on parameter estimation in a fixed causal structure, rather than learning the actual structure. Therefore, I consider the experimental question: can people learn complex causal structures? The data from those experiments reveal that people appear to use a wide array of learning strategies. The epistemology of our causal beliefs is substantially more complex than has previously been thought. ;Finally, at the end of this dissertation, I step back and examine the implications of these experimental results for descriptive theories of causal judgment. I also point towards several open research questions, both descriptive and normative, raised by the work in this dissertation.

Learning by artificial intelligence systems-what I will typically call machine learning-has a distinguished history, and the field has experienced something of a renaissance in the past twenty years. Machine learning consists principally of a diverse set of algorithms and techniques that have been applied to problems in a wide range of domains. Any overview of the methods and applications will inevitably be incomplete, at least at the level of specific algorithms and techniques. There are many excellent introductions to the formal and statistical details of machine learning algorithms and techniques available elsewhere. The present chapter focuses on machine learning as a general way of “thinking about the world,” and provides a high-level characterization of the major goals of machine learning. There are a number of philosophical concerns that have been raised about machine learning, but upon closer examination, it is not always clear whether the objections really speak against machine learning specifically. Many seem rather to be directed towards machine learning as a particular instantiation of some more general phenomenon or process. One of the general morals of this chapter is that machine learning is, in many ways, less unusual or peculiar than is sometimes thought.

Most learning models assume, either implicitly or explicitly, that the goal of learning is to acquire a complete and veridical representation of the world, but this view assumes away the possibility that pragmatic goals can play a central role in learning. We propose instead that people are relatively frugal learners, acquiring goal-relevant information while ignoring goal-irrelevant features of the environment. Experiment 1 provides evidence that learning is goal-dependent, and that people are relatively frugal when given a specific, practical goal. Experiment 2 investigates possible mechanisms underlying this effect, and finds evidence that people exhibit goal-driven attention allocation, but not goaldriven reasoning. We conclude by examining how frugality can be integrated into Bayesian models of learning

In the latter half of the twentieth century, philosophers of science have argued (implicitly and explicitly) that epistemically rational individuals might compose epistemically irrational groups and that, conversely, epistemically rational groups might be composed of epistemically irrational individuals. We call the conjunction of these two claims the Independence Thesis, as they together imply that methodological prescriptions for scientific communities and those for individual scientists might be logically independent of one another. We develop a formal model of scientific inquiry, define four criteria for individual and group epistemic rationality, and then prove that the four definitions diverge, in the sense that individuals will be judged rational when groups are not and vice versa. We conclude by explaining implications of the inconsistency thesis for (i) descriptive history and sociology of science and (ii) normative prescriptions for scientific communities.

Bayesian models of human learning are becoming increasingly popular in cognitive science. We argue that their purported confirmation largely relies on a methodology that depends on premises that are inconsistent with the claim that people are Bayesian about learning and inference. Bayesian models in cognitive science derive their appeal from their normative claim that the modeled inference is in some sense rational. Standard accounts of the rationality of Bayesian inference imply predictions that an agent selects the option that maximizes the posterior expected utility. Experimental confirmation of the models, however, has been claimed because of groups of agents that probability match the posterior. Probability matching only constitutes support for the Bayesian claim if additional unobvious and untested (but testable) assumptions are invoked. The alternative strategy of weakening the underlying notion of rationality no longer distinguishes the Bayesian model uniquely. A new account of rationality—either for inference or for decision-making—is required to successfully confirm Bayesian models in cognitive science.

In many people, caffeine causes slight muscle tremors, particularly in their hands. In general, the Caffeine → Muscle Tremors causal connection is a noisy one: someone can drink coffee and experience no hand shaking, and there are many other factors that can lead to muscle tremors. Now suppose that Jane drinks several cups of coffee and then notices that her hands are trembling; an obvious question is: did this instance of coffee drinking cause this instance of hand-trembling? Structurally similar questions arise throughout everyday life: Did this pressing of the ‘k’ key cause this change in the pixels on the computer monitor? Did these episodes of smoking cause this lung cancer? Did this studying cause this test score? And so on. These questions all ask about singular causation in a particular situation, in contrast with general causation across multiple cases. They are thus particularly salient in situations in which we care about that specific case, as in many legal contexts, social interactions, physical explanations of anomalous events, and more.

Our concept of actual causation plays a deep, ever-present role in our experiences. I first argue that traditional philosophical methods for understanding this concept are unlikely to be successful. I contend that we should instead use functional analyses and an understanding of the cognitive bases of causal cognition to gain insight into the concept of actual causation. I additionally provide initial, programmatic steps towards carrying out such analyses. The characterization of the concept of actual causation that results is quite different from many standard views: it is graded, context-sensitive, and extrinsic.

There is now substantial agreement about the representational component of a normative theory of causal reasoning: Causal Bayes Nets. There is less agreement about a normative theory of causal discovery from data, either computationally or cognitively, and almost no work investigating how teaching the Causal Bayes Nets representational apparatus might help individuals faced with a causal learning task. Psychologists working to describe how naïve participants represent and learn causal structure from data have focused primarily on learning from single trials under a variety of conditions. In contrast, one component of the normative theory focuses on learning from a sample drawn from a population under some experimental or observational study regime. Through a virtual Causality Lab that embodies the normative theory of causal reasoning and which allows us to record student behavior, we have begun to systematically explore how best to teach the normative theory. In this paper we explain the overall project and report on pilot studies which suggest that students can quickly be taught to (appear to) be quite rational

Erratum to: Synthese DOI 10.1007/s11229-014-0408-3Appendix 1: NotationLet \(X\) represent a sequence of data, and let \(X_B^t\) represent an i.i.d. subsequence of length \(t\) of data generated from distribution \(B\).We conjecture that the i.i.d. assumption could be eliminated by defining probability distributions over sequences of arbitrary length, though this complication would not add conceptual clarity. Let \(\mathbf{F}\) be a framework (in this case, a set of probability distributions or densities).Let any \(P(\,)\) functions be either a probability distribution function or probability density function, as appropriate. Let \(M_\mathbf{F}\) be a method that takes a data sequence \(X\) as input and outputs a distribution

Even if one can experiment on relevant factors, learning the causal structure of a dynamical system can be quite difficult if the relevant measurement processes occur at a much slower sampling rate than the “true” underlying dynamics. This problem is exacerbated if the degree of mismatch is unknown. This paper gives a formal characterization of this learning problem, and then provides two sets of results. First, we prove a set of theorems characterizing how causal structures change under undersampling. Second, we develop an algorithm for inferring aspects of the causal structure at the “true” timescale from the causal structure learned from the undersampled data. Research on causal learning in dynamical contexts has largely ignored the challenges of undersampling, but this paper provides a framework and foundation for learning causal structure from this type of complex time series data.

We evaluate the asymptotic performance of boundedly-rational strategies in multi-armed bandit problems, where performance is measured in terms of the tendency (in the limit) to play optimal actions in either (i) isolation or (ii) networks of other learners. We show that, for many strategies commonly employed in economics, psychology, and machine learning, performance in isolation and performance in networks are essentially unrelated. Our results suggest that the appropriateness of various, common boundedly-rational strategies depends crucially upon the social context (if any) in which such strategies are to be employed.

We argue that current discussions of criteria for actual causation are ill-posed in several respects. (1) The methodology of current discussions is by induction from intuitions about an infinitesimal fraction of the possible examples and counterexamples; (2) cases with larger numbers of causes generate novel puzzles; (3) "neuron" and causal Bayes net diagrams are, as deployed in discussions of actual causation, almost always ambiguous; (4) actual causation is (intuitively) relative to an initial system state since state changes are relevant, but most current accounts ignore state changes through time; (5) more generally, there is no reason to think that philosophical judgements about these sorts of cases are normative; but (6) there is a dearth of relevant psychological research that bears on whether various philosophical accounts are descriptive. Our skepticism is not directed towards the possibility of a correct account of actual causation; rather, we argue that standard methods will not lead to such an account. A different approach is required

A pervasive feature of the sciences, particularly the applied sciences, is an experimental focus on a few (often only one) possible causal connections. At the same time, scientists often advance and apply relatively broad models that incorporate many different causal mechanisms. We are naturally led to ask whether there are normative rules for integrating multiple local experimental conclusions into models covering many additional variables. In this paper, we provide a positive answer to this question by developing several inference rules that use local causal models to place constraints on the integrated model, given quite general assumptions. We also demonstrate the practical value of these rules by applying them to a case study from ecology. Experimental scope in applied sciences Fusing the results of experiments A concrete example of the inference rules Application to a case study.

Arguments, claims, and discussions about the “level of description” of a theory are ubiquitous in cognitive science. Such talk is typically expressed more precisely in terms of the granularity of the theory, or in terms of Marr’s three levels. I argue that these ways of understanding levels of description are insufficient to capture the range of different types of theoretical commitments that one can have in cognitive science. When we understand these commitments as points in a multi-dimensional space, we find that we must also reconsider our understanding of intertheoretic relations. In particular, we should understand cognitive theories as constraining one another, rather than reducing to one another.

Our best sciences are frequently held to be one way, perhaps the optimal way, to learn about the world’s higher-level ontology and structure. I first argue that which scientific theory is “best” depends in part on our goals or purposes. As a result, it is theoretically possible to have two scientific theories of the same domain, where each theory is best for some goal, but where the two theories posit incompatible ontologies. That is, it is possible for us to have goal-dependent pluralism in our scientific ontologies. This ontological pluralism arises simply from our inability to directly know the world’s objects, rather than any particular claims about our cognitive limits, values, or social structures. I then present two case studies in which this possibility actually occurs—one based on simulations and theoretical analyses of constructed causal systems, and one from actual scientific investigations into the proper ontology for ocean regions.

Research on adaptive rationality has focused principally on inference, judgment, and decision-making that lead to behaviors and actions. These processes typically require cognitive representations as input, and these representations must presumably be acquired via learning. Nonetheless, there has been little work on the nature of, and justification for, adaptively rational learning processes. In this paper, we argue that there are strong reasons to believe that some learning is adaptively rational in the same way as judgment and decision-making. Indeed, overall adaptive rationality can only properly be assessed for pairs of learning and decision processes. We thus present a formal framework for modeling such pairs of cognitive processes, and thereby assessing their adaptive rationality relative to the environment and the agent’s goals. We then use this high-level formal framework on specific cases by demonstrating how natural formal constraints on decision-making can lead to substantive predictions about adaptively rational learning and representation; and characterizing adaptively rational learning for fast-and-frugal one-reason decision-making.