John Pollock

As a high school student, I rediscovered Hume’s problem of induction on my own. For a while, I was horrified. I thought, “We cannot know anything!” After a couple of weeks I calmed down and reasoned that there had to be something wrong with my thinking, and that led me quickly to the realization that good reasons need not be deductive, and to the discovery of defeasible reasoning. From there it was a short jump to a more general interest in how rational cognition works. I am interested in rational cognition in general. Epistemology is one constituent of rational cognition, practical cognition (rational decision making) another. Much of the work on rational cognition begins with the supposition that only ideal agents can be truly rational. Real agents have limited powers of reasoning and limited memory capacity. It is often supposed that such resource-bounded agents can only approximate rationality, and that as philosophers we should confine our attention to ideal agents. If one wishes, one can of course define “rationality” in this way, but this has never been what interested me. We come to philosophy wondering what we should believe, what we should do, and how we should go about deciding these matters. These are questions about ourselves, with all of our cognitive limitations. For example, it is often claimed that ideal agents, with unlimited cognitive powers, should believe all of the logical consequences of their beliefs. But we, as real resource-bounded agents, cannot do that, so that is not something we should do. What I want to know is how I, as a real agent, should go about deciding what to believe and what to do. Thus my topic is real rationality as opposed to ideal rationality. In the realm of practical decision making, I have explored this distinction at great length in my recent book (2006). Here I will focus on its implications for epistemology. For many years epistemology was derailed by the Gettier problem..

Internalism in epistemology is the view that all the factors relevant to the justification of a belief are importantly internal to the believer, while externalism is the view that at least some of those factors are external. This extremely modest first approximation cries out for refinement (which we undertake below), but is enough to orient us in the right direction, namely that the debate between internalism and externalism is bound up with the controversy over the correct account of the distinction between justified beliefs and unjustified beliefs.1 Understanding that distinction has occasionally been obscured by attention to the analysis of knowledge and to the Gettier problem, but our view is that these problems, while interesting, should not completely seduce philosophers away from central questions about epistemic justification. A plausible starting point in the discussion of justification is that the distinction between justified beliefs and unjustified beliefs is not the same as the distinction between true beliefs and false beliefs. This follows from the mundane observation that it is possible to rationally believe..

The “grand problem” of AI has always been to build artificial agents with human-like intelligence. That is the stuff of science fiction, but it is also the ultimate aspiration of AI. In retrospect, we can understand what a difficult problem this is, so since its inception AI has focused more on small manageable problems, with the hope that progress there will have useful implications for the grand problem. Now there is a resurgence of interest in tackling the grand problem head-on. Perhaps AI has made enough progress on the little problems that we can fruitfully address the big problem. The objective is to build agents of human-level intelligence capable of operating in environments of real-world complexity. I will refer to these as GIAs — “generally intelligent agents”. OSCAR is a cognitive architecture for GIAs, implemented in LISP.1 OSCAR draws heavily on my work in philosophy concerning both epistemology (Pollock 1974, 1986, 1990, 1995, 1998, 2008b, 2008; Pollock and Cruz 1999; Pollock and Oved, 2005) and rational decision making (2005, 2006, 2006a).

The “grand problem” of AI has always been to build artificial agents of human-level intelligence, capable of operating in environments of real-world complexity. OSCAR is a cognitive architecture for such agents, implemented in LISP. OSCAR is based on my extensive work in philosophy concerning both epistemology and rational decision making. This paper provides a detailed overview of OSCAR. The main conclusions are that such agents must be capablew of operating against a background of pervasive ignorance, because the real world is too complex for them to know more than a small fraction of what is true. This is handled by giving the agent the power to reason defeasibily. The OSCAR system of defeasible reasoning is sketched. It is argued that if epistemic cognition must be defeasible, planning must also be done defeasibly, and the best way to do that is to reason defeasibly about plans. A sketch is given about how this might work.

Bayesians take “definite” or “single-case” probabilities to be basic. Definite probabilities attach to closed formulas or propositions. We write them here using small caps: PROB(P) and PROB(P/Q). Most objective probability theories begin instead with “indefinite” or “general” probabilities (sometimes called “statistical probabilities”). Indefinite probabilities attach to open formulas or propositions. We write indefinite probabilities using lower case “prob” and free variables: prob(Bx/Ax). The indefinite probability of an A being a B is not about any particular A, but rather about the property of being an A. In this respect, its logical form is the same as that of relative frequencies. For instance, we might talk about the probability of a human baby being female. That probability is about human babies in general — not about individuals. If we examine a baby and determine conclusively that she is female, then the definite probability of her being female is 1, but that does not alter the indefinite probability of human babies in general being female. Most objective approaches to probability tie probabilities to relative frequencies in some way, and the resulting probabilities have the same logical form as the relative frequencies. That is, they are indefinite probabilities. The simplest theories identify indefinite probabilities with relative frequencies.3 It is often objected that such “finite frequency theories” are inadequate because our probability judgments often diverge from relative frequencies. For example, we can talk about a coin being fair (and so the indefinite probability of a flip landing heads is 0.5) even when it is flipped only once and then destroyed (in which case the relative frequency is either 1 or 0). For understanding such indefinite probabilities, it has been suggested that we need a notion of probability that talks about possible instances of properties as well as actual instances..

This paper investigates decision-theoretic planning in sophisticated autonomous agents operating in environments of real-world complexity. An example might be a planetary rover exploring a largely unknown planet. It is argued th a t existing algorithms for decision-theoretic planning are based on a logically incorrect theory of rational decision making. Plans cannot be evaluated directly in terms of their expected values, because plans can be of different scopes, and they can interact with other previously adopted plans. Furthermore, in the real world, the search for optimal plans is completely intractable. An alternative theory of rational decision making is proposed, called “locally global planning”.

It is argued that we cannot build a sophisticated autonomous planetary rover just by implementing sophisticated planning algorithms. Planning must be based on information, and the agent must have the cognitive capability of acquiring new information about its environment. That requires the implementation of a sophisticated epistemology. Epistemological considerations indicate that the rover cannot be assumed to have a complete probability distribution at its disposal. Its planning must be based upon “thin” knowledge of probabilities, and that has important implications for what planning algorithms might be employed.

It’s morning. You sit down at your desk, cup of coffee in hand, and prepare to begin your day. First, you turn on your computer. Once it is running, you check your e-mail. Having decided it is all spam, you trash it. You close the window on your e-mail program, but leave the program running so that it will periodically check the mail server to see whether you have new mail. If it finds new mail it will alert you by playing a musical tone. Next you start your word processor. You have in mind to write a paper in moral philosophy about whether people who send spam deserve capital punishment. So you open a new window and type several paragraphs of text into it. You like what you wrote, so you save it, creating a file. Later, you have more thoughts about spam and capital punishment, so you open the file again and make some changes. Then it is time to go to class. You turn off your word processor, but leave your computer running so that your email program can collect your e-mail. This mundane sequence of events can seem philosophically puzzling when we think about it carefully. While in your word processor, you opened several windows, entered text into them, and created files. What sorts of things are these files, windows, and text? It might seem that windows are easy to understand. You can, after all, see windows. That is the whole point of them. You see a window by seeing a pattern on the surface of your monitor. Isn’t the window identical with that physical pattern? But that is too quick. First, you can turn your monitor off. The window is still open. You can type text into it, and if you turn the monitor back on you can verify that you made that change. Second, you can drag another window in front of the original window. The original window disappears from view, but it still exists. Things may be happening in it that you cannot see. For example, if it is an e-mail window, new messages may be listed in it as they are 1 downloaded..

In the past, few mainstream epistemologists have endorsed Bayesian epistemology, feeling that it fails to capture the complex structure of epistemic cognition. The defenders of Bayesian epistemology have tended to be probability theorists rather than epistemologists, and I have always suspected they were more attracted by its mathematical elegance than its epistemological realism. But recently Bayesian epistemology has gained a following among younger mainstream epistemologists. I think it is time to rehearse some of the simpler but still quite devastating objections to Bayesian epistemology. Most of these objections are familiar, but have never been adequately addressed by the Bayesians.

Direct inference derives values for definite (single-case) probabilities from those of related indefinite (general) probabilities. But direct inference is less useful than might be supposed, because we often have too much information, with the result that we can make conflicting direct inferences, and hence they all undergo collective defeat, leaving us without any conclusion to draw about the value of the definite probabilities. This paper presents reason for believing that there is a function — the Y- function — that can be used to combine different indefinite probabilities to yield a single value for the definite probability. Thus we get a kind of “computational” direct inference.

This paper addresses the logical foundations of goal-regression planning in autonomous rational agents. It focuses mainly on three problems. The first is that goals and subgoals will often be conjunctions, and to apply goal-regression planning to a conjunction we usually have to plan separately for the conjuncts and then combine the resulting subplans. A logical problem arises from the fact that the subplans may destructively interfere with each other. This problem has been partially solved in the AI literature (e.g., in SNLP and UCPOP), but the solutions proposed there work only when a restrictive assumption is satisfied. This assumption pertains to the computability of threats. It is argued that this assumption may fail for an autonomous rational agent operating in a complex environment. Relaxing this assumption leads to a theory of defeasible planning. The theory is formulated precisely and an implementation in the OSCAR architecture is discussed.

Decision-theoretic planning is normally based on the assumption that plans can be compared by comparing their expected-values, and the objective is to find an optimal plan. This is typically defended by reference to classical decision theory. However, classical decision theory is actually incompatible with this “simple plan-based decision theory”. A defense of plan-based decision theory must begin by showing that classical decision theory is incorrect insofar as the two theories conflict, so this paper begins by raising objections to classical decision theory. First, there is a discussion of the considerations arising out of the Newcomb problem that have given rise to causal decision theory. Next, counterexamples are constructed for classical decision theory turning on the fact that an agent may be unable to perform an action, and may even be unable to try to perform an action. A proposal is made for how to repair classical decision theory in light of these counterexamples. But then turning to the concept of an “alternative” that is presupposed by classical decision theory, it is argued that actions must often be chosen in groups rather than individually, i.e., the objects of rational choice are plans. It is argued that optimality cannot be defined for plans, and even if it could be, it would not be reasonable to require rational agents to find optimal plans. So simple plan-based decision theory must also be rejected. An alternative called “locally global planning” is proposed as a replacement for both classical decision theory and simple plan-based decision theory

There are two general approaches to handling contingencies in decision-theoretic planning. State-space planners reason globally, building a map of the parts of the world relevant to the planning problem, and then attempt to distill a plan out of the map. POCL planners reason locally, attempting to build the plan up from local relationships. A planning problem is constructed that humans find trivial, but no state-space planner can solve. This motivates an investigation of decision-theoretic POCL contingency planners. Existing POCL contingency planners attempt to generalize the results of classical POCL contingency planning. However, this paper argues that the nature of contingency planning changes dramatically in decision-theoretic contexts, and results from classical contingency planning are of little relevance. In particular, in classical planning contingencies can only be attached to conditional forks, but in most uses of contingencies in decision-theoretic planning they are attached to single branches of the plan rather than to conditional forks. A criterion of adequacy for contingency planners is formulated, following from ordinary completeness, and it is shown that existing decision-theoretic POCL contingency planners do not satisfy it. Some tentative suggestions are made regarding how to construct a planner that does satisfy the adequacy condition.

The objective of this paper is to construct an implementable theory of rational decision-making for cognitive agents subject to realistic resource constraints. It is argued that decision-making should select actions indirectly by selecting plans that prescribe them. It is also argued that although expected values provide the tool for evaluating plans, plans cannot be compared straightforwardly in terms of their expected values, and the objective of a realistic agent cannot be to find optimal plans. The theory of Locally Global planning is proposed as a realistic alternative to standard "maximizing" theories of rational decision-making.

In concrete applications of probability, statistical investigation gives us knowledge of some probabilities, but we generally want to know many others that are not directly revealed by our data. For instance, we may know prob(P/Q) (the probability of P given Q) and prob(P/R), but what we really want is prob(P/Q&R), and we may not have the data required to assess that directly. The probability calculus is of no help here. Given prob(P/Q) and prob(P/R), it is consistent with the probability calculus for prob(P/Q&R) to have any value between 0 and 1. Is there any way to make a reasonable estimate of the value of prob(P/Q&R)? A related problem occurs when probability practitioners adopt undefended assumptions of statistical independence simply on the basis of not seeing any connection between two propositions. This is common practice, but its justification has eluded probability theorists, and researchers are typically apologetic about making such assumptions. Is there any way to defend the practice? This paper shows that on a certain conception of probability — nomic probability — there are principles of “probable probabilities” that license inferences of the above sort. These are principles telling us that although certain inferences from probabilities to probabilities are not deductively valid, nevertheless the second-order probability of their yielding correct results is 1. This makes it defeasibly reasonable to make the inferences. Thus I argue that it is defeasibly reasonable to assume statistical independence when we have no information to the contrary. And I show that there is a function Y(r,s:a) such that if prob(P/Q) = r, prob(P/R) = s, and prob(P/U) = a (where U is our background knowledge) then it is defeasibly reasonable to expect that prob(P/Q&R) = Y(r,s:a). Numerous other defeasible inferences are licensed by similar principles of probable probabilities..

Since Gettier, much of epistemology has focused on analyzing “S knows that P”, but that is not my interest. My general interest is in rational cognition — both in what it is to be rational, and in how rational cognition works. The traditional epistemological question, “How do you know?”, can be taken as addressing part of the more general problem of producing a theory of rational cognition. It is about specifically epistemic rationality. I interpret this question literally, as a question about how we should proceed in our epistemic endeavors. Epistemological theories that try to answer this question are theories of procedural epistemology (see my 1998), and when, from this perspective, we assess beliefs in terms of their justifiedness, the concept of justification is one of procedural epistemic justification. Whether this has anything to do with the analysis of knowledge is an open question, and not one that I have much interest in addressing.

When combining information from multiple sources and attempting to estimate the probability of a conclusion, we often find ourselves in the position of knowing the probability of the conclusion conditional on each of the individual sources, but we have no direct information about the probability of the conclusion conditional on the combination of sources. The probability calculus provides no way of computing such joint probabilities. This paper introduces a new way of combining probabilistic information to estimate joint probabilities. It is shown that on a particular conception of objective probabilities, clear sense can be made of second-order probabilities (probabilities of probabilities), and these can be related to combinatorial theorems about proportions in finite sets as the sizes of the sets go to infinity. There is a rich mathematical theory consisting of such theorems, and the theorems generate corresponding theorems about secondorder probabilities. Among the latter are a number of theorems to the effect that certain inferences from probabilities to probabilities, although not licensed by the probability calculus, have probability 1 of producing correct results. This does not mean that they will always produce correct results, but the set of cases in which the inferences go wrong form a set of measure 0. Among these inferences are some enabling us to reasonably estimate the values of joint probabilities in a wide range of cases. A function called the Y-function is defined. The central theorem is the Y-Theorem, which tells us that if we know the individual probabilities for the different information sources and estimate the joint probability using the Y-function, the second-order probability of getting the right answer is 1. This mathematical result is tested empirically using a simple multi-sensor example. The Y-theorem agrees with Dempster's rule of combination in special cases, but not in general.

I argue here that sophisticated AI systems, with the exception of those aimed at the psychological modeling of human cognition, must be based on general philosophical theories of rationality and, conversely, philosophical theories of rationality should be tested by implementing them in AI systems. So the philosophy and the AI go hand in hand. I compare human and generic rationality within a broad philosophy of AI and conclude by suggesting that ultimately, virtually all familiar philosophical problems will turn out to be at least indirectly relevant to the task of building an autonomous rational agent, and conversely, the AI enterprise has the potential to throw light at least indirectly on most philosophical problems.

Stuart Russell [14] describes rational agents as --œthose that do the right thing--�. The problem of designing a rational agent then becomes the problem of figuring out what the right thing is. There are two approaches to the latter problem, depending upon the kind of agent we want to build. On the one hand, anthropomorphic agents are those that can help human beings rather directly in their intellectual endeavors. These endeavors consist of decision making and data processing. An agent that can help humans in these enterprises must make decisions and draw conclusions that are rational by human standards of rationality. Anthropomorphic agents can be contrasted with goal-oriented agents --” those that can carry out certain narrowly-defined tasks in the world. Here the objective is to get the job done, and it makes little difference how the agent achieves its design goal.