Parker DuVall

Why do we distrust conversational AI (CAI) testifiers? That is to say, why do many have the intuition that CAI are less epistemically trustworthy than a standard human testifier? While it seems like we have such a pre-theoretical intuition, we are often lacking a philosophical defense of this attitude. In this paper, I argue that testimony is more trustworthy if it is based in a first-person experience of consciousness. I start by laying out an account of CAI testimony, including considering what it would mean for a CAI system to have knowledge. I then use a number of pragmatic encroachment cases to show that when the stakes are high and we have to choose between two instances of testimony to ground our belief about p, we always prefer the piece of testimony from the speaker with a first-person experience of p. Since CAI systems cannot, and possibly never will, have a first-person experience of consciousness (i.e., there is nothing it is “like” to be a CAI), we will always view CAI testimony as less trustworthy. Finally, I consider and respond to a number of objections to my arguments.