Scott Brown

Evidence-accumulation models (EAMs) are powerful tools for making sense of human and animal decision-making behavior. EAMs have generated significant theoretical advances in psychology, behavioral economics, and cognitive neuroscience and are increasingly used as a measurement tool in clinical research and other applied settings. Obtaining valid and reliable inferences from EAMs depends on knowing how to establish a close match between model assumptions and features of the task/data to which the model is applied. However, this knowledge is rarely articulated in the EAM literature, leaving beginners to rely on the private advice of mentors and colleagues and inefficient trial-and-error learning. In this article, we provide practical guidance for designing tasks appropriate for EAMs, relating experimental manipulations to EAM parameters, planning appropriate sample sizes, and preparing data and conducting an EAM analysis. Our advice is based on prior methodological studies and the our substantial collective experience with EAMs. By encouraging good task-design practices and warning of potential pitfalls, we hope to improve the quality and trustworthiness of future EAM research and applications.

In the modern world, many important tasks have become too complex for a single unaided individual to manage. Teams conduct some safety-critical tasks to improve task performance and minimize the risk of error. These teams have traditionally consisted of human operators, yet, nowadays, artificial intelligence and machine systems are incorporated into team environments to improve performance and capacity. We used a computerized task modeled after a classic arcade game to investigate the performance of human–machine and human–human teams. We manipulated the group conditions between team members; sometimes, they were instructed to collaborate, compete, or work separately. We evaluated players' performance in the main task (gameplay) and, in post hoc analyses, participant behavioral patterns to inform group strategies. We compared game performance between team types (human–human vs. human–machine) and group conditions (competitive, collaborative, independent). Adapting workload capacity analysis to human–machine teams, we found performance under both team types and all group conditions suffered a performance efficiency cost. However, we observed a reduced cost in collaborative over competitive teams within human–human pairings, but this effect was diminished when playing with a machine partner. The implications of workload capacity analysis as a powerful tool for human–machine team performance measurement are discussed.

Discrete choice experiments—selecting the best and/or worst from a set of options—are increasingly used to provide more efficient and valid measurement of attitudes or preferences than conventional methods such as Likert scales. Discrete choice data have traditionally been analyzed with random utility models that have good measurement properties but provide limited insight into cognitive processes. We extend a well-established cognitive model, which has successfully explained both choices and response times for simple decision tasks, to complex, multi-attribute discrete choice data. The fits, and parameters, of the extended model for two sets of choice data (involving patient preferences for dermatology appointments, and consumer attitudes toward mobile phones) agree with those of standard choice models. The extended model also accounts for choice and response time data in a perceptual judgment task designed in a manner analogous to best–worst discrete choice experiments. We conclude that several research fields might benefit from discrete choice experiments, and that the particular accumulator-based models of decision making used in response time research can also provide process-level instantiations for random utility models

For decisions between many alternatives, the benchmark result is Hick's Law: that response time increases log-linearly with the number of choice alternatives. Even when Hick's Law is observed for response times, divergent results have been observed for error rates—sometimes error rates increase with the number of choice alternatives, and sometimes they are constant. We provide evidence from two experiments that error rates are mostly independent of the number of choice alternatives, unless context effects induce participants to trade speed for accuracy across conditions. Error rate data have previously been used to discriminate between competing theoretical accounts of Hick's Law, and our results question the validity of those conclusions. We show that a previously dismissed optimal observer model might provide a parsimonious account of both response time and error rate data. The model suggests that people approximate Bayesian inference in multi-alternative choice, except for some perceptual limitations

Understanding individual differences in cognitive performance is an important part of understanding how variations in underlying cognitive processes can result in variations in task performance. However, the exploration of individual differences in the components of the decision process—such as cognitive processing speed, response caution, and motor execution speed—in previous research has been limited. Here, we assess the heritability of the components of the decision process, with heritability having been a common aspect of individual differences research within other areas of cognition. Importantly, a limitation of previous work on cognitive heritability is the underlying assumption that variability in response times solely reflects variability in the speed of cognitive processing. This assumption has been problematic in other domains, due to the confounding effects of caution and motor execution speed on observed response times. We extend a cognitive model of decision‐making to account for relatedness structure in a twin study paradigm. This approach can separately quantify different contributions to the heritability of response time. Using data from the Human Connectome Project, we find strong evidence for the heritability of response caution, and more ambiguous evidence for the heritability of cognitive processing speed and motor execution speed. Our study suggests that the assumption made in previous studies—that the heritability of cognitive ability is based on cognitive processing speed—may be incorrect. More generally, our methodology provides a useful avenue for future research in complex data that aims to analyze cognitive traits across different sources of related data, whether the relation is between people, tasks, experimental phases, or methods of measurement.

Miller (1956) identified his famous limit of 7 ± 2 items based in part on absolute identification—the ability to identify stimuli that differ on a single physical dimension, such as lines of different length. An important aspect of this limit is its independence from perceptual effects and its application across all stimulus types. Recent research, however, has identified several exceptions. We investigate an explanation for these results that reconciles them with Miller’s work. We find support for the hypothesis that the exceptional stimulus types have more complex psychological representations, which can therefore support better identification. Our investigation uses data sets with thousands of observations for each participant, which allows the application of a new technique for identifying psychological representations: the structural forms algorithm of Kemp and Tenenbaum (2008). This algorithm supports inferences not possible with previous techniques, such as multidimensional scaling.

The ability to imagine objects undergoing rotation (mental rotation) improves markedly with practice, but an explanation of this plasticity remains controversial. Some researchers propose that practice speeds up the rate of a general-purpose rotation algorithm. Others maintain that performance improvements arise through the adoption of a new cognitive strategy—repeated exposure leads to rapid retrieval from memory of the required response to familiar mental rotation stimuli. In two experiments we provide support for an integrated explanation of practice effects in mental rotation by combining behavioral and EEG measures in a way that provides more rigorous inference than is available from either measure alone. Before practice, participants displayed two well-established signatures of mental rotation: Both response time and EEG negativity increased linearly with rotation angle. After extensive practice with a small set of stimuli, both signatures of mental rotation had all but disappeared. In contrast, after the same amount of practice with a much larger set both signatures remained, even though performance improved markedly. Taken together, these results constitute a reversed association, which cannot arise from variation in a single cause, and so they provide compelling evidence for the existence of two routes to expertise in mental rotation. We also found novel evidence that practice with the large but not the small stimulus set increased the magnitude of an early visual evoked potential, suggesting increased rotation speed is enabled by improved efficiency in extracting three-dimensional information from two-dimensional stimuli