Alexander Krauss

Few things have impacted our lives as much as science and technology, but how we developed science and civilisation is one of the most challenging questions that has not yet been well explained. Attempting to identify the central driver, leading scientists have highlighted the role of culture, cooperation and geography. They focus thus on broad factors that are important basic preconditions but that we cannot directly influence. To better address the question, this paper integrates evidence from evolutionary biology, cognitive science, methodology, archaeology and anthropology. The paper identifies 9 main preconditions necessary for contemporary science, which include 6 main preconditions for civilisation. Using a kind of quasi-experimental research design we observe that some cultures (experimental groups) met the preconditions while other cultures (control groups) did not. Among the preconditions, we explain how our mind's evolved methodological abilities (to observe, solve problems and experiment) have directly enabled acquiring knowledge about the world and collectively developing increasingly sophisticated methods (such as mathematics and more systematic experimentation) that have enabled science and civilisation. We have driven the major revolutions throughout our history – the palaeolithic technological and agricultural revolutions and later the so-called scientific, industrial and digital revolutions – by using our methodological abilities in new ways and developing new methods and tools, i.e. through methodological revolutions. Viewing our methods as the main mechanism through which we have directly developed scientific and technological knowledge, and thus science and civilisation, provides a new framework for understanding science and the history of science. Viewing humans as homo methodologicus, using an expanding methodological toolbox, provides a nuanced explanation of how we have been directly able to meet our needs, solve problems and develop vast bodies of technological and scientific knowledge. By better understanding the origin and foundations of science, we can better understand their limits and, most importantly, how to push those limits. We can do so especially by addressing the evolved cognitive constraints and biases we face and improving the methods we use.

Biomedical research, especially pharmaceutical research, has been criticised for engaging in practices that lead to over-estimations of the effectiveness of medical treatments. A central issue concerns the reporting of absolute and relative measures of medical effectiveness. In this paper we critically examine proposals made by Jacob Stegenga to (a) give priority to the reporting of absolute measures over relative measures, and (b) downgrade the measures of effectiveness (effect sizes) of the treatments tested in clinical trials (Stegenga, 2015a). After exposing significant flaws in a central case study used by Stegenga to bolster his first proposal (a), we go on to argue that neither of these proposals is defensible (a or b). We defend the practice, in line with the New England Journal of Medicine, of reporting both absolute and relative measures whenever feasible.

Background: Randomised controlled trials (RCTs) are commonly viewed as the best research method to inform public health and social policy. Usually they are thought of as providing the most rigorous evidence of a treatment’s effectiveness without strong assumptions, biases and limitations. Objective: This is the first study to examine that hypothesis by assessing the 10 most cited RCT studies worldwide. Data sources: These 10 RCT studies with the highest number of citations in any journal were identified by searching Scopus (the largest database of peer-reviewed journals). Results: This study shows that these world-leading RCTs that have influenced policy produce biased results by illustrating that participants’ background traits that affect outcomes are often poorly distributed between trial groups, that the trials often neglect alternative factors contributing to their main reported outcome and, among many other issues, that the trials are often only partially blinded or unblinded. The study here also identifies a number of novel and important assumptions, biases and limitations not yet thoroughly discussed in existing studies that arise when designing, implementing and analysing trials. Conclusions: Researchers and policymakers need to become better aware of the broader set of assumptions, biases and limitations in trials. Journals need to also begin requiring researchers to outline them in their studies. We need to furthermore better use RCTs together with other research methods.

Climate change, nutrition, poverty and medical drugs are widely discussed and pressing issues in science, policy and society. Despite these issues being of great importance for the quality of our lives it remains unclear how well people understand them. Specifically, do particular demographic and socioeconomic factors explain variation in public understanding of these four concepts? To what extent are people’s changes in understanding associated with changes in their behaviour? Do people judge scientific practices relying on the more descriptive concepts of climate change and effective medical drugs to be more objective (less controversial) than practices relying on the more value-laden concepts of poverty and healthy nutrition? To address these questions, an experimental survey and regression analyses are conducted using data collected from about one thousand participants across different continents. The study finds that public understanding of science is generally low. A smaller proportion of people were able to correctly identify the common explanation accepted internationally among the scientific community for climate change and effectiveness of medical drugs (42% and 43% of participants in the study, respectively) than for poverty and healthy nutrition (61% and 65% of participants, respectively). Older age and political non-conservativeness were the strongest predictors of correctly understanding these four concepts. Greater levels of education and political non-conservativeness were in turn the strongest predictors of people’s reported changes in their behaviour based on their improved understanding of these concepts. Because climate change is among the least understood scientific concepts but is arguably the greatest challenge of our time, better efforts are needed to improve how media, awareness campaigns and education systems mediate information on the topic in order to tackle the large knowledge deficits that constrain behavioural change.

Many scientists routinely generalize from study samples to larger populations. It is commonly assumed that this cognitive process of scientific induction is a voluntary inference in which researchers assess the generalizability of their data and then draw conclusions accordingly. We challenge this view and argue for a novel account. The account describes scientific induction as involving by default a generalization bias that operates automatically and frequently leads researchers to unintentionally generalize their findings without sufficient evidence. The result is unwarranted, overgeneralized conclusions. We support this account of scientific induction by integrating a range of disparate findings from across the cognitive sciences that have until now not been connected to research on the nature of scientific induction. The view that scientific induction involves by default a generalization bias calls for a revision of the current thinking about scientific induction and highlights an overlooked cause of the replication crisis in the sciences. Commonly proposed interventions to tackle scientific overgeneralizations that may feed into this crisis need to be supplemented with cognitive debiasing strategies against generalization bias to most effectively improve science.

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed.

This paper outlines the methodological and empirical limitations of analysing the potential relationship between complex social phenomena such as democracy and inequality. It shows that the means to assess how they may be related is much more limited than recognised in the existing literature that is laden with contradictory hypotheses and findings. Better understanding our scientific limitations in studying this potential relationship is important for research and policy because many leading economists and other social scientists such as Acemoglu and Robinson mistakenly claim to identify causal linkages between inequality and democracy but at times still inform policy. In contrast to the existing literature, the paper argues that ‘structural’ or ‘causal’ mechanisms that may potentially link the distribution of economic wealth and different political regimes will remain unknown given reasons such as their highly complex and idiosyncratic characteristics, fundamental econometric constraints and analysis at the macro-level. Neither new data sources, different analysed time periods nor new data analysis techniques can resolve this question and provide robust, general conclusions about this potential relationship across countries. Researchers are thus restricted to exploring rough correlations over specific time periods and geographic contexts with imperfect data that are very limited for cross-country comparisons.