Alexander Krauss

Many scientists routinely generalize from study samples to larger populations. It is commonly assumed that this cognitive process of scientific induction is a voluntary inference in which researchers assess the generalizability of their data and then draw conclusions accordingly. We challenge this view and argue for a novel account. The account describes scientific induction as involving by default a generalization bias that operates automatically and frequently leads researchers to unintentionally generalize their findings without sufficient evidence. The result is unwarranted, overgeneralized conclusions. We support this account of scientific induction by integrating a range of disparate findings from across the cognitive sciences that have until now not been connected to research on the nature of scientific induction. The view that scientific induction involves by default a generalization bias calls for a revision of the current thinking about scientific induction and highlights an overlooked cause of the replication crisis in the sciences. Commonly proposed interventions to tackle scientific overgeneralizations that may feed into this crisis need to be supplemented with cognitive debiasing strategies against generalization bias to most effectively improve science.

How can scientific progress be conceived best? Does science mainly undergo revolutionary paradigm shifts? Or is the evolution of science mainly cumulative? Understanding whether science advances through cumulative evolution or through paradigm shifts can influence how we approach scientific research, education and policy. The most influential and cited account of science was put forth in Thomas Kuhn’s seminal book The structure of scientific revolutions. Kuhn argues that science does not advance cumulatively but goes through fundamental paradigm changes in the theories of a scientific field. There is no consensus yet on this core question of the nature and advancement of science that has since been debated across science. Examining over 750 major scientific discoveries (all Nobel Prize and major non-Nobel Prize discoveries), we systematically test this fundamental question about scientific progress here. We find that three key measures of scientific progress—major discoveries, methods and fields—each demonstrate that science evolves cumulatively. First, we show that no major scientific methods or instruments used across fields (such as statistical methods, X-ray methods or chromatography) have been completely abandoned, i.e. subject to paradigm shifts. Second, no major scientific fields (such as biomedicine, chemistry or computer science) have been completely abandoned. Rather, they have all continuously expanded over time, often over centuries, accumulating extensive bodies of knowledge. Third, scientific discoveries including theoretical discoveries are also predominately cumulative, with only 1% of over 750 major discoveries having been abandoned. The continuity of science is most compellingly evidenced by our methods and instruments, which enable the creation of discoveries and fields. We thus offer here a new perspective and answer to this classic question in science and the philosophy and history of science by utilizing methods from statistics and empirical sciences.

What are the limits of science and where do we draw their boundaries? Influential theories about the size of the universe, superstrings, the historical origin of life, the general causes of democratisation and the emergence of our consciousness exist across science. But can these influential theories be scientifically reliable if we cannot yet empirically test them rigorously? There is a vast literature on the foundations and boundaries of our mind, but it has not yet been directly linked to how our mind and methods shape the foundations and boundaries of our knowledge and science. In doing so, this paper aims to provide a novel explanation of how our cognitive, sensory and methodological abilities and constraints drive the present limits of science – influencing the theories about the world we are able to develop and test and those we are not yet able to. By integrating evidence from our evolved mind and the constraints of the methods and instruments developed using our mind we explain what types of phenomena are accessible to scientific investigation, and what phenomena are presently not. The central conclusion is that we reach the present limits of science, and what science itself is, when our theories involve phenomena that are not observable and thus the theories are not verifiable and empirically reliable using our mind, methods and instruments (thus called here the OVER criterion of science). These are the features of phenomena that fall beyond our sensory range that we are not able to perceive and are not presently accessible to our mind including the methods and instruments we develop with our mind – such as the phenomena outlined above. This criterion applies to all scientific theories across the natural and social sciences that aim to explain real phenomena in the world. Scientific theories and their present limits can be better explained and advanced in light of this interdisciplinary and methods-driven framework. The paper concludes by outlining how we can develop new methods and instruments that reduce our cognitive and methodological constraints and thus push the boundaries of science.

Scientific, medical, and technological knowledge has transformed our world, but we still poorly understand the nature of scientific methodology. Science textbooks, science dictionaries, and science institutions often state that scientists follow, and should follow, the universal scientific method of testing hypotheses using observation and experimentation. Yet, scientific methodology has not been systematically analyzed using large-scale data and scientific methods themselves as it is viewed as not easily amenable to scientific study. Using data on all major discoveries across science including all Nobel Prize and major non-Nobel Prize discoveries, we can address the question of the extent to which “the scientific method” is actually applied in making science’s groundbreaking research and whether we need to expand this central concept of science. This study reveals that 25% of all discoveries since 1900 did not apply the common scientific method (all three features)—with 6% of discoveries using no observation, 23% using no experimentation, and 17% not testing a hypothesis. Empirical evidence thus challenges the common view of the scientific method. Adhering to it as a guiding principle would constrain us in developing many new scientific ideas and breakthroughs. Instead, assessing all major discoveries, we identify here a general, common feature that the method of science can be reduced to: making all major discoveries has required using sophisticated methods and instruments of science. These include statistical methods, particle accelerators, and X-ray methods. Such methods extend our mind and generally make observing, experimenting, and testing hypotheses in science possible, doing so in new ways and ensure their replicability. This provides a new perspective to the scientific method—embedded in our sophisticated methods and instruments—and suggests that we need to reform and extend the way we view the scientific method and discovery process.

What are the limits of science and where do we draw their boundaries? Influential theories about the size of the universe, superstrings, the historical origin of life, the general causes of democratisation and the emergence of our consciousness exist across science. But can these influential theories be scientifically reliable if we cannot yet empirically test them rigorously? There is a vast literature on the foundations and boundaries of our mind, but it has not yet been directly linked to how our mind and methods shape the foundations and boundaries of our knowledge and science. In doing so, this paper aims to provide a novel explanation of how our cognitive, sensory and methodological abilities and constraints drive the present limits of science – influencing the theories about the world we are able to develop and test and those we are not yet able to. By integrating evidence from our evolved mind and the constraints of the methods and instruments developed using our mind we explain what types of phenomena are accessible to scientific investigation, and what phenomena are presently not. The central conclusion is that we reach the present limits of science, and what science itself is, when our theories involve phenomena that are not observable and thus the theories are not verifiable and empirically reliable using our mind, methods and instruments (thus called here the OVER criterion of science). These are the features of phenomena that fall beyond our sensory range that we are not able to perceive and are not presently accessible to our mind including the methods and instruments we develop with our mind – such as the phenomena outlined above. This criterion applies to all scientific theories across the natural and social sciences that aim to explain real phenomena in the world. Scientific theories and their present limits can be better explained and advanced in light of this interdisciplinary and methods-driven framework. The paper concludes by outlining how we can develop new methods and instruments that reduce our cognitive and methodological constraints and thus push the boundaries of science.

Powerful new methods and tools drive scientific progress—but how do we actually make such innovations? No theory yet explains how we invent major tools across fields. To address this gap, we examine all nobel-prize-winning method discoveries—that enabled breakthrough findings not possible without them—and we trace science’s most influential toolmakers across fields, from Ernst Ruska’s electron microscope and Kary Mullis’s PCR method to Ernest Lawrence’s particle accelerator. Here we lay out the critical pathways taken to create these groundbreaking tools. We introduce a taxonomy of science’s methods and tools: a scientific table of discovery methods that is a map of underexplored and unexplored method opportunities. By mapping the method landscape, we reveal gaps and possibilities that help guide where methods can be adapted, recombined or strategically developed to catalyse discovery. It can help identify and predict untapped combinations of tools—and where the next breakthroughs can come from. What if we no longer wait for new discovery tools to emerge by chance but begin deliberately prioritising their development? How many big breakthroughs are we missing because we have not yet strategically focused on designing the needed tools? We also outline the need for establishing methods labs and hubs—as incubators of innovation—that catalyse tool creation.

Scientific discoveries have revolutionised our lives, but how new discoveries emerge is one of the big questions of science that has not yet been explained. Answering it is key because we can vastly accelerate how we trigger new breakthroughs. To date, there exists no established theory or general empirical analysis of how major discoveries across science arise. This is the first study to assess science’s major discoveries that cover all nobel-prize and major non-nobel-prize discoveries throughout history. These over 750 discoveries are linked here to the methods and tools used to make them and the traits of the discoverers. This enables establishing which factors are more important to catalyse new breakthroughs. Contrary to common belief, we find a surprising pattern: science’s major discoveries are driven by a new method or tool that enables us to study the world through new lenses and discover what we often did not even know existed—from atoms and galaxies to microorganisms. We find that after designing powerful tools—from X-ray methods and the electron microscope to spectrometers and chromatography—we have sparked dozens of major breakthroughs that were impossible without them. What unites the diverse discoveries that each of these tools made possible is not a shared theory, research teams, or more funding, but using the shared powerful tool, each uncovering groundbreaking findings across different fields. We also uncover that most discoveries since 1975 are triggered within 4 years of creating the needed tool, and many are triggered simultaneously. Building on the evidence of science’s major discoveries, we develop a general new methods-driven discovery theory. The theory explains how new methods and instruments are the common mechanism driving new discoveries by allowing us to observe, measure and understand the world in ways that are not possible without them. Establishing this mechanism of scientific progress provides a powerful new framework to explain and speed up discoveries. We can apply this principle widely across science, from physics to biology, and it can provide a foundation for the science of science—and a new way to predict discovery itself.

How do we spark new scientific discoveries? Why do some breakthroughs seem even accidental? And most importantly, how can we accelerate them and push the boundaries of science? These are some of the biggest unsolved questions in science. Many believe breakthroughs arise by chance or serendipity. But here we show that it is powerful new tools and methods that enable discovering what we often did not even know existed: improved microscopes uncovered the hidden world of microorganisms and viruses, x-ray methods unlocked the structure of our DNA, particle accelerators detected subatomic particles that make up our world, and advanced telescopes revealed galaxies we never imagined. The Engine of Scientific Discovery explores, for the first time, science’s biggest discoveries—spanning all nobel-prize and major non-nobel discoveries throughout history. What we find is surprising: we have triggered science’s over 750 major discoveries by first inventing a new method or tool that made the breakthroughs possible—often within just a few years. This is because new tools enable us to see, measure and understand the world in ways that are impossible without them. From discovering planets beyond our solar system that reshape how we understand the universe, to CRISPR gene editing that transforms how we fight disease and cancer. This consistent pattern reveals a powerful insight: our transformative new methods and tools are The Engine of Scientific Discovery—a fundamental principle of scientific progress that has been overlooked until now. By shifting our focus to inventing new discovery tools, we can spark a method revolution in science. Instead of waiting for breakthroughs, we can actively design and build new tools of discovery. What if the next great breakthroughs depend not just on asking better questions, but on developing better tools to ask and answer them—on entirely new ways of discovering? A new theory of discovery emerges here, offering a roadmap to accelerate the pace of new breakthroughs across science. This book is for anyone who wants to understand how we make discoveries—and what drives science.

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed.

Climate change, nutrition, poverty and medical drugs are widely discussed and pressing issues in science, policy and society. Despite these issues being of great importance for the quality of our lives it remains unclear how well people understand them. Specifically, do particular demographic and socioeconomic factors explain variation in public understanding of these four concepts? To what extent are people's changes in understanding associated with changes in their behaviour? Do people judge scientific practices relying on the more descriptive concepts of climate change and effective medical drugs to be more objective (less controversial) than practices relying on the more value-laden concepts of poverty and healthy nutrition? To address these questions, an experimental survey and regression analyses are conducted using data collected from about one thousand participants across different continents. The study finds that public understanding of science is generally low. A smaller proportion of people were able to correctly identify the common explanation accepted internationally among the scientific community for climate change and effectiveness of medical drugs (42% and 43% of participants in the study, respectively) than for poverty and healthy nutrition (61% and 65% of participants, respectively). Older age and political non-conservativeness were the strongest predictors of correctly understanding these four concepts. Greater levels of education and political non-conservativeness were in turn the strongest predictors of people's reported changes in their behaviour based on their improved understanding of these concepts. Because climate change is among the least understood scientific concepts but is arguably the greatest challenge of our time, better efforts are needed to improve how media, awareness campaigns and education systems mediate information on the topic in order to tackle the large knowledge deficits that constrain behavioural change.

The emergence of scientific discoveries is often seen as the most extraordinary and puzzling feature of science. Many think that discoveries arise due to serendipity or chance—that they are unexplainable or unpredictable. Existing studies have not yet explained the extent and role of serendipity in science, because they generally only study a sample of discoveries that does not allow for generalising about science. To do so, this is the first study to assess science’s major discoveries, defined as all nobel-prize and major non-nobel-prize discoveries across fields and history. By examining over 750 major discoveries, we find that discoveries commonly labelled as serendipitous are actually sparked by using—for the first time—a powerful new tool that makes the unexpected observation possible that we are not even looking for: an improved microscope revealed cells, a discharge tube uncovered X-rays, gel diffusion revealed the Hepatitis B virus and a cutting-edge spectrograph detected the first planet outside our solar system. What seems like chance is often just using an entirely new lens—just as in each of these discoveries. In other words, we show that breakthroughs that seem serendipitous follow shortly after we create the new tool, with other researchers able to arrive at the same findings using these same discovery tools. While a serendipitous moment was reported in about one in six major discoveries, there is a pattern behind the surprise: using new tools to make the breakthrough finding commonly through exploratory research. This is important because it changes how we think about discovery: we can generally better understand individual serendipitous discoveries as tool-triggered discoveries that multiple researchers can make. Crucially, new method innovations provide explanatory power to help predict discoveries—whether labelled serendipitous or not. So here we reframe discovery: because new tools are what commonly unlock surprise, we can actually strategically design and accelerate discovery, shifting it from a more passive outcome of serendipity.

Science and knowledge are studied by researchers across many disciplines, examining how they are developed, what their current boundaries are and how we can advance them. By integrating evidence across disparate disciplines, the holistic field of science of science can address these foundational questions. This field illustrates how science is shaped by many interconnected factors: the cognitive processes of scientists, the historical evolution of science, economic incentives, institutional influences, computational approaches, statistical, mathematical and instrumental foundations of scientific inference, scientometric measures, philosophical and ethical dimensions of scientific concepts, among other influences. Achieving a comprehensive overview of a multifaceted field like the science of science requires pulling together evidence from the many sub-fields studying science across the natural and social sciences and humanities. This enables developing an interdisciplinary perspective of scientific practice, a more holistic understanding of scientific processes and outcomes, and more nuanced perspectives to how scientific research is conducted, influenced and evolves. It enables leveraging the strengths of various disciplines to create a holistic view of the foundations of science. Different researchers study science from their own disciplinary perspective and use their own methods, and there is a large divide between quantitative and qualitative researchers as they commonly do not read or cite research using other methodological approaches. A broader, synthesizing paper employing a qualitative approach can however help provide a bridge between disciplines by pulling together aspects of science (economic, scientometric, psychological, philosophical etc.). Such an approach enables identifying, across the range of fields, the powerful role of our scientific methods and instruments in shaping most aspects of our knowledge and science, whereas economic, social and historical influences help shape what knowledge we pursue. A unifying theory is then outlined for science of science – the new-methods-drive-science theory.

What are the unique features and characteristics of the scientists who have made the greatest discoveries in science? To address this question, we assess all major scientific discoverers, defined as all nobel-prize and major non-nobel-prize discoverers, and their demographic, institutional and economic traits. What emerges is a general profile of the scientists who have driven over 750 of science’s greatest advances. We find that interdisciplinary scientists who completed two or more degrees in different academic fields by the time of discovery made about half—54%—of all nobel-prize discoveries and 42% of major non-nobel-prize discoveries over the same period; this enables greater interdisciplinary methodological training for making new scientific achievements. Science is also becoming increasingly elitist, with scientists at the top 25 ranked universities accounting for 30% of both all nobel-prize and non-nobel-prize discoveries. Scientists over the age of 50 made only 7% of all nobel-prize discoveries and 15% of non-nobel-prize discoveries and those over the age of 60 made only 1% and 3%, respectively. The gap in years between making nobel-prize discoveries and receiving the award is also increasing over time across scientific fields—illustrating that it is taking longer to recognise and select major breakthroughs. Overall, we find that those who make major discoveries are increasingly interdisciplinary, older and at top universities. We also assess here the role and distribution of factors like geographic location, gender, religious affiliation and country conditions of these leading scientists, and how these factors vary across time and scientific fields. The findings suggest that more discoveries could be made if science agencies and research institutions provide greater incentives for researchers to work against the common trend of narrow specialisation and instead foster interdisciplinary research that combines novel methods across fields.

Scientific, medical, and technological knowledge has transformed our world, but we still poorly understand the nature of scientific methodology. Science textbooks, science dictionaries, and science institutions often state that scientists follow, and should follow, the universal scientific method of testing hypotheses using observation and experimentation. Yet, scientific methodology has not been systematically analyzed using large-scale data and scientific methods themselves as it is viewed as not easily amenable to scientific study. Using data on all major discoveries across science including all Nobel Prize and major non-Nobel Prize discoveries, we can address the question of the extent to which “the scientific method” is actually applied in making science's groundbreaking research and whether we need to expand this central concept of science. This study reveals that 25% of all discoveries since 1900 did not apply the common scientific method (all three features)—with 6% of discoveries using no observation, 23% using no experimentation, and 17% not testing a hypothesis. Empirical evidence thus challenges the common view of the scientific method. Adhering to it as a guiding principle would constrain us in developing many new scientific ideas and breakthroughs. Instead, assessing all major discoveries, we identify here a general, common feature that the method of science can be reduced to: making all major discoveries has required using sophisticated methods and instruments of science. These include statistical methods, particle accelerators, and X-ray methods. Such methods extend our mind and generally make observing, experimenting, and testing hypotheses in science possible, doing so in new ways and ensure their replicability. This provides a new perspective to the scientific method—embedded in our sophisticated methods and instruments—and suggests that we need to reform and extend the way we view the scientific method and discovery process.

How can scientific progress be conceived best? Does science mainly undergo revolutionary paradigm shifts? Or is the evolution of science mainly cumulative? Understanding whether science advances through cumulative evolution or through paradigm shifts can influence how we approach scientific research, education and policy. The most influential and cited account of science was put forth in Thomas Kuhn’s seminal book The structure of scientific revolutions. Kuhn argues that science does not advance cumulatively but goes through fundamental paradigm changes in the theories of a scientific field. There is no consensus yet on this core question of the nature and advancement of science that has since been debated across science. Examining over 750 major scientific discoveries (all Nobel Prize and major non-Nobel Prize discoveries), we systematically test this fundamental question about scientific progress here. We find that three key measures of scientific progress—major discoveries, methods and fields—each demonstrate that science evolves cumulatively. First, we show that no major scientific methods or instruments used across fields (such as statistical methods, X-ray methods or chromatography) have been completely abandoned, i.e. subject to paradigm shifts. Second, no major scientific fields (such as biomedicine, chemistry or computer science) have been completely abandoned. Rather, they have all continuously expanded over time, often over centuries, accumulating extensive bodies of knowledge. Third, scientific discoveries including theoretical discoveries are also predominately cumulative, with only 1% of over 750 major discoveries having been abandoned. The continuity of science is most compellingly evidenced by our methods and instruments, which enable the creation of discoveries and fields. We thus offer here a new perspective and answer to this classic question in science and the philosophy and history of science by utilizing methods from statistics and empirical sciences.

How do we drive new knowledge and science? What are their present boundaries? And how can we improve science? We still do not understand these essential questions about science well, even though science is at the foundation of modern society. The field of science of science can provide answers to these foundational questions. The central challenge of the field is integrating the different empirical and theoretical knowledge across disciplines into a holistic field and uncovering the general mechanism driving science across fields. This is the first book to offer an integrated framework for the science of science and thus aims to provide a comprehensive understanding of the foundations and limits of science. The book integrates 14 scientific fields and illustrates how our evolved mind (that enables us to observe, experiment and solve problems) makes doing science possible but also shapes what and how we observe. Our scientific methods and instruments (such as statistics and telescopes) enable us to study a much larger range of phenomena but also have constraints to how we measure them. Institutions and funding shape what knowledge we produce and how we evaluate our evidence, among other influences. By integrating the fields together, we are able to identify the common mechanism that underpins the different factors studied across all these fields: our powerful scientific methods and instruments. The book explains how the sophisticated scientific tools we develop are the main driving force for creating new knowledge and advancing science. This methodological toolbox of ours sets the scope and present limits of what we can know and what is possible in science—while economic, social and historical influences help shape what we study within that scope and those limits. The book offers a unifying theory for the field of science of science—the new-methods-drive-science theory. By better understanding the foundations of science we will also show how we can reduce the constraints and biases that we and our scientific methods and instruments face to advance science and push its present boundaries. This book is written in an easily accessible way for readers interested in understanding how science works.

This chapter proposes a novel approach to addressing irreplicability encountered in randomised controlled trials (RCTs). Irreplicability affects, to different degrees, all fields of science. Some of the most common explanations for non-replicability are low sample size, low statistical power, p-hacking, publication bias and HARKing. Such issues face some studies but not others. Focusing on RCTs, this chapter highlights that, while it is well-recognised that many types of bias and other types of constraint generally face studies across the biomedical, behavioural and social sciences, they are not commonly discussed collectively. In response to this gap in the literature on RCTs, we shall consider a novel approach to addressing non-replicability. The chapter helps explain how a degree of inevitable variation between the outcomes of different studies is driven by a combined set of biases and constraints underlying the scientific process that relate to methods, people, contexts, and time points at which data are collected in studies. It outlines how causal evidence in such studies is inferred from estimated statistical results that are the outcome of multiple complex processes: sampling, randomising, blinding, controlling, delivering treatments, and so on. These processes involve many actors making many decisions at different steps when designing, implementing and analysing studies within a particular context and time period. Correspondingly, the chapter considers the influence of the combined set of constraints on the quality and replicability of RCTs, and how to better tackle this issue. This entails integrating comparability of groups, treatments, contexts, etc. within a single study—called here a multi-study. Implications are then drawn for practitioners.

Drawing on the statistical and philosophical expertise of its authors, this book is designed to improve understanding and use of randomised controlled trials (RCTs) among health professionals. It is intended for use primarily by medical educators involved in teaching statistics and evidence-based medicine (EBM) to medical students, junior doctors and other health professionals. However, each of the chapters serves a wider range of interests, including the practical needs of physicians in interpreting research evidence to support clinical decision making and the teaching needs of philosophers of medicine who want to more fully appreciate how RCTs work in practice and provide engaging examples for their students. Rather than compete with the proliferating methodological literature on RCT designs, this book focuses on cultivating a healthy skepticism among developing health professionals to support critical appraisal of their own and published work on RCTs at a fundamental level, including through a more informed understanding of the place of subgroup analyses in sound statistical inference. Management of the positive predictive value in the statistical analysis of RCT findings is included as an important topic for contemporary medical curricula. In comparing RCTs with non-randomised studies, a search for empirical evidence for the superiority of RCTs is initiated, pointing to the need for further work to confirm what form this evidence should take. Medical educators will find a wealth of reasons to encourage their students to think more critically about how the RCT operates in practice as a gold standard.

Few things have impacted our lives as much as science and technology, but how we developed science and civilisation is one of the most challenging questions that has not yet been well explained. Attempting to identify the central driver, leading scientists have highlighted the role of culture, cooperation and geography. They focus thus on broad factors that are important basic preconditions but that we cannot directly influence. To better address the question, this paper integrates evidence from evolutionary biology, cognitive science, methodology, archaeology and anthropology. The paper identifies 9 main preconditions necessary for contemporary science, which include 6 main preconditions for civilisation. Using a kind of quasi-experimental research design we observe that some cultures (experimental groups) met the preconditions while other cultures (control groups) did not. Among the preconditions, we explain how our mind's evolved methodological abilities (to observe, solve problems and experiment) have directly enabled acquiring knowledge about the world and collectively developing increasingly sophisticated methods (such as mathematics and more systematic experimentation) that have enabled science and civilisation. We have driven the major revolutions throughout our history – the palaeolithic technological and agricultural revolutions and later the so-called scientific, industrial and digital revolutions – by using our methodological abilities in new ways and developing new methods and tools, i.e. through methodological revolutions. Viewing our methods as the main mechanism through which we have directly developed scientific and technological knowledge, and thus science and civilisation, provides a new framework for understanding science and the history of science. Viewing humans as homo methodologicus, using an expanding methodological toolbox, provides a nuanced explanation of how we have been directly able to meet our needs, solve problems and develop vast bodies of technological and scientific knowledge. By better understanding the origin and foundations of science, we can better understand their limits and, most importantly, how to push those limits. We can do so especially by addressing the evolved cognitive constraints and biases we face and improving the methods we use.

Biomedical research, especially pharmaceutical research, has been criticised for engaging in practices that lead to over-estimations of the effectiveness of medical treatments. A central issue concerns the reporting of absolute and relative measures of medical effectiveness. In this paper we critically examine proposals made by Jacob Stegenga to (a) give priority to the reporting of absolute measures over relative measures, and (b) downgrade the measures of effectiveness (effect sizes) of the treatments tested in clinical trials (Stegenga, 2015a). After exposing significant flaws in a central case study used by Stegenga to bolster his first proposal (a), we go on to argue that neither of these proposals is defensible (a or b). We defend the practice, in line with the New England Journal of Medicine, of reporting both absolute and relative measures whenever feasible.

Background: Randomised controlled trials (RCTs) are commonly viewed as the best research method to inform public health and social policy. Usually they are thought of as providing the most rigorous evidence of a treatment’s effectiveness without strong assumptions, biases and limitations. Objective: This is the first study to examine that hypothesis by assessing the 10 most cited RCT studies worldwide. Data sources: These 10 RCT studies with the highest number of citations in any journal were identified by searching Scopus (the largest database of peer-reviewed journals). Results: This study shows that these world-leading RCTs that have influenced policy produce biased results by illustrating that participants’ background traits that affect outcomes are often poorly distributed between trial groups, that the trials often neglect alternative factors contributing to their main reported outcome and, among many other issues, that the trials are often only partially blinded or unblinded. The study here also identifies a number of novel and important assumptions, biases and limitations not yet thoroughly discussed in existing studies that arise when designing, implementing and analysing trials. Conclusions: Researchers and policymakers need to become better aware of the broader set of assumptions, biases and limitations in trials. Journals need to also begin requiring researchers to outline them in their studies. We need to furthermore better use RCTs together with other research methods.

Climate change, nutrition, poverty and medical drugs are widely discussed and pressing issues in science, policy and society. Despite these issues being of great importance for the quality of our lives it remains unclear how well people understand them. Specifically, do particular demographic and socioeconomic factors explain variation in public understanding of these four concepts? To what extent are people’s changes in understanding associated with changes in their behaviour? Do people judge scientific practices relying on the more descriptive concepts of climate change and effective medical drugs to be more objective (less controversial) than practices relying on the more value-laden concepts of poverty and healthy nutrition? To address these questions, an experimental survey and regression analyses are conducted using data collected from about one thousand participants across different continents. The study finds that public understanding of science is generally low. A smaller proportion of people were able to correctly identify the common explanation accepted internationally among the scientific community for climate change and effectiveness of medical drugs (42% and 43% of participants in the study, respectively) than for poverty and healthy nutrition (61% and 65% of participants, respectively). Older age and political non-conservativeness were the strongest predictors of correctly understanding these four concepts. Greater levels of education and political non-conservativeness were in turn the strongest predictors of people’s reported changes in their behaviour based on their improved understanding of these concepts. Because climate change is among the least understood scientific concepts but is arguably the greatest challenge of our time, better efforts are needed to improve how media, awareness campaigns and education systems mediate information on the topic in order to tackle the large knowledge deficits that constrain behavioural change.

Many scientists routinely generalize from study samples to larger populations. It is commonly assumed that this cognitive process of scientific induction is a voluntary inference in which researchers assess the generalizability of their data and then draw conclusions accordingly. We challenge this view and argue for a novel account. The account describes scientific induction as involving by default a generalization bias that operates automatically and frequently leads researchers to unintentionally generalize their findings without sufficient evidence. The result is unwarranted, overgeneralized conclusions. We support this account of scientific induction by integrating a range of disparate findings from across the cognitive sciences that have until now not been connected to research on the nature of scientific induction. The view that scientific induction involves by default a generalization bias calls for a revision of the current thinking about scientific induction and highlights an overlooked cause of the replication crisis in the sciences. Commonly proposed interventions to tackle scientific overgeneralizations that may feed into this crisis need to be supplemented with cognitive debiasing strategies against generalization bias to most effectively improve science.

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed.

This paper outlines the methodological and empirical limitations of analysing the potential relationship between complex social phenomena such as democracy and inequality. It shows that the means to assess how they may be related is much more limited than recognised in the existing literature that is laden with contradictory hypotheses and findings. Better understanding our scientific limitations in studying this potential relationship is important for research and policy because many leading economists and other social scientists such as Acemoglu and Robinson mistakenly claim to identify causal linkages between inequality and democracy but at times still inform policy. In contrast to the existing literature, the paper argues that ‘structural’ or ‘causal’ mechanisms that may potentially link the distribution of economic wealth and different political regimes will remain unknown given reasons such as their highly complex and idiosyncratic characteristics, fundamental econometric constraints and analysis at the macro-level. Neither new data sources, different analysed time periods nor new data analysis techniques can resolve this question and provide robust, general conclusions about this potential relationship across countries. Researchers are thus restricted to exploring rough correlations over specific time periods and geographic contexts with imperfect data that are very limited for cross-country comparisons.