In empirical research in the social sciences expressions of statistical significance are meant to capture and summarise the evidence implied by data. To evaluate the evidential content of statements such as ‘the difference between means is significant at α=5%’, we consider the Bayesian probability of the hypotheses tested, where the conditioning event is an announcement of general formsignificant at α. By proceeding as if neither observed effects nor their exactP-values are reported, the meaning…
Read moreIn empirical research in the social sciences expressions of statistical significance are meant to capture and summarise the evidence implied by data. To evaluate the evidential content of statements such as ‘the difference between means is significant at α=5%’, we consider the Bayesian probability of the hypotheses tested, where the conditioning event is an announcement of general formsignificant at α. By proceeding as if neither observed effects nor their exactP-values are reported, the meaning of such descriptionsof themselves is revealed. It is demonstrated, for large samples particularly, that a report merely that data aresignificant at α has no objective meaning, and under some conditions should be interpreted not as evidence against the null hypothesis, as is usually supposed, but as strong evidence in its favor. This conclusion is supported by both algebraic arguments and example calculations for the special, but important case of the normal mean. It is also found that significance at one level tends to imply significance at much lower levels, the more strongly the larger the sample.
