Ina Jäntgen

We often want to choose the intervention that most effectively achieves our goals. But what are the most effective interventions? Here, we often turn to advice from scientists who report the effectiveness of tested interventions, using the mean difference: the difference in expected outcomes given a tested intervention and given a control intervention. I identify a problem with this practice: even perfectly accurate mean differences omit information about how interventions change the probability of an outcome of interest, and this omitted information can be vital for decision-makers to choose the best intervention—the one they would rationally choose if they were to learn all the information available in studies about how the interventions change the probability distribution over an outcome of interest. These interventions’ mean differences omit some of this information and, in doing so, may fail to inform the decision-maker sufficiently to identify the best intervention. That is the bad news. The good news is: I identify sufficient conditions for mean differences to inform rational agents sufficiently (specifically, those agents abiding by expected utility theory or risk-weighted expected utility theory). Based on my results, I advise how researchers and decision-makers can use mean differences to facilitate rational decision-making.

Absolute and relative outcome measures measure a treatment’s effect size, purporting to inform treatment choices. I argue that absolute measures are at least as good as, if not better than, relative ones for informing rational decisions across choice scenarios. Specifically, this dominance of absolute measures holds for choices between a treatment and a control group treatment from a trial and for ones between treatments tested in different trials. This distinction has hitherto been neglected, just like the role of absolute and baseline risks in rational decision-making that my analysis reveals. Recognizing both aspects advances the discussion on reporting outcome measures.