Methods for estimating how confident we should be about statistical hypotheses

What is it about?

A research study has found that a sample of Oscar winners lived 3.9 years longer than a sample of other performers, suggesting that public recognition has the power to extend life. However conclusions based on a limited sample may not give an accurate picture of the overall situation. The conventional statistical approach to this issue is to cite a p value (or significance level): in this case the authors of the research study state that p=0.003. However p values are frequently misunderstood, and do not not tell us how likely it is that a hypothesis is true. This article proposes using confidence levels, which can be regarded tentative probabilities, instead: based on the same data our confidence that Oscar winners do, on average, live longer is 99.85%, and our confidence that they live more than a year longer is 98.6%. This is far more useful and easily interpreted than the p value.

Featured Image

Photo by Edge2Edge Media on Unsplash

Photo by Edge2Edge Media on Unsplash

Why is it important?

Null hypothesis significance tests, and their associated p values, have been the target of strong criticism for more than 50 years, but they are still very widely used, and in some fields almost a requirement for getting research published. This has serious implications for both the communication of research (people are likely to jump to erroneous conclusions), and the progress of research itself. The proposed alternative to p values is a simple extension of the well established idea of confidence intervals. It would avoid the problems of p values while acknowledging the tentative nature of any conclusions in this area.