What is it about?
We critique the widespread use of multiple hypothesis test corrections in scientific research, arguing that these methods fail to control false positives effectively in real-world studies. The actual number of false positives often exceeds the intended threshold due to issues such as arbitrary definitions of test groups, flexibility in error rate control, and unchecked exploratory analyses. We propose to move away from p-value adjustments toward shrinking parameter estimates. This approach directly targets exaggerated effect sizes that lead to false positives, providing a more robust solution for controlling error rates. Shrinkage methods, while less common and requiring more effort, offer advantages in reducing biases and are better suited for real-world complexities.
Featured Image
Photo by Bozhin Karaivanov on Unsplash
Why is it important?
We addresses a foundational problem in scientific research: the reliability and reproducibility of findings when multiple hypotheses are tested.
Perspectives
One reason for writing this paper was to encourage quantitative researchers to investigate the usefulness of shrinking parameter estimates instead of adjusting p-values as a way to control false positives.
Stanley Lazic
Read the Original
This page is a summary of: Why multiple hypothesis test corrections provide poor control of false positives in the real world., Psychological Methods, November 2024, American Psychological Association (APA),
DOI: 10.1037/met0000678.
You can read the full text:
Resources
Contributors
The following have contributed to this page
