A new relevance measure to replace the p-value

What is it about?

Empirical research usually explores the effect(s) of one or more "input" variables--quanitative or qualitative--on a "target" variable, e.g., the effect of a medication on a disease symptom, or of a person's social characteristics on income. The common way of reporting the result focusses on the p-value of a test for the null hypothesis that there is no effect. This practice has been widely criticized, culminating in an official statement of the American Statistical Association on the limited usefulness of the p-value. A basic caveat is the fact that even negligibly small effects can be made statistically significant by generating a large enough number of observations. A reasonable scientific question about an effect asks if it is relevant in practice rather than just testing if it is non-zero. This insight leads to the necessity of specifying a threshold of relevance. This paper introduces a simple quantitative measure of relevance, for which the threshold is one. It then develops a way to summarize the result of a quanitative study in a simple and easily interpretable fashion, thereby answering the question about the relevance of the effect. The concept is developed for the most commonly used statistical models.

Featured Image

Photo by Myriam Jessier on Unsplash

Photo by Myriam Jessier on Unsplash

Why is it important?

Even though the common practice of summarizing the result of an empirical stuy by the p-value of a test for zero effect has been depricated, alternative ways of analyzing data have been scarce. This paper introduces a straightforward concept and ready-to-use method to answer the scientifically meaningful question: "Is the effect under study of practical relevance?"

Perspectives