How should we quantify the strength of relationships between variables?

What is it about?

The probably most widely used statistical measure of the strength of a relationship between variables is the explained variance, also called R-squared. This measure ranges between 0 and 1 (under some common assumptions), and is often interpreted as a percentage to which the variation in one variable can be explained by the variation in another variable, or by a model of other variables. If R-squared equals 0.10 when height is used to predict weight, say, then one would often say that the variation in weight can to 10% be explained by the variation in height. One common, but improper, way to summarize this is therefore to say that for each 1 kg variation in weight explained by height, another 9 kg variation in weight remains unexplained. However, this 10% figure is only true for the variance of the variable, which measures squared variation, rather than variation on the original scale. As an example, if two people weigh 70 and 80 kg, then the difference in kg is 10, but the squared difference in kg is 100. The latter number is the basis for calculations of R-squared. A truer summary would therefore be that for each 1 squared kg variation explained, another 9 squared kg variation remains unexplained, or that for each 1 kg variation explained, another 3 kg variation remains unexplained. Interest often lies in explaining variation on the original scale of an examined variable. It is, for example, often of greater interest how many kg more or less different people weigh in the example above, rather than how many squared kg. Therefore, three different coefficients that capture relative variation on the original scale of variables are presented in this article, as substitutes of R-squared. The third and new measure summarizes variation in the manner above: if for each 1 kg variation in weight explained by height, another 3 kg variation remains unexplained, then this measure equals 1 / (1 + 3) = 0.25, meaning 25% of the variation in weight is explained in this way. It is argued that this often provides a more applicable and intuitive measure of the strength of an association compared to R-squared.

Featured Image

Photo by Julien Photo on Unsplash

Photo by Julien Photo on Unsplash

Why is it important?

It is not straightforward how the strength of a relationship should be measured. However, R-squareds ubiquity means its percentage explanation is easily misapplied, as in the example above about how much variation that is and is not captured. Misunderstanding of statistical measures are very common, and may lead to improper interpretations of the results of scientific studies. The measures in this article help combat such misunderstandings around R-squared, by introducing alternative measures that capture useful properties of results, that readers may otherwise mistakenly believe are captured by R-squared.