Testing statistical hypothesis methods under difficult conditions

What is it about?

Several common statistical hypothesis tests are applied to simulated series with varying degrees (and types) of temporal dependence and contemporaneous correlation. It is shown that the Hering and Genton (HG) test is the most accurate in terms of type-I errors, and is also a powerful test. The circular block bootstrap approach, while not as accurate as the HG test, is generally reasonably accurate when the series are temporally dependent, and the iid bootstrap when they are independent; like the HG test, both are robust to contemporaneous correlation. Other tests, such as the usual t-test and normal approximation, even when a variance-inflation factor is applied, are not robust to the contemporaneous correlation.

Featured Image

Photo by Nancy Cantwell on Unsplash

Photo by Nancy Cantwell on Unsplash

Why is it important?

In weather forecast verification, the it is fairly uncommon to account for dependence, especially contemporaneous correlation, when testing whether one forecast model is better than another on average. Sometimes the variance-inflation factor (VIF) is used, but in this case, usually only the AR(1) version is used, and it can be very inaccurate if the series does not follow an AR(1) process.