Estimating credibility and true quantity discernibility

In this study, we depart from common statistical practice in one important way that we hope improves our scientific communication. The term “significant” is largely misunderstood as meaning “scientifically, clinically, or practically important.” Statistical significance is unrelated to scientific significance but has been ubiquitously misunderstood as meaning “significant.” This well-documented and long-standing phenomenon is called the “significance fallacy,”^72–75 a key contributor to the replication crisis in biomedicine and psychology. Hence, leading statistical authorities have strongly recommended abandoning the phrase “statistical significance” entirely,^76–79 necessitating a search for another phrase to describe the amount of statistical evidence in research findings.

vInstead, we will proceed as follows. We will continue to describe a P value in terms of its “statistical significance.” However, we will describe its corresponding estimate in terms of “statistical discernibility.” For example, if an estimated effect of 2 has a P value of .001, we might describe the estimate as being statistically discernible for the true, unknown effect (or simply say the estimate is “discernible”). That is, there is sufficient statistical evidence that 2 is a statistically valid estimate of the true, unknown value. If that estimate of 2 had a P value of .83, we might say 2 is not statistically discernible as the true effect ⁸⁰ is an example of a publication that successfully used this lexical strategy.

Our hope in taking this approach is to avoid committing the common error of making scientifically unsupported claims (i.e. based on the statistical qualities of an estimate, rather than on the size and direction of the estimate itself). For example, we might incorrectly claim that “there was a significant effect of getting more sleep on step count the next day, i.e. more sleep causes an increase of 2 steps (p  =  .001)”, when in fact the true finding is, “there was a discernible effect of getting more sleep on step count the next day; i.e. more sleep causes a credible increase of 2 steps (p  =  .001), but this small increase may not be practically significant.”