Each week, we select a recently published Open Access article to feature. This week’s article comes from Statistica Neerlandica and studies a statistical framework for replicability based on the sceptical p-value.

The article’s abstract is given below, with the full article available to read here.

Micheloud, C., Balabdaoui, F., & Held, L. (2023). Assessing replicability with the sceptical p-value: Type-I error control and sample size planning. Statistica Neerlandica, 1– 19. https://doi.org/10.1111/stan.12312

We study a statistical framework for replicability based on a recently proposed quantitative measure of replication success, the sceptical p-value. A recalibration is proposed to obtain exact overall Type-I error control if the effect is null in both studies and additional bounds on the partial and conditional Type-I error rate, which represent the case where only one study has a null effect. The approach avoids the double dichotomization for significance of the two-trials rule and has larger project power to detect existing effects over both studies in combination. It can also be used for power calculations and requires a smaller replication sample size than the two-trials rule for already convincing original studies. We illustrate the performance of the proposed methodology in an application to data from the Experimental Economics Replication Project.