Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - MMCTest-A safe algorithm for implementing multiple monte carlo tests
AU - Gandy, Axel
AU - Hahn, Georg
PY - 2014/12/1
Y1 - 2014/12/1
N2 - Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest, a sequential algorithm that gives, with arbitrarily high probability, the same classification as a specific multiple testing procedure applied to ideal p-values. The method can be used with a class of multiple testing procedures that include the Benjamini and Hochberg false discovery rate procedure and the Bonferroni correction controlling the familywise error rate. One of the key features of the algorithm is that it stops sampling for all the hypotheses that can already be decided as being rejected or non-rejected. MMCTest can be interrupted at any stage and then returns three sets of hypotheses: the rejected, the non-rejected and the undecided hypotheses. A simulation study motivated by actual biological data shows that MMCTest is usable in practice and that, despite the additional guarantee, it can be computationally more efficient than other methods.
AB - Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest, a sequential algorithm that gives, with arbitrarily high probability, the same classification as a specific multiple testing procedure applied to ideal p-values. The method can be used with a class of multiple testing procedures that include the Benjamini and Hochberg false discovery rate procedure and the Bonferroni correction controlling the familywise error rate. One of the key features of the algorithm is that it stops sampling for all the hypotheses that can already be decided as being rejected or non-rejected. MMCTest can be interrupted at any stage and then returns three sets of hypotheses: the rejected, the non-rejected and the undecided hypotheses. A simulation study motivated by actual biological data shows that MMCTest is usable in practice and that, despite the additional guarantee, it can be computationally more efficient than other methods.
KW - Benjamini-Hochberg
KW - Bonferroni correction
KW - Bootstrap
KW - False discovery rate
KW - Multiple comparisons
KW - Resampling
KW - Sequential algorithm
U2 - 10.1111/sjos.12085
DO - 10.1111/sjos.12085
M3 - Journal article
AN - SCOPUS:84920367101
VL - 41
SP - 1083
EP - 1101
JO - Scandinavian Journal of Statistics
JF - Scandinavian Journal of Statistics
SN - 0303-6898
IS - 4
ER -