Rights statement: © Springer Science+Business Media New York 2016. The final publication is available at Springer via http://dx.doi.org/10.1007/s11222-016-9656-z
Accepted author manuscript, 141 KB, PDF document
Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License
Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - QuickMMCTest
T2 - quick multiple Monte Carlo testing
AU - Gandy, Axel
AU - Hahn, Georg
N1 - © Springer Science+Business Media New York 2016. The final publication is available at Springer via http://dx.doi.org/10.1007/s11222-016-9656-z
PY - 2017/5/4
Y1 - 2017/5/4
N2 - Multiple hypothesis testing is widely used to evaluate scientific studies involving statistical tests. However, for many of these tests, p values are not available and are thus often approximated using Monte Carlo tests such as permutation tests or bootstrap tests. This article presents a simple algorithm based on Thompson Sampling to test multiple hypotheses. It works with arbitrary multiple testing procedures, in particular with step-up and step-down procedures. Its main feature is to sequentially allocate Monte Carlo effort, generating more Monte Carlo samples for tests whose decisions are so far less certain. A simulation study demonstrates that for a low computational effort, the new approach yields a higher power and a higher degree of reproducibility of its results than previously suggested methods.
AB - Multiple hypothesis testing is widely used to evaluate scientific studies involving statistical tests. However, for many of these tests, p values are not available and are thus often approximated using Monte Carlo tests such as permutation tests or bootstrap tests. This article presents a simple algorithm based on Thompson Sampling to test multiple hypotheses. It works with arbitrary multiple testing procedures, in particular with step-up and step-down procedures. Its main feature is to sequentially allocate Monte Carlo effort, generating more Monte Carlo samples for tests whose decisions are so far less certain. A simulation study demonstrates that for a low computational effort, the new approach yields a higher power and a higher degree of reproducibility of its results than previously suggested methods.
KW - Multiple hypothesis testing
KW - Monte Carlo
KW - Thompson sampling
KW - Bonferroni correction
KW - Benjamini-Hochberg procedure
KW - FALSE DISCOVERY RATE
KW - BONFERRONI PROCEDURE
KW - ASSOCIATION
KW - VALUES
U2 - 10.1007/s11222-016-9656-z
DO - 10.1007/s11222-016-9656-z
M3 - Journal article
VL - 27
SP - 823
EP - 832
JO - Statistics and Computing
JF - Statistics and Computing
SN - 0960-3174
IS - 3
ER -