Home > Research > Publications & Outputs > A Framework for Monte Carlo based Multiple Testing

Links

Text available via DOI:

View graph of relations

A Framework for Monte Carlo based Multiple Testing

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

A Framework for Monte Carlo based Multiple Testing. / Gandy, Axel; Hahn, Georg.

In: Scandinavian Journal of Statistics, Vol. 43, No. 4, 01.12.2016, p. 1046-1063.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

Gandy, A & Hahn, G 2016, 'A Framework for Monte Carlo based Multiple Testing', Scandinavian Journal of Statistics, vol. 43, no. 4, pp. 1046-1063. https://doi.org/10.1111/sjos.12228

APA

Gandy, A., & Hahn, G. (2016). A Framework for Monte Carlo based Multiple Testing. Scandinavian Journal of Statistics, 43(4), 1046-1063. https://doi.org/10.1111/sjos.12228

Vancouver

Gandy A, Hahn G. A Framework for Monte Carlo based Multiple Testing. Scandinavian Journal of Statistics. 2016 Dec 1;43(4):1046-1063. https://doi.org/10.1111/sjos.12228

Author

Gandy, Axel ; Hahn, Georg. / A Framework for Monte Carlo based Multiple Testing. In: Scandinavian Journal of Statistics. 2016 ; Vol. 43, No. 4. pp. 1046-1063.

Bibtex

@article{35aaf9a611144ec6b0fa92a67ab6e29e,
title = "A Framework for Monte Carlo based Multiple Testing",
abstract = "We are concerned with a situation in which we would like to test multiple hypotheses with tests whose p-values cannot be computed explicitly but can be approximated using Monte Carlo simulation. This scenario occurs widely in practice. We are interested in obtaining the same rejections and non-rejections as the ones obtained if the p-values for all hypotheses had been available. The present article introduces a framework for this scenario by providing a generic algorithm for a general multiple testing procedure. We establish conditions that guarantee that the rejections and non-rejections obtained through Monte Carlo simulations are identical to the ones obtained with the p-values. Our framework is applicable to a general class of step-up and step-down procedures, which includes many established multiple testing corrections such as the ones of Bonferroni, Holm, Sidak, Hochberg or Benjamini–Hochberg. Moreover, we show how to use our framework to improve algorithms available in the literature in such a way as to yield theoretical guarantees on their results. These modifications can easily be implemented in practice and lead to a particular way of reporting multiple testing results as three sets together with an error bound on their correctness, demonstrated exemplarily using a real biological dataset.",
keywords = "algorithm, framework, hypothesis testing, Monte Carlo, multiple testing procedure, p-value",
author = "Axel Gandy and Georg Hahn",
year = "2016",
month = dec,
day = "1",
doi = "10.1111/sjos.12228",
language = "English",
volume = "43",
pages = "1046--1063",
journal = "Scandinavian Journal of Statistics",
issn = "0303-6898",
publisher = "Blackwell-Wiley",
number = "4",

}

RIS

TY - JOUR

T1 - A Framework for Monte Carlo based Multiple Testing

AU - Gandy, Axel

AU - Hahn, Georg

PY - 2016/12/1

Y1 - 2016/12/1

N2 - We are concerned with a situation in which we would like to test multiple hypotheses with tests whose p-values cannot be computed explicitly but can be approximated using Monte Carlo simulation. This scenario occurs widely in practice. We are interested in obtaining the same rejections and non-rejections as the ones obtained if the p-values for all hypotheses had been available. The present article introduces a framework for this scenario by providing a generic algorithm for a general multiple testing procedure. We establish conditions that guarantee that the rejections and non-rejections obtained through Monte Carlo simulations are identical to the ones obtained with the p-values. Our framework is applicable to a general class of step-up and step-down procedures, which includes many established multiple testing corrections such as the ones of Bonferroni, Holm, Sidak, Hochberg or Benjamini–Hochberg. Moreover, we show how to use our framework to improve algorithms available in the literature in such a way as to yield theoretical guarantees on their results. These modifications can easily be implemented in practice and lead to a particular way of reporting multiple testing results as three sets together with an error bound on their correctness, demonstrated exemplarily using a real biological dataset.

AB - We are concerned with a situation in which we would like to test multiple hypotheses with tests whose p-values cannot be computed explicitly but can be approximated using Monte Carlo simulation. This scenario occurs widely in practice. We are interested in obtaining the same rejections and non-rejections as the ones obtained if the p-values for all hypotheses had been available. The present article introduces a framework for this scenario by providing a generic algorithm for a general multiple testing procedure. We establish conditions that guarantee that the rejections and non-rejections obtained through Monte Carlo simulations are identical to the ones obtained with the p-values. Our framework is applicable to a general class of step-up and step-down procedures, which includes many established multiple testing corrections such as the ones of Bonferroni, Holm, Sidak, Hochberg or Benjamini–Hochberg. Moreover, we show how to use our framework to improve algorithms available in the literature in such a way as to yield theoretical guarantees on their results. These modifications can easily be implemented in practice and lead to a particular way of reporting multiple testing results as three sets together with an error bound on their correctness, demonstrated exemplarily using a real biological dataset.

KW - algorithm

KW - framework

KW - hypothesis testing

KW - Monte Carlo

KW - multiple testing procedure

KW - p-value

U2 - 10.1111/sjos.12228

DO - 10.1111/sjos.12228

M3 - Journal article

VL - 43

SP - 1046

EP - 1063

JO - Scandinavian Journal of Statistics

JF - Scandinavian Journal of Statistics

SN - 0303-6898

IS - 4

ER -