Home > Research > Publications & Outputs > A simple method for implementing Monte Carlo tests

Keywords

View graph of relations

A simple method for implementing Monte Carlo tests

Research output: Contribution to Journal/MagazineJournal article

Published
Close
<mark>Journal publication date</mark>5/11/2016
<mark>Journal</mark>arxiv.org
Publication StatusPublished
<mark>Original language</mark>English

Abstract

We consider a statistical test whose p-value can only be approximated using Monte Carlo simulations. We are interested in deciding whether the p-value for an observed data set lies above or below a given threshold such as 5%. We want to ensure that the resampling risk, the probability of the (Monte Carlo) decision being different from the true decision, is uniformly bounded. This article introduces a simple method with this property, the confidence sequence method (CSM). We compare our approach to SIMCTEST (Gandy, 2009, JASA) which also guarantees a uniform bound on the resampling risk. A key advantage of CSM is its simplicity, which comes at the expense of being conservative and leads to seemingly uniformly larger stopping boundaries in CSM compared to SIMCTEST. Since the stopping boundaries in CSM are only marginally larger, we nevertheless conclude that the simplicity of CSM makes it a useful method for practical applications.