Home > Research > Publications & Outputs > A permutation test for assessing the presence o...

Electronic data

  • Permutation final

    Rights statement: The final, definitive version of this article has been published in the Journal, Statistical Methods in Medical Research, 30 (11), 2021, © SAGE Publications Ltd, 2021 by SAGE Publications Ltd at the Statistical Methods in Medical Research page: https://journals.sagepub.com/doi/10.1177/09622802211033640 on SAGE Journals Online: http://journals.sagepub.com/

    Accepted author manuscript, 621 KB, PDF document

    Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License

Links

Text available via DOI:

View graph of relations

A permutation test for assessing the presence of individual differences in treatment effects

Research output: Contribution to journalJournal articlepeer-review

Published
  • C. Chang
  • T. Jaki
  • M.S. Sadiq
  • A. Kuhlemeier
  • D. Feaster
  • N. Cole
  • A. Lamont
  • D. Oberski
  • Y. Desai
  • M. Lee Van Horn
Close
<mark>Journal publication date</mark>1/11/2021
<mark>Journal</mark>Statistical Methods in Medical Research
Issue number11
Volume30
Number of pages13
Pages (from-to)2369-2381
Publication StatusPublished
Early online date27/09/21
<mark>Original language</mark>English

Abstract

An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.

Bibliographic note

The final, definitive version of this article has been published in the Journal, Statistical Methods in Medical Research, 30 (11), 2021, © SAGE Publications Ltd, 2021 by SAGE Publications Ltd at the Statistical Methods in Medical Research page: https://journals.sagepub.com/doi/10.1177/09622802211033640 on SAGE Journals Online: http://journals.sagepub.com/