Home > Research > Publications & Outputs > A permutation test for assessing the presence o...

Electronic data

  • Permutation final

    Rights statement: The final, definitive version of this article has been published in the Journal, Statistical Methods in Medical Research, 30 (11), 2021, © SAGE Publications Ltd, 2021 by SAGE Publications Ltd at the Statistical Methods in Medical Research page: https://journals.sagepub.com/doi/10.1177/09622802211033640 on SAGE Journals Online: http://journals.sagepub.com/

    Accepted author manuscript, 621 KB, PDF document

    Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License

Links

Text available via DOI:

View graph of relations

A permutation test for assessing the presence of individual differences in treatment effects

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

A permutation test for assessing the presence of individual differences in treatment effects. / Chang, C.; Jaki, T.; Sadiq, M.S. et al.
In: Statistical Methods in Medical Research, Vol. 30, No. 11, 01.11.2021, p. 2369-2381.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

Chang, C, Jaki, T, Sadiq, MS, Kuhlemeier, A, Feaster, D, Cole, N, Lamont, A, Oberski, D, Desai, Y & Lee Van Horn, M 2021, 'A permutation test for assessing the presence of individual differences in treatment effects', Statistical Methods in Medical Research, vol. 30, no. 11, pp. 2369-2381. https://doi.org/10.1177/09622802211033640

APA

Chang, C., Jaki, T., Sadiq, M. S., Kuhlemeier, A., Feaster, D., Cole, N., Lamont, A., Oberski, D., Desai, Y., & Lee Van Horn, M. (2021). A permutation test for assessing the presence of individual differences in treatment effects. Statistical Methods in Medical Research, 30(11), 2369-2381. https://doi.org/10.1177/09622802211033640

Vancouver

Chang C, Jaki T, Sadiq MS, Kuhlemeier A, Feaster D, Cole N et al. A permutation test for assessing the presence of individual differences in treatment effects. Statistical Methods in Medical Research. 2021 Nov 1;30(11):2369-2381. Epub 2021 Sept 27. doi: 10.1177/09622802211033640

Author

Chang, C. ; Jaki, T. ; Sadiq, M.S. et al. / A permutation test for assessing the presence of individual differences in treatment effects. In: Statistical Methods in Medical Research. 2021 ; Vol. 30, No. 11. pp. 2369-2381.

Bibtex

@article{f77319bd4a9e4c5d9c964330b080541d,
title = "A permutation test for assessing the presence of individual differences in treatment effects",
abstract = "An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.",
keywords = "Predicted individual treatment effects, heterogeneity in treatment effects, personalized medicine, permutation test, Random Forests",
author = "C. Chang and T. Jaki and M.S. Sadiq and A. Kuhlemeier and D. Feaster and N. Cole and A. Lamont and D. Oberski and Y. Desai and {Lee Van Horn}, M.",
note = "The final, definitive version of this article has been published in the Journal, Statistical Methods in Medical Research, 30 (11), 2021, {\textcopyright} SAGE Publications Ltd, 2021 by SAGE Publications Ltd at the Statistical Methods in Medical Research page: https://journals.sagepub.com/doi/10.1177/09622802211033640 on SAGE Journals Online: http://journals.sagepub.com/",
year = "2021",
month = nov,
day = "1",
doi = "10.1177/09622802211033640",
language = "English",
volume = "30",
pages = "2369--2381",
journal = "Statistical Methods in Medical Research",
issn = "0962-2802",
publisher = "SAGE Publications Ltd",
number = "11",

}

RIS

TY - JOUR

T1 - A permutation test for assessing the presence of individual differences in treatment effects

AU - Chang, C.

AU - Jaki, T.

AU - Sadiq, M.S.

AU - Kuhlemeier, A.

AU - Feaster, D.

AU - Cole, N.

AU - Lamont, A.

AU - Oberski, D.

AU - Desai, Y.

AU - Lee Van Horn, M.

N1 - The final, definitive version of this article has been published in the Journal, Statistical Methods in Medical Research, 30 (11), 2021, © SAGE Publications Ltd, 2021 by SAGE Publications Ltd at the Statistical Methods in Medical Research page: https://journals.sagepub.com/doi/10.1177/09622802211033640 on SAGE Journals Online: http://journals.sagepub.com/

PY - 2021/11/1

Y1 - 2021/11/1

N2 - An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.

AB - An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.

KW - Predicted individual treatment effects

KW - heterogeneity in treatment effects

KW - personalized medicine

KW - permutation test

KW - Random Forests

U2 - 10.1177/09622802211033640

DO - 10.1177/09622802211033640

M3 - Journal article

VL - 30

SP - 2369

EP - 2381

JO - Statistical Methods in Medical Research

JF - Statistical Methods in Medical Research

SN - 0962-2802

IS - 11

ER -