Final published version
Licence: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - A Good-Turing estimator for feature allocation models
AU - Ayed, Fadhel
AU - Battiston, Marco
AU - Camerlenghi, Federico
AU - Favaro, Stefano
PY - 2019/10/1
Y1 - 2019/10/1
N2 - Feature allocation models generalize classical species sampling models by allowing every observation to belong to more than one species, now called features. Under the popular Bernoulli product model for feature allocation, we assume n observable samples and we consider the problem of estimating the expected number Mn of hitherto unseen features that would be observed if one additional individual was sampled. The interest in estimating Mn is motivated by numerous applied problems where the sampling procedure is expensive, in terms of time and/or financial resources allocated, and further samples can be only motivated by the possibility of recording new unobserved features. We consider a nonparametric estimator M^n of Mn which has the same analytic form of the popular Good-Turing estimator of the missing mass in the context of species sampling models. We show that M^n admits a natural interpretation both as a jackknife estimator and as a nonparametric empirical Bayes estimator. Furthermore, we give provable guarantees for the performance of M^n in terms of minimax rate optimality, and we provide with an interesting connection between M^n and the Good-Turing estimator for species sampling. Finally, we derive non-asymptotic confidence intervals for M^n, which are easily computable and do not rely on any asymptotic approximation. Our approach is illustrated with synthetic data and SNP data from the ENCODE sequencing genome project.
AB - Feature allocation models generalize classical species sampling models by allowing every observation to belong to more than one species, now called features. Under the popular Bernoulli product model for feature allocation, we assume n observable samples and we consider the problem of estimating the expected number Mn of hitherto unseen features that would be observed if one additional individual was sampled. The interest in estimating Mn is motivated by numerous applied problems where the sampling procedure is expensive, in terms of time and/or financial resources allocated, and further samples can be only motivated by the possibility of recording new unobserved features. We consider a nonparametric estimator M^n of Mn which has the same analytic form of the popular Good-Turing estimator of the missing mass in the context of species sampling models. We show that M^n admits a natural interpretation both as a jackknife estimator and as a nonparametric empirical Bayes estimator. Furthermore, we give provable guarantees for the performance of M^n in terms of minimax rate optimality, and we provide with an interesting connection between M^n and the Good-Turing estimator for species sampling. Finally, we derive non-asymptotic confidence intervals for M^n, which are easily computable and do not rely on any asymptotic approximation. Our approach is illustrated with synthetic data and SNP data from the ENCODE sequencing genome project.
U2 - 10.1214/19-EJS1614
DO - 10.1214/19-EJS1614
M3 - Journal article
VL - 13
SP - 3775
EP - 3804
JO - Electronic Journal of Statistics
JF - Electronic Journal of Statistics
SN - 1935-7524
IS - 2
ER -