Estimating the local false discovery rate via a bootstrap solution to the reference class problem

Abstract

The local false discovery rate (LFDR) can be utilized as a statistical approach for simultaneously analyzing thousands of tests. We present a model for multiple hypothesis testing that incorporates a covariate into each test. Incorporating the covariates may improve the performance of testing procedures, because each covariate contains additional information based on the scientific context of the corresponding test. This method provides different LFDR estimates depending on a tuning parameter that determines a reference class of hypotheses from the covariate. We estimate the optimal value of that parameter by choosing the one that minimizes the estimated LFDR resulting from bias and the variance in a bootstrap approach. Such an estimation method is called an adaptive reference class (ARC) method since the class of hypotheses depends on the data. We apply this method to brain data to detect dyslexic-non-dyslexic difference voxels.

We prove a result for the asymptotic performance of the ARC method under certain assumptions concerning the prior probability of each hypothesis test as a function of the covariate and the LFDR estimator. For finite numbers of hypotheses, we use simulation data to evaluate the performance of the estimator associated with the ARC method. The simulations assuming a large covariate effect indicate that the LFDR estimator has a smaller mean squared error under the ARC method than that under the method that uses the entire set of hypotheses without regard for the covariate.