A Variational Spike-and-Slab Approach for Group Variable Selection

Abstract

We propose a parameter-expanded Coordinate-Ascent Variational Inference algorithm for high-dimensional linear regression models with grouped variables under spike-and-slab priors. This approach provides a variational Bayes approximation that is computationally more efficient than classical methods. Theoretically, we establish conditions for a more general class of spike-and-slab priors under which contraction rates can be derived for both the true posterior distribution and its variational approximation. Several existing theoretical results can be viewed as special cases of this framework. The methodology and theoretical results are further extended to additive models. Simulation studies and real-world applications demonstrate the proposed method’s superiority over existing approaches in both variable selection and parameter estimation. The implementation can be found at https://github.com/HowardGech/GVSSB.

Changhao Ge

PhD candidate of Applied Math

My research interests include Statistical Algorithms, Bayesian Statistics, Machine Learning, Computation and Optimization