Each week, we publish layman’s abstracts of new articles from our prestigious portfolio of journals in statistics. The aim is to highlight the latest research to a broader audience in an accessible format.The article featured today is from Stat with the full article now available to read here.
, & (2022). Confidence intervals that utilize sparsity. Stat, 11( 1), e434. https://doi.org/10.1002/sta4.434
The sparsity assumption is that many of the coefficients in a model are zero, without any knowledge of which particular coefficients are zero. Statistical inference that successfully utilizes sparsity is a hot topic. Much has been written about penalized maximum likelihood point estimators, such as hard-thresholding, LASSO and SCAD estimators, which successfully utilize sparsity. However, for fully informative inference, confidence intervals that successfully utilize sparsity are required. Far less is known about such confidence intervals. In fact, finding such confidence intervals is particularly difficult in view of the following negative result. It is known that confidence intervals, with the desired minimum coverage probability and centered on penalized maximum likelihood estimators, necessarily have very unattractive expected length properties. The authors consider a linear regression model with orthogonal regressors and Gaussian random errors with known variance, in the low-dimensional setting that the length of the regression parameter vector does not exceed the length of the response vector. For the first time, confidence intervals are described that have the desired coverage probability and attractive expected length properties, particularly when sparsity holds. In fact, these confidence intervals dominate the usual confidence intervals with the same coverage, whenever the degree of sparsity exceeds a known value.