Scalable confidence intervals of precision matrices in high dimensions

doi:10.3969/j.issn.0253-2778.2020.06.006

Abstract

Abstract: In order to solve the problem of the computational inefficiency in confidence intervals of high-dimensional precision matrices, the De-SCIO was proposed. Compared with other methods, the computational efficiency of the confidence intervals based on De-SCIO statistic are greatly improved, and their average coverage is closer to the true level. The construction of the De-SCIO statistic is simple and avoids complicated theoretical derivation. Under reasonable assumptions, the asymptotic normality of the De-SCIO statistic was proved. The advantages of this method in average coverage and computational efficiency were demonstrated by the numerical studies and real data example.

Key words: precision matrix, confidence intervals, sparsity, de-sparsified statistic

CLC Number:

O212.2

ZHOU Huiting, ZHOU Jia, ZHENG Zemin. Scalable confidence intervals of precision matrices in high dimensions[J]. Journal of University of Science and Technology of China, 2020, 50(6): 752-757.

References

［1］
SCHFER J, STRIMMER K. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics[J]. Statistical Applications in Genetics and Molecular Biology, 2005, 4(1): Article 32.
[2] CHEN X, LIU Y, LIU H, et al. Learning spatial-temporal varying graphs with applications to climate data analysis[C]// Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence. Palo Alto, CA: Association for the Advancement of Artificial Intelligence, 2010: 425-430.
[3] FAN J, LIAO Y, LIU H. An overview of the estimation of large covariance and precision matrices [J]. Econometrics Journal, 2016, 19(1): C1-C32.
[4] WAINWRIGHT M J, JORDAN M I. Graphical Models, Exponential Families, and Variational Inference [M]. Hanover,MA: Now, 2008.
[5] LIU H, LAFFERTY J, WASSERMAN L. The nonparanormal: Semiparametric estimation of high dimensional undirected graphs [J]. Journal of Machine Learning Research, 2009, 10(3): 2295-2328.
[6] LAURITZEN S. Graphical Models [M]. New York: Oxford Univ Press, 1996.
[7] MEINSHAUSEN N, BHLMANN P. High-dimensional graphs and variable selection with the Lasso [J]. The Annals of Statistics, 2006, 34(3): 1436-1462.
[8] BICKEL P J, LEVINA E. Covariance regularization by thresholding[J]. The Annals of Statistics, 2008, 36(6): 2577-2604.
[9] RAVIKUMAR P, WAINWRIGHT M J, RASKUTTI G, et al. High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence[J]. Electronic Journal of Statistics, 2011, 5: 935-980.
[10] LIU W, LUO X. Fast and adaptive sparse precision matrix estimation in high dimensions[J]. Journal of Multivariate Analysis, 2015, 135:153-162.
[11] REN Z, SUN T, ZHANG C, et al. Asymptotic normality and optimalities in estimation of large Gaussian graphical models[J]. The Annals of Statistics, 2015, 43(3): 991-1026.
[12] FAN Y, LV J. Innovated scalable efficient estimation in ultra-large Gaussian graphical models[J]. The Annals of Statistics, 2016, 44(5): 2098-2126.
[13] CAI T, LIU W, LUO X. A constrained 1 minimization approach to sparse precision matrix estimation[J]. Journal of the American Statistical Association, 2011, 106: 594-607.
[14] NICKL R,VAN DE GEER S. Confidence sets in sparse regression[J]. The Annals of Statistics, 2013, 41(6): 2852-2876.
[15] VAN DE GEER S, BUHLMANN P, RITOV Y, et al. On asymptotically optimal confidence regions and tests for high dimensional models[J]. The Annals of Statistics, 2014, 42(3): 1166-1202.
[16] MEINSHAUSEN N. Assumption-free confidence intervals for groups of variables in sparse high-dimensional regression [DB/OL]. [2019-12-01]. https://arxiv.org/abs/1309.3489.
[17] ZHANG C H, ZHANG S S. Confidence intervals for low-dimensional parameters in high dimensional liner models[J]. Journal of the Royal Statistical Society: Series B, 2014, 76: 217-242.
[18] JANKOVA J, VAN DE GEER S. Confidence intervals for high-dimensional inverse covariance estimation [J]. Electronic Journal of Statistics, 2015, 9(1): 1205-1229.
[19] JANKOVA J,VAN DE GEER S. Honest confidence regions and optimality in high-dimensional precision matrix estimation[J]. TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, 2017, 26(1): 143-162.
[20] HUANG X, LI M. Confidence intervals for sparse precision matrix estimation via Lasso penalized D-trace loss[J]. Communications in Statistics: Theory and Methods, 2017, 46(24): 12299-12316.
[21] JANKOVA J,VAN DE GEER S. Inference in high dimensional graphical models [DB/OL]. [2019-12-01]. https://arxiv.org/abs/1801.08512.
[22] YUAN M, LIN Y. Model selection and estimation in the Gaussian graphical model[J]. Biometrika, 2007, 94(1): 19-35.
[23] STRANGER B E, NICA A C, FORREST M S, et al. Population genomics of human gene expression[J]. Nature Genetics, 2007, 39(10): 1217-1224.
[24] BHADRA A, MALLICK B K. Joint high-dimensional Bayesian variable and covariance selection with an application to eQTL analysis[J]. Biometrics, 2013, 69(2): 447-457.
[25] DURRETT R. Probability:Theory and Examples [M]. Cambridge: Cambridge University Press, 2010.
[26] WANG C, JIANG B. An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss[J]. Computational Statistics & Data Analysis, 2020, 142: Article 106812.

附录
A.1 引理

引理A.1[18]引理5] 令α,β∈Rp, 有‖α‖2≤M,‖β‖2≤M. 令Xk∈Rp并且Xk～N(0,Σ*). 则对于m≥2, 有下式成立:

E|αTXkXTkβ-EαTXkXTkβ|m/(2M2K2)m≤m!2.

引理A.2[9，引理1] 设X∈Rp且X～N(0,Σ*), 令δτ(n,r)=8(1+12K2)maxi Σ*ii
2log(4r)/n,
则对于所有的K>0,γ>2,δτ(n,pγ)
<8(1+12K2)maxi Σ*ii有下式成立:


P(|Σ︿-Σ*|∞≥ δτ(n,pγ))≤1pγ-2.

引理A.3[18,引理1] 设X∈Rp且X～N(0,Σ*). 假设Θ*=(Σ*)-1存在并且满足条件2.1, 则有下式成立:

|Θ* W |∞=maxi=1,…,p|Θ*iWej| =OP(logp/n).

引理A.4[10, 定理2-3] 设X∈Rp且X～N(0,Σ*).假设Θ*=(Σ*)-1存在并且满足条件2.1和条件2.2. 设Θ︿为SCIO估计量, 调节参数λni logp/n. 则以大于1-O(p-1+n-δ/8)的概率有下式成立:

(a) Θ︿S∈Θ*S.
(b) |Θ︿-Θ*|∞≤CMplogpn.
A.2 证明定理2.1

证明将式(3)简单变化得出以下分解式:

n(T︿ij-Θ*ij)σij=-n(Θ*WΘ*)ijσij+n(rem)ijσij(A1)

随后将分别证明
n(Θ*WΘ*)ij/σijdN(0,1)和
n|rem|∞/σij=oP(1).

A.2.1 证明n(Θ*WΘ*)ij/σijdN(0,1)


n(Θ*WΘ*)ijσij=

nσij∑nk=11n(Θ*TiXkXTkΘ*j-Θ*ij)=

1σijn∑nk=1 Zijk,

这里Zijk=Θ*TiXkXkTΘ*j-Θ*ij.

由引理A.1,可得对于m≥2,σ0>0(条件2.1得出‖Θ*i‖2≤L,‖Θ*j‖2≤L)有下式成立:

E|Zij，k|m/(2σ0L2K)m≤m!2(K/σ0)m-2.

即对于每个固定的m有:supΘ*E|Zij，k|m=O(1). 则存在常数w>0, 有E|Zij，k|2=σ2ij≥w2.

最后由文献[25]中的Berry-Esseen定理, 可得

|Fn(z)-Φ(z)|≤3E|Z1|3(E|Z1|2)3/2n≤Cn,

其中,Fn(z)是1σijn∑nk=1Zijk的分布函数,Φ(z)是N(0,1)的分布函数, 并且常数C独立于Θ*和n.

由此可以证明：

n(Θ*WΘ*)ijσijdN(0,1).

A.2.2 证明n|rem|∞/σij=oP(1)
由矩阵范数的运算可得

|rem|∞
≤|(Θ︿-Θ*)WΘ*|∞
+|(Θ︿Σ︿-I)(Θ︿-Θ*)|∞≤‖Θ︿-Θ*‖∞|WΘ*|∞
+|Θ︿Σ︿-I|∞‖Θ︿-Θ*‖∞,
其中,


|Θ︿Σ︿-I|∞=|(Θ︿-Θ*)(Σ︿-Σ*)+(Θ︿-Θ*)Σ*+Θ*(Σ︿-Σ*) |∞≤‖Θ︿-Θ*‖∞|Σ︿-Σ*|∞+
|Θ︿-Θ*|∞‖Σ*‖∞+|Θ* W|∞.

因此可得


|rem|∞
≤2‖Θ︿-Θ*‖∞|WΘ*|∞+|Σ︿-Σ*|∞‖Θ︿-Θ*‖∞2+|Θ︿-Θ*|∞‖Σ*‖∞‖Θ︿-Θ*‖∞.

令


|rem1|∞=‖Θ︿-Θ*‖∞|WΘ* |∞,|rem2|∞= |Σ︿-Σ*|∞‖Θ︿-Θ*‖∞2,

|rem3|∞=|Θ︿-Θ*|∞‖Σ*‖∞‖Θ︿-Θ*‖∞.

又有:

①由引理A.2, K=O(1),maxi Σ*ii=O(1), 可得|Σ︿-Σ*|∞=OP (logp/n).

②由引理A.4得‖Θ︿-Θ*‖∞≤d|Θ︿-Θ*|∞(Θ︿S∈Θ*S)及|Θ︿-Θ*|∞=OP(logp/n)(Mp=O(1)).

③在条件2.1成立下, 由引理A.3得|Θ*W|∞=OP(logp/n).

则由①,②,③可得


|rem|∞
=OPmax dlogpn, d2logpn3/2, d‖Σ*‖∞logpn
=OPdlogpn.

再由定理2.1中的稀疏性假设d=o(n/logp)以及1/σij=O(1),可得

n|rem|∞/σij=oP(1).

综合A.2.1和A.2.2可以证明

n(T︿ij-Θ*ij)σijdN(0,1).

证毕.
A.3 证明引理2.1

证明因为X～N(0,Σ*), 所以Θ* X～N(0,Θ*).
令Z=Θ*TX, 则Z～N(0,Θ*).

则有

σ2ij=Var(Θ*TiXΘ*Tj X)
=Var(eTiΘ*TXXTΘ*ej)=Var(eTiZZTej).

从而，

σ2ij
=Var(ZiZj)=E((Zi)2(Zj)2)-(E(ZiZj))2
=Θ*iiΘ*jj+2Θ*ij2-Θ*ij2=Θ*iiΘ*jj+Θ*ij2.

再由条件2.1中1/L≤Λmin(Θ*)≤Λmax(Θ*)≤L, 可得Θ*iiΘ*jj+Θ*ij2≥Λ2min(Θ*)≥1/L2>0, 这里L1, 因此1/σij=O(1).

通过引理A.4以及Mp=O(1)可得|Θ︿-Θ*|∞=OP(logp/n), 因此有下式成立:


|σ︿2ij-σ2ij|
≤|Θ︿iiΘ︿jj-Θ*iiΘ*jj|+|Θ︿2ij-Θ*ij2|
≤|ΔiiΔjj+Θ*iiΔjj+Θ*jjΔii|+|Δij(Δij+2Θ*ij)|=OP(logp/n),

此处定义Δij=Θ︿ij-Θ*ij.

证毕.

()
()