Movatterモバイル変換

Introduction

Precision matrix estimation requires selecting appropriate regularization parameterλ to balance sparsity (number of edges) and model fit (likelihood), and a mixing parameterα to trade off between element-wise (individual-level) and block-wise (group-level) penalties.

Background: Negative Log-Likelihood

In a Gaussian graphical model (GGM), the data matrixX_n × d consists ofn independent and identically distributed observationsX₁, …, X_n drawn fromN_d(μ, Σ). LetΩ = Σ⁻¹ denote the precision matrix, and define the empirical covariance matrix as$S = n^{-1} \sum_{i=1}^n (X_i-\bar{X})(X_i-\bar{X})^\top$. Up to an additive constant, the negative log-likelihood (nll) forΩ simplified to$$\mathrm{nll}(\Omega) = \frac{n}{2}[-\log\det(\Omega) + \mathrm{tr}(S\Omega)].$$ The edge setE(Ω) is determined by the non-zero off-diagonal entries: an edge(i, j) is included if and only ifω_ij ≠ 0 fori < j. The number of edges is therefore given by|E(Ω)|.

Selection Criteria

1. AIC: Akaike information criterion(Akaike 1973)

Ω̂_AIC = arg min _Ω{2 nll(Ω) + 2 |E(Ω)|}.

2. BIC: Bayesian information criterion(Schwarz 1978)

Ω̂_BIC = arg min _Ω{2 nll(Ω) + log (n) |E(Ω)|}.

3. EBIC: Extended Bayesian information criterion(Chen and Chen 2008; Foygel and Drton 2010)

Ω̂_EBIC = arg min _Ω{2 nll(Ω) + log (n) |E(Ω)| + 4 ξ log (d) |E(Ω)|},

whereξ ∈ [0, 1] is a tuning parameter. Settingξ = 0 reduces EBIC to the classic BIC.

4. HBIC: High dimensional Bayesian information criterion(Wang, Kim, and Li 2013; Fan et al. 2017)

Ω̂_HBIC = arg min _Ω{2 nll(Ω) + log [log (n)] log (d) |E(Ω)|}.

5. K-fold cross validation with negative log-likelihood loss.

Figure 1 illustrates theK-fold cross-validation procedure used for tuning the parametersλ andα. The notation#λ and#α denotes the number of candidate values considered forλ andα, respectively, forming a grid of#λ × #α total parameter combinations. For each of theK iterations, negative log-likelihood loss is evaluated for all parameter combinations, yieldingK performance values per combination. The optimal parameter pair is selected as the one achieving the lowest average loss across theK iterations.

Reference