.2024 May 10;24(1):111.

doi: 10.1186/s12874-024-02217-2.

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

Yixin Zhao^#¹, Bilal Khan^#¹, Zelalem F Negeri²

Affiliations

PMID:38730436
PMCID: PMC11084104
DOI: 10.1186/s12874-024-02217-2

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

Yixin Zhao et al. BMC Med Res Methodol.2024.

.2024 May 10;24(1):111.

doi: 10.1186/s12874-024-02217-2.

Authors

Yixin Zhao^#¹, Bilal Khan^#¹, Zelalem F Negeri²

Affiliations

¹ Department of Statistics and Actuarial Science, University of Waterloo, 200 University Ave W, Waterloo, N2L 3G1, Ontario, Canada.
² Department of Statistics and Actuarial Science, University of Waterloo, 200 University Ave W, Waterloo, N2L 3G1, Ontario, Canada. znegeri@uwaterloo.ca.

^# Contributed equally.

PMID:38730436
PMCID: PMC11084104
DOI: 10.1186/s12874-024-02217-2

Abstract

Background: A Generalized Linear Mixed Model (GLMM) is recommended to meta-analyze diagnostic test accuracy studies (DTAs) based on aggregate or individual participant data. Since a GLMM does not have a closed-form likelihood function or parameter solutions, computational methods are conventionally used to approximate the likelihoods and obtain parameter estimates. The most commonly used computational methods are the Iteratively Reweighted Least Squares (IRLS), the Laplace approximation (LA), and the Adaptive Gauss-Hermite quadrature (AGHQ). Despite being widely used, it has not been clear how these computational methods compare and perform in the context of an aggregate data meta-analysis (ADMA) of DTAs.

Methods: We compared and evaluated the performance of three commonly used computational methods for GLMM - the IRLS, the LA, and the AGHQ, via a comprehensive simulation study and real-life data examples, in the context of an ADMA of DTAs. By varying several parameters in our simulations, we assessed the performance of the three methods in terms of bias, root mean squared error, confidence interval (CI) width, coverage of the 95% CI, convergence rate, and computational speed.

Results: For most of the scenarios, especially when the meta-analytic data were not sparse (i.e., there were no or negligible studies with perfect diagnosis), the three computational methods were comparable for the estimation of sensitivity and specificity. However, the LA had the largest bias and root mean squared error for pooled sensitivity and specificity when the meta-analytic data were sparse. Moreover, the AGHQ took a longer computational time to converge relative to the other two methods, although it had the best convergence rate.

Conclusions: We recommend practitioners and researchers carefully choose an appropriate computational algorithm when fitting a GLMM to an ADMA of DTAs. We do not recommend the LA for sparse meta-analytic data sets. However, either the AGHQ or the IRLS can be used regardless of the characteristics of the meta-analytic data.

Keywords: Adaptive Gauss-Hermite; Computational methods; Diagnostic test accuracy; Generalized linear mixed models; IRLS; Laplace approximation; Meta-analysis.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

**Fig. 1**
Forest plots of sensitivity (left) and specificity (right) of the meta-analysis from Vonasek et al. (2021) [12]. The a and b in Schwoebel 2020 denote the two distinct screening tests, “One or more of cough, fever, or poor weight gain in tuberculosis contacts” and “One or more of cough, fever, or decreased playfulness in children aged under five years, inpatient or outpatient,” respectively, utilized in the study

**Fig. 2**
Forest plots of sensitivity (left) and specificity (right) of the meta-analysis from Jullien et al. (2020) [16]

**Fig. 3**
Bias for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when $σ_{1}^{2} = 1.59$ , $σ_{2}^{2} = 1.83$ , $σ_{12} = - 0.34$ , $n_{1} = 300$ , and $n_{2} = 500$

**Fig. 4**
Bias for between-study variances based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when $σ_{1}^{2} = 1.59$ , $σ_{2}^{2} = 1.83$ , $σ_{12} = - 0.34$ , $n_{1} = 300$ , and $n_{2} = 500$

**Fig. 5**
RMSE for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when $σ_{1}^{2} = 1.59$ , $σ_{2}^{2} = 1.83$ , $σ_{12} = - 0.34$ , $n_{1} = 300$ , and $n_{2} = 500$

**Fig. 6**
CI width for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when $σ_{1}^{2} = 1.59$ , $σ_{2}^{2} = 1.83$ , $σ_{12} = - 0.34$ , $n_{1} = 300$ , and $n_{2} = 500$

**Fig. 7**
Coverage for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when $σ_{1}^{2} = 1.59$ , $σ_{2}^{2} = 1.83$ , $σ_{12} = - 0.34$ , $n_{1} = 300$ , and $n_{2} = 500$

See this image and copyright information in PMC

References

1. Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3–8. doi: 10.2307/1174772. - DOI
1. Negeri ZF, Beyene J. Statistical methods for detecting outlying and influential studies in meta-analysis of diagnostic test accuracy studies. Stat Methods Med Res. 2020;29(4):1227–1242. doi: 10.1177/0962280219852747. - DOI - PubMed
1. Negeri ZF, Beyene J. Robust bivariate random-effects model for accommodating outlying and influential studies in meta-analysis of diagnostic test accuracy studies. Stat Methods Med Res. 2020;29(11):3308–3325. doi: 10.1177/0962280220925840. - DOI - PubMed
1. Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59(12):1331–1332. doi: 10.1016/j.jclinepi.2006.06.011. - DOI - PubMed
1. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982–990. doi: 10.1016/j.jclinepi.2005.02.022. - DOI - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Related information

MedGen

LinkOut - more resources

Full Text Sources
Medical
- MedlinePlus Health Information

Movatterモバイル変換

Account

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Full text links

Actions

Share

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

Affiliations

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Similar articles

References

Publication types

MeSH terms

Related information

LinkOut - more resources

Full Text Sources

Medical