Movatterモバイル変換

Title:	Statistical Tests for Assessing Trinormal ROC Data
Version:	0.7
Date:	2024-10-02
Description:	Several statistical test functions as well as a function for exploratory data analysis to investigate classifiers allocating individuals to one of three disjoint and ordered classes. In a single classifier assessment the discriminatory power is compared to classification by chance. In a comparison of two classifiers the null hypothesis corresponds to equal discriminatory power of the two classifiers. See also "ROC Analysis for Classification and Prediction in Practice" by Nakas, Bantis and Gatsonis (2023), ISBN 9781482233704.
Depends:	R (≥ 3.3.0)
Imports:	ggplot2, rgl, gridExtra
License:	LGPL-2.1
Encoding:	UTF-8
LazyData:	true
RoxygenNote:	7.2.3
Suggests:	testthat, knitr, rmarkdown, MASS, reshape
VignetteBuilder:	knitr
URL:	https://www.math.uzh.ch/pages/trinROC/
BugReports:	https://git.math.uzh.ch/reinhard.furrer/trinROC/-/issues
NeedsCompilation:	no
Packaged:	2024-10-01 12:34:46 UTC; furrer
Author:	Samuel Noll [aut], Reinhard Furrer [aut, cre], Benjamin Reiser [ctb], Christos T. Nakas [ctb], Annina Cincera [aut]
Maintainer:	Reinhard Furrer <reinhard.furrer@uzh.ch>
Repository:	CRAN
Date/Publication:	2024-10-04 15:20:02 UTC

trinROC: Statistical Tests for Assessing Trinormal ROC Data

Description

Several statistical test functions as well as a function for exploratory data analysis to investigate classifiers allocating individuals to one of three disjoint and ordered classes. In a single classifier assessment the discriminatory power is compared to classification by chance. In a comparison of two classifiers the null hypothesis corresponds to equal discriminatory power of the two classifiers. See also "ROC Analysis for Classification and Prediction in Practice" by Nakas, Bantis and Gatsonis (2023), ISBN 9781482233704.

Details

Seevignette("Overview", package = "trinROC") for an overview of the package.Further,sd(),var() andcov() are chosen withoptions(trinROC.MLE = TRUE) according to the maximum likelihood estimates (default). Change to sampleestimates by settingoptions(trinROC.MLE = FALSE)

Author(s)

Maintainer: Reinhard Furrerreinhard.furrer@uzh.ch (ORCID)

Authors:

• Samuel Nolluncle.sam@gmx.net

• Annina Cinceraannina.cincera@sunrise.ch

Other contributors:

• Benjamin Reiser [contributor]

• Christos T. Nakascnakas@uth.gr [contributor]

References

Noll, S., Furrer, R., Reiser, B. and Nakas, C. T. (2019).Inference in ROC surface analysis via a trinormal model-based testing approach.Stat,8(1), e249.

See Also

Useful links:

• https://www.math.uzh.ch/pages/trinROC/

• Report bugs athttps://git.math.uzh.ch/reinhard.furrer/trinROC/-/issues

Bootstrap test for three-class ROC data

Description

A statistical test function to assess three-class ROC data. It can be usedfor assessment of a single classifier or comparison of two independent /correlated classifiers, using the Bootstrap test.

Usage

boot.test(  x1,  y1,  z1,  x2 = 0,  y2 = 0,  z2 = 0,  dat = NULL,  paired = FALSE,  n.boot = 1000,  conf.level = 0.95,  alternative = c("two.sided", "less", "greater"))

Arguments

Details

Based on the reference standard, the Bootstrap test assesses thediscriminatory power of classifiers by comparing the volumes under the ROCsurfaces (VUS). It distinguishes between single classifier assessment,where a classifier is compared to the chance plane with VUS=1/6, andcomparison between two classifiers. The latter case tests the equalitybetween VUS_1 and VUS_2. The data can arise in a unpaired or pairedsetting. Ifpaired isTRUE, a correlation is introduced whichhas to be taken into account. Therefore the sets of the two classifiershave to have classwise equal size. The data can be input as the dataframedat or as single vectorsx1, y1, z1, .... Theimplemented methods to evaluate theVUS andvar(VUS), cov(vus.1,vus.2) are based on the empirical model assumptions andresampling techniques. This means, there are no underlying distributionsassumed in any of the classes.

Value

A list of class"htest" containing the following components:

References

Nakas, C. T. and C. T. Yiannoutsos (2004). Ordered multiple-classROC analysis with continuous measurements.Statistics inMedicine,23(22), 3437–3449.

See Also

trinROC.test,trinVUS.test.

Examples

data(cancer)data(krebs)# investigate a single marker:boot.test(dat = krebs[,c(1,2)], n.boot=500)# result is equal to:x1 <- with(krebs, krebs[trueClass=="healthy", 2])y1 <- with(krebs, krebs[trueClass=="intermediate", 2])z1 <- with(krebs, krebs[trueClass=="diseased", 2])boot.test(x1, y1, z1, n.boot=500) # comparison of marker 2 and 6:boot.test(dat = krebs[,c(1,2,5)], paired = TRUE) # result is equal to:x2 <- with(krebs, krebs[trueClass=="healthy", 5])y2 <- with(krebs, krebs[trueClass=="intermediate", 5])z2 <- with(krebs, krebs[trueClass=="diseased", 5])boot.test(x1, y1, z1, x2, y2, z2, paired = TRUE)

Box-Cox transformation on three-class ROC data

Description

A transformation function for three-class ROC data in order to obtain normallydistributed classes.

Usage

boxcoxROC(  x,  y,  z,  lambda = seq(-2, 2, 0.05),  lambda2 = NULL,  eps = 0.02,  verbose = TRUE)

Arguments

Details

A Box-Cox transformation computing

X^{(\lambda)} = \left\{ \begin{array}{ll} (X^\lambda -1)/\lambda, &\mbox{if } \; \lambda \neq 0,\\ \log(X),& \mbox{else } \; \lambda = 0,\end{array} \right.

with optimal\lambda estimated from the likelihood kernel function,as formally described in the supplementarymaterial in Bantis et al. (2017). If the data include any nonpositiveobservations, a shifting parameterlambda2 can be included in thetransformation given by:

X^{(\lambda)} = \left\{ \begin{array}{ll} ((X+\lambda_2)^\lambda -1)/\lambda, &\mbox{if }\, \lambda \neq 0,\\ \log(X+\lambda_2),& \mbox{else } \; \lambda = 0. \end{array}\right.\\

Value

A list with components:

References

Bantis LE, Nakas CT, Reiser B, Myall D and Dalrymple-Alford JC(2015) Construction of joint confidence regions for the optimal true classfractions of receiver operating characteristic (roc) surfaces andmanifolds.Statistical Methods in Medical Research26(3): 1429–1442.

Box, G. E. P. and Cox, D. R. (1964). An analysis oftransformations (with discussion).Journal of the Royal Statistical Society,Series B,26, 211–252.

See Also

shapiro.test andboxcox from the packageMASS.

Examples

data(cancer)x1 <- with(cancer, cancer[trueClass=="healthy", 9])y1 <- with(cancer, cancer[trueClass=="intermediate", 9])z1 <- with(cancer, cancer[trueClass=="diseased", 9])boxcoxROC(x1, y1, z1)

Synthetic data set to investigate three-class ROC data.

Description

A dataset containing randomly generated measurements from three diagnostic classes as they mayarise in a cancer investigation.

Usage

cancer

Format

A data frame with 100 rows and 10 variables (9 classifiers):

trueClass

A factor, indicating the class membership of theindividuals.

Class1, Class2, Class3, Class4, Class5, Class6, Class7, Class8, Class9

Measurements of a specific classifier obtained from the patients that underwent theclinical study.

Empirical VUS calculation

Description

This function computes the empirical Volume Under the Surface (VUS)of three-class ROC data.

Usage

emp.vus(x, y, z, dat = NULL, old.version = TRUE)

Arguments

Details

This function computes the empirical VUS of three-class ROC datausing theexpand.grid function. It has been shown to be faster thancomputation using themerge function (VUS.merge()) or directgeometrical imlementation. The measurements can be input as seperate vectorsx, y, z or as a data framedat.

Value

It returns the numericVUS of the data.

References

Scurfield, B. K. (1996). Multiple-event forced-choice tasks inthe theory of signal detectability.Journal of MathematicalPsychology40.3, 253–269.

Nakas CT and Yiannoutsos CT (2004) Ordered multiple-class rocanalysis with continuous measurements.Statistics in Medicine23(22):3437–3449.

Examples

data(krebs)x1 <- with(krebs, cancer[trueClass=="healthy", 4])y1 <- with(krebs, cancer[trueClass=="intermediate", 4])z1 <- with(krebs, cancer[trueClass=="diseased", 4])emp.vus(x1, y1, z1)# Alternatively:emp.vus(dat = krebs[,c(1,4)])

Determine equidistant means of trinormal ROC data simulation

Description

A function that computes the equidistant meansmuy andmuz fora specificmux. The VUS as well as the set of standard errors aregiven as arguments to the function.

Usage

findmu(mux = 0, sdx = 1, sdy = 1, sdz = 1, VUS = 1/6, step = 0.001)

Arguments

Details

Defaults are: VUS = 1/6, standard errors for all three classes equal1. The searching algorithm is stepwise increasing the differencesmuy-mux andmuz-mux according to the variablestep.The algorithm stops when the computed VUS exceeds the preferred VUS. Theaccording parametersmux, muy, muz are returned with the computedVUS.

Remark: The biggerVUS and the smallerstep is chosen, thelonger the computation lasts.

Value

A data frame with the following components:

Examples

# find equidistant means with mux=2.7 and VUS = 0.45:findmu(mux = 2.7, VUS = 0.45)# specify standard errors:findmu(mux = 2.7, sdx = 1.1, sdy = 1.3, sdz = 1.5, VUS = 0.45)

Synthetic small data set to investigate three-class ROC data.

Description

A dataset containing randomly generated measurements from three diagnostic classes as they mayarise in a cancer investigation. For illustration, this dataset has beenchosen to be smaller than the data setcancer.

Usage

krebs

Format

A data frame with 50 rows and 5 variables (4 classifiers):

trueClass

A factor, indicating the class membership of theindividuals.

Fac1, Fac2, Fac3, Fac4

Measurements obtained from the patients that underwent theclinical study.

Exploratory data analysis for a three-class ROC marker

Description

A function that investigates data that arose from a single marker andcontains the reference standard of the three classes "healthy","intermediate" and "diseased".

Usage

roc.eda(  x,  y,  z,  dat = NULL,  type = c("empirical", "trinormal"),  plotVUS = FALSE,  saveVUS = FALSE,  sep.dens = FALSE,  scatter = FALSE,  conf.level = 0.95,  n.boot = 1000,  verbose = TRUE,  alternative = c("two.sided", "less", "greater"),  class.labels = c("healthy", "intermediate", "diseased"))

Arguments

Details

For the preliminary assessment of a classifier, exporatorydata analysis (EDA) on the markers is necessary. This function assessesmeasurements from a single marker and computes the VUS, statistical testsand returns a summary table as well as some plots of the data.

Value

A list with class "htest" containing the following components:

Warning

Iftype = "empirical", computation may take a while, asroc.eda callsthe functionboot.test().

See Also

trinROC.test,trinVUS.test for trinormaldata investigation,boot.test for empirical data analysis.rocsurf.emp,rocsurf.trin for the surface plot.

Examples

data(krebs)# empirical EDA:x <- with(krebs, krebs[trueClass=="healthy", 5])y <- with(krebs, krebs[trueClass=="intermediate", 5])z <- with(krebs, krebs[trueClass=="diseased", 5])roc.eda(x, y, z, type = "e", plotVUS = FALSE)# equal data input via:# roc.eda(dat = krebs[,c(1,5)], type = "e", plotVUS = FALSE)data(cancer)# trinormal EDA:roc.eda(dat = cancer[,c(1,10)], type = "trin", plotVUS = FALSE)# trinormal EDA with different plots:roc.eda(dat = cancer[,c(1,5)], type = "t", sep.dens = TRUE, scatter = TRUE)

Statistical test function for computing multiple tests on three-class ROC data

Description

A statistical test function that assesses three-class ROC data with thetrinormal based ROC test, the trinormal VUS test and the Bootstrap test.

Usage

roc3.test(  dat,  type = c("ROC", "VUS", "Bootstrap"),  paired = FALSE,  conf.level = 0.95,  n.boot = 1000,  p.adjust = FALSE)

Arguments

Details

For the preliminary assessment of a classifier, differentstatistical tests have been proposed in the literature. This function canbe used for either comparison of single classifiers to a null hypothesis ofbeing not better than a random allocation function or comparison of twoclassifiers under the null hypothesis of having equal discriminatory power.Depending on the specification of the user, (s)he can apply the trinormalbased ROC test (LINK), the test developed by Xiong et. al. or the Bootstraptest or any combination of these tests. More information of the specifictests can be obtained by calling?functionname. If more than twomarkers are present, a pairwise comparison between each marker is realized.

Value

A list with components:

Note

Iftype = "Bootstrap", the Bootstrap test is evaluated. Thismay take some time, especially with sample sizes > 100.

Examples

data(krebs)roc3.test(krebs, type = c("ROC", "VUS"), paired = TRUE)[c("Overview","P.values")]

Empirical ROC surface plot

Description

Function for computation of the empirical ROC surface.

Usage

rocsurf.emp(x, y, z, plot = TRUE, saveVUS = FALSE)

Arguments

Details

This function takes three-class ROC data and computes the threedimentional surface using the R-packagergl. The ROC surface isdefined as

z = ROCs(t_-,t_+) = F_0(c_+) - F_0(c_-)=F_0(G_+^{-1}(t_+) ) -F_0(F_-^{-1}(t_-) ),

wherec_-, c_+ are the two cut-off points andF_-, F_0, F_+ thecdf of the three classes withG = 1-F.

Value

A list with the following components:

See Also

surface3d.

Examples

data(cancer)x1 <- with(cancer, cancer[trueClass=="healthy", 9])y1 <- with(cancer, cancer[trueClass=="intermediate", 9])z1 <- with(cancer, cancer[trueClass=="diseased", 9])rocsurf.emp(x1, y1, z1)

Trinormal ROC surface plot

Description

Function for computation of the trinormal ROC surface.

Usage

rocsurf.trin(x, y, z, p = 300, plot = TRUE, saveVUS = FALSE)

Arguments

Details

This function takes three-class ROC data and computes the threedimentional surface using the R-packagergl. The ROC surface isdefined as

z = ROCs(t_-,t_+) = F_0(c_+) - F_0(c_-)=F_0(G_+^{-1}(t_+) ) -F_0(F_-^{-1}(t_-) ),

wherec_-, c_+ are the two cut-off points andF_-, F_0, F_+ thecdf of the three classes withG = 1-F.

Value

A list with the following components:

References

Xiong, C., G. Van Belle, et al. (2006). Measuring and estimatingdiagnostic accuracy when there are three ordinal diagnostic groups.Statistics in Medicine 25(7), 1251–1273.

Examples

data(cancer)x1 <- with(cancer, cancer[trueClass=="healthy", 8])y1 <- with(cancer, cancer[trueClass=="intermediate", 8])z1 <- with(cancer, cancer[trueClass=="diseased", 8])rocsurf.trin(x1, y1, z1)

Trinormal based ROC test

Description

A statistical test function to assess three-class ROC data. It is possible toinvestigate a single classifier or make a comparison of two independent /correlated classifiers.

Usage

trinROC.test(  x1,  y1,  z1,  x2 = 0,  y2 = 0,  z2 = 0,  dat = NULL,  paired = FALSE,  conf.level = 0.95)

Arguments

Details

The trinormal ROC model is a parametric model in three-class ROCanalysis. It is based on normality in each of the trhee classes D_-(healthy), D_0 (intermediate) and D_+ (diseased) with denoted distributionsN(\mu_-,\sigma_-^2),N(\mu_0,\sigma_0^2) andN(\mu_+,\sigma_+^2). A classifier of a trinormal ROC model classifiesindividuals into one of the three ordered classes based on two cut-off pointsc_- < c_+. We definet_-=F_-(c_-) andt_+=1-F_+(c_+)=G_+(c_+). Now, the ROC surface can be written as

ROCs(t_-,t_+) = \Phi \left(\frac{\Phi^{-1} (1-t_+) +d}{c} \right) -\Phi \left(\frac{\Phi^{-1} (t_-)+b}{a} \right)

whith parameters a, b, c and c given bya =\frac{\hat{\sigma}_0}{\hat{\sigma}_-}, b = \frac{ \hat{\mu}_- -\hat{\mu}_0}{\hat{\sigma}_-}, c = \frac{\hat{\sigma}_0}{\hat{\sigma}_+}, d =\frac{ \hat{\mu}_+ - \hat{\mu}_0}{\hat{\sigma}_+}. It is a surface in theunit cube that plots the probability of a measurement to get assigned to theintermediate class as the two thresholdsc_-,c_+ are varying.

Based on the reference standard, the trinormal based ROC test can be usedto assess the discriminatory power of such classifiers. It distinguishesbetween single classifier assessment, where a classifier is compared to somehypothetical distributions in the classes, and comparison between twoclassifiers. The latter case tests for equality between the parameters a, b,c and d of the ROC curves. The data can arise in a unpaired or pairedsetting. Ifpaired isTRUE, a correlation is introduced whichhas to be taken into account. Therefore the sets of the two classifiers haveto have classwise equal size. The data can be input as the data framedat or as single vectorsx1, y1, z1, ....

As the Chi-squared test is by definition a one-sided test, the variablealternative cannot be specified in this test. For this 'goodness offit' test, we assume the parametersa_1, \dots , d_1 anda_2, \dots , d_2 to have apairwise equivalent normal distribution (in large sample sets).

Value

A list of class"htest" containing the following components:

References

Noll, S., Furrer, R., Reiser, B. and Nakas, C. T. (2019).Inference in ROC surface analysis via a trinormal model-based testing approach.Stat,8(1), e249.

See Also

trinVUS.test,boot.test.

Examples

data(cancer)data(krebs)# investigate a single marker:trinROC.test(dat = cancer[,c(1,3)])trinROC.test(dat = krebs[,c(1,5)])# result is equal to:x1 <- with(cancer, cancer[trueClass=="healthy", 3])y1 <- with(cancer, cancer[trueClass=="intermediate", 3])z1 <- with(cancer, cancer[trueClass=="diseased", 3])trinROC.test(x1, y1, z1)# comparison of marker 2 and 6:trinROC.test(dat = cancer[,c(1,3,5)], paired = TRUE)trinROC.test(dat = cancer[,c(1,3,5)], paired = FALSE)# result is equal to:x2 <- with(cancer, cancer[trueClass=="healthy", 5])y2 <- with(cancer, cancer[trueClass=="intermediate", 5])z2 <- with(cancer, cancer[trueClass=="diseased", 5])trinROC.test(x1, y1, z1, x2, y2, z2, paired = TRUE)

Trinormal VUS test

Description

A statistical test function to assess three-class ROC data. It can be usedfor assessment of a single classifier or comparison of two independent /correlated classifiers, using the statistical test developed by Xiong et al.(2007).

Usage

trinVUS.test(  x1,  y1,  z1,  x2 = 0,  y2 = 0,  z2 = 0,  dat = NULL,  paired = FALSE,  conf.level = 0.95,  alternative = c("two.sided", "less", "greater"))

Arguments

Details

Based on the reference standard, this trinormal VUS test assesses thediscriminatory power of classifiers by comparing the volumes under the ROCsurfaces (VUS). It distinguishes between single classifier assessment,where a classifier is compared to the chance plane with VUS=1/6, andcomparison between two classifiers. The latter case tests the equalitybetween VUS_1 and VUS_2. The data can arise in a unpaired or pairedsetting. Ifpaired isTRUE, a correlation is introduced whichhas to be taken into account. Therefore the sets of the two classifiershave to have classwise equal size. The data can be input as the dataframedat or as single vectorsx1, y1, z1, ....

Value

A list of class"htest" containing the following components:

References

Xiong, C., Van Belle, G. Miller J. P., Morris, J. C. (2006). Measuring and estimatingdiagnostic accuracy when there are three ordinal diagnostic groups.Statistics in Medicine,25(7), 1251–1273.

Xiong, C., van Belle, G., Miller, J. P., Yan, Y., Gao, F., Yu, K., and Morris, J. C. (2007). A parametric comparisonof diagnostic accuracy with three ordinal diagnostic groups.Biometrical Journal,49(5), 682–693.doi:10.1002/bimj.200610359.

See Also

trinROC.test,boot.test.

Examples

data(cancer)data(krebs)# investigate a single marker:trinVUS.test(dat = cancer[,c(1,3)])trinVUS.test(dat = krebs[,c(1,5)])# result is equal to:x1 <- with(cancer, cancer[trueClass=="healthy", 3])y1 <- with(cancer, cancer[trueClass=="intermediate", 3])z1 <- with(cancer, cancer[trueClass=="diseased", 3])trinVUS.test(x1, y1, z1)# comparison of marker 2 and 6:trinVUS.test(dat = cancer[,c(1,3,5)], paired = TRUE)trinVUS.test(dat = cancer[,c(1,3,5)], paired = FALSE)# result is equal to:x2 <- with(cancer, cancer[trueClass=="healthy", 5])y2 <- with(cancer, cancer[trueClass=="intermediate", 5])z2 <- with(cancer, cancer[trueClass=="diseased", 5])trinVUS.test(x1, y1, z1, x2, y2, z2, paired = TRUE)