Movatterモバイル変換

Md Riaz Ahmed Khanmdriazahmed.khan@jacks.sdstate.eduThomas Brandenburger, PhDthomas.brandenburger@sdstate.edu

Maintainer

ROCit

The goal of ROCit is to easily compute, summarize, and visualize theperformance of a binary classifier. ROCit package provides flexibilityto easily evaluate threshold-bound metrics, such as, accuracy,misclassification rate, sensitivity, specificity, F-Score. . Also, ROCcurve, along with AUC, can be obtained using different methods, such asempirical, binormal and non-parametric. ROCit encompasses a wide varietyof methods for constructing confidence interval of ROC curve and AUC.ROCit features the option of constructing empirical gains table, whichis a handy tool for direct marketing. The package offers options forcommonly used visualization, such as, ROC curve, KS plot, lift plot.

Example

This is a basic example which shows the workflow of assessing abinary classifier.

library(ROCit)data("Loan")summary(Loan$Status)#>  CO  FP#> 131 769

Make a response

Use score as a predictorof defaulting

Calculate threshold-boundmetrics

m<-measureit(score = score,class = class,measure =c("ACC","SENS","SPEC","FSCR"))mymetrics<-as.data.frame(cbind(Cutoff = m$Cutoff,Depth = m$Depth,Accuracy = m$ACC,Sensitivity = m$SENS,Specificity = m$SPEC,`F-Score`= m$FSCR))head(mymetrics)#>     Cutoff       Depth  Accuracy Sensitivity Specificity    F-Score#> 1      Inf 0.000000000 0.8544444 0.000000000   1.0000000        NaN#> 2 269.1711 0.001111111 0.8555556 0.007633588   1.0000000 0.01515152#> 3 267.3342 0.002222222 0.8544444 0.007633588   0.9986996 0.01503759#> 4 266.5938 0.003333333 0.8533333 0.007633588   0.9973992 0.01492537#> 5 265.2553 0.004444444 0.8522222 0.007633588   0.9960988 0.01481481#> 6 263.8303 0.005555556 0.8533333 0.015267176   0.9960988 0.02941176tail(mymetrics)#>         Cutoff     Depth  Accuracy Sensitivity Specificity   F-Score#> 897 103.236891 0.9955556 0.1500000           1  0.00520156 0.2551120#> 898  93.941202 0.9966667 0.1488889           1  0.00390117 0.2548638#> 899  70.836943 0.9977778 0.1477778           1  0.00260078 0.2546161#> 900  -2.673036 0.9988889 0.1466667           1  0.00130039 0.2543689#> 901  -5.449033 1.0000000 0.1455556           1  0.00000000 0.2541222#> 902       -Inf 1.0000000 0.1455556           1  0.00000000 0.2541222

ROC curve

rocit_object_empirical<-rocit(score = score,class = class)summary(rocit_object_empirical)#>#>  Empirical ROC curve#>  Number of postive responses :  131#>  Number of negative responses :  769#>  Area under curve :  0.666315925311945

Plot ROC curve

Confidence interval of AUC

ci.auc<-ciAUC(rocit_object_empirical,level =0.95)print(ci.auc)#>#>    estimated AUC : 0.666315925311945#>    AUC estimation method : empirical#>#>    CI of AUC#>    confidence level = 95%#>    lower = 0.612575921206968     upper = 0.720055929416921

Confidence interval of ROCcurve

ci.roc<-ciROC(rocit_object_empirical,level =0.9)# plot(ci.roc)

Gains table

gainstable<-gainstable(rocit_object_empirical,ngroup =8)print(gainstable,maxdigit =2)#>   Bucket Obs CObs Depth Resp CResp RespRate CRespRate CCapRate Lift CLift#> 1      1 112  112  0.12   30    30     0.27      0.27     0.23 1.84  1.84#> 2      2 113  225  0.25   25    55     0.22      0.24     0.42 1.52  1.68#> 3      3 113  338  0.38   21    76     0.19      0.22     0.58 1.28  1.54#> 4      4 112  450  0.50   18    94     0.16      0.21     0.72 1.10  1.44#> 5      5 112  562  0.62   10   104     0.09      0.19     0.79 0.61  1.27#> 6      6 113  675  0.75   16   120     0.14      0.18     0.92 0.97  1.22#> 7      7 113  788  0.88    5   125     0.04      0.16     0.95 0.30  1.09#> 8      8 112  900  1.00    6   131     0.05      0.15     1.00 0.37  1.00# plot(gainstable)