Repository files navigation

The aim of theprecrec package is to provide an integrated platformthat enables robust performance evaluations of binary classifiers.Specifically,precrec offers accurate calculations of ROC (ReceiverOperator Characteristics) and precision-recall curves. All the maincalculations ofprecrec are implemented withC++/Rcpp.

• Package website – GitHub pagesthat contain all precrec documentation.

• Introduction toprecrec– a package vignette that contains the descriptions of the functionswith several useful examples. View the vignette withvignette("introduction", package = "precrec") in R. The HTML versionis also available on theGitHubPages.

• Help pages – all thefunctions including the S3 generics except forprint have their ownhelp pages with plenty of examples. View the main help page withhelp(package = "precrec") in R. The HTML version is also availableon theGitHub Pages.

precrec provides accurate precision-recall curves.

• Non-linear interpolation
• Elongation to the y-axis to estimate the first point when necessary
• Use of score-wise threshold values instead of fixed bins

precrec also calculates AUC scores with high accuracy.

precrec calculates curves in a matter of seconds even for a fairlylarge dataset. It is much faster than most other tools that calculateROC and precision-recall curves.

In addition to precision-recall and ROC curves,precrec offers basicevaluation measures.

• Error rate
• Accuracy
• Specificity
• Sensitivity, true positive rate (TPR), recall
• Precision, positive predictive value (PPV)
• Matthews correlation coefficient
• F-score

precrec calculates confidence intervals when multiple test sets aregiven. It automatically shows confidence bands about the averaged curvein the corresponding plot.

precrec calculates partial AUCs for specified x and y ranges. It canalso draw partial ROC and precision-recall curves for the specifiedranges.

precrec provides several useful functions that lack in most otherevaluation tools.

• Handling multiple models and multiple test sets
• Handling tied scores and missing scores
• Pre- and post-process functions of simple data preparation and curveanalysis

• Install the release version ofprecrec from CRAN withinstall.packages("precrec").

• Alternatively, you can install a development version ofprecrec fromour GitHub repository. Toinstall it:

  1. Make sure you have a working development environment.

     • Windows: Install Rtools (available on the CRAN website).
     • Mac: Install Xcode from the Mac App Store.
     • Linux: Install a compiler and various development libraries(details vary across different flavors of Linux).

  2. Installdevtools from CRAN withinstall.packages("devtools").

  3. Installprecrec from the GitHub repository withdevtools::install_github("evalclass/precrec").

Theprecrec package provides the following six functions.

Moreover, theprecrec package provides nine S3 generics for the S3object created by theevalmod function.N.B. The R languagespecifies S3 objects and S3 generic functions as part of the most basicobject-oriented system in R.

Following two examples show the basic usage ofprecrec functions.

Theevalmod function calculates ROC and Precision-Recall curves andreturns an S3 object.

library(precrec)# Load a test datasetdata(P10N10)# Calculate ROC and Precision-Recall curvessscurves<- evalmod(scores=P10N10$scores,labels=P10N10$labels)

Theautoplot function outputs ROC and Precision-Recall curves by usingtheggplot2 package.

# The ggplot2 package is requiredlibrary(ggplot2)# Show ROC and Precision-Recall plotsautoplot(sscurves)

Precrec: fast and accurate precision-recall and ROC curve calculationsin R

Takaya Saito; Marc Rehmsmeier

Bioinformatics 2017; 33 (1): 145-147.

doi:10.1093/bioinformatics/btw570

• Classifier evaluation with imbalanceddatasets - our web site thatcontains several pages with useful tips for performance evaluation onbinary classifiers.

• The Precision-Recall Plot Is More Informative than the ROC Plot WhenEvaluating Binary Classifiers on ImbalancedDatasets - our paperthat summarized potential pitfalls of ROC plots with imbalanceddatasets and advantages of using precision-recall plots instead.