R

# install dcurves to perform DCA from CRANinstall.packages("dcurves")# install other packages used in this tutorialinstall.packages(  c(    "tidyverse", "survival", "gt", "broom",    "gtsummary", "rsample", "labelled"  ))# load packageslibrary(dcurves)library(tidyverse)library(gtsummary)

Stata

* install dca functions from GitHub.comnet install dca, from("https://raw.github.com/ddsjoberg/dca.stata/master/") replace

SAS

/* source the dca macros from GitHub.com *//* you can also navigate to GitHub.com and save the macros locally */FILENAME dca URL "https://raw.githubusercontent.com/ddsjoberg/dca.sas/main/dca.sas";FILENAME stdca URL "https://raw.githubusercontent.com/ddsjoberg/dca.sas/main/stdca.sas";%INCLUDE dca;%INCLUDE stdca;

Python

# install dcurves to perform DCA (first install package via pip)# pip install dcurvesfrom dcurves import dca, plot_graphs# install other packages used in this tutorial# pip install pandas numpy statsmodels lifelinesimport pandas as pdimport numpy as npimport statsmodels.api as smimport lifelines

Via logistic regression with cancer as the outcome, we can see thatfamily history is related to biopsy outcome with odds ratio 2.32 (95%CI: 1.44, 3.71; p<0.001). The decision curve analysis can help usaddress the clinical utility of using family history to predict biopsyoutcome.

First, note that there are many threshold probabilities shown herethat are not of interest. For example, it is unlikely that a patientwould demand that they had at least a 50% risk of cancer before theywould accept a biopsy. Let’s do the decision curve analysis again, thistime restricting the output to threshold probabilities a more clinicallyreasonable range. We think it would be not make sense if a patient optedfor biopsy if their risk of cancer was less than 5%; similarly, it wouldbe irrational not to get a biopsy if risk was above 35%. So we want tolook at the range between 5% and 35%. Because 5% is pretty close to 0%,we will show the range between 0% and 35%.

Now that the graph is showing a more reasonable range of thresholdprobabilities, let’s assess the clinical utility of family historyalone. We can see here that although family history is significantlyassociated with biopsy outcome, it only adds value to a small range ofthreshold probabilities near 13% - 20%. If your personal thresholdprobability is 15% (i.e. you would only undergo a biopsy if yourprobability of cancer was 15% or more), then family history alone can bebeneficial in making the decision to undergo biopsy. However, if yourthreshold probability is less than 13% or higher than 20%, then usingfamily history to decide on biopsy has less benefit than choosing tobiopsy or not biopsy respectively. Hence, we would conclude that usingfamily history to determine who to biopsy would help some patients butharm some others, and so should not be used in the clinic.

We now want to compare our different approaches to cancer detection:biopsying everyone, biopsying no-one, biopsying on the basis of familyhistory, or biopsying on the basis of a multivariable statistical modelincluding the marker, age and family history of cancer.

The key aspect of decision curve analysis is to look at whichstrategy leads to the largest net benefit (i.e. the “highest” line),which in this example would correspond to the model that includes age,family history of cancer, and the marker. It is clear that thestatistical model has the highest net benefit across the entire range ofthreshold probabilities. Accordingly, we would conclude that using themodel to decide whether to biopsy a patient would lead to betterclinical outcomes.

Second, the legend can be switched on – useful for initial evaluation– and off - better for producing publication-ready figures. The default(shown in all the examples above) is to include the legend. To avoidshowing the legend:

Finally, it can sometimes be important to modify the y axis,especially when event rates – and therefore net benefit – is low. Wewill demonstrate by first creating a data set where the incidence ofcancer is 90% lower than in the original data set. Running a decisioncurve ends up with a large amount of white space where net benefit isless than 0, which is uninteresting.

A better alternative would be to use an option that truncates they-axis. We can also truncate the x-axis on the grounds that net benefitis below zero for threshold probabilities above 0.1.

An alternative is to calculate net benefit at wider intervals. Bydefault, the software calculates net benefit at every 1% of thresholdprobability, but we this can cause artifacts unless the data set is verylarge. You can specify instead to calculate net benefit every 5%.

A few additional points are worth noting. First, look at the greenline, the net benefit for “treat all”, that is, biopsy everyone. Thiscrosses the y axis at the prevalence. Imagine that a man had a riskthreshold of 14%, and asked his risk under the “biopsy all” strategy. Hewould be told that his risk was the prevalence (14%) and so would beundecided about whether to biopsy or not. When a patient’s riskthreshold is the same as his predicted risk, the net benefit ofbiopsying and not biopsying are the same.

Second, the decision curve for the binary variable (family history ofcancer, the brown line) crosses the “biopsy all men” line at 1 –negative predictive value. Again, this is easily explained: the negativepredictive value is 87%, so a patient with no family history of cancerhas a probability of disease of 13%; a patient with a thresholdprobability less than this – for example, a patient who would opt forbiopsy even if risk was 10% - should therefore be biopsied even if theyhad no family history of cancer. The net benefit for a binary variableis equivalent to biopsy no-one at the positive predictive value. This isbecause for a binary variable, a patient with the characteristic isgiven a risk at the positive predictive value.

This decision curve suggests that although the model might be usefulin the most risk averse patients, it is actually harmful in patientswith more moderate threshold probabilities, that is, it has a netbenefit lower than a reasonable clinical alternative. As such, the Brownet al. model should not be used in clinical practice. This effect, amodel being harmful, occurs due to miscalibration, that is, whenpatients are given risks that are too high or too low. Note thatmiscalibration only occurs rarely when models are created and tested onthe same data set, such as in the example where we created a model withboth family history and the marker.

Now, let’s compare the performance of these approaches using decisioncurve analysis:

In general, the “joint” approach does better than the “conditional”approach, something which shouldn’t be surprising as we are measuringthe cancer marker on everyone, not just those at intermediate risk, andso are getting more information. Clearly, however, we are not takinginto account the downside of having to measure the marker on morepatients, which is the whole justification for using the “conditional”approach. This issue is dealt with in the “incorporating harms” sectionof the tutorial.

The saved table lists the range of threshold probability that wespecified, followed by the net benefits of treating all, none, ourspecified model, intervention avoided (we discuss this below), and thenewly created variable which represent the increase in net benefits ofour model using only the marker.

Net benefit has a ready clinical interpretation. The value of 0.03 ata threshold probability of 20% can be interpreted as: “Comparing toconducting no biopsies, biopsying on the basis of the marker is theequivalent of a strategy that found 3 cancers per hundred patientswithout conducting any unnecessary biopsies.”

At a probability threshold of 10%, the net reduction in interventionsis about 0.15. In other words, at this probability threshold, biopsyingpatients on the basis of the marker is the equivalent of a strategy thatled to an absolute 15% reduction in the number of biopsies withoutmissing any cancers. You can also report the net interventions avoidedper 100 patients (or any number of patients).

The survival probability to any time-point can be derived from anytype of survival model; here we use a Cox as this is the most commonmodel in statistical practice. You don’t necessarily need to know thisto run a decision curve, but the formula for a survival probability froma Cox model is given by:

\[S(t|X) = S_0(t)^{e^{X\beta}}\]

Where\(X\) is matrix of covariatesin the Cox model,\(\beta\) is a vectorcontaining the parameter estimates from the Cox model, and\(S_0(t)\) is the baseline survivalprobability to time t.

To run a decision curve analysis, we will create a Cox model withage, family history, and the marker, as predictors, save out thebaseline survival function in a new variable, and obtaining the linearprediction from the model for each subject.

We then obtain the baseline survival probability to our time point ofinterest. If no patient was observed at the exact time of interest, wecan use the baseline survival probability to the observed time closestto, but not after, the time point. We can then calculate the probabilityof failure at the specified time point. For our example, we will use atime point of 1.5, which would corresponds to the eighteen months thatwe are interested in.

The code for running the decision curve analysis is straightforwardafter the probability of failure is calculated. All we have to do isspecify the time point we are interested in. For our example, let us notonly set the threshold from 0% to 50%, but also add smoothing.

This shows that using the model to inform clinical decisions willlead to superior outcomes for any decision associated with a thresholdprobability of above 2% or so.

R

# import datadf_cancer_dx_case_control <-  readr::read_csv(    file = "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_cancer_dx_case_control.csv"  ) %>%  # assign variable labels. these labels will be carried through in the `dca()` output  labelled::set_variable_labels(    patientid = "Patient ID",    casecontrol = "Case-Control Status",    risk_group = "Risk Group",    age = "Patient Age",    famhistory = "Family History",    marker = "Marker",    cancerpredmarker = "Prediction Model"  )# summarize datadf_cancer_dx_case_control %>%  select(-patientid) %>%  tbl_summary(    by = casecontrol,    type = all_dichotomous() ~ "categorical"  ) %>%  modify_spanning_header(all_stat_cols() ~ "**Case-Control Status**")

Stata

* import dataimport delimited "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_cancer_dx_case_control.csv", clearlabel variable patientid "Patient ID"label variable casecontrol "Case-Control Status"label variable risk_group "Risk Group"label variable age "Patient Age"label variable famhistory "Family History"label variable marker "Marker"label variable cancerpredmarker "Probability of Cancer Diagnosis"

SAS

FILENAME case_con URL "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_cancer_dx_case_control.csv";PROC IMPORT FILE = case_con OUT = work.data_case_control DBMS = CSV;RUN;DATA data_case_control;  SET data_case_control;  LABEL patientid = "Patient ID"        casecontrol = "Case-Control Status"        risk_group = "Risk Group"        age = "Patient Age"        famhistory = "Family History"        marker = "Marker"        cancerpredmarker = "Probability of Cancer Diagnosis";RUN;

Python

df_case_control = pd.read_csv(    "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_cancer_dx_case_control.csv")# Summarize Data With Column Medians# drop 'patientid', then group and take medians of numeric columns onlymedians = (    df_case_control.drop(columns="patientid")    .groupby("casecontrol")    .median(numeric_only=True))print(medians)

R

dca(casecontrol ~ cancerpredmarker,  data = df_cancer_dx_case_control,  prevalence = 0.20,  thresholds = seq(0, 0.5, 0.01)) %>%  plot(smooth = TRUE)

Stata

dca casecontrol cancerpredmarker, prevalence(0.20) xstop(0.50)

SAS

%DCA(data = data_case_control, outcome = casecontrol,     predictors = cancerpredmarker, prevalence = 0.20);

Python

dca_case_control_df = dca(    data=df_case_control,    outcome="casecontrol",    modelnames=["cancerpredmarker"],    prevalence=0.20,    thresholds=np.arange(0, 0.51, 0.01),)plot_graphs(    plot_df=dca_case_control_df,    graph_type="net_benefit",    y_limits=[-0.05, 0.25],    smooth_frac=0.1,)

R

# Run the decision curve incorporating simple harm of marker measurementdca(cancer ~ marker,  data = df_cancer_dx,  thresholds = seq(0, 0.35, 0.01),  as_probability = "marker",  harm = list(marker = 0.0333)) %>%  plot(smooth = TRUE)

Stata

dca cancer marker, probability(no) harm(0.0333) xstop(0.35) xlabel(0(0.05)0.35)

SAS

%DCA(data = data_cancer, outcome = cancer,     predictors = marker,     probability=no,     harm = 0.0333,     graph = yes, xstop = 0.35);

Python

dca_harm_simple_df = dca(    data=df_cancer_dx,    outcome="cancer",    modelnames=["marker"],    thresholds=np.arange(0, 0.36, 0.01),    harm={"marker": 0.0333},    models_to_prob=["marker"],)plot_graphs(    plot_df=dca_harm_simple_df,    graph_type="net_benefit",    y_limits=[-0.05, 0.15],    color_names=["blue", "red", "green"],    smooth_frac=0.2,)

R

# the harm of measuring the marker is stored in a scalarharm_marker <- 0.0333# in the conditional test, only patients at intermediate risk# have their marker measured# harm of the conditional approach is proportion of patients who have the marker# measured multiplied by the harmharm_conditional <- mean(df_cancer_dx$risk_group == "intermediate") * harm_marker# Run the decision curvedca(cancer ~ risk_group,  data = df_cancer_dx,  thresholds = seq(0, 0.35, 0.01),  as_probability = "risk_group",  harm = list(risk_group = harm_conditional)) %>%  plot(smooth = TRUE)

Stata

* the harm of measuring the marker is stored in a locallocal harm_marker = 0.0333* in the conditional test, only patients at intermediate risk have their marker measuredg intermediate_risk = (risk_group=="intermediate")* harm of the conditional approach is proportion of patients who have the marker measured multiplied by the harm of measuringsum intermediate_risklocal harm_conditional = r(mean)*`harm_marker'*convert risk group to a numerical variable to use in the decision curve analysisencode (risk_group), g(risk_category)* Run the decision curvedca cancer risk_category, ///probability(no) harm(`harm_conditional') xstop(0.35) xlabel(0(0.05)0.35)

SAS

* the harm of measuring the marker is stored as a macro variable;%LET harm_marker = 0.0333;*in the conditional test, only patients at intermediate risk have their marker measured;DATA data_cancer;  SET data_cancer;  intermediate_risk = (risk_group = "intermediate");RUN;* calculate the proportion of patients who have the marker and save out mean risk;PROC MEANS DATA = data_cancer; VAR intermediate_risk; OUTPUT OUT = meanrisk MEAN = meanrisk;RUN;DATA _NULL_;  SET meanrisk;  CALL SYMPUT("meanrisk", meanrisk);RUN;* harm of the conditional approach is proportion of patients who have the marker measured multiplied by the harm;%LET harm_conditional = %SYSEVALF(&meanrisk.*&harm_marker.);* Run the decision curve;%DCA(data = data_cancer, outcome = cancer, predictors = risk_group,     probability=no,     harm = &harm_conditional., xstop = 0.35);

Python

harm_marker = 0.0333harm_conditional = (df_cancer_dx["risk_group"] == "intermediate").mean() * harm_markerdca_harm_df = dca(    data=df_cancer_dx,    outcome="cancer",    modelnames=["risk_group"],    models_to_prob=["risk_group"],    thresholds=np.arange(0, 0.36, 0.01),    harm={"risk_group": harm_conditional},)plot_graphs(    plot_df=dca_harm_df,    y_limits=[-0.05, 0.15],    color_names=["blue", "red", "green"],    smooth_frac=0.2,)

R

# set seed for random processset.seed(112358)formula <- cancer ~ marker + age + famhistorydca_thresholds <- seq(0, 0.36, 0.01)# create a 10-fold cross validation set, 1 repeat which is the base case, change to suit your use casecross_validation_samples <- rsample::vfold_cv(df_cancer_dx, v = 10, repeats = 1)df_crossval_predictions <-  cross_validation_samples %>%  # for each cut of the data, build logistic regression on the 90% (analysis set)  rowwise() %>%  mutate(    # build regression model on analysis set    glm_analysis =      glm(        formula = formula,        data = rsample::analysis(splits),        family = binomial      ) %>%        list(),    # get predictions for assessment set    df_assessment =      broom::augment(        glm_analysis,        newdata = rsample::assessment(splits),        type.predict = "response"      ) %>%        list()  ) %>%  ungroup() %>%  # pool results from the 10-fold cross validation  pull(df_assessment) %>%  bind_rows() %>% # Concatenate all cross validation predictions  group_by(patientid) %>%  summarise(cv_pred = mean(.fitted), .groups = "drop") %>% # Generate mean prediction per patient  ungroup()df_cv_pred <-  df_cancer_dx %>%  left_join(    df_crossval_predictions,    by = "patientid"  )dcurves::dca( # calculate net benefit scores on mean cross validation predictions  data = df_cv_pred,  formula = cancer ~ cv_pred,  thresholds = dca_thresholds,  label = list(    cv_pred = "Cross-validated Prediction Model"  ))$dca |>  # plot cross validated net benefit values  ggplot(aes(x = threshold, y = net_benefit, color = label)) +  stat_smooth(method = "loess", se = FALSE, formula = "y ~ x", span = 0.2) +  coord_cartesian(ylim = c(-0.014, 0.14)) +  scale_x_continuous(labels = scales::percent_format(accuracy = 1)) +  labs(x = "Threshold Probability", y = "Net Benefit", color = "") +  theme_bw()

Stata

* Load Original Datasetimport delimited "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_cancer_dx.csv", clear* Start by specifying whether cross validation will be done once or multiple times* The global variable cv specifies the number of repeated crossvalidations.* Set global cv to 1 for a single crossvalidation, but, say, 25 for multiple crossvalidationglobal cv=25forvalues i=1(1)$cv {  * Local macros to store the names of model.  local prediction1 = "model"  * Create variables to later store probabilities from each prediction model  quietly g `prediction1'=.  * Create a variable to be used to `randomize' the patients.  quietly g u = uniform()  * Sort by the event to ensure equal number of patients with the event are in each  * group  sort cancer u  * Assign each patient into one of ten groups  g group = mod(_n, 10) + 1  * Loop through to run through for each of the ten groups  forvalues j=1(1)10 {    * First for the "base" model:    * Fit the model excluding the jth group.    quietly logit cancer age famhistory marker if group!=`j'    * Predict the probability of the jth group.    quietly predict ptemp if group==`j'    * Store the predicted probabilities of the jth group (that was not used in    * creating the model) into the variable previously created    quietly replace `prediction1' = ptemp if group==`j'    * Dropping the temporary variable that held predicted probabilities for all    * patients    drop ptemp  }  * Creating a temporary file to store the results of each of the iterations of our  * decision curve for the multiple the 10 fold cross validation  * This step may omitted if the optional forvalues loop was excluded.  tempfile dca`i'  * Run decision curve, and save the results to the tempfile.  * For those excluding the optional multiple cross validation, this decision curve  * (to be seen by excluding "nograph") and the results (saved under the name of your  * choosing) would be the decision curve corrected for overfit.  quietly dca cancer `prediction1', xstop(.5) nograph ///  saving("`dca`i''")  drop u group `prediction1'} // This closing bracket ends the initial loop for the multiple cross validation.* It is also necessary for those who avoided the multiple cross validation* by changing the value of the forvalues loop from 200 to 1*/* The following is only used for the multiple 10 fold cross validations.use "`dca1'", clearforvalues i=2(1)$cv {  * Append all values of the multiple cross validations into the first file  append using "`dca`i''"}* Calculate the average net benefit across all iterations of the multiple* cross validationcollapse all none model model_i, by(threshold)save "Cross Validation DCA Output.dta", replace* Labeling the variables so that the legend will have the proper labelslabel var all "Treat All"label var none "Treat None"label var model "Cross-validated Prediction Model"* Plotting the figure of all the net benefits.twoway (line all threshold if all>-0.05, sort) || (line none model threshold, sort)

SAS

%MACRO CROSSVAL(data=,             /*Name of input dataset*/                out=,              /*Name of output dataset containing calculated net benefit*/                outcome=,          /*outcome variable, 1=event, 0=nonevent*/                predictors=,       /*Covariates in the model*/                v = 10,            /*number of folds in the cross validation*/                repeats = 200,     /*Number of repeated cross validations that will be run*/                /*arguments passed to %DCA() */                xstart=0.01, xstop=0.99, xby=0.01, harm=);  %DO x = 1 %TO &repeats.;    *Load original data and create a variable to be used to 'randomize' patients;    DATA CROSSVAL_dca_of;      SET &data.;      u = RAND("Uniform");    RUN;    *Sort by the event to ensure equal number of patients with the event are in each group;    PROC SORT DATA = CROSSVAL_dca_of;      BY &outcome. u;    RUN;    *Assign each patient into one of ten groups;    DATA CROSSVAL_dca_of;      SET CROSSVAL_dca_of;      group = MOD(_n_, &v.) + 1;    RUN;    *Loop through to run through for each of the ten groups;    %DO y = 1 %TO &v.;      * First, fit the model excluding the yth group.;      PROC LOGISTIC DATA = CROSSVAL_dca_of OUTMODEL = CROSSVAL_df_model&y. DESCENDING NOPRINT;        MODEL &outcome. = &predictors.;        WHERE group ne &y.;      RUN;      *Put yth group into base test dataset;      DATA CROSSVAL_df_test&y.;        SET CROSSVAL_dca_of;        WHERE group = &y.;      RUN;       *Apply the model to the yth group and save the predicted probabilities of the yth group (that was not used in creating the model);      PROC LOGISTIC INMODEL=CROSSVAL_df_model&y. NOPRINT;        SCORE DATA = CROSSVAL_df_test&y. OUT = CROSSVAL_model_pr&y.;      RUN;    %END;    * Combine model predictions for all 10 groups;    DATA CROSSVAL_model_pr(RENAME = (P_1=model_pred));      SET CROSSVAL_model_pr1-CROSSVAL_model_pr&v.;    RUN;    * Run decision curve and save out results;    * For those excluding the optional multiple cross validation, this decision curve (to be seen by using "graph=yes") and the results (saved under the name of your choosing) would be the decision curve corrected for overfit;    %DCA(data=CROSSVAL_model_pr, out=CROSSVAL_dca&x., outcome=&outcome., predictors=model_pred,         graph=no, xstart=&xstart., xstop=&xstop., xby=&xby., harm=&harm.);  *This "%END" statement ends the initial loop for the multiple cross validation. It is also necessary for those who avoided the multiple cross validation by changing the value in the DO loop from 200 to 1;  %END;  DATA CROSSVAL_allcv_pr;    SET CROSSVAL_dca1-CROSSVAL_dca&repeats.;  RUN;  * Calculate the average net benefit across all iterations of the multiple cross validation;  PROC MEANS DATA=CROSSVAL_allcv_pr NOPRINT;    CLASS threshold;    VAR all none model_pred;    OUTPUT OUT=CROSSVAL_minfinal MEAN=all none model_pred;  RUN;  * Save out average net benefit and label variables so that the plot legend will have the proper labels.;  DATA &out. (KEEP=model_pred all none threshold);    SET CROSSVAL_minfinal;    LABEL all="Treat All";    LABEL none="Treat None";    LABEL model_pred="Cross-validated Prediction Model";  RUN;  PROC DATASETS LIB=WORK;    DELETE CROSSVAL_:;  RUN;  QUIT;%MEND CROSSVAL;* Run the crossvalidation macro;%CROSSVAL(data = data_cancer,          out = allcv_pr,          outcome = cancer,          predictors= age famhistory marker,          xstop = 0.5);* Plotting the figure of all the net benefits;PROC GPLOT DATA=allcv_pr;  axis1 ORDER=(-0.05 to 0.15 by 0.05) LABEL=(ANGLE=90 "Net Benefit") MINOR=none;  axis2 ORDER=(0 to 0.5 by 0.1) LABEL=("Threshold Probability") MINOR=none;  legend LABEL=NONE ACROSS=1 CBORDER=BLACK;  PLOT all*threshold  none*threshold  model_pred*threshold /  VAXIS=axis1  HAXIS=axis2  LEGEND=legend OVERLAY;  SYMBOL INTERPOL=JOIN;RUN;QUIT;

Python

# Load dependencies for cross validationimport random  # Library to generate random seedfrom sklearn.model_selection import (    RepeatedKFold,)  # Cross validation data selection and segregation function# Set seed for random processesrandom.seed(112358)# Load simulation datadf_cancer_dx = pd.read_csv(    "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_cancer_dx.csv")# Define the formula (make sure the column names in your DataFrame match these)formula = "cancer ~ marker + age + famhistory"# Create cross-validation objectrkf = RepeatedKFold(n_splits=10, n_repeats=1, random_state=112358)# Placeholder for predictionscv_predictions = []# Perform cross-validationfor train_index, test_index in rkf.split(df_cancer_dx):    # Split data into training and test sets    train = df_cancer_dx.iloc[train_index]    test = df_cancer_dx.iloc[test_index].copy()  # ← make an explicit copy    # Fit the model    model = sm.Logit.from_formula(formula, data=train).fit(disp=0)    # Make predictions on the test set    test["cv_prediction"] = model.predict(test)    # Store predictions    cv_predictions.append(test[["patientid", "cv_prediction"]])# Concatenate predictions from all foldsdf_predictions = pd.concat(cv_predictions)# Calculate mean prediction per patientdf_mean_predictions = (    df_predictions.groupby("patientid")["cv_prediction"].mean().reset_index())# Join with original datadf_cv_pred = pd.merge(df_cancer_dx, df_mean_predictions, on="patientid", how="left")# Decision curve analysisdf_dca_cv = dca(    data=df_cv_pred,    modelnames=["cv_prediction"],    outcome="cancer",    thresholds=np.arange(0, 0.36, 0.01),)# Plot DCA curvesplot_graphs(    plot_df=df_dca_cv,    graph_type="net_benefit",    y_limits=[-0.01, 0.15],    color_names=["blue", "red", "green"],    smooth_frac=0.2,)