Skip to contents

Statistical metrics with confidence intervals

Source: R/raptools.R
CI.raplot.Rd

The CI.raplot function produces summary metrics for risk assessment. Outputs the NRI, IDI, weighted NRI and category Free NRI all for those with events and those without events. Also the AUCs of the two models and the comparison (DeLong) between AUCs. Output includes confidence intervals. Uses statistics.raplot. Displayed graphically by raplot.

Usage

CI.raplot(
  x1,
  x2 = NULL,
  y = NULL,
  t = NULL,
  NRI_return = FALSE,
  conf.level = 0.95,
  n.boot = 1000,
  dp = 3
)

Arguments

x1

Either a logistic regression fitted using glm (base package) or lrm (rms package) or calculated probabilities (eg through a logistic regression model) of the baseline model. Must be between 0 & 1

x2

Either a logistic regression fitted using glm (base package) or lrm (rms package) or calculated probabilities (eg through a logistic regression model) of the new (alternative) model. Must be between 0 & 1

y

Binary of outcome of interest. Must be 0 or 1 (if fitted models are provided this is extracted from the fit which for an rms fit must have x = TRUE, y = TRUE).

t

The risk threshold(s) for groups. eg t<-c(0,0.1,1) is a two group model with a threshold of 0.1 & t<-c(0,0.1,0.3,1) is a three group model with thresholds at 0.1 and 0.3.

NRI_return

If NRI statistics are required (default = FALSE).

conf.level

The confidence interval expressed as a fraction of 1 (ie 0.95 is the 95% confidence interval )

n.boot

The number of "bootstraps" to use. Performance slows down with more bootstraps. For trialling result, use a low number (eg 5), for accuracy use a large number (eg 2000)

dp

The number of decimal places to display

Value

A list with the following elements:

meta_data

A data.frame with thresholds, confidence interval, number of bootstraps, input data type and decimal places.

Metrics

Point estimates of the statistical metrics (see function docs).

Each_bootstrap_metrics

List of per-bootstrap metric results.

Summary_metrics

A table of summary metrics with confidence intervals (e.g. Total, Events, Non-events, NRI, IDI, AUCs, Brier scores, etc.).

References

Pencina, M. J., D'Agostino, R. B., & Vasan, R. S. (2008). Evaluating the added stats::predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statistics in Medicine, 27(2), 157-172. doi:10.1002/sim.2929

Examples

if (FALSE) { # \dontrun{
data(data_risk)
y<-data_risk$outcome 
x1<-data_risk$baseline
x2<-data_risk$new
t<-c(0,0.19,1) 
#e.g.
output<-CI.raplot(x1, x2, y, t, conf.level = 0.95, n.boot = 5, dp = 2) 
} # }