Movatterモバイル変換

zcurve: An Implementation of Z-Curves

Description

An implementation of z-curves - a method for estimating expected discovery and replicability rates on the bases of test-statistics of published studies. The package provides functions for fitting the density, EM, and censored EM version (Bartoš & Schimmack, 2022,doi:10.15626/MP.2021.2720; Schimmack & Bartoš, 2023,doi: 10.1371/journal.pone.0290084), as well as the original density z-curve (Brunner & Schimmack, 2020,doi:10.15626/MP.2018.874). Furthermore, the package provides summarizing and plotting functions for the fitted z-curve objects. See the aforementioned articles for more information about the z-curves, expected discovery and replicability rates, validation studies, and limitations.

Author(s)

Maintainer: František Bartošf.bartos96@gmail.com

Authors:

• Ulrich Schimmackulrich.schimmack@utoronto.ca

See Also

Useful links:

• https://fbartos.github.io/zcurve/

• Report bugs athttps://github.com/FBartos/zcurve/issues

Z-scores from subset of original studies featured in OSC 2015reproducibility project

Description

The dataset contains z-scores from subset of originalstudies featured in psychology reproducibility project(Collaboration and others 2015). Only z-scores from studies with unambiguousoriginal outcomes are supplied (eliminating 7 studies with marginallysignificant results). The real replication rate for those studies is35/90 (the whole project reports 36/97).

Usage

OSC.z

Format

A vector with 90 observations

References

Collaboration OS, others (2015).“Estimating the reproducibility of psychological science.”Science,349(6251).doi:10.1126/science.aac4716.

Control settings for the zcurve EM algorithm

Description

All these settings are passed to the Expectation Maximizationfitting algorithm. All unspecified settings are set to the default value.Settingmodel = "EM" sets all settings to the defaultvalue irrespective of any other setting and fits z-curve as described inBartoš and Schimmack (2022)

Arguments

References

Bartoš F, Schimmack U (2022).“Z-curve 2.0: Estimating replication rates and discovery rates.”Meta-Psychology,6.doi:10.15626/MP.2021.2720.

See Also

zcurve(),control_density

Examples

# to increase the number of starting fits# and change the means of the mixture componentsctrl <- list(   fit_reps  = 50,   mu = c(0, 1.5, 3, 4.5, 6))## Not run: zcurve(OSC.z, method = "EM", control = ctrl)

Control settings for the z-curve 2.0 density algorithm

Description

All settings are passed to the density fittingalgorithm. All unspecified settings are set to the default value.Settingmodel = "KD2" sets all settings to the defaultvalue irrespective of any other setting and fits z-curve asdescribe in Bartoš and Schimmack (2022). In order to fit thez-curve 1.0 density algorithm, setmodel = "KD1" and go tocontrol_density_v1

Arguments

References

Bartoš F, Schimmack U (2022).“Z-curve 2.0: Estimating replication rates and discovery rates.”Meta-Psychology,6.doi:10.15626/MP.2021.2720.

See Also

zcurve(),control_density_v1,control_EM

Examples

# to decrease the criterion and increase the number of iterationsctrl <- list(   max_iter  = 300,   criterion = 1e-4)## Not run: zcurve(OSC.z, method = "density", control = ctrl)

Control settings for the original z-curve density algorithm

Description

All settings are passed to the density fittingalgorithm. All unspecified settings are set to the default value.Settingmodel = "KD1" sets all settings to the defaultvalue irrespective of any other setting and fits z-curve as describedin Brunner and Schimmack (2020).

Arguments

References

Brunner J, Schimmack U (2020).“Estimating population mean power under conditions of heterogeneity and selection for significance.”Meta-Psychology,4.doi:10.15626/MP.2018.874.

See Also

zcurve(),control_density,control_EM

Examples

# to increase the number of iterationsctrl <- list(   version   = 1,   max_iter  = 300)## Not run: zcurve(OSC.z, method = "density", control = ctrl)

Prints first few rows of a z-curve data object

Description

Prints first few rows of a z-curve data object

Usage

## S3 method for class 'zcurve_data'head(x, ...)

Arguments

See Also

zcurve_data()

Reports whether x is a zcurve object

Description

Reports whether x is a zcurve object

Usage

is.zcurve(x)

Arguments

Plot fitted z-curve object

Description

Plot fitted z-curve object

Usage

## S3 method for class 'zcurve'plot(  x,  annotation = FALSE,  CI = FALSE,  extrapolate = FALSE,  plot_type = "base",  y.anno = c(0.95, 0.88, 0.78, 0.71, 0.61, 0.53, 0.43, 0.35),  x.anno = 0.6,  cex.anno = 1,  ...)

Arguments

See Also

zcurve()

Examples

## Not run: # simulate some z-statistics and fit a z-curvez <- abs(rnorm(300,3))m.EM <- zcurve(z, method = "EM", bootstrap = 100)# plot the z-curveplot(m.EM)# add annotation text and model fit CIplot(m.EM, annotation = TRUE, CI = TRUE)# change the location of the annotation to the leftplot(m.EM, annotation = TRUE, CI = TRUE, x_text = 0)## End(Not run)

Compute z-score corresponding to a power

Description

A function for computing z-scores of two-sided testscorresponding to powerpower for a given significance levelalphaalpha (or corresponding cut-off z-statistica).

Usage

power_to_z(  power,  alpha = 0.05,  a = stats::qnorm(alpha/2, lower.tail = FALSE),  two.sided = TRUE,  nleqslv_control = list(xtol = 1e-15, maxit = 300, stepmax = 0.5))

Arguments

Examples

# z-scores corresponding to the (aproximate) power of components of EM2power_to_z(c(0.05, 0.20, 0.40, 0.60, 0.80, 0.974, 0.999), alpha = .05)

Prints estimates from z-curve object

Description

Prints estimates from z-curve object

Usage

## S3 method for class 'zcurve'print.estimates(x, ...)

Arguments

See Also

zcurve()

Prints summary object for z-curve method

Description

Prints summary object for z-curve method

Usage

## S3 method for class 'zcurve'print.summary(x, ...)

Arguments

See Also

zcurve()

Prints a fitted z-curve object

Description

Prints a fitted z-curve object

Usage

## S3 method for class 'zcurve'print(x, ...)

Arguments

See Also

zcurve()

Prints a z-curve data object

Description

Prints a z-curve data object

Usage

## S3 method for class 'zcurve_data'print(x, ...)

Arguments

See Also

zcurve_data()

Summarize fitted z-curve object

Description

Summarize fitted z-curve object

Usage

## S3 method for class 'zcurve'summary(  object,  type = "results",  all = FALSE,  ERR.adj = 0.03,  EDR.adj = 0.05,  round.coef = 3,  conf.level = 0.95,  ...)

Arguments

Value

Summary of a z-curve object

See Also

zcurve()

Compute power corresponding to z-scores

Description

A function for computing power of two-sided testscorresponding to z-scores for a given significance level.alpha (or corresponding cut-off z-scorea)

Usage

z_to_power(  z,  alpha = 0.05,  a = stats::qnorm(alpha/2, lower.tail = FALSE),  two.sided = TRUE)

Arguments

Examples

# mean powers corresponding to the mean components of KD2z_to_power(0:6, alpha = .05)

Fit a z-curve

Description

zcurve is used to fit z-curve models. The functiontakes input of z-statistics or two-sided p-values and returns object ofclass"zcurve" that can be further interrogated by summary and plotfunction. It default to EM model, but different version of z-curves canbe specified using themethod andcontrol arguments. See'Examples' and 'Details' for more information.

Usage

zcurve(  z,  z.lb,  z.ub,  p,  p.lb,  p.ub,  data,  method = "EM",  bootstrap = 1000,  parallel = FALSE,  control = NULL)

Arguments

Details

The function returns the EM method by default and changingmethod = "density" gives the KD2 version of z-curve as outlined inBartoš and Schimmack (2022). For the original z-curve(Brunner and Schimmack 2020), referred to as KD1, specify'control = "density", control = list(model = "KD1")'. Specifyingthe lower and upper bounds of z-scores or p-values will fit the censoredversion of z-curve described in (Schimmack and Bartoš 2023).

Value

The fitted z-curve object

References

Bartoš F, Schimmack U (2022).“Z-curve 2.0: Estimating replication rates and discovery rates.”Meta-Psychology,6.doi:10.15626/MP.2021.2720.

Brunner J, Schimmack U (2020).“Estimating population mean power under conditions of heterogeneity and selection for significance.”Meta-Psychology,4.doi:10.15626/MP.2018.874.

Schimmack U, Bartoš F (2023).“Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values.”PLoS ONE,18(8), e0290084.doi:10.1371/journal.pone.0290084.

See Also

summary.zcurve(),plot.zcurve(),control_EM,control_density

Examples

# load data from OSC 2015 reproducibility projectOSC.z# fit an EM z-curve (with disabled bootstrap due to examples times limits)m.EM <- zcurve(OSC.z, method = "EM", bootstrap = FALSE)# a version with 1000 boostraped samples would looked like:m.EM <- zcurve(OSC.z, method = "EM", bootstrap = 1000)# or KD2 z-curve (use larger bootstrap for real inference)m.D <- zcurve(OSC.z, method = "density", bootstrap = FALSE)# inspect the resultssummary(m.EM)summary(m.D)# see '?summary.zcurve' for more output options# plot the resultsplot(m.EM)plot(m.D)# see '?plot.zcurve' for more plotting options# to specify more options, set the control arguments# ei. increase the maximum number of iterations and change alpha levelctr1 <- list(  "max_iter" = 9999,  "alpha"    = .10  )## Not run: m1.EM <- zcurve(OSC.z, method = "EM", bootstrap = FALSE, control = ctr1)# see '?control_EM' and '?control_density' for more information about different# z-curves specifications

z-curve estimates

Description

The following functions extract estimatesfrom the z-curve object.

Usage

ERR(object, round.coef = 3)EDR(object, round.coef = 3)ODR(object, round.coef = 3)Soric(object, round.coef = 3)file_drawer_ration(object, round.coef = 3)expected_n(object, round.coef = 0)missing_n(object, round.coef = 0)significant_n(object)included_n(object)

Arguments

Details

Technically, ODR, significant n, and included nare not z-curve estimates but they are grouped in thiscategory for convenience.

See Also

zcurve()

Fit a z-curve to clustered data

Description

zcurve_clustered is used to fit z-curve models toclustered data. The function requires a data object created with thezcurve_data() function as the input (where id denotes clusters).Two different methods that account for clustering ar implemented viathe EM model:"w" for down weighting the likelihood of the teststatistics proportionately to the number of repetitions in the clusters,and"b" for a nested bootstrap where only a single study from eachbootstrap is selected for model fitting.

Usage

zcurve_clustered(  data,  method = "b",  bootstrap = 1000,  parallel = FALSE,  control = NULL)

Arguments

Value

The fitted z-curve object

References

There are no references for Rd macro⁠\insertAllCites⁠ on this help page.

See Also

zcurve(),summary.zcurve(),plot.zcurve(),control_EM,control_density

Prepare data for z-curve

Description

zcurve_data is used to prepare data for thezcurve() function. The function transform strings containingreported test statistics"z", "t", "f", "chi", "p" into two-sidedp-values. Test statistics reported as inequalities are as consideredto be censored as well as test statistics reported with low accuracy(i.e., rounded to too few decimals). See details for more information.

Usage

zcurve_data(data, id = NULL, rounded = TRUE, stat_precise = 2, p_precise = 3)

Arguments

Details

By default, the function extract the type of test statistic:

"F(df1, df2)=x"

F-statistic with df1 and df2 degrees of freedom,

"chi(df)=x"

Chi-square statistic with df degrees of freedom,

"t(df)=x"

for t-statistic with df degrees of freedom,

"z=x"

for z-statistic,

"p=x"

for p-value.

The input is not case sensitive and automatically removes empty spaces. Furthermore,inequalities ("<" and">") can be used to denote censoring. I.e., thatthe p-value is lower than"x" or that the test statistic is larger than"x"respectively. The automatic de-rounding procedure (ifrounded = TRUE) treatsp-values with less decimal places than specified inp_precise or test statisticswith less decimal places than specified instat_precise as censored on an intervalthat could result in a given rounded value. I.e., a"p = 0.03" input would bede-rounded as a p-value lower than 0.035 but larger than 0.025.

Value

An object of type"zcurve_data".

See Also

zcurve(),print.zcurve_data(),head.zcurve_data()

Examples

# Specify a character vector containing the test statisticsdata <- c("z = 2.1", "t(34) = 2.21", "p < 0.03", "F(2,23) > 10", "p = 0.003")# Obtain the z-curve data objectdata <- zcurve_data(data)# inspect the resulting objectdata

Options for the zcurve package

Description

A placeholder object and functions for the zcurve package.(adapted from the runjags R package).

Usage

zcurve.options(...)zcurve.get_option(name)

Arguments

Value

The current value of all available zcurve options (after applying anychanges specified) is returned invisibly as a named list.