Movatterモバイル変換

Title:	Power for Meta-Analysis of Dependent Effects
Version:	0.2.0
BugReports:	https://github.com/MikkelVembye/POMADE/issues
Description:	Provides functions to compute and plot power levels, minimum detectable effect sizes, and minimum required sample sizes for the test of the overall average effect size in meta-analysis of dependent effect sizes.
Depends:	R (≥ 4.1.0)
License:	MIT + file LICENSE
Encoding:	UTF-8
LazyData:	true
RoxygenNote:	7.2.3
Imports:	ggplot2, dplyr, magrittr, purrr, future, furrr, stats,stringr, utils, tidyr, tibble
Suggests:	covr, roxygen2, testthat (≥ 3.0.0)
Config/testthat/edition:	3
Language:	en-US
URL:	https://mikkelvembye.github.io/POMADE/
NeedsCompilation:	no
Packaged:	2024-02-13 18:45:25 UTC; B199526
Author:	Mikkel H. Vembye [aut, cre], James E. Pustejovsky [aut]
Maintainer:	Mikkel H. Vembye <mikkel.vembye@gmail.com>
Repository:	CRAN
Date/Publication:	2024-02-13 20:41:20 UTC

Co-Teaching Dataset

Description

Data from a meta-analysis on the effects of collaborative modelsof instruction on student achievement from Vembye, Weiss, andBhat (2023).

Usage

VWB23_pilot

Format

A tibble with 76 rows/studies and 9 variables

study_year

Study author and year of publication

studyid

Unique study ID

esid

Unique effect size ID

kj

Number of effect sizes per study

N_meanj

Average sample size of study

Nt_meanj

Average sample size of treatment group within study

Nc_meanj

Average sample size of control group within study

ESS_meanj

Roughly approximated effective sample sizes

vg_ms_mean

Average cluster bias corrected sampling variance estimates

Source

Find background material onVembye's OSF page,and the preprint athttps://osf.io/preprints/metaarxiv/mq5v7/.

References

Vembye, M. H., Weiss, F., & Bhat, B. H. (2023). The EffectsCo-Teaching and Related Collaborative Models of Instruction onStudent Achievement: A Systematic Review and Meta-Analysis.Review ofEducational Research,doi:10.3102/00346543231186588

Cluster Bias Correction

Description

Function to conduct cluster bias correction of sampling variance estimatesobtained from cluster-randomized studies in which the reported variance does not account for clustering.

Usage

cluster_bias_adjustment(sigma2js, cluster_size = 22, icc = 0.2)

Arguments

Value

Returns a vector of cluster bias adjusted variance estimates

Examples

cbc_var <- cluster_bias_adjustment(  sigma2js = c(0.04, 0.06, 0.08, 0.1),  cluster_size = 15,  icc = 0.15)cbc_var

Approximate Effective Sample Sizes

Description

Approximate Effective Sample Sizes

Usage

effective_sample_sizes(  sample_sizes_raw = NULL,  Nt_raw = NULL,  Nc_raw = NULL,  cluster_size = 22,  icc = 0.22)

Arguments

Details

N_j/DE

Value

A vector of effective sample sizes, adjusted for cluster-dependence.

Examples

sample_sizes <- sample(50:1000, 50, replace = TRUE)effective_sample_sizes(  sample_sizes_raw = sample_sizes,  cluster_size = 20,  icc = 0.15)

Minimum Detectable Effect Size (MDES) for Meta-Analysis With DependentEffect Sizes

Description

Compute the minimum detectable effect size in a meta-analysis ofdependent effect size estimates, given a specified number of studies, powerlevel, estimation method, and further assumptions about the distribution ofstudies.

Usage

mdes_MADE(  J,  tau,  omega,  rho,  alpha = 0.05,  target_power = 0.8,  d = 0,  model = "CHE",  var_df = "RVE",  sigma2_dist = NULL,  n_ES_dist = NULL,  iterations = 100,  seed = NULL,  warning = TRUE,  upper = 2,  show_lower = FALSE)

Arguments

Value

Returns atibble with information about the expectation of thenumber of studies, the between-study and within-study variance components,the sample correlation, the contrast effect, the level of statisticalsignificance, the target power value(s), the minimum detectable effectsize, the number of iterations, the model to handle dependent effect sizes,and the methods used to obtain sampling variance estimates as well as thenumber effect sizes per study.

Examples

mdes_MADE(  J = 30,  tau = 0.05,  omega = 0.02,  rho = 0.2,  model = "CHE",  var_df = "RVE",  sigma2_dist = 4 / 100,  n_ES_dist = 6,  seed = 10052510)

Finding the Number of Studies Needed to Obtain a Certain Amount ofPower

Description

Compute the minimum number of studies needed to obtain aspecified power level in a meta-analysis of dependent effect sizeestimates, given an effect size of practical concern, estimation method,and further assumptions about the distribution of studies.

Usage

min_studies_MADE(  mu,  tau,  omega,  rho,  alpha = 0.05,  target_power = 0.8,  d = 0,  model = "CHE",  var_df = "RVE",  sigma2_dist = NULL,  n_ES_dist = NULL,  iterations = 100,  seed = NULL,  warning = TRUE,  upper = 100,  show_lower = FALSE)

Arguments

Value

Returns atibble with information about the expectation of theeffect size of practical concern, the between-study and within-studyvariance components, the sample correlation, the contrast effect, the levelof statistical significance, the target power value(s), the number ofstudies needed, the number of iterations, the model to handle dependenteffect sizes, and the methods used to obtain sampling variance estimates aswell as the number effect sizes per study.

Examples

min_studies_MADE(  mu = 0.3,  tau = 0.05,  omega = 0.01,  rho = 0.2,  target_power = .7,  alpha = 0.05,  model = "CE",  var_df = "RVE",  sigma2_dist = 4 / 200,  n_ES_dist = 5.5,  seed = 10052510)

Generic plot function for 'MADE' objects

Description

Create a faceted plot displaying the results of a set of poweranalyses. This is a generic function to make facet_grid plots, withspecific methods defined forpower_MADE,mdes_MADE, andmin_studies_MADE objects.

Usage

plot_MADE(  data,  v_lines,  legend_position,  color,  numbers,  number_size,  numbers_ynudge,  caption,  x_lab,  x_breaks,  x_limits,  y_breaks,  y_limits,  y_expand = NULL,  warning,  traffic_light_assumptions,  ...)

Arguments

Value

Aggplot object

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (Inpreparation). Conducting power analysis for meta-analysis of dependenteffect sizes: Common guidelines and an introduction to the POMADE Rpackage.

See Also

plot_MADE.power,plot_MADE.mdes,plot_MADE.min_studies

Examples

power_dat <-  power_MADE(    J = c(50, 56),    mu = 0.15,    tau = 0.1,    omega = 0.05,    rho = 0,    sigma2_dist = 4 / 200,    n_ES_dist = 6  )power_example <-  plot_MADE(   data = power_dat,   power_min = 0.8,   expected_studies = c(52, 54),   warning = FALSE,   caption = TRUE,   color = TRUE,   model_comparison = FALSE,   numbers = FALSE   )power_example

Plot function for a 'mdes' object

Description

Creates a faceted plot for minimum detectable effect size (mdes)analyses calculated usingmdes_MADE.

Usage

## S3 method for class 'mdes'plot_MADE(  data,  v_lines = NULL,  legend_position = "bottom",  color = TRUE,  numbers = TRUE,  number_size = 2.5,  numbers_ynudge = NULL,  caption = TRUE,  x_lab = NULL,  x_breaks = NULL,  x_limits = NULL,  y_breaks = ggplot2::waiver(),  y_limits = NULL,  y_expand = NULL,  warning = TRUE,  traffic_light_assumptions = NULL,  es_min = NULL,  expected_studies = NULL,  ...)

Arguments

Details

In general, it can be rather difficult to guess/approximate the truemodel parameters and sample characteristics a priori. Calculating theminimum detectable effect size under just a single set of assumptions caneasily be misleading even if the true model and data structure onlyslightly diverge from the yielded data and model assumptions. To maximizethe informativeness of the analysis, Vembye, Pustejovsky, & Pigott (Inpreparation) suggest accommodating the uncertainty of the powerapproximations by reporting or plotting minimum detectable effect sizeestimates across a range of possible scenarios, which can be done usingplot_MADE.mdes.

Value

Aggplot plot showing the minimum detectable effectsize across the expected number of studies, faceted by the between-study andwithin-study SDs, with different colors, lines, and shapes corresponding todifferent values of the assumed sample correlation.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (Inpreparation). Conducting power analysis for meta-analysis of dependenteffect sizes: Common guidelines and an introduction to the POMADE Rpackage.

See Also

plot_MADE

Examples

mdes_MADE(  J = c(25, 35),  tau = 0.05,  omega = 0,  rho = 0,  target_power = .6,  alpha = 0.1,  sigma2_dist = 4 / 200,  n_ES_dist = 8,  seed = 10052510) |>  plot_MADE(expected_studies = c(28, 32), numbers = FALSE)

Plot function for a 'min_studies' object

Description

Creates a faceted plot with analyses of the minimum number ofstudies needed to obtained a given effect size with specified levels ofpower, as calculated usingmin_studies_MADE.

Usage

## S3 method for class 'min_studies'plot_MADE(  data,  v_lines = NULL,  legend_position = "bottom",  color = TRUE,  numbers = TRUE,  number_size = 2.5,  numbers_ynudge = NULL,  caption = TRUE,  x_lab = NULL,  x_breaks = NULL,  x_limits = NULL,  y_breaks = ggplot2::waiver(),  y_limits = NULL,  y_expand = NULL,  warning = TRUE,  traffic_light_assumptions = NULL,  v_shade = NULL,  h_lines = NULL,  ...)

Arguments

Details

In general, it can be rather difficult to guess/approximate the truemodel parameters and sample characteristics a priori. Calculating theminimum number of studies needed under just a single set of assumptions caneasily be misleading even if the true model and data structure onlyslightly diverge from the yielded data and model assumptions. To maximizethe informativeness of the analysis, Vembye, Pustejovsky, &Pigott (In preparation) suggest accommodating the uncertainty of the powerapproximations by reporting or plotting power estimates across a range ofpossible scenarios, which can be done usingplot_MADE.power.

Value

Aggplot plot showing the minimum number of studies needed toobtain a given effect size with a certain amount of power and level-alpha, facetedacross levels of the within-study SD and the between-study SD,with different colors, lines, and shapes corresponding to different valuesof the assumed sample correlation. Iflength(unique(data$mu)) > 1, itreturns aggplot plot showing the minimum studies neededto obtained a given effect size with a certain amount of power andlevel-alpha across effect sizes of practical concern, faceted by thebetween-study and within-study SDs, with different colors, lines, andshapes corresponding to different values of the assumed sample correlation.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (Inpreparation). Conducting power analysis for meta-analysis of dependenteffect sizes: Common guidelines and an introduction to the POMADE Rpackage.

See Also

plot_MADE

Examples

min_studies_MADE(  mu = c(0.25, 0.35),  tau = 0.05,  omega = 0.02,  rho = 0.2,  target_power = .7,  sigma2_dist = 4 / 200,  n_ES_dist = 6,  seed = 10052510) |>  plot_MADE(y_breaks = seq(0, 10, 2), numbers = FALSE)

Plot function for a 'power' object

Description

Creates a faceted plot or plots for power analyses conductedwithpower_MADE.

Usage

## S3 method for class 'power'plot_MADE(  data,  v_lines = NULL,  legend_position = "bottom",  color = TRUE,  numbers = TRUE,  number_size = 2.5,  numbers_ynudge = 0,  caption = TRUE,  x_lab = NULL,  x_breaks = NULL,  x_limits = NULL,  y_breaks = seq(0, 1, 0.2),  y_limits = c(0, 1),  y_expand = NULL,  warning = TRUE,  traffic_light_assumptions = NULL,  power_min = NULL,  expected_studies = NULL,  model_comparison = FALSE,  ...)

Arguments

Details

In general, it can be rather difficult to guess/approximate the truemodel parameters and sample characteristics a priori. Calculating powerunder only a single set of assumptions can easily be misleading even if thetrue model and data structure only slightly diverge from the yielded dataand model assumptions. To maximize the informativeness of the powerapproximations, Vembye, Pustejovsky, & Pigott (In preparation) suggestaccommodating the uncertainty of the power approximations by reporting orplotting power estimates across a range of possible scenarios, which can bedone usingplot_MADE.power.

Value

Aggplot plot showing power across the expected number ofstudies, faceted by the between-study and within-study SDs, with differentcolors, lines, and shapes corresponding to different values of the assumedsample correlation. Ifmodel_comparison = TRUE, it returns aggplot plot showing power across the expected number of studies,faceted by the between-study and within-study SDs, with different colors,lines, and shapes corresponding to different working models for dependenteffect size estimates

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (Inpreparation). Conducting power analysis for meta-analysis of dependenteffect sizes: Common guidelines and an introduction to the POMADE Rpackage.

See Also

plot_MADE

Examples

power_dat <-  power_MADE(    J = c(50, 56),    mu = 0.15,    tau = 0.1,    omega = 0.05,    rho = 0,    sigma2_dist = 4 / 200,    n_ES_dist = 6  )power_example <-  plot_MADE(   data = power_dat,   power_min = 0.8,   expected_studies = c(52, 54),   warning = FALSE,   caption = TRUE,   color = TRUE,   model_comparison = FALSE,   numbers = FALSE   )power_example

Power Approximation for Overall Average Effects in Meta-Analysis With Dependent Effect Sizes

Description

Compute power of the test of the overall average effect size ina meta-analysis of dependent effect size estimates, given a specifiednumber of studies, effect size of practical concern, estimation method, andfurther assumptions about the distribution of studies.

Usage

power_MADE(  J,  mu,  tau,  omega,  rho,  alpha = 0.05,  d = 0,  model = "CHE",  var_df = "RVE",  sigma2_dist = NULL,  n_ES_dist = NULL,  iterations = 100,  seed = NULL,  warning = TRUE,  average_power = TRUE)

Arguments

Details

Find all background material behind the power approximations inVembye, Pustejovsky, & Pigott (2022), including arguments for why it is suggestedneither to conduct power analysis based on balanced assumptions aboutthe number of effects per study and the study variance nor to use the originalpower approximation assuming independence among effect sizes (Hedges & Pigott, 2001).

Value

Returns atibble with information about the expectation of thenumber of studies, the effect size of practical concern, the between-studyand within-study variance components, the sample correlation, the contrasteffect, the level of statistical significance, the sampling variance ofoverall average effect size of practical concern, the degrees of freedom,the power, the mcse, the number of iterations, the model to handledependent effect sizes, and the methods used to obtain sampling varianceestimates as well as the number effect sizes per study.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (2022).Power approximations for overall average effects in meta-analysis with dependent effect sizes.Journal of Educational and Behavioral Statistics, 1–33.doi:10.3102/10769986221127379

Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis.Psychological Methods, 6(3), 203–217.doi:10.1037/1082-989X.6.3.203

Examples

power <- power_MADE(   J = c(40, 60),   mu = 0.2,   tau = 0.2,   omega = 0.1,   rho = 0.7,   sigma2_dist = \(x) rgamma(x, shape = 5, rate = 10),   n_ES_dist = \(x) 1 + stats::rpois(x, 5.5 - 1),   model = c("CHE", "MLMA", "CE"),   var_df = c("Model", "Satt", "RVE"),   alpha = .05,   seed = 10052510,   iterations = 5 )power

Between-Study Variance Approximation Function

Description

Rough approximation of the between-study variance based onassumption about the typical sample size of studies included in thesynthesis

Usage

tau2_approximation(sample_size = 100, es, df_minus2 = TRUE)

Arguments

Value

Atibble with small, medium, and large magnitudes of tau2

Examples

tau2_approximation(sample_size = 50,es = 0.1,df_minus2 = TRUE)