| Version: | 1.14 |
| Type: | Package |
| Title: | Computerized Adaptive Testing with Multidimensional ItemResponse Theory |
| Description: | Provides tools to generate HTML interfaces for adaptive and non-adaptive tests using the shiny package (Chalmers (2016) <doi:10.18637/jss.v071.i05>). Suitable for applying unidimensional and multidimensional computerized adaptive tests (CAT) using item response theory methodology and for creating simple questionnaires forms to collect response data directly in R. Additionally, optimal test designs (e.g., "shadow testing") are supported for tests that contain a large number of item selection constraints. Finally, package contains tools useful for performing Monte Carlo simulations for studying test item banks. |
| Depends: | mirt (≥ 1.37), shiny (≥ 1.0.1) |
| Imports: | lattice, stats, Rcpp, methods, markdown, pbapply, lpSolve |
| Suggests: | shinythemes, parallel, knitr |
| Encoding: | UTF-8 |
| ByteCompile: | yes |
| LazyLoad: | yes |
| VignetteBuilder: | knitr |
| LinkingTo: | Rcpp, RcppArmadillo |
| License: | GPL (≥ 3) |
| Repository: | CRAN |
| Maintainer: | Phil Chalmers <rphilip.chalmers@gmail.com> |
| URL: | https://github.com/philchalmers/mirtCAT,https://github.com/philchalmers/mirtCAT/wiki,https://groups.google.com/forum/#!forum/mirt-package |
| BugReports: | https://github.com/philchalmers/mirtCAT/issues?state=open |
| RoxygenNote: | 7.3.2 |
| NeedsCompilation: | yes |
| Packaged: | 2024-07-12 18:29:49 UTC; phil |
| Author: | Phil Chalmers |
| Date/Publication: | 2024-07-12 19:50:05 UTC |
Computerized Adaptive Testing with Multidimensional Item Response Theory
Description
Computerized Adaptive Testing with Multidimensional Item Response Theory
Details
Provides tools to generate an HTML interface for creating adaptive and non-adaptive educational and psychological tests using the shiny package. Suitable for applying unidimensional and multidimensional computerized adaptive tests using item response theory methodology and for creating simple questionnaires forms to collectresponse data directly in R.
Users interested in the most recent version of this package can visithttps://github.com/philchalmers/mirtCAT and follow the instructionsfor installing the package from source (additional details about installing from Github can be found athttps://github.com/philchalmers/mirt). Questions regarding the package can be sent to the mirt-package Google Group, located athttps://groups.google.com/forum/#!forum/mirt-package.
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
Compute the values given the criteria and internal objects
Description
A function that returns a named vector of evaluated criteria for each respective item in the test bank. The names are associated with the item number in the bank. Note that criteria values are returned such that the maximum value always represents the most optimal item (e.g., maximum information). In cases where the minimum value is typically selected (e.g., minimum variance) all values are multiplied by -1 to turn it into a maximizationproblem.
Usage
computeCriteria( x, criteria, person = NULL, test = NULL, design = NULL, subset = NULL, info_mats = FALSE)Arguments
x | an object of class 'mirtCAT_design' returned from the |
criteria | item selection criteria (see |
person | (required when |
test | (required when |
design | (required when |
subset | an integer vector indicating which items should be included in the optimal search;the default |
info_mats | logical; if more than one trait is present in the test, should the respective informationmatricies be returned instead of the scalar summary statistics (e.g., D-rule). When TRUE will return a list of matricies associated with each respective item |
Value
a vector of criteria values for each respective item
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
mirtCAT,updateDesign,extract.mirtCAT,findNextItem
Examples
## Not run: # test defined in mirtCAT help file, first exampleCATdesign <- mirtCAT(df, mod, design_elements = TRUE)computeCriteria(CATdesign, criteria = 'MI')computeCriteria(CATdesign, criteria = 'MEI')## End(Not run)Create a unique GUI session name from a string of characters
Description
This is used inmirtCAT to create a random session name so thatshiny knows which environment to select objects from when multiple CATsessions have been initialized.
Usage
createSessionName(n = 30, datetime = TRUE)Arguments
n | number of upper/lower characters to sample |
datetime | logical; include the current date/time the function was calledin the string as well? This further helps with the uniqueness of the generatedstring |
Value
a list containing the internal environmental components for mirtCAT
Function returning an object used by shiny
Description
This function returns the GUI setup results by callingshinyApp. Primarily, this is only useful when hosting the application publicly, such as throughhttps://www.shinyapps.io/. The functionmirtCAT_preamble must be runbefore this function is called. The object is executed by callingrunApp.
Usage
createShinyGUI(ui = NULL, host_server = TRUE)Arguments
ui | a shiny UI function used to define the interface. If |
host_server | logical; is |
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
mirtCAT,mirtCAT_preamble,getPerson
Examples
## Not run: mirtCAT_preamble(df=df)runApp(createShinyGUI(host_server = FALSE), port = 8000) # run locallyperson <- getPerson()summary(person)runApp(createShinyGUI(), port = 8000) # for remote server hosting## End(Not run)Extract elements from the internal person, test, and design objects
Description
This function extracts elements, as well as builds a few convenient elements, from the three internalperson,design, ortestobjects that are accessible through acustomNextItem function definition (seemirtCAT for details).
Usage
extract.mirtCAT(x, what)Arguments
x | either the |
what | a character vector extracting the desired element (see the Details section) |
Details
Depending on which object is supplied, the following elements can be extracted.
The 'person' argument
IDa scalar value indicating the ID of the participant (generally only needed in Monte Carlo simulations)
responsesan integer vector indicating how items that have been responded to. Each element pertains to the associated item location (e.g.,
responses[100]is associated with the 100th item), and isNAif the item has not been responded toraw_responsesof the same form as
responses, pertaining to the observed responsesin a character vectoritems_in_bankan integer vector indicating items which have not been administered yet and are also valid candidates for administration
items_answeredan integer vector indicating the order in which items have been responded to
thetasthe current ability/latent trait estimates given the previously administered items
thetas_SEthe current ability/latent trait standard error estimates given the previously administered items
thetas_historyhistory of the ability/latent trait estimates
thetas_SE_historyhistory of the latent trait standard error estimates
item_timeof the same form as
items_answered, pertaining to the amount of time it took the participant to response to the itemdemographicsa data.frame containing the (optional) prior survey information from the GUI interface
clientDataa list of useful information from shiny's
session$clientData
The 'design' argument
items_not_scoredan integer vector indicating items which should be included but not scored in the test (these are experimental items)
min_itemsminimum number of items to administer
max_itemsmaximum number of items to administer
max_timemaximum amount of time alloted to the GUI
met_SEMlogical vector indicating whether the SEM criteria has been met
met_delta_thetaslogical vector indicating whether the delta_thetas criteria has been met
met_classifylogical vector indicating whether the classify criteria has been met
exposureexposure control elements of the same form as
responsescontentcontent constraint information
content_propcontent proportions
test_propertiesuser-defined
data.frameof test-based propertiesperson_propertiesuser-defined
data.frameof person-based properties
The 'test' argument
moextract the defined model from the
mirtpackage. Afterward, users can use theextract.mirtfunction to pull out a large number of internal elements for easy use
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
mirt,mirtCAT,extract.mirt,findNextItem
Examples
## Not run: #example testset.seed(1234)nitems <- 25itemnames <- paste0('Item.', 1:nitems)a <- matrix(rlnorm(nitems, .2, .3))d <- matrix(rnorm(nitems))dat <- simdata(a, d, 500, itemtype = 'dich')colnames(dat) <- itemnamesmod <- mirt(dat, 1, verbose = FALSE, TOL = .01)# simple math itemsquestions <- answers <- character(nitems)choices <- matrix(NA, nitems, 5)spacing <- floor(d - min(d)) + 1 #easier items have more variation in the optionsfor(i in 1:nitems){ n1 <- sample(1:50, 1) n2 <- sample(51:100, 1) ans <- n1 + n2 questions[i] <- paste0(n1, ' + ', n2, ' = ?') answers[i] <- as.character(ans) ch <- ans + sample(c(-5:-1, 1:5) * spacing[i,], 5) ch[sample(1:5, 1)] <- ans choices[i, ] <- as.character(ch)}df <- data.frame(Question=questions, Option=choices, Type = 'radio', stringsAsFactors = FALSE)df$Answer <- answerspat <- generate_pattern(mod, Theta = 0, df)#------------------------------------------------# administer items in sequencecustomNextItem <- function(person, design, test){ # browser() items_left_2_choose_from <- extract.mirtCAT(person, 'items_in_bank') min(items_left_2_choose_from)}res <- mirtCAT(df, local_pattern=pat, design = list(customNextItem=customNextItem))summary(res)#------------------------------------------------# administer items in order, but stop after 10 itemscustomNextItem <- function(person, design, test){ items_left_2_choose_from <- extract.mirtCAT(person, 'items_in_bank') items_answered <- extract.mirtCAT(person, 'items_answered') total <- sum(!is.na(items_answered)) ret <- if(total < 10) min(items_left_2_choose_from) else return(NA) ret}res <- mirtCAT(df, local_pattern=pat, design = list(customNextItem=customNextItem))summary(res)#------------------------------------------------# using findNextItem() and stopping after 10 itemscustomNextItem <- function(person, design, test){ items_answered <- extract.mirtCAT(person, 'items_answered') total <- sum(!is.na(items_answered)) ret <- NA if(total < 10) ret <- findNextItem(person=person, test=test, design=design, criteria = 'MI') ret}res <- mirtCAT(df, mod, local_pattern=pat, start_item = 'MI', design = list(customNextItem=customNextItem))summary(res)# equivalent to the followingres2 <- mirtCAT(df, mod, local_pattern=pat, start_item = 'MI', criteria = 'MI', design = list(max_items = 10))summary(res2)## End(Not run)Find next CAT item
Description
A function that returns the next item in the computerized adaptive, optimal assembly, or shadow test.For direction manipulation of the internal objects this function should be used in conjunctionwith theupdateDesign andcustomNextItem.Finally, the raw input forms can be used when acustomNextItem function has beendefined inmirtCAT.
Usage
findNextItem( x, person = NULL, test = NULL, design = NULL, criteria = NULL, objective = NULL, subset = NULL, all_index = FALSE, ...)Arguments
x | an object of class 'mirtCAT_design' returned from the |
person | (required when |
test | (required when |
design | (required when |
criteria | item selection criteria (see |
objective | a vector of values used as the optimization criteria to be passed to |
subset | an integer vector indicating which items should be included in the optimal search;the default |
all_index | logical; return all items instead of just the most optimal?When |
... | additional arguments to be passed to |
Details
When a numericobjective is supplied the next item in the computerized adaptive test is found viaan integer solver through searching for a maximum. The raw input forms can be usedwhen acustomNextItem function has been defined inmirtCAT, and requiresthe definition of aconstr_fun (see the associated element inmirtCAT for details,as well as the examples below). Can be used to for 'Optimal Test Assembly',as well as 'Shadow Testing' designs (van der Linden, 2005),by using thelp function. Whenobjective is not supplied the result follows thetypical maximum criteria of more standard adaptive tests.
Value
typically returns an integer value indicating the index of the next item to be selected or avalue ofNA to indicate that the test should be terminated. However, see the arguments forfurther returned object descriptions
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
van der Linden, W. J. (2005). Linear models for optimal test design. Springer.
See Also
mirtCAT,updateDesign,extract.mirtCAT
Examples
## Not run: # test defined in mirtCAT help file, first example # equivalent to criteria = 'MI'customNextItem <- function(design, person, test){ item <- findNextItem(person=person, design=design, test=test, criteria = 'MI') item } set.seed(1)nitems <- 100itemnames <- paste0('Item.', 1:nitems)a <- matrix(rlnorm(nitems, .2, .3))d <- matrix(rnorm(nitems))dat <- simdata(a, d, 500, itemtype = 'dich')colnames(dat) <- itemnamesmod <- mirt(dat, 1, verbose = FALSE)# simple math itemsquestions <- answers <- character(nitems)choices <- matrix(NA, nitems, 5)spacing <- floor(d - min(d)) + 1 #easier items have more variation in the optionsfor(i in 1:nitems){ n1 <- sample(1:50, 1) n2 <- sample(51:100, 1) ans <- n1 + n2 questions[i] <- paste0(n1, ' + ', n2, ' = ?') answers[i] <- as.character(ans) ch <- ans + sample(c(-5:-1, 1:5) * spacing[i,], 5) ch[sample(1:5, 1)] <- ans choices[i, ] <- as.character(ch)}df <- data.frame(Question=questions, Option=choices, Type = 'radio', stringsAsFactors = FALSE) response <- generate_pattern(mod, 1)result <- mirtCAT(mo=mod, local_pattern = response, design = list(customNextItem=customNextItem)) -----------------------------------------------------------# direct manipulation of internal objectsCATdesign <- mirtCAT(df=df, mo=mod, criteria = 'MI', design_elements = TRUE)# returns number 1 in this case, since that's the starting itemfindNextItem(CATdesign)# determine next item if item 1 and item 10 were answered correctlyCATdesign <- updateDesign(CATdesign, new_item = 1, new_response = 1)extract.mirtCAT(CATdesign$person, 'thetas') # updated thetasCATdesign <- updateDesign(CATdesign, new_item = 10, new_response = 1)extract.mirtCAT(CATdesign$person, 'thetas') # updated thetas againfindNextItem(CATdesign)findNextItem(CATdesign, all_index = TRUE) # all items rank in terms of most optimal#-------------------------------------------------------------## Integer programming example (e.g., shadow testing)# find maximum information subject to constraints# sum(xi) <= 5 ### 5 or fewer items# x1 + x2 <= 1 ### items 1 and 2 can't be together# x4 == 0 ### item 4 not included# x5 + x6 == 1 ### item 5 or 6 must be included, but not both# constraint functionconstr_fun <- function(design, person, test){ # left hand side constrains # - 1 row per constraint, and ncol must equal number of items mo <- extract.mirtCAT(test, 'mo') nitems <- extract.mirt(mo, 'nitems') lhs <- matrix(0, 4, nitems) lhs[1,] <- 1 lhs[2,c(1,2)] <- 1 lhs[3, 4] <- 1 lhs[4, c(5,6)] <- 1 # relationship direction dirs <- c("<=", "<=", '==', '==') #right hand side rhs <- c(5, 1, 0, 1) #all together constraints <- data.frame(lhs, dirs, rhs) constraints}CATdesign <- mirtCAT(df=df, mo=mod, design_elements = TRUE, design = list(constr_fun=constr_fun))# MI criteria value associated with each respective itemobjective <- computeCriteria(CATdesign, criteria = 'MI')# most optimal item, given constraintsfindNextItem(CATdesign, objective=objective)# all the items which solve the problemfindNextItem(CATdesign, objective=objective, all_index = TRUE)## within a customNextItem() definition the above code would look like# customNextItem <- function(design, person, test){# objective <- computeCriteria(person=person, design=design, test=test,# criteria = 'MI')# item <- findNextItem(person=person, design=design, test=test,# objective=objective)# item# }## End(Not run)Generate a mirt object from population parameters
Description
This function generates amirt object from known population parameters, which is then passed tomirtCAT for running CAT applications.
Usage
generate.mirt_object( parameters, itemtype, latent_means = NULL, latent_covariance = NULL, key = NULL, min_category = rep(0L, length(itemtype)))Arguments
parameters | a matrix or data.frame of parameters corresponding to the model definitionslisted in |
itemtype | a character vector indicating the type of item with which the parameters refer. See the |
latent_means | (optional) a numeric vector used to define the population latent meanstructure. By default the mean structure is centered at a 0 vector |
latent_covariance | (optional) a matrix used to define the population variance-covariance structure between the latent traits. By default the relationship is assumed to be orthogonal standard normal (i.e., an identity matrix) |
key | scoring key required for nested-logit models. See |
min_category | the value representing the lowest category index. By default this is 0,therefore the response suitable for the first category is 0, second is 1, and so on up to |
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
Examples
## Not run: ### build a unidimensional test with all 3PL itemsnitems <- 50a1 <- rlnorm(nitems, .2,.2)d <- rnorm(nitems)g <- rbeta(nitems, 20, 80)pars <- data.frame(a1=a1, d=d, g=g)head(pars)obj <- generate.mirt_object(pars, '3PL')coef(obj, simplify = TRUE)plot(obj, type = 'trace')### build a two-dimensional test ## all graded items with 5 response categoriesnitems <- 30as <- matrix(rlnorm(nitems*2, .2, .2), nitems)diffs <- t(apply(matrix(runif(nitems*4, .3, 1), nitems), 1, cumsum)) diffs <- -(diffs - rowMeans(diffs)) ds <- diffs + rnorm(nitems)pars2 <- data.frame(as, ds)colnames(pars2) <- c('a1', 'a2', paste0('d', 1:4))head(pars2)obj <- generate.mirt_object(pars2, 'graded')coef(obj, simplify = TRUE)### unidimensional mixed-item testlibrary(plyr)pars3 <- rbind.fill(pars, pars2) #notice the NA's where parameters do not existobj <- generate.mirt_object(pars3, itemtype = c(rep('2PL', 50), rep('graded', 30)))coef(obj)itemplot(obj, 51)itemplot(obj, 1, drop.zeros=TRUE)## End(Not run)Generate a CAT patterns
Description
Generate a CAT pattern given various inputs. Returns a character vector or numeric matrix (depending on whether adf input was supplied) with columns equal to the test size androws equal to the number of rows inTheta. For simulation studies, supplying aTheta input with more than 1 row will generate a matrix of responses forrunning independent CAT session when passed tomirtCAT(..., local_pattern). Whenthe returned object is an integer vector then theTheta values will be stored as an attribute'Theta' to be automatically used in Monte Carlo simulations.
Usage
generate_pattern(mo, Theta, df = NULL)Arguments
mo | single group object defined by the |
Theta | a numeric vector indicating the latent theta values for a single person |
df | (optional) data.frame object containing questions, options, and scoringkeys. See |
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
Examples
## Not run: # return real response vector given choices and (optional) answers pat <- generate_pattern(mod, Theta = 0, df=df)# mirtCAT(df, mo=mod, local_pattern = pat)# generate single pattern observed in dataset used to define modpat2 <- generate_pattern(mod, Theta = 0)# mirtCAT(mo=mod, local_pattern = pat2)# generate multiple patterns to be analyzed independently pat3 <- generate_pattern(mod, Theta = matrix(c(0, 2, -2), 3))# mirtCAT(mo=mod, local_pattern = pat3)## End(Not run)Retrieve person object after running createShinyGUI
Description
This function returns a suitable person object identical to the result returned bymirtCAT,and is only required when the GUI is launched by thecreateShinyGUI method.
Usage
getPerson()Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
mirtCAT,mirtCAT_preamble,createShinyGUI
Examples
## Not run: mirtCAT_preamble(df=df)runApp(createShinyGUI(), port = 8000)person <- getPerson()summary(person)## End(Not run)Get the internal working environment state during mirtCAT session
Description
This function is used to access the internal state of the mirtCAT GUI session. It is only useful when designing a customized GUI using theshinyGUI$uiinput tomirtCAT.
Usage
get_mirtCAT_env(sessionName)Arguments
sessionName | the name of the session defined in |
Value
a list containing the internal environmental components for mirtCAT
Generate an adaptive or non-adaptive test HTML interface
Description
Provides tools to generate an HTML interface for creating adaptive and non-adaptive educational and psychological tests using theshiny package. Suitable for applying unidimensional and multidimensional computerized adaptive tests using item response theory methodology. Test scoring is performed using themirt package.However, if no scoring is required (i.e., a standard survey) then defining amirt object may be omitted.
Usage
mirtCAT( df = NULL, mo = NULL, method = "MAP", criteria = "seq", start_item = 1, local_pattern = NULL, AnswerFuns = list(), design_elements = FALSE, cl = NULL, progress = FALSE, primeCluster = TRUE, customTypes = list(), design = list(), shinyGUI = list(), preCAT = list(), ...)## S3 method for class 'mirtCAT'print(x, ...)## S3 method for class 'mirtCAT'summary(object, sort = TRUE, ...)## S3 method for class 'mirtCAT'plot( x, pick_theta = NULL, true_thetas = TRUE, SE = 1, main = NULL, par.strip.text = list(cex = 0.7), par.settings = list(strip.background = list(col = "#9ECAE1"), strip.border = list(col = "black")), scales = list(x = list(rot = 90)), ...)Arguments
df | a
|
mo | single group object defined by the |
method | argument passed to |
criteria | adaptive criteria used, default is to administer each item sequentially using Possible inputs for unidimensional adaptive tests include: Possible inputs for multidimensional adaptive tests include: Applicable to both unidimensional and multidimensional tests are the Non-adaptive methods applicable even when no |
start_item | two possible inputs to determine the starting item are available. Passing a number will indicate the specific item to be used as the start item;default is 1, which selects the first item in the defined test/survey. If a character string is passed then the item will be selected from one of the item selections criteria available (see the |
local_pattern | a character/numeric matrix of response patterns used to run the CAT application without generating the GUI interface. This option requires complete response pattern(s) to be supplied. |
AnswerFuns | a list with the length equal to the number of items in the item bank consisting of user-defined functions. These functions are used to determine whether a givenresponse obtained from the GUI is 'correct' or 'incorrect' by returning a logical scalar value, while For example, the following provides a customized response function for the first item. AnswerFuns <- as.list(rep(NA, nrow(df))) AnswerFuns[[1]] <- function(input) input == '10' || to.lower(input) == 'ten' |
design_elements | logical; return an object containing the test, person, and design elements? Primarily this is to be used with the |
cl | an object definition to be passed to the parallel package (see |
progress | logical; print a progress bar to the console with the |
primeCluster | logical; when a |
customTypes | an optional list input containing functions for Designing Original Graphical Stimuli (DOGS).DOGS elements in the input list must contain a unique name, and the item with which it is associated must bedeclared in the a myDOGS <- function(inputId, df_row) ... and must return, at the very minimum, an associated The following is a simple example of DOGS for a true-false question and how it is passed: good_dogs <- function(inputId, df_row){ return(list(h2('This statement is false'), radioButtons(inputId = inputId, label='', choices = c('True', 'False'), selected = '') )) } df <- data.frame(Question = '', ..., Type = 'Doug') results <- mirtCAT(df=df, customTypes = list(Doug = good_dogs))IMPORTANT: When using the custom inputs the select defined |
design | a list of design based control parameters for adaptive and non-adaptive tests. These can be
|
shinyGUI | a list of GUI based parameters to be over-written. These can be
|
preCAT | a list object which can be used to specify a pre-CAT block in which different test properties may be applied prior to beginning the CAT session. If thelist is empty, no preCAT block will be used. All of the following elements are required to use the
|
... | additional arguments to be passed to |
x | object of class |
object | object of class |
sort | logical; sort the response patterns based on the order they were administered? If FALSE, the raw response patterns containing NAs will be returnedfor items that were not administered |
pick_theta | a number indicating which theta to plot (only applicable for multidimensional tests). The default is to facet each theta on one plot, but to plot only the first factor pass |
true_thetas | logical; include a horizontal line indicating where the population-level theta values are? Only applicable to Monte Carlo simulations because this value would not be known otherwise |
SE | size of the standard errors to plot. The default is 1, and therefore plots thestandard error. To obtain the 95% interval use |
main | title of the plot. Will default to |
par.strip.text | plotting argument passed to |
par.settings | plotting argument passed to |
scales | plotting argument passed to |
Details
All tests will stop once the'min_SEM' criteria has been reached or classificationabove or below the specified cutoffs can be made. If all questions shouldbe answered, users should specify an extremely small'min_SEM' or, equivalently, a large'min_items' criteria to thedesign list input.
Value
Returns a list object of class'Person' containing the following elements:
raw_responsesA character vector indicating the raws responses to the respectiveitems, where NA indicates the item was not answered
scored_responsesAn integer vector of scored responses if the
item_answersinputwas used for each respective itemitems_answeredAn integer vector indicating the order in which the items were answered
thetasA numeric vector indicating the final theta estimates
SE_thetasA numeric vector indicating the standard errors of the final theta estimates
thetas_historyA matrix indicating the progression of updating the theta valuesduring the test
thetas_SE_historyA matrix indicating the standard errors for theta after eachsuccessive item was answered
item_timeA numeric vector indicating how long the respondent took to answereach question (in seconds)
demographicsA data.frame object containing the information collected on the first page of the shiny GUI. This is used to store the demographic information for eachparticipant
classificationA character vector indicating whether the traits could be classified as 'above' or 'below' the desired cutoffs
HTML help files, exercises, and examples
To access examples, vignettes, and exercise files that have been generated withknitr pleasevisithttps://github.com/philchalmers/mirtCAT/wiki.
Modifying thedesign object directly throughcustomNextItem() (advanced)
In addition to providing a completely defined item-selection map via thecustomNextItem() function, users may also wish to control some of the more fine-grained elements of thedesign object to adjust the general control parameters of the CAT (e.g., modifying the maximum number of items to administer, stoppingthe CAT if something peculiar has been detected in the response patterns, etc). Note that this feature is rarely required for most applications, though more advanced users may wish to modify these various low-level elements of thedesign object directly to change the flow of the CATto suit their specific needs.
While theperson object is defined as aReference Class (seesetRefClass) the design object is generally considered a fixed S4 class, meaning that, unlike theperson object, it's elements are not mutable. Therefore, in order to make changes directly to thedesign object the users should follow these steps:
Within the defined
customNextItemfunction, thedesignobject slots are first modified (e.g.,design@max_items <- 20L).Along with the desired next item scalar value from
customNextItem(), the scalar object should also contain an attribute with the name'design'which holds the newly defineddesignobject(e.g.,attr(ret, 'design') <- design; return(ret)).
Following the above process the work-flow inmirtCAT will use the newdesign object in place of theold one, even in Monte Carlo simulations.
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
generate_pattern,generate.mirt_object,extract.mirtCAT,findNextItem,computeCriteria
Examples
## Not run: ### unidimensional scored example with generated items# create mo from estimated parametersset.seed(1234)nitems <- 50itemnames <- paste0('Item.', 1:nitems)a <- matrix(rlnorm(nitems, .2, .3))d <- matrix(rnorm(nitems))dat <- simdata(a, d, 1000, itemtype = 'dich')mod <- mirt(dat, 1)coef(mod, simplify=TRUE)# alternatively, define mo from population values (not run)pars <- data.frame(a1=a, d=d)mod2 <- generate.mirt_object(pars, itemtype='2PL')coef(mod2, simplify=TRUE)# simple math itemsquestions <- answers <- character(nitems)choices <- matrix(NA, nitems, 5)spacing <- floor(d - min(d)) + 1 #easier items have more variation in the optionsfor(i in 1:nitems){ n1 <- sample(1:50, 1) n2 <- sample(51:100, 1) ans <- n1 + n2 questions[i] <- paste0(n1, ' + ', n2, ' = ?') answers[i] <- as.character(ans) ch <- ans + sample(c(-5:-1, 1:5) * spacing[i,], 5) ch[sample(1:5, 1)] <- ans choices[i, ] <- as.character(ch)}df <- data.frame(Question=questions, Option=choices, Type = 'radio', stringsAsFactors = FALSE)head(df)(res <- mirtCAT(df)) #collect response only (no scoring or estimating thetas)summary(res)# include scoring by providing Answer keydf$Answer <- answers(res_seq <- mirtCAT(df, mod)) #sequential scoring (res_random <- mirtCAT(df, mod, criteria = 'random')) #random(res_MI <- mirtCAT(df, mod, criteria = 'MI', start_item = 'MI')) #adaptive, MI starting itemsummary(res_seq)summary(res_random)summary(res_MI)#-----------------------------------------# HTML tags for better customization, coerced to characters for compatibility# help(tags, package='shiny')options <- matrix(c("Strongly Disagree", "Disagree", "Neutral", "Agree", "Strongly Agree"), nrow = 3, ncol = 5, byrow = TRUE)shinyStems <- list(HTML('Building CATs with mirtCAT is difficult.'), div(HTML('mirtCAT requires a'), br(), HTML('substantial amount of coding.')), div(strong('I would use'), HTML('mirtCAT in my research.')))questions <- sapply(shinyStems, as.character)df <- data.frame(Question=questions, Option = options, Type = "radio", stringsAsFactors=FALSE)res <- mirtCAT(df)res#-----------------------------------------# run locally, random response pattern given Thetaset.seed(1)pat <- generate_pattern(mod, Theta = 0, df=df)head(pat)# seq scoring with character pattern for the entire test (adjust min_items)res <- mirtCAT(df, mod, local_pattern=pat, design = list(min_items = 50)) summary(res)# same as above, but using special input vector that doesn't require df inputset.seed(1)pat2 <- generate_pattern(mod, Theta = 0)head(pat2)print(mirtCAT(mo=mod, local_pattern=pat2))# run CAT, and save results to object called person (start at 10th item)person <- mirtCAT(df, mod, item_answers = answers, criteria = 'MI', start_item = 10, local_pattern = pat)print(person)summary(person)# plot the sessionplot(person) #standard errorsplot(person, SE=1.96) #95 percent confidence intervals#-----------------------------------------### save response object to temp directory in case session ends earlywdf <- paste0(getwd(), '/temp_file.rds')res <- mirtCAT(df, mod, shinyGUI = list(temp_file = wdf))# resume test this way if test was stopped early (and temp files were saved)res <- mirtCAT(df, mod, shinyGUI = list(temp_file = wdf))print(res)## End(Not run)Preamble function called by mirtCAT
Description
This is largely an internal function called bymirtCAT, however it is made public for better use with external web-hosting interfaces (like shinyapps.io).For more information seehttps://shiny.rstudio.com/articles/persistent-data-storage.html for further information about saving output remotely when usingshiny.
Usage
mirtCAT_preamble(..., final_fun = NULL)Arguments
... | arguments passed to |
final_fun | a function called just before the shiny GUI has been terminated, primarily forsaving results externally with packages such as |
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
mirtCAT,createShinyGUI,getPerson
Examples
## Not run: mirtCAT_preamble(df = df)## End(Not run)Update design elements
Description
A function that will update the object returned fromfindNextItem. This can be used to run the CAT session manually in a set-by-step manner.
Usage
updateDesign(x, new_item, new_response, updateTheta = TRUE)Arguments
x | an object of class 'mirtCAT_design' returned from the |
new_item | a numeric vector indicating which items to select |
new_response | a numeric vector indicating the responses the the selected items |
updateTheta | logical; update the internal ability terms after the new item response has beenadded to the internal objects? |
Value
returns an object of class 'mirtCAT_design' with updated elements.
Author(s)
Phil Chalmersrphilip.chalmers@gmail.com
References
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response TheoryPackage for the R Environment.Journal of Statistical Software, 48(6), 1-29.doi:10.18637/jss.v048.i06
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications.Journal of Statistical Software, 71(5), 1-39.doi:10.18637/jss.v071.i05
See Also
Examples
## Not run: set.seed(1)nitems <- 100itemnames <- paste0('Item.', 1:nitems)a <- matrix(rlnorm(nitems, .2, .3))d <- matrix(rnorm(nitems))dat <- simdata(a, d, 500, itemtype = 'dich')colnames(dat) <- itemnamesmod <- mirt(dat, 1, verbose = FALSE)# test defined in mirtCAT help file, first exampleCATdesign <- mirtCAT(mo = mod, criteria = 'MI', design_elements = TRUE, start_item = 2)# returns 2 in this case, since that was the starting itemfindNextItem(CATdesign) # first iteration, no answered itemsCATdesign$person$items_answered# update when next item is item 2 and answered correctlyCATdesign <- updateDesign(CATdesign, new_item = 2, new_response = 1)CATdesign$person$items_answered # item 2 answered firstCATdesign$person$responses # in item 2 element response was = 1 CATdesign$person$thetas # current estimatefindNextItem(CATdesign) # determine next item if item 70 were also answered correctly nextCATdesign <- updateDesign(CATdesign, new_item = 70, new_response = 1)CATdesign$person$items_answered CATdesign$person$responses findNextItem(CATdesign) # continue on, now with item 95 added next (answered incorrectly)CATdesign <- updateDesign(CATdesign, new_item = 95, new_response = 0)CATdesign$person$thetasCATdesign$person$thetas_historyCATdesign$person$thetas_SE_historyfindNextItem(CATdesign)## End(Not run)