Wright Map Tutorial - Part 3
Using Conquest Output and Making Thresholds
Updated March 18, 2016 for changes in version 1.2.
In this part of the tutorial, we’ll show how to load ConQuest output to make a CQmodel object and then WrightMaps. We’ll also show how to turn deltas into thresholds. All the example files here are available in the /inst/extdata folder of our github site. If you download the latest version of the package, they should be in a folder called /extdata wherever your R packages are stored. You can set this folder as your working directory with setwd() or use the system.file() command—as in the next set of examples—to run them.
Let’s load a model. The first parameter should be the name of the person estimates file, while the second should be the name of the show file. Both are necessary for creating Wright maps (although the CQmodel function will run fine with only one or the other, provided that they are properly passed).
We start by defining a path to the WrightMap example files.
fpath <- system.file("extdata", package="WrightMap")
And we load the example output.
model1 <- CQmodel(p.est = file.path(fpath,"ex2.eap"), show = file.path(fpath,"ex2.SHW"))
This (model1) is a CQmodel object. Enter the name of the object to see the names of all the tables & information stored within this object.
## ## ConQuest Output Summary: ## ======================== ## Partial Credit Analysis ## ## The item model: item+item*step ## 1 dimension ## 582 participants ## Deviance: 9272.597 (21 parameters) ## ## Additional information available: ## Summary of estimation: $SOE ## Response model parameter estimates: $RMP ## Regression coefficients: $reg.coef ## Variances: $variances ## Reliabilities: $rel.coef ## GIN tables (thresholds): $GIN ## EAP table: $p.est ## Additional details: $run.details
Type the name of any of these tables to see the information stored there.
## ## Summary of estimation ## ## Estimation method: Gauss-Hermite Quadrature with 15 nodes ## Assumed population distribution: Gaussian ## Constraint: DEFAULT ## ## Termination criteria: ## 1000 iterations ## 0.0001 change in parameters ## 0.0001 change in deviance ## 100 iterations without a deviance improvement ## 10 Newton steps in M-step ## Estimation terminated after 27 iterations because the deviance convergence criteria was reached. ## ## Random number generation seed: 1 ## 2000 nodes used for drawing 5 plausible values ## 200 nodes used when computing fit ## Value for obtaining finite MLEs for zero/perfects: 0.3
## CONSTANT ## Main dimension 0.972 ## S. errors 0.062
## MLE Person separation RELIABILITY ## Main dimension NA ## WLE Person separation RELIABILITY EAP/PV RELIABILITY ## Main dimension NA 0.813
## errors ## [1,] 2.162 NA
The most relevant for our purposes are the RMP, GIN, and p.est tables. The RMP tables contain the Response Model Parameters. These are item parameters. Typing model1$RMP would display them, but they’re a little long, so I’m just going to ask for the names and then show the first few rows of each table.
## [1] "item" "item*step"
For this model, the RMPs have item and item*step parameters. We could add these to get the deltas. Let’s see what the tables look like.
## n_item item est error U.fit U.Low U.High U.T W.fit W.Low W.High W.T ## 1 1 1 0.753 0.055 1.11 0.88 1.12 1.8 1.10 0.89 1.11 1.8 ## 2 2 2 1.068 0.053 1.41 0.88 1.12 6.0 1.37 0.89 1.11 6.0 ## 3 3 3 -0.524 0.058 0.82 0.88 1.12 -3.2 0.87 0.88 1.12 -2.3 ## 4 4 4 -1.174 0.060 0.76 0.88 1.12 -4.3 0.85 0.88 1.12 -2.7 ## 5 5 5 -0.389 0.057 0.95 0.88 1.12 -0.9 0.95 0.89 1.11 -0.9 ## 6 6 6 0.067 0.055 1.03 0.88 1.12 0.6 1.02 0.89 1.11 0.3
head(model1$RMP$"item*step")
## n_item item step est error U.fit U.Low U.High U.T W.fit W.Low W.High ## 1 1 1 0 NA NA 2.03 0.88 1.12 13.3 1.18 0.89 1.11 ## 2 1 1 1 -1.129 0.090 0.99 0.88 1.12 -0.1 1.00 0.95 1.05 ## 3 1 1 2 1.129 NA 0.80 0.88 1.12 -3.5 0.95 0.89 1.11 ## 4 2 2 0 NA NA 2.25 0.88 1.12 15.4 1.40 0.90 1.10 ## 5 2 2 1 -0.626 0.093 1.04 0.88 1.12 0.7 1.04 0.94 1.06 ## 6 2 2 2 0.626 NA 1.08 0.88 1.12 1.2 1.08 0.89 1.11 ## W.T ## 1 3.0 ## 2 0.0 ## 3 -0.9 ## 4 7.1 ## 5 1.3 ## 6 1.4
Let’s look at a more complicated example.
model2 <- CQmodel(file.path(fpath,"ex4a.mle"), file.path(fpath,"ex4a.shw")) model2$equation
## [1] "rater+topic+criteria+rater*topic+rater*criteria+topic*criteria+rater*topic*criteria*step"
## [1] "rater" "topic" ## [3] "criteria" "rater*topic" ## [5] "rater*criteria" "topic*criteria" ## [7] "rater*topic*criteria*step"
head(model2$RMP$"rater*topic*criteria*step")
## n_rater rater n_topic topic n_criteria criteria step est error ## 1 1 Amy 1 Sport 1 spelling 1 NA NA ## 2 1 Amy 1 Sport 1 spelling 2 0.299 0.398 ## 3 1 Amy 1 Sport 1 spelling 3 -0.299 NA ## 4 2 Beverely 1 Sport 1 spelling 0 NA NA ## 5 2 Beverely 1 Sport 1 spelling 1 -0.184 0.491 ## 6 2 Beverely 1 Sport 1 spelling 2 0.051 0.461 ## U.fit U.Low U.High U.T W.fit W.Low W.High W.T ## 1 0.43 0.70 1.30 -4.7 0.99 0.00 2.00 0.1 ## 2 1.34 0.70 1.30 2.1 1.05 0.42 1.58 0.3 ## 3 1.28 0.70 1.30 1.7 1.05 0.51 1.49 0.3 ## 4 0.41 0.74 1.26 -5.8 1.47 0.00 2.09 0.9 ## 5 3.23 0.74 1.26 10.9 0.95 0.30 1.70 0.0 ## 6 0.87 0.74 1.26 -1.0 1.30 0.62 1.38 1.5
The GIN tables show the threshold parameters.
## [,1] [,2] ## Item_1 -0.469 1.977 ## Item_2 0.234 1.906 ## Item_3 -1.789 0.742 ## Item_4 -2.688 0.336 ## Item_5 -1.656 0.883 ## Item_6 -1.063 1.195 ## Item_7 -1.969 1.047 ## Item_8 -1.617 1.289 ## Item_9 -0.957 1.508 ## Item_10 -0.992 2.094
## $Amy ## $Amy$Sport ## [,1] [,2] [,3] ## spelling -31.996 -1.976 -1.250 ## coherence -1.447 -1.446 -1.209 ## structure -2.247 -0.911 -0.172 ## grammar -0.885 -0.773 -0.107 ## content -0.486 0.104 0.627 ## ## $Amy$Family ## [,1] [,2] [,3] ## spelling -31.996 -2.516 -0.912 ## coherence -1.401 -1.280 -1.103 ## structure -1.966 -1.260 -0.294 ## grammar -1.069 -0.380 -0.106 ## content -0.728 -0.012 0.950 ## ## $Amy$Work ## [,1] [,2] [,3] ## spelling -2.055 -2.051 -1.128 ## coherence -1.515 -1.320 -0.862 ## structure -1.402 -1.158 -0.631 ## grammar -0.816 -0.550 0.122 ## content -0.430 0.212 0.762 ## ## $Amy$School ## [,1] [,2] [,3] ## spelling -31.996 -2.059 -0.997 ## coherence -1.403 -1.402 -0.999 ## structure -1.629 -1.148 -0.462 ## grammar -0.967 -0.421 0.070 ## content -0.782 -0.027 1.121 ## ## ## $Beverely ## $Beverely$Sport ## [,1] [,2] [,3] ## spelling -2.054 -1.339 -0.663 ## coherence -1.751 -1.129 -0.674 ## structure -1.042 -0.437 0.013 ## grammar -0.502 -0.082 0.529 ## content -0.253 0.613 1.184 ## ## $Beverely$Family ## [,1] [,2] [,3] ## spelling -31.996 -2.264 -0.718 ## coherence -1.524 -1.357 -0.684 ## structure -1.326 -0.577 0.164 ## grammar -0.796 0.118 0.599 ## content -0.469 0.690 1.230 ## ## $Beverely$Work ## [,1] [,2] [,3] ## spelling -2.366 -1.465 -0.672 ## coherence -1.388 -1.088 -0.925 ## structure -1.115 -0.621 0.197 ## grammar -0.345 0.045 0.495 ## content -0.212 0.482 1.282 ## ## $Beverely$School ## [,1] [,2] [,3] ## spelling -1.826 -1.611 -0.873 ## coherence -1.632 -1.222 -0.794 ## structure -1.270 -0.865 0.321 ## grammar -0.491 -0.037 0.413 ## content -0.361 0.449 1.137 ## ## ## $Colin ## $Colin$Sport ## [,1] [,2] [,3] ## spelling -1.660 -0.685 0.564 ## coherence -0.612 -0.168 0.362 ## structure -0.485 0.519 1.512 ## grammar 0.611 1.275 1.698 ## content 1.037 1.853 2.343 ## ## $Colin$Family ## [,1] [,2] [,3] ## spelling -1.477 -0.677 -0.022 ## coherence -0.441 -0.277 0.332 ## structure -0.318 0.265 1.299 ## grammar 0.361 1.252 1.839 ## content 1.009 1.683 2.374 ## ## $Colin$Work ## [,1] [,2] [,3] ## spelling -1.697 -1.002 0.089 ## coherence -0.654 -0.105 0.192 ## structure -0.502 0.502 1.205 ## grammar 0.662 1.218 1.573 ## content 0.766 1.806 2.357 ## ## $Colin$School ## [,1] [,2] [,3] ## spelling -1.595 -0.788 0.095 ## coherence -0.629 -0.389 0.123 ## structure -0.470 0.122 1.237 ## grammar 0.385 1.010 1.679 ## content 0.698 1.520 2.310 ## ## ## $David ## $David$Sport ## [,1] [,2] [,3] ## spelling -1.405 -0.482 0.412 ## coherence -0.357 0.136 0.581 ## structure 0.023 0.724 1.811 ## grammar 0.714 1.454 1.959 ## content 1.256 2.031 2.912 ## ## $David$Family ## [,1] [,2] [,3] ## spelling -1.271 -0.404 0.741 ## coherence 0.028 0.415 0.977 ## structure 0.474 1.069 1.756 ## grammar 1.177 1.733 2.085 ## content 1.284 2.169 3.596 ## ## $David$Work ## [,1] [,2] [,3] ## spelling -1.378 -0.587 0.498 ## coherence -0.119 0.260 0.795 ## structure 0.173 1.003 1.885 ## grammar 1.199 1.592 2.008 ## content 1.437 2.174 3.117 ## ## $David$School ## [,1] [,2] [,3] ## spelling -0.815 -0.330 0.424 ## coherence 0.062 0.293 0.805 ## structure 0.295 1.012 1.955 ## grammar 1.035 1.642 2.260 ## content 1.312 2.107 3.407
Finally, the p.est table shows person parameters.
head(model1$p.est) ##EAPs
## casenum est (d1) error (d1) pop (d1) ## 1 1 -0.08240 0.50495 0.88205 ## 2 2 1.75925 0.55966 0.85510 ## 3 3 0.16483 0.49122 0.88838 ## 4 4 3.57343 0.82692 0.68367 ## 5 5 -0.62303 0.52908 0.87051 ## 6 6 0.16483 0.49122 0.88838
head(model2$p.est) ##MLEs
## casenum sscore (d1) max (d1) est (d1) error (d1) ## 1 1 23 60 -0.49687 0.25349 ## 2 2 36 60 0.69311 0.26051 ## 3 3 24 60 -0.26371 0.26378 ## 4 4 52 60 1.85869 0.37825 ## 5 5 47 60 1.91466 0.28843 ## 6 6 47 60 0.53122 0.28348
Ok, we have person parameters and item parameters: Let’s make a Wright Map
## Using GIN table for threshold parameters
The above uses the GIN table as thresholds. But you may want to use RMP tables. For example, if you have an item table and an itemstep table, you might want to combine them to make deltas. You could do this yourself, but you could also let the make.deltas function do it for you. This function reshapes the itemstep parameters, checks the item numbers to see if there are any dichotomous items, and then adds the steps and items. This can be especially useful if you didn’t get a GIN table from ConQuest (see below).
model3 <- CQmodel(file.path(fpath,"ex2a.eap"), file.path(fpath,"ex2a.shw")) model3$GIN
This model has no GIN table, but it does have item and item*step tables. The make.deltas function will read the model equation and look for the appropriate tables.
## 1 2 3 ## Earth shape -0.961 -0.493 NA ## Earth pictu.. -0.650 0.256 2.704 ## Falling off -1.416 1.969 1.265 ## What is Sun -0.959 1.343 NA ## Moonshine 0.157 -0.482 -0.128 ## Moon and ni.. -0.635 0.861 NA ## Night and d.. 0.157 -0.075 -0.739 ## Breathe on .. 0.657 1.152 -3.558
When sent a model with no GIN table, wrightMap will automatically send it to make.deltas without the user having to ask.
wrightMap(model3, label.items.row = 2)
The make.deltas function can also handle rating scale models.
model4 <- CQmodel(file.path(fpath,"ex2b.eap"), file.path(fpath,"ex2b-2.shw")) model4$GIN
This rating scale model again has no GIN table (always the first thing wrightMap looks for) so we’ll need to make deltas.
## 1 2 ## Curriculum .. -0.468 1.900 ## Not Until E.. -0.123 2.245 ## Financial R.. -1.743 0.625 ## Staff Commi.. -2.230 0.138 ## Commitment .. -1.609 0.759 ## Run for som.. -1.193 1.175 ## Achievable .. -1.570 0.798 ## Principals .. -1.317 1.051 ## Parents sup.. -0.952 1.416 ## Student mot.. -0.636 1.732
Or let wrightMap make them automatically.
wrightMap(model4, label.items.row = 2)
In the above examples, we let wrightMap decide what parameters to graph. WrightMap starts by looking for a GIN table. If it finds that, it assumes they are thresholds and graphs them accordingly. If there is no GIN table, it then sends the function to make.deltas, which will examine the model equation to see if it knows how to handle it. Make.deltas can handle equations of the form
A + B (e.g. item + step [RSM])
A + A * B (e.g. item + item * step [PCM])
A + A * B + B (e.g item + item * gender + gender)
(It will also notice if there are minus signs rather than plus signs and react accordingly.)
But sometimes we may want something other than the default. Let’s look at model2 again.
## [1] "rater+topic+criteria+rater*topic+rater*criteria+topic*criteria+rater*topic*criteria*step"
Here’s the default Wright Map, using the GIN table:
wrightMap(model2, min.logit.pad = -29)
## Using GIN table for threshold parameters
This doesn’t look great. Instead of showing all these estimates, we can specify a specific RMP table to use using the item.table parameter.
wrightMap(model2, item.table = "rater")
That shows just the rater parameters. Here’s just the topics.
wrightMap(model2, item.table = "topic")
What I really want, though, is to show the rater*topic estimates. For this, we can use the interactions and step.table parameters.
wrightMap(model2, item.table = "rater", interactions = "rater*topic" , step.table = "topic")
Switch the item and step names to graph it the other way:
wrightMap(model2, item.table = "topic", interactions = "rater*topic" , step.table = "rater")
You can leave out the interactions to have more of a rating scale-type model.
wrightMap(model2, item.table = "rater", step.table = "topic")
Or leave out the step table:
wrightMap(model2, item.table = "rater", interactions = "rater*topic")
Again, make.deltas is reading the model equation to decide whether to add or subtract. If, for some reason, you want to specify a different sign for one of the tables, you can use item.sign, step.sign, and inter.sign for that.
wrightMap(model2, item.table = "rater", interactions = "rater*topic" , step.table = "topic", step.sign = -1)
The last few examples might not make sense for this model, but are just to illustrate how the function works. Note that all three of these parameters must be the exact name of specific RMP tables, and you can’t specify an interactions table or a step table without also specifying an item table (although JUST an item table is fine). And if your model equation is more complicated than the ones specified above, you will have to either use a GIN table or specify in the function call which tables to use for what. A model of the form item + item * step + booklet, for example, will not run unless there is a GIN table or you have defined at least the item.table.
So far, we’ve seen how to use the GIN table to graph thresholds, or the RMP tables to graph deltas. We have one use case left: Making thresholds out of those RMP-generated deltas. Coulter (Dan) Furr has provided a lovely function for exactly this purpose. The example below uses the model3 deltas, but you can send it any matrix with items as rows and steps as columns.
deltas <- make.deltas(model3)
## 1 2 3 ## Earth shape -0.961 -0.493 NA ## Earth pictu.. -0.650 0.256 2.704 ## Falling off -1.416 1.969 1.265 ## What is Sun -0.959 1.343 NA ## Moonshine 0.157 -0.482 -0.128 ## Moon and ni.. -0.635 0.861 NA ## Night and d.. 0.157 -0.075 -0.739 ## Breathe on .. 0.657 1.152 -3.558
## Assuming partial credit model
## [,1] [,2] [,3] ## Earth shape -1.3229164 -0.1310804 NA ## Earth pictu.. -0.9241595 0.4451567 2.7832333 ## Falling off -1.4503041 1.3141486 1.9728871 ## What is Sun -1.0466830 1.4306938 NA ## Moonshine -0.6759150 -0.2252513 0.4156190 ## Moon and ni.. -0.8076978 1.0336795 NA ## Night and d.. -0.6343026 -0.1937096 0.1852925 ## Breathe on .. -0.7007363 -0.5078997 -0.4741583
Alternately, we can just send the model object directly:
## Assuming partial credit model
## [,1] [,2] [,3] ## Earth shape -1.3229164 -0.1310804 NA ## Earth pictu.. -0.9241595 0.4451567 2.7832333 ## Falling off -1.4503041 1.3141486 1.9728871 ## What is Sun -1.0466830 1.4306938 NA ## Moonshine -0.6759150 -0.2252513 0.4156190 ## Moon and ni.. -0.8076978 1.0336795 NA ## Night and d.. -0.6343026 -0.1937096 0.1852925 ## Breathe on .. -0.7007363 -0.5078997 -0.4741583
You don’t have to do any of this to make a Wright Map. You can just send the model to wrightMap, and use the type parameter to ask it to calculate the thresholds for you.
wrightMap(model3, type = "thresholds", label.items.row = 2)
Again, the default type is to use the GIN table if present, and to make deltas if not. You can also force it to make deltas (and ignore the GINs) by setting type to deltas. Alternately, if you specify an item.table, the type will switch to deltas unless you then set type to thresholds.
Last important time-saving note
Finally: If all you want is the Wright Maps, you can skip CQmodel entirely and just send your files to wrightMap:
wrightMap(file.path(fpath,"ex2a.eap"), file.path(fpath,"ex2.shw"), label.items.row = 3)
## Using GIN table for threshold parameters
