# Item Equating with same Group – SAT, ACT example

August 5, 2013
By

(This article was first published on Econometrics by Simulation, and kindly contributed to R-bloggers)

`# Item equating is the practice of making the results from two # different assessments equivalent.  This can be done by either# 1. having the same group take both assessments# 2. having equivalent groups take the different assessments# 3. having non-equivalent groups which use common items take the different#    assessments. # In this post I will cover topic 1. # For this code I will use the catR package to generate my assessments # and responses.library("catR") # Let's attempt the first proceedure:# First let's generate our item parameters for assessment 1. # Let's create an item bank with 100 items for the assessmentsnitems <- 100bank1 <- createItemBank(model="2PL", items=nitems)\$itemParbank2 <- createItemBank(model="2PL", items=nitems)\$itemPar # Now let's generate a 1000 person population sample to take our assessmentnpeep <- 1000theta <- rnorm(npeep) # Calculate the score on both assessmentsresp1 <- resp2 <- matrix(0, nrow=npeep, ncol=nitems)for (i in 1:npeep) {  resp1[i,Pi(theta[i],bank1)\$Pi<runif(nitems)] <- 1 # Test 1  resp2[i,Pi(theta[i],bank2)\$Pi<runif(nitems)] <- 1 # Test 2} # To calculate total score on the tests we can sum the results of each rowscore1 <- apply(resp1, 1, sum)score2 <- apply(resp2, 1, sum) # Since we know both forms of the test are parrellel we can check the correlation# between scores on the different forms of the test as a measure of reliability. cor(score1,score2)# This gives me a correlation estimate of .95 which is very good. # However, we are not interested in reliability right now.  We would like to # equate the two tests using the information thus far garnished. # First let's estimate our parameters.  # I will use the ltm command in the package ltm for this.library("ltm") est2pl1 <- ltm(resp1~z1)est2pl2 <- ltm(resp2~z1) # Now we have two tests with the different items.# We want to make sure the items are on the same scale# so that it does not matter which test individuals take.# Their expected score will be the same. cbind(score1,score2) # Because we simulated the generation of both tests which are very lengthy# the tests are pretty much already equated by design which is frankly# much easier to do by simulation and perhaps impossible to do with# real assessments. # Nevertheless, let's act as if our tests were not already parrellel and# equate them. # Let's first do some linear equating which is done by setting the standardized# scores of the two exames equal to each other. (page 33)# See http://ncme.org/linkservid/65DCF34D-1320-5CAE-6E896C93330B9C73/showMeta/ # X1 and X2 refers to total scores for each individual on exams 1 and 2# (X1 - mean(X1))/Sd(X1) = (X2 - mean(X2))/Sd(X2)# X1 = Sd(X1)/Sd(X2)*X2 + (mean(X1)-Sd(X1)/Sd(X2)*mean(X2)) = A*X2 + B# Where A=Sd(x1)/Sd(x2) and B=mean(X1)-A*mean(X2) (A <- sd(score1)/sd(score2))# A being close to 1 indicates that the tests are scaled similary (B <- mean(score1)-A*mean(score2))# B being close to 0 indicates the tests are of similar difficulty # Now let's calculate what score2 would be if scaled on assessment 1. score2scaled <- A*score2 + B summary(cbind(score1,score2,score2scaled))# We can see that score2scaled looks slightly closer to score1. # Let's try the same thing with slightly more interesting. # Let's imagine that score1 is for ACT from 0 to 32 and score 2 is from SAT 200 to 800 # The standard deviation is 6 and average score around 20 for the ACTACT <- (score1-mean(score1))/sd(score1)*6 + 20ACT[ACT>32] <- 32ACT[ACT<4]  <- 4ACT <- round(ACT) # ACT rounds to whole numbers # The standard deviation is 100 and average score around 500 for the ACTSAT <- (score2-mean(score2))/sd(score2)*100 + 500SAT[SAT>800] <- 800SAT[SAT<200]  <- 200SAT <- round(SAT/10)*10 # SAT rounds to nearest 10  summary(cbind(SAT,ACT)) # Now let's see if we can transform our SAT scores to be on our ACT scale(A <- sd(ACT)/sd(SAT))(B <- mean(ACT)-A*mean(SAT)) SATscaled <- A*SAT + Bsummary(cbind(SAT,ACT,SATscaled)) # The results from taking parrellel tests should fall on a linear formplot(ACT,SATscaled, main="SAT results placed on ACT scale") `

`# This is the easiest method of equating two tests.  # However, it is not usually the most practical since it is costly to get the same# group of individuals to take two different tests.  In addition, there# may be issues with fatigue which could be alleviated somewhat if for half of the# group the first assessment was given first and for a different half `
`# the second assessment assessment was given first.`

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...