(Guest post by Achim Zeileis)

Development of the R package exams for automatic generation of (statistical) exams in R started in 2006 and version 1 was published in JSS by Grün and Zeileis (2009). It was based on standalone `Sweave `exercises, that can be combined into exams, and then rendered into different kinds of PDF output (exams, solutions, self-study materials, etc.). Now, a major revision of the package has been released that extends the capabilities and adds support for learning management systems. It is still based on the *same* type of

`Sweave` files for each exercise but can also render them into output formats like HTML (with various options for displaying mathematical content) and XML specifications for online exams in learning management systems such as Moodle or OLAT. Supplementary files such as graphics or data are

handled automatically. Here, I give a brief overview of the new capabilities. A detailed discussion is in the working paper by Zeileis, Umlauf, and Leisch (2012) that is also contained in the package as a vignette.

### The basic idea…

…is to have standard standalone `Sweave` files for each exercise. These typically comprise (1) R code chunks (as usual within `<<>>=` and `@`) for random data generation. (2) Question and solution descriptions contained in LaTeX environments of corresponding names, typically with `\Sexpr{}` for including data

for the question. (3) Metainformation about the exercise type (numeric, multiple choice, …), its correct solution etc. Then many different replications of (collections of) such an exercise can be easily created by repeating the following four steps:

*Weaving* the exercise (i.e., executing the R code for random data generation and embedding that into the text), *reading* the resulting LaTeX into R, *transforming* the LaTeX to another format such as HTML (if necessary), *writing* the corresponding output files, e.g., in PDF, HTML, XML, …

The package provides convenience interfaces `exams2pdf()`, `exams2html()`, `exams2moodle()`, and `exams2qti12()` (for OLAT) that directly carry out all of these steps but the package also provides the underlying building blocks such that new interfaces can be easily created.

### A simple example

To get a quick impression of the look and feel of the package, consider the following simple example: computation of a *t* statistic from given mean, variance, and sample size. The corresponding `Sweave` template `tstat.Rnw` is provided within the package. After installing the package, you can produce random versions of this exercise in PDF and HTML by running the following code:

1 2 3 | library("exams") exams2pdf("tstat.Rnw") exams2html("tstat.Rnw") |

The former needs a working LaTeX installation and then pops up a PDF with the exercise.

The latter does not need LaTeX but just the R package tth (interfacing Ian Hutchinson’s wonderful TtH

TeX-to-HTML converter) and then pops up the HTML with the exercise in a browser.

MathML support is needed for this version of the HTML exercise, e.g., by using the Firefox browser.

### How it works

The `tstat.Rnw` is a relatively simple `Sweave` file: It first draws some random data and then uses the textbook formula to compute the correct *t* statistic. Subsequently, there are two LaTeX environments with the `{question}` and `{solution}`, respectively. Finally, some simple metainformation is provided

in LaTeX style (but commented):

<>= ## DATA GENERATION n n}}} = \frac{\Sexpr{Mean} - \Sexpr{mu}}{\sqrt{\frac{\Sexpr{Var}}{\Sexpr{n}}}} = \Sexpr{tstat}. \end{eqnarray*} The absolute value of the $t$~test statistic is thus equal to $\Sexpr{format(abs(tstat), nsmall = 3)}$. \end{solution} %% META-INFORMATION %% \extype{num} %% \exsolution{\Sexpr{format(abs( tstat), nsmall = 3)}} %% \exname{t statistic} %% \extol{0.01}

After running `Sweave()` on it, the resulting LaTeX code has the random numbers filled in and the data-generating code omitted:

\begin{question} A machine fills milk into $500$ml packages. It is suspected that the machine is not working correctly and that the amount of milk filled differs from the setpoint $\mu_0 = 500$. A sample of $226$ packages filled by the machine are collected. The sample mean $\bar{y}$ is equal to $517.2$ and the sample variance $s^2_{n-1}$ is equal to $262.56$. Test the hypothesis that the amount filled corresponds on average to the setpoint. What is the absolute value of the $t$~test statistic? \end{question} \begin{solution} The $t$~test statistic is calculated by: \begin{eqnarray*} t & = & \frac{\bar y - \mu_0}{\sqrt{\frac{s^2_{n-1}}{n}}} = \frac{517.2 - 500}{\sqrt{\frac{262.56}{226}} } = 15.958. \end{eqnarray*} The absolute value of the $t$~test statistic is thus equal to $15.958$. \end{solution} %% META-INFORMATION %% \extype{num} %% \exsolution{15.958} %% \exname{t statistic} %% \extol{0.01}

The LaTeX code is then read into R, can be transformed into HTML using `tth()`, embedded into a simple HTML file, and then shown in a browser:

### Support for learning management systems

The same HTML code shown in the example above can also be embedded into learning management

systems. These typically have XML-based exchange formats, e.g., Moodle uses its own Moodle XML

while OLAT and several other systems use the international IMS QTI 1.2 standard. The package provides functions `exams2moodle()` and `exams2qti12() `which provide output files that can be easily imported into Moodle and OLAT, respectively.

As a result, the `tstat.Rnw` looks in Moodle like this:

Note that this now has a field for entering the answer (and only after that the correct solution

can optionally be shown).

To generate a Moodle quiz with three random replications for five exercises drawn from collections

of different templates, consider the following example (based on templates provided within the

package):

1 2 3 4 5 6 7 | myexam <- list( "boxplots", c("confint", "ttest", "tstat"), c("anova", "regression"), "scatterplot", "relfreq") exams2moodle(myexam, n = 3) |

This produces a file `moodlequiz.xml` that can be imported into the question bank in Moodle. (In this step, the HTML markup for formulas and graphics is not formatted by Moodle but don’t worry: the final quiz will render correctly.) Subsequently, all replications of each exercise can be added as “random” questions into the quiz. Some more configuration of the quiz as a whole can then be done in Moodle but otherwise everything else is ready for the participants of the quiz.

### If you want to use this for your own course…

- You need an installation of a suitable learning management system but many universities have

support for this anyway. - You need to build your collection of
`Sweave`exercise templates. This is, of course,

the most work. However, the package provides a suite of examples that can be helpful as

starting points. - You need to export the exams/quizzes/assessments from R to the correct format for the

learning management system. Moodle and OLAT are directly supported. But through QTI 1.2

other systems could, in principle, be interfaced as well. We haven’t tested this yet, though.

More details are provided in the working paper and we also provide a support forum on R-Forge if you experience any problems etc.