[Ruler-commits] r36 - pkg/ruleR/inst/doc

[noreply at r-forge.r-project.org]

Author: doebler
Date: 2012-08-20 16:44:11 +0200 (Mon, 20 Aug 2012)
New Revision: 36

Added:
   pkg/ruleR/inst/doc/ruleR.bib
Modified:
   pkg/ruleR/inst/doc/ruleR.Rnw
Log:
Started writing vignette.

Modified: pkg/ruleR/inst/doc/ruleR.Rnw
===================================================================
--- pkg/ruleR/inst/doc/ruleR.Rnw	2012-08-10 16:02:02 UTC (rev 35)
+++ pkg/ruleR/inst/doc/ruleR.Rnw	2012-08-20 14:44:11 UTC (rev 36)
@@ -28,7 +28,20 @@
 \maketitle
 
 \section{Introduction}
+The success of a psychological test is largely determined by the quality of its items. In classic linear testing, the number of items needed to measure a single latent trait with acceptable reliability is often in the range of 20 to 60 items. While it is certainly possible to construct this number of items with the help of heuristics it is often possible to indentify rules governing the item construction process. For example for the classic Advanced Progressive Matrices Test (\cite{raven1962advanced}) five (broad) rules used in its construction have been identified (\cite{carpenter1990one}). These rules and their combinations are used on the rows and/or columns of a matrix resulting in many different stimuli.
 
+Also number sequences used in intelligence tests are often derived from basic rules like addition and subtraction (of constants or two previous numbers), multiplication and the digitsum (sum of the digits of a number).
+
+Recently \emph{automated item generation} (AIG) has been explored in various contexts (TODO: Citations). Here the idea is to identify the underlying template or rule(s) that constitute an item and to generate new items (potentially infinetly many) of the same type from that. Two main approaches can be identified: \emph{item cloning} (IC) and \emph{rule-based item generation} (RIG). The first approach uses an existing item (a parent), typically with known psychometric qualities, and produces a clone (a child) of that item by changing its \emph{surface features}, e.g. in a statistics exam for university students, the cover story of the example is changed, but the student is nevertheless to make the same calculations (TODO: citation). RIG on the other hand focuses on the rules that govern the item construction process. Once the rules and their relations (and their relation to surface features of items) are known, a new item can be generated from a (combination of) rule(s). Often it is possible to predict the difficulty of an item by using an LLTM (TODO: citation fisher and applications of LLTM in AIG).
+
+There are several situations in which automated item generation is favourable:
+\begin{enumerate}
+\item Linear tests, especially in high stakes situations like college admission, are often used only in one year since the test security can not be guaranteed once the test has been exposed to a large population. Here automated item generation leads to tests for which the answers can not be learnt by heart.
+\item Computer adaptive testing (CAT; \cite{elements} TODO: add more citations) relies on large pools to cover a wide range of potential person abilities. It is often expensive to produce items, so automating the process is certainly desirable here. Also if the psychometric properties of parent items or rules are known, the CAT algorithm can generate items on the fly.
+\end{enumerate}
+
+While the identification of rules does not directly imply that an rule-based generation is possible it is a necessary step. The \texttt{ruleR} package provides a framework to generate number sequence items and matrix type items in \texttt{R}. While \texttt{R} itself is not a frontend for computer based testing, it's applicability has been successfully demonstrated, for example by the concerto testing platform (\cite{concerto}).
+
 \section{Number sequences}
 
 \section{Matrix items}

Added: pkg/ruleR/inst/doc/ruleR.bib
===================================================================
--- pkg/ruleR/inst/doc/ruleR.bib	                        (rev 0)
+++ pkg/ruleR/inst/doc/ruleR.bib	2012-08-20 14:44:11 UTC (rev 36)
@@ -0,0 +1,34 @@
+ at article{carpenter1990one,
+  title={{What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test}},
+  author={Carpenter, P.A. and Just, M.A. and Shell, P.},
+  journal={{Psychological Review}},
+  volume={97},
+  number={3},
+  pages={404--431},
+  year={1990},
+  publisher={American Psychological Association}
+}
+
+ at book{raven1962advanced,
+  title={Advanced Progressive Matrices},
+  author={Raven, J.C.},
+  year={1962},
+  publisher={London: H.K. Lewis}
+}
+
+ at misc{concerto,
+title={{Concerto: Open-source Online R-based Adaptive Testing Platform}},
+author={{The Psychometrics Centre}},
+howpublished={Software package},
+year={2012},
+note={Available at:  \url{http://code.google.com/p/concerto-platform}},
+}
+
+ at book {elements,
+  title = {{Elements of Adaptive Testing}},
+	year = {2010},
+	publisher = {Springer},
+	address = {New York},
+	doi = {10.1007/978-0-387-85461-8},
+	author = {van der Linden, W. J. and Glas, C. A. W.}
+}