Reviewing Peer Reviews - A Rule-Based Approach

Peer reviews may be used as an effective tool in non-rote learning, especially if one wishes to provide learners with computer-assisted learning environments within the general theory of pedagogical constructivism. Our past research and development efforts have resulted in the creation of an innovative approach to statistics education in which peer review activities, based on reproducible computing technology, play an important role in the constructivist learning of statistical concepts. The associated research has also shown that many types of objective measurements are available and that these are of the utmost importance when explaining students' learning outcomes. The first problem that is addressed in the current paper is how these peer review measurements can be used to predict learning outcomes. In order to engage students in taking peer reviews seriously they should be motivated. This can be achieved by reviewing and grading the peer reviews that they submit during the course. Any such attempt is likely to raise the problem, however, of how the educator should review the reviews. This is the second problem that will be discussed in our paper. Using a theoretical framework as our starting point, we are able to derive empirical rules that allow us to perform Peer Review-Reviews under the principles of implementability, predictability, comparability and purposefulness. It may be argued that the review of peer reviews becomes obsolete if students have to write a term paper which is going to be evaluated by the educator. However, the underlying rationale is that a term paper involves all concepts that are deemed important while the peer review of an assignment only focuses on a small number of topics. The third problem that is investigated in this paper focuses on the relative importance of grading term papers versus reviewing peer reviews. In other words, is it more efficient to have students submit a term paper which is graded by the educator, or is it better to review the Peer Reviews of weekly assignments? Our findings will be based on a detailed quantitative analysis of the peer review and student assessment data that were collected over a period of three years, involving 285 university-level students who took an introductury statistics course.