Aggregating Student Peer Assessment During Capstone Projects

Student assessment of other student's work has many potential benefits to learning for both the assessor and the assessed. However, sources of peer assessment provide, quite often, subjective evidence, which can be conflicting, uncertain and even ignorant. One of the key elements to providing an overall quality assessment of a student's work derived from the assessment by his/her peers is the use of an appropriate method of combining or fusing these heterogeneous evidence sources. Since the development of the belief theory introduced by Shafer in the 1970s, many combination rules have been proposed in the literature with two main methods selected here. The first is an evidential reasoning (ER) approach, the kernel of which is an ER algorithm developed on the basis of the framework and the evidence combination rule of the Dempster-Shafer (DS) theory. It has been claimed also in the literature that Dempster's rule generates counter-intuitive and unacceptable results in practical situations, so an approach based on the Dezert-Smarandache (DSmT) theory of fusion will also be explored, in particular the PCR6 rule of proportional conflict redistribution. Results for peer assessment marks allocated by a student cohort, consisting of 20 students, during their capstone projects, and, aggregated using each of these two approaches are compared with each other and with results obtained by the more traditional Averaging Rule (AR) approach. It is clear from the findings that when subjective evidence is aggregated then the simple AR approach as the accepted combination method is in doubt. It also seems that the DS method of aggregation seems the best alternative to traditional averaging.