On portfolio assessment, group work, and quasi-anonymization: What structural information do anonymized reports reveal?

Abstract

With traditional summative exams students’ identities are often hidden from the examiners. However, achieving anonymity can be challenging with portfolio and coursework-based assessments. This study was motivated by a discussion of whether students should, or can, be anonymized in courses with portfolio assessments, especially if the teacher is both actively involved in the students’ learning process and the assessment. With portfolios the student will work on a set of projects throughout a course and submit the final product for assessment. One may argue that as the teacher gains familiarity with students’ work throughout the semester it is challenging to achieve true anonymity even if the final submissions are technically anonymized. To provide empirical evidence of this an experiment was designed to assess the degree of simplicity to which non-anonymous coursework submitted for formative evaluation can be matched with anonymized coursework submitted for the final summative evaluation. The experiment was also designed to assess whether it was possible to automatically detect student group structures based solely on the reports. The results based on 1098 reports show that students’ writings could be successfully matched across versions, while it was only possible to partially detect group structures. Implications of this work is that conscious decisions need to be taken about the quality of the student-teacher dialogue quality versus the need for anonymous assessments.