Automatic marking of descriptive questions of online examinations using NLP

Abstract

Online examinations are increasingly being integrated into universities using low-level questions on Bloom's taxonomy such as True-False and Multiple-Choice questions. Unfortunately, the summative assessments in Tanzania rely on descriptive questions that are higher in Bloom Taxonomy which are not easily marked by computers. To achieve a true online examination in Tanzania a system that can mark descriptive questions is necessary. This paper presents a Natural Language Processing (NLP) Technique employing Cosine Similarity that can automatically mark descriptive questions without human intervention. The Cosine Similarity compares between candidate answer and the marking scheme after having them transformed into mathematical vectors by the NLP technique called Term frequency–inverse document frequency (TF-IDF).   Evaluation done by comparing the marking of short answers by NLP-based online exam system and Moodle as well as feedback from the Computer Science students from the Open University of Tanzania who used the NLP based online exam system indicates positive performance. 86% of questions were marked correctly by the NLP system and got a similar score to the one in Moodle. 66% of students to trusted the score returned by the system while only 4% did not trust the score which confirmed the acceptability of Cosine Similarity for marking mark descriptive questions automatically.