An Online System for Assignment Marking

Greg Baker
Simon Fraser University

Abstract

In an attempt to streamline the process of marking electronically submitted assignments for large classes, an online marking system has been implemented that can be used by instructors and teaching assistants. This system has had several benefits in terms of efficiency of marking time and for student feedback.

The marking system itself is described, along with a summary of experiences using it and possible improvements that can be made.

Keywords: grading, marking automation, web application

1. Introduction

The School of Computing Science at Simon Fraser University has for some time had an “Assignment Submission Server” available for use in classes. This allows students to submit files over the Internet to the instructor and teaching assistants. When received, the submitted code or other work can be evaluated.

Getting feedback to students has been tricky with this system. When written assignments are submitted, the marker can annotate and give feedback on the question paper itself, which is then returned to the student. With electronic submission, the common solutions are to print out and annotate the assignment, fill out a marking form with comments, or send email to the student with their marks.

Printing out assignments that has been submitted electronically seems counter-productive and returning printed assignments is problematic. It also leaves the instructor with no copies of the marks and comments once they are returned to the students. Sending email to each student individually takes a lot of time—the marker must not only send each email, but also record the marks separately.

2. Automated Marking

In order to address this problem, an online marking system was created. This is a web-based application created in Perl and using MySQL for data storage. Being a web-based system, it use is not limited to a particular location or group of machines. All administration and configuration of the system can also be done with a web-based interface.

Markers can log into the system and access any courses for which they have been granted access. When they select a course, they are presented with a list of assignments. Assignments can be marked by any combination of authorized users. To mark a student's assignment, the marker enters the student's userid (which is assumed to uniquely identify a student), their student number and email address.

Each assignment has a list of criteria which the students are expected to meet. These can be configured by the instructor or any of the markers using an online interface. For each criterion the marker can enter a mark and comments. The comments are arbitrary text and can be quite long (up to 64k characters). This allows students to get very detailed feedback on their work.

It was recognized that often many students make the same mistakes on their assignments. This means that the marker will often have to enter the same comment many times. There are also occasionally problems with consistency—a marker can change the mark penalty for a particular error part way through the marking or different markers could apply different standards. To address both of these problems, the concept of “common problems” was added to the system.

Common problems are configurable on a per-criteria basis. The marker can indicate a mark penalty and comment for a particular problem. Then on the marking screen, a button appears associated with that problem. When clicked, it deducts the mark penalty and inserts the comment for the student. Once a marker has configured a reasonably complete set of common problems, an individual assignment can be completed quite quickly. This uses less marking time but also provides better feedback to students—the marker only has to type each comment once, so can take more time in crafting useful feedback for the problem.

Below is a screenshot of the marking system with a sample assignment. At the top are the fields to enter student information, ie. their userid, student number and email address. Following are the assignment criteria. For each criterion, its name and description are displayed. The marker can enter a mark for that component and a comment. Finally, the common problems that the marker has configured for this assignment are represented by the buttons to the right of the criteria. The marker can also enter a late penalty and overall comments on the assignment (not shown).

Along with simple marks and comments, a file can be stored along with each marked assignment, allowing for return of annotated code. A student's code can be converted to an Acrobat PDF file and the marker can add comments with Acrobat. This file can then be returned to the student, allowing the same kinds of comments that might be made on printed code. Below is part of an example PDF file.

While this was created for annotating code, any other type of file could be included along with the marking as well. When the data are submitted, they are then stored in the system for later processing.

3. Output

The most visible output from the marking system is the email that is sent to the student. This is used to give feedback, as well as to report marks to the students. A sample email is shown below.

To: thestudent@sfu.ca
From: John Doe <jdoe@cs.sfu.ca>
Subject: CMPT 000 Assignment 1 results

SFU Userid: thestudent
Student number: 000000000

=================================================================

Program Creation: 2 out of 2
  Criteria: The first.c program was created properly.

Compiling and Execution: 0 out of 3
  Criteria: The program was compiled and run correctly.
  Comments: no output provided

Modified program: 5 out of 5
  Criteria: The program was modified and correctly calculates 
    the area.

Algorithm: 3.5 out of 5
  Criteria: The algorithm description is complete and unambiguous.
  Comments: Decomposition Outline is not complete. How do you know 
    that the equation has 0, 1 or 2 real roots?

=================================================================

TOTAL: 10.5 out of 15

[Marked Jan 29 2003, 11:43 by John Doe.]

Typically, all of the emails to the class are sent when the marker has completed the marking. The system can also be configured on a per-assignment basis to send emails immediately when individual marks are submitted.

Other forms of output can be added to the system as necessary. At the moment the following are available:

an import file for WebCT
an import file for the SFU Computing Science Gradebook system
a spreadsheet import file

These files can be used to export marks from this system and into another course-management system.

4. Experiences

My experience using the system has been good. The TAs have been happy with it because it streamlines the feedback process and eliminates some of the repetitiveness of marking many assignments. It also keeps all of the marking information organized, ie. there is no possibility of misplacing assignments or completed marking.

I have found that having the marks stored electronically is very valuable when students come with concerns about their marking. I can find their marks in the system and quickly review the comments the TA made. This does not rely on the students to bring in a marking summary or on their memory of the comments they received (which is often inaccurate).

If I want to change the marks for a student, I can quickly load the data currently in the system, edit the marks and comments and then resubmit. The entire process can also be done asynchronously, by email.

An unexpected benefit, as an instructor, is that I can keep track of the marking while it is in progress. I can easily check to see how many assignments the TAs have marked and thus have a good idea of when they will be finished. I can also spot-check a few assignments early in the process to make sure the marking is being done with due care and attention. I have also found it useful to be able to look back on the details of a student's performance over the semester when making difficult pass/fail decisions for borderline students, and when writing letters of reference.

5. Future Work

While the marking system is currently in a usable state, there are still improvements to be made.

The user interface needs improvement to allow the most common tasks to be performed more quickly. In particular, it should be easier to add common problems directly from the marking screen and the process of entering and looking up marks should be more tightly tied together.

5.1. Feature Additions

Currently, the system does not import any data about the class. It simply records marks for the userids that the marker enters. It is easy to enter a userid incorrectly, which then causes failed emails and missing marks. It would be useful to incorporate a class list into the system and warn when information is entered incorrectly, or could allow them to choose from a list of students. This would also allow markers to know how many students do not have marks and who they are.

The current authentication method is HTTP basic authentication. This is handled by the web server, so that users arrive into the system with an authenticated userid. This may not be possible for all systems. It would be possible to extend the system to keep track of user's passwords and authenticate internally.

The system was designed to focus on mark recording and giving feedback to the students by email. It would be possible to add student access to the system, allowing them to see their marks via a web interface. This would require students to be given accounts and passwords for the system; at SFU their Gradebook accounts could be used. In general, I have found that email feedback is sufficient.

5.2. Portability

The current system makes several assumptions that are local to SFU. These include the format of student numbers and userids. There are also several places where an “@sfu.ca” email address is assumed for the students and markers. The code has been written so it can be easily generalized to other institutions. However, there still remains some work to be done to implement this.

5.3. Release

I would like the code to be reviewed by a third party for security before it is released. Once this is done, the code will be released under the GNU Public License. I am currently using the software in several courses and consider it secure Anyone wishing to experiment with the system before its general release can contact me for more information.

6. Conclusion

Overall experiences with online marking have been positive, particularly when coupled with online assignment submission. It has eliminated the problem of processing and returning hard copies of assignments.

As an instructor, it is one more tool that helps in keeping track of large classes. With many students and TAs, the online marking system provides a central location for all of the assignment marks and comments.

Greg Baker
School of Computing Science
Simon Fraser University
Burnaby, BC, Canada
V5A 1S6
ggbaker@cs.sfu.ca
http://www.cs.sfu.ca/~ggbaker/

Return to Greg's Home Page.