Overview¶

Here’s an overview of the scripts and the grading workflow.

Scripts¶

The three scripts that you will use are:

grade.py
- Organizes the submission files and generates the grading files for you. For more information, see the Grading structure section.
open_reports_in_browser.py
- Opens mulitiple codecheck report files at once for quickly viewing the reports while grading. For more information, see the Open reports in the browser section.
grade_upload.py
- Parses the grades.txt generated by grade.py and uploads the data to the assignment on Canvas. For more information, see the Uploading the grades section.

Canvas API¶

This script communicates with Canvas through the Canvas API with the following API calls:

List assignments
- In grade.py, this is used to get a listing of all the assignments in the course to find the match with the assignment name that is passed in when you run the script. It will grab the assignment ID and the assignment’s maximum score from Canvas.
- In grade_upload.py, This is used to get the Canvas assignment ID of the assignment that matches the assignment name that you pass to the script.
Grade or comment on a submission
- In grade_upload.py, this is used to upload the scores and your comments for each student written in the grades.txt file produced by this script.

Workflow¶

The overall grading workflow is as follows:

Download the submission files from Canvas. There should be a “Download Submissions” link on the righthand sidebar on the assignment page on Canvas.
Extract the files to a new directory.
Run the grade.py script on the directory. The typical command looks like:
```
python grade.py "The Assignment Name on Canvas" path/to/submissionfiles
```
Where "The Assignment Name on Canvas" is the name of assignment name you want to grade, and path/to/submissionfiles is path to the files that you extracted to in Step 2 above.
Grade the files by looking at the report files for each student with the open_reports_in_browser.py script. For more detail about this script, see the Open reports in the browser section.
When you’re done grading, upload the grades.txt file to Canvas. The typical command looks like:
```
python grade_upload.py path/to/grades.txt
```
Where path/to/grades.txt is the filepath to the grades.txt file.

What to watch out for¶

These grading scripts don’t do everything, but it tries to catch some common problems.

What the script detects¶

The script automatically detects the following issues and informs you if anything was found in the _notes_and_score_changes section of grades.txt:

Duplicate submissions, such as two identical submissions of the same part. For example, If an assignment had students submit a Robot.java file for one of the parts, and a student submitted two codecheck reports for Robot.java, then the script will only count one of them. In grades.txt, this problem will be described as “javafile already submitted.”
Draft/Final submissions. Usually this problem happens when students submit draft files for a final homework. First, the script counts all of the submissions that are “draft” level and “final” level. The script then chooses the level with the most submissions (because most students are going to submit the correct files), a la the-majority-is-probably-correct. In grades.txt, this problem will be described as “Incorrect problem level”.
Irrelevant submissions. Just as the draft/final homework level is detected with the majority-is-probably-correct heuristic, the irrelevant submissions are also found the same way. Every codecheck submission has a problem ID, so all of them are counted, so the top three (or, more generally, the top number-of-parts-for-the-assignment) IDs are chosen for the assignment. Problem IDs that are not part of this top list are not scored. In grades.txt, this problem will be described as “javafile is not part of this assignment”.
Cheating. If multiple students submitted the same codecheck submissions, then they will not be given credit for those submissions. In grades.txt, this problem will be described as “This exact codecheck submission has been turned in by another student. Submission ID = xxxxxx”, where xxxxxx is the submission ID. You can search for xxxxxx in the same grades.txt file to see the other students who have submitted the same files.

What you should look for¶

Code structure. This includes the logic of the code, instance variables, naming, and the like. These scripts don’t check for style, code logic, hard coding, etc.