Detailed usage¶

Here’s a detailed page on what happens to the files when you use these scripts.

Structure¶

Important

The script should only be run on a fresh directory of submission files from Canvas. In other words, don’t run the script more than once on the same set of data. If you want to redo the script again for the same homework submissions, start over from the beginning with the initial set of submission files untouched.

For help and more options for running the script, run python grade.py --help

$ python grade.py --help

usage: grade.py [-h] [-v] [-n NUMBER_PARTS] [--no-verify] [--no-checks] [-r]
                assignment_name directory

positional arguments:
  assignment_name       The assignment name. Must match an assignment found on
                        Canvas.
  directory             the directory containing subdirectories of student
                        submissions.

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         print verbose output to stdout.
  -n NUMBER_PARTS, --number-parts NUMBER_PARTS
                        provide the number of parts for this assignment
  --no-verify           skip jarsigner verification and unarchive all
                        submission files
  --no-checks           Don't check for warnings/cheating submissions (checks
                        by default)
  -r, --remove-existing
                        removes grading files from the directory from a
                        previous run

The grading script requires a directory of submissions from Canvas.

To grade an assignment, download the .zip file from Canvas and extract its contents to a directory. The directory structure may look something like

directory_with_submissions
|
|--studentname1_studentid1_submissionid1_filename1.ext
|--studentname1_studentid1_submissionid2_filename2.ext
|--studentname1_studentid1_submissionid3_filename3.ext
|--studentname2_studentid2_submissionid4_filename4.ext
|--studentname2_studentid2_submissionid5_filename5.ext
|--studentname3_studentid3_submissionid3_filename.ext
+--...

where ext is either signed.zip, zip, or jar. (Note: There were three possible extensions with codecheck because it first gave out jar files, and then zip files. Now, codecheck only provides students with a signed.zip file).

To run the script on the directory, the command will be something like

python grade.py "Assignment Name" directory_with_submissions

Where "Assignment Name" matches the name of the assignment on Canvas. Don’t forget to enclose the name in quotation marks if it contains spaces.

When the script runs on a submissions directory, it organizes the directory by separating each student’s submissions to an individual directory. So the directory tree above will become

directory_with_submissions
|
|--studentname1_studentid1
| |
| |--studentname1_studentid1_submissionid1_filename1.ext
| |--studentname1_studentid1_submissionid2_filename2.ext
| +--studentname1_studentid1_submissionid3_filename3.ext
|
|--studentname2_studentid2
| |
| |--studentname2_studentid2_submissionid4_filename4.ext
| |--studentname2_studentid2_submissionid5_filename5.ext
|
|--studentname3_studentid3
| |
| +--studentname3_studentid3_submissionid3_filename.ext
|
+--...

Then, the script will extract the files to their own directory (with the same name as the .ext file). If the --no-verify option is specified, then all files will be extracted. Otherwise, only those that are verified by jarsigner will be extracted.

Once all the files are extracted and in their own directories, the script will begin totaling the codecheck scores from the report.html files and writing the scores to a file called total_grade.txt. Then, the script will walk through the directory tree and aggregate the scores from the total_grade.txt files in each student directory into a file called grades.txt, which will be found at the root of directory_with_submissions.

Summary of the structure¶

Each student’s submissions are moved to his or her own directory named lastname--firstname-miscname_canvasid. Then, each codecheck file is unarchived to its own directory. Each student will have a total_grade.txt file found in his or her own directory, and a grades.txt file will be found in the directory_of_submissions (the directory you ran the script with).

Aggregate Java files¶

Also, the script aggregates the java files and the report files. The script will aggregate all of the java submission files into one file called aggregate_java_files.txt in each student directory for easier viewing of the source files that a student as submitted. Here’s an example of an aggregated file for a student:

public class StringDemo
{
   public static void main(String[] args)
   {
       String word = "surprise"; //do not change this line

       System.out.println(word.length());

   }

}
/**************************************************/
public class TextDemo
{
     public static void main(String[] args)
    {
          Text message = new Text(10, 50, "Hello, World!");
          message.draw();

    }
}
/**************************************************/
public class Rainbow
{
    public static void main(String[] args)
    {
          Rectangle box = new Rectangle(0, 0, 100, 20);
          box.setColor(Color.RED);
          box.fill();

    }
}

A /**************************************************/ is the separator between different files.

Aggregate report files¶

The report.html files will be aggregated into one aggregate_report.html file per student. These aggregated files are useful to see the reports of one student’s homework on one page instead of a page for each part of the homework.

After the above steps, the directory structure will become

directory_with_submissions
|
|--grades.txt
|
|--studentname1_studentid1
| |
| |--studentname1_studentid1_submissionid1_filename1
| | |
| | +--(codecheck files)
| |
| |--studentname1_studentid1_submissionid2_filename2
| | |
| | +--(codecheck files)
| |
| |--studentname1_studentid1_submissionid3_filename3
| | |
| | +--(codecheck files)
| |
| |--studentname1_studentid1_submissionid1_filename1.ext
| |--studentname1_studentid1_submissionid2_filename2.ext
| |--studentname1_studentid1_submissionid3_filename3.ext
| |--aggregate_java_submissions.txt
| |--aggregate_report.html
| +--total_grade.txt
|
+--...

Open reports in the browser¶

With all of these “aggregate_*” files created to help with looking at the grades, it sure would be helpful to have a fast way to open all of these files at once. The script open_reports_in_browser.py will do that for you.

The script opens the aggregate_report.html files for a given directory with the default application for html files (probably your browser).

There are optional arguments (documented below) to provide the script if you only want to open reports for a certain alphabetical range for students’ last names. This is helpful if you’re taking a break for grading and you don’t want to open all the report files, but just the student you left off with and the rest.

To open all the report files, you can run

python open_reports_in_browser.py path/to/directory

Opening all of the report files at once is probably not what you want to do though, because that’s 100+ tabs in your browser at once.

There are -s and -e options that are optional. They’re shorthand for --start-letter and --end-letter. (Run python open_reports_in_browser.py --help for clarification).

For example, to open report files starting with students whose last names start with ‘G’ and the rest of the students, run

python open_reports_in_browser.py path/to/directory -s G

which will open the reports from the ‘G’ students to the last students (‘Z’ students, implicitly).

To open report files from the beginning of the list until the ‘N’ students, run

python open_reports_in_browser.py path/to/directory -e N

which will open the reports from the beginning of the alphabet (‘A’ students) to the ‘N’ students.

To open reports in a range, say, from ‘G’ to ‘N’, just combine the options -s and -e

python open_reports_in_browser.py path/to/directory -s G -e N

There’s a shortcut option, -se, that can be used if you just want to start and end with the same letter, which will only open reports within that single letter’s range.

python open_reports_in_browser.py path/to/directory -se A

which is equivalent to the call

python open_reports_in_browser.py path/to/directory -s A -e A

(Note: the last-name letters are case-insensitive. so -s G and -s g are the same)

And if you would rather open the aggregate_java_files.txt files instead of the report files, then supply the -j option to the script. They’ll probably open up in your text editor (because the script opens the files with the default application depending on the file extension. “txt” usually opens in text editors). The java files take up less vertical space overall compared to the report files, but the report files have colors like red that indicate test cases that failed and the test results of the program (including graphical results for graphics programs, so the report files are probably the way to go, but aggregate java files are also an option.

Checking for duplicate submissions¶

Usage:

python check_duplicate_submission_ids.py <directory>

check_duplicate_submission_ids.py is a script that will check the submissions directory (that you will pass as an argument to the script) for any duplicate submission ids in the directory. If any duplicates are found, then the script will display them like so:

Here are the students who have the same submission id:
Submission id: xxxxxxxxxxxxxxxxxxxxxxxxxxxx
      Students:
      lastname1--firstname1
      lastname2--firstname2
Submission id: yyyyyyyyyyyyyyyyyyyyyyyyyyyy
      Students:
      lastname3--firstname3
      lastname4--firstname4
[... and so on]

If there are no duplicate submission ids, the script will output:

There are no duplicate submission ids.

Grading structure¶

Each student’s submissions are moved to his or her own directory named lastname--firstname-miscname_canvasid. Then, each codecheck file is unarchived to its own directory. Each student will have a total_grade.txt file found in his or her own directory, and a grades.txt file will be found in the directory_of_submissions (the directory you ran the script with).

At the top of grades.txt there are two fields that are used to identify the assignment:

_canvas_assignment_name: The name of this assignment.
_canvas_assignment_id: The ID of this assignment.

These are used for uploading grades to the correct assignment later when you use grade_upload.py.

For each student, grades.txt file has several fields:

_dir shows the name of the directory of the student’s submissions
_name shows the name of the student
_canvas_id shows the Canvas ID of the student
_total_score shows the score that the student received for the assignment
_comment holds the comments that you put in for a student’s submission. This will be sent to Canvas and appear as a comment on the student’s assignment. It will only be treated as text, with the beginning and ending whitespace stripped out. This field does not affect the score in any way.
_notes_and_score_changes holds the options to mutate the score. If you want to change the score, the first token of a line must be an integer, and that integer must be followed by a single space. You can also write your own personal notes here which will not sent to Canvas.

Both the _comment and _notes_and_score_changes fields can be empty, but you should comment on something with each student. Give some good feedback :). All other fields should be treated as read-only. You should only need to write in the _comment and _notes_and_score_changes fields.

Students who’ve received full credit for the assignment will automatically have a comment complimenting them for their good work, including their first name (and middle name, if any. Sometimes including the middle name is appropriate for a name such as “Yin Yan”, but not as much for names such as “Chris Joseph”). Examples include “Good job, Joe.” “Good work, Sally.”, and “Well done, Jessica.”

Example¶

Here’s an example student submission in grades.txt:

_dir: last_first_0123456
_name: last_first
_canvas_id: 0123456
_total_score: 18/18
_comment:

Good job, but remember to make your instance variables private.

_notes_and_score_changes:

-1 didn't make instance variables private
-2 let's deduct more points for this example
+1 we can add points too
Here's a note. This line won't change the score at all.
--------

The student last_first with the ID 0123456 would receive 16 (= 18 - 1 - 2 + 1) and see the above comment on Canvas. The _notes_and_score_changes field doesn’t get published. It is only used to modify the score (or write notes), and only the numerical value at the beginning of a line matters. The label is optional; its only purpose is to give the score change context. For instance, the above _notes_and_score_changes field would function exactly the same as

_notes_and_score_changes:
-1
-2
+1
--------

Uploading the grades¶

Uploading the grades to Canvas will publish the grades automatically with the grades given by the grades.txt file for students who submitted the grades. For students who did not do the homework (i.e., they did not submit anything), they will receive a “No submission.” comment along with a score of 0 automatically when the grades are submitted.

Run grades_upload.py to upload the grades to Canvas.

python grades_upload.py path/to/grades.txt

path/to/grades.txt is the filepath to the grades.txt file you edited with your grades and comments.

Script Optional Arguments¶

Here’s some detailed description for the optional arguments you can pass to the scripts.

`grade.py`¶

Indicate the number of parts with -n NUMBER (or --number-parts NUMBER)
- If you don’t want to deal with the prompt and provide the number of parts to the assignment at runtime, you can provide the number of parts as an option when you run the script with this option, with NUMBER being the number of parts for this assignment. (e.g., 3)
Skip verifying the submission files with --no-verify
- Instead of checking if the codecheck file is verified before extracting it, the script will skip verifying any files and just extract it. If you want the script to extract all the submission files, regardless of whether or not the codecheck file is actually signed, then use this option. Beware that the script will grade any codecheck file that’s structured correctly even if it’s not signed.

`grade_upload.py`¶

Don’t upload comments with --no-comments
- By default the upload script will upload both scores and comments. If you only want to upload the grades, you can use the --no-comments option when running the script. Say, you want to reupload all the grades again due to a correction after already submitting grades. If you upload the grades again with comments, the comments will be duplicated. This options avoids duplication.
Verbose output with --verbose
- Use this option to have the script print out the grades before confirming the upload and print out more information with the URLs during the upload process. For more information on the possible options, run python grade_upload.py --help.