Generally, the responsibility of composing the exam falls upon the instructor of record, though question ideas may sometimes be opened to course staff. Teaching Professor Dan Garcia offers some advice on how to design exams:
Dan’s Five Finger Test for Exam Questions
Please visit the CS-ED Podcast to listen or read the full transcript of Dan Garcia’s interview on designing exams.
A few additional logistical concerns:
- For questions that would be tedious to grade, include a bubble-in option to receive a small fraction of the points in exchange for not grading the question.
- For code-writing questions, include scaffolding and fill-in blanks to help structure responses and reduce variation in answers. This can also help students by providing hints to the structure of the problem.
- Prefer multiple choice questions where possible. Integrate multiple-choice options for true-false, even when an explanation is required, to trim the effort needed to identify incorrect-choice answers.
Pledge of Academic Integrity
In 2012, a team of psychologists and behavioral economists presented a study “Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end”, proposing that when people sign to a pledge of honesty to their reportings before, rather than after, their submission, the opportunity to cheat makes ethics salient when they are needed most and significantly reduces dishonesty. The authors suggest that when we become aware of the possibility of immoral behavior, we reflect on our own morality. Since this publication, organizations from around the world have adopted this practice, including many professors at UC Berkeley.
Here is an example taken from Professor Babak Ayazifar:
By my honor, I affirm that
- this document—which I have produced for the evaluation of my performance—reflects my original, bona fide work, and that I have neither provided to, nor received from, anyone excessive or unreasonable assistance that produces unfair advantage for me or for any of my peers;
- as a member of the UC Berkeley community, I have acted with honesty, integrity, respect for others, and professional responsibility—and in a manner consistent with the letter and intent of the campus Code of Student Conduct;
- I have not violated—nor aided or abetted anyone else to violate—the instructions for this exam given by the course staff, including, but not limited to, those on the cover page of this document; and
- More generally, I have not committed any act that violates—nor aided or abetted anyone else to violate—UC Berkeley, state, or Federal regulations, during this exam.
(10 Points) In the space below, hand-copy the text of the pledge above—verbatim—and then write your name in legible letters, sign, include your full SID, and date before uploading your work to Gradescope.
Screenshot of Rubric Used for Grading Exams in Gradescope
Use Gradescope! Visit the Help Center to learn how to use Gradescope most effectively.
When grading with multiple graders on an ungrouped problem, design the rubric to reduce ambiguity. Use Shift+Return to enter multiple lines of text and organize ideas. The clearer the rubric, the more consistent the grading and the fewer confused regrades from students. When there is ambiguity, consult with other graders instead of assuming one way or the other. Gradescope’s rubrics support LaTeX (use $$ or $$$), including bolding, colors, etc, which can help communicate some answers.
Positive grading rubrics are the default for many questions, though you may want to consider scenarios in which negative grading rubrics may be more helpful or decompose knowledge in a different way. (Note that you can mix-and-match styles within a rubric, and between questions.)
Make sure new staff members are sufficiently introduced to the grading process. Grade in pairs, perhaps. Rubric creation should also be done as a group first to establish a consistent grading scheme amongst all graders.
A few practical tips:
- Plan a tiny bit before creating a rubric.
- Quickly review the first 5-10 submissions to get an overview and use it to iterate and improve the rubric before committing resources to grading.
- Use the magnifying glass to spot-check.
- Create a rubric item to “flag” certain submissions that are difficult (or take longer than expected) to grade or otherwise don’t fit the rubric. Grade them in a second pass using the magnifying glass. Be sure to delete this before releasing grades.
When regrading, it is important to keep in mind equity with other students. Before reading the student’s plea, regrade the problem first to avoid coloring judgement with student bias. Add or deduct points accordingly. Consider your regrading strategy as a course. If two TAs graded a question, have the other TA address the regrade.