Exams for Remote Instruction - Faculty

This is a guide for faculty who plan to administer exams remotely.

Overview

As you plan to administer exams remotely, consider if an exam is necessary, or if alternatives would better allow you to evaluate what your students have learned.  For example, is a project-based assessment or a series of lower-stakes assessments appropriate?

For most courses, the best option for delivering an exam is via a Canvas quiz, which supports both quantitative and qualitative exam formats. You can specify the amount of time a student has to take an exam and mix-and-match questions from among the following types:

• Automatically graded questions: Numeric, multiple choice, multiple answers, true/false, dropdown, matching, categorization, fill-in-the-blank, hot spot
  
• Manually graded questions: Short answer/essay, file upload

Since there is currently no option for remote proctoring, develop exams with the assumption that they are open note.  Allowing students to use course materials during an exam eliminates unenforceable restrictive policies. 

Best Practices

• Reinforce academic integrity by beginning with a 0-point question asking students to abide by the University's honor code.
• Consider what course content is available to students given their location. Don't assume all students have access to the same materials unless they are available online.
• Consider what, if any, content should be visible after students complete an exam. Make sure to restrict the student result view of exam questions, responses, feedback, and correct/incorrect answers until after all students have completed the exam.
• Set extended time for students needing accommodations.

Maximizing Exam Security

The strongest approach to exam integrity is to design an exam experience that delivers variations of the exam to each student. You can:

• Shuffle question order so that students see the exam questions in a different order.
• Shuffle answer option order so that answers and distractors appear in a different order, and optionally lock all of the above-type options.
  
• Create a larger pool of exam questions and draw from  an item bank; questions drawn from an item bank will be presented in random order.
  
• Create multiple versions of questions placed in an item bank.

Other approaches to enhance exam security include:

• Time limits, which minimize the amount of time students have to look up answers or consult with their peers.
• Multiple versions of an exam, when questions are linked or depend on each other and cannot be drawn from an item bank.
• Open-ended, complex questions, which require manual grading.
• Requiring that students show written work, by taking photos of their work, creating work digitally in Excel or a document, then uploading the files to Canvas.
• Show one question at a time and, optionally, lock questions after answering to prevent students from returning to earlier questions.

Exam Timing

Although reducing the amount of time that students have to take an exam increases exam security, it also increases the chances that students will encounter technical difficulties that hinder their ability to complete the exam. Multiple factors should be weighed when considering time constraints on your exam: 

• If you have an uncomplicated, easy-to-complete exam, you can simply estimate how long a prepared student would take to complete the exam and give your class that amount of time to complete it.  

• If you have students in multiple time zones, we suggest giving a wide window of time in which students can complete the exam to accommodate everyone. Based on your level of comfort, that window can be anywhere from 8-24 hours. Note that this window can still be used in conjunction with a timed exam so that students can take the exam anytime within the window of availability, but still only receive a finite number of minutes to complete it.  

• If you have a very complex exam that involves students using multiple technology platforms, uploading files, or answering different types of questions, we recommend granting up to 50% more time than you would in an in-person setting. This provides time for students to resolve technical issues before their time runs out, without requiring support from others.  

• If you’d like students to upload an image that shows their work, consider having students submit this to a separate assignment that accepts file uploads, instead of to the quiz. This avoids problems encountered when attempting to upload work during the timed exam.  

Regardless of what type of exam you’re administering, always remember to set a final due date so that the exam appears on students’ Canvas calendars, To-Do Lists, and other prominent places in Canvas. 

Introducing New Quizzes

Beginning Spring 2020, the Courseware team recommends that most courses use Canvas's New Quizzes (rather than the legacy Classic Quizzes). New Quizzes has been in use in select courses at Wharton since Spring 2019, and it will eventually be the only quiz option available in Canvas. New Quizzes offers many benefits for remote exams, including an easy-to-use quiz building environment.

• New Quizzes offers:
  • A more intuitive question-writing environment for you and TAs, as well as a friendlier test-taking experience for your students
  • Simpler procedures for extra-time accommodations
  • Onscreen calculator options (basic or scientific) which you can make available selectively
  • Regrading of most automatically graded question types, including numeric questions
  • Specialized question types including categorization, ordering, and image "hot spots"
  • Association of adjacent questions with a "stimulus" (a problem description, optionally with figures or graphs)
  • Configurable essay questions, including spell check, rich text formatting, word count/limit, and notes for graders
  • Selective shuffling of answer options per question, as well as flexible random question selection from item banks
• Classic Quizzes currently provides options that are not yet available in New Quizzes, but is more limited in terms of regrading.  This quiz engine is appropriate if you:
  • Rely on a downloadable Excel file of student responses
  • Prefer to bulk-download all student submissions to file-upload questions
  • Need to allow students to record audio or video in their responses
  • Link to files as part of the instructions
  • Want to revise quizzes in multiple Canvas sites controlled by a Blueprint Courses template

Questions?

If you need more assistance, please contact courseware@wharton.upenn.edu for support.