In this article, you'll learn the basics of using the WHOOPPEE app.
Before you start:
You will need the following before you can use WHOOPPEE:
- A laptop or desktop PC (tablets and mobile are not supported)
- A modern web browser (such as Chrome, Firefox, or Safari)
- Software to generate PDFs for Canvas submission (Acrobat is recommended)
What is WHOOPPEE?
WHOOPPEE is the Wharton Online Ordinal Peer Performance Evaluation Engine which allows students to anonymously submit, rank, and review assignments from their peers. The app operates through 4 phases: Confirm Submission, Rank & Review, Respond, and Results. Its basic workflow is:
- Submit an assignment through Canvas. This assignment must be stripped of all Personally Identifiable Information (PII) and completely anonymous. The good news is: WHOOPPEE will do that for you!
- When the Submission phase is closed, WHOOPPEE then distributes random batches of other submissions from your peers back to you.
- Once you have the batch of submissions, you can review them, comment on them, and rank them in order of strength.
- After ranking and reviewing, you will then be able to view your peers' comments on your own submission, and decide if the comments are helpful or unhelpful.
- The results of the assignment (including your grade) are displayed.
- WHOOPPEE also provides some helpful context in addition to the grade of your submission itself; examples include the strength of the batch your submission was in, as well as the strength of your grading of your peers (and/or a Gold Standard of ranks and reviews). All of this combined helps determine your grade for the assignment.
For your convenience, Wharton Computing has provided a simple walk-through video for most standard WHOOPPEE assignments. You can view this video here (best viewed in full screen mode for text readability purposes).
Please note the following changes since this video was created:
- WHOOPPEE no longer requires you to use a tool (manual or automatic) to review and remove PII. The app does that for you.
- The Class Distribution page in the Results phase is no longer available; only My Paper and My Results are shown.
Here is an example of a student going through an assignment with WHOOPPEE.
- Once your submission has been uploaded through Canvas, log into the WHOOPPEE app through the link provided in the assignment.
Note: The file must be in PDF format. This allows for the PII check to run successfully.
- After logging in, select the course you're currently enrolled in to advance to the Confirm Submission phase:
- Once your submission is confirmed, you will see a confirmation page. In the interim, just sit back for the time being and wait for other students to finish these previous steps, and the Rank & Review phase will automatically advance.
Rank and Review Phase
Once the Rank and Review phase begins, the WHOOPPEE app will automatically display the Rank and Review section when you're logged in. This is where you will review the assignments in the Not Ranked section (the lower field) and then drag them to the Ranked section (the upper field) in the order of your preference.
- Click the three dots under the Actions column on the assignment you'd like to review. Click Review to begin the review process.
- You can also download the assignment for later viewing, or report a concern (for example, if you think you know the identity of the person that submitted this assignment).
- The download link will appear on the left, and when you're done reading, you can submit your comments on the right.
- When you're finished reviewing the assignment, click Submit Review and you'll be taken back to the landing page, with the assignment status changed to Reviewed. Continue until you have reviewed all assignments.
- When you've completed review of all the assignments, click and drag the six dots on the left side of the assignment to move the assignments into the Ranked section in the order you like. (You'll see the cursor turn from an arrow to a hand when you're on the right spot.)
- When you're satisfied with the order of your reviews, click Finalize My Rankings and Reviews in the top right corner.
In the respond phase, you will be able to read the reviews of your assignment, and mark down whether the comments were helpful, unhelpful, or neither.
- Read the reviews of your submission.
- Click the face that corresponds with your opinion (you'll see the color change).
- When you're done, click the I'm all set, now take me to the results! button below the final review box.
Finally, you'll land on the Results page, which allows you to see your rank in the class for the assignment. You will also see the class mean, median, and standard deviation of scores next to your rank.
This page is divided into two sections: My Paper and My Rankings.
- My Paper: This is a table that includes all of the ranks that your paper received. This is broken down by each student's class rank as a ranker, the strength of the batch your paper was in, and your paper's rank within that batch.
- My Rankings: This is a table that includes all of the papers you reviewed and ranked. This is broken down by each paper's overall paper rank within the class, the strength of the batch of papers it was in, and the rank you gave it.
And that's it! Any further questions about your results and how they were calculated can be directed to your teaching team.
Because of the way web browsers interact with web applications, sometimes data can become "stuck" or otherwise not function as expected. This happens sometimes in WHOOPPEE, as well.
The simplest way to resolve most technical issues is to perform the following steps:
- Refresh your web browser.
- Clear your browser's cache and/or cookies.
- Use a different web browser, since different web browsers can render pages in slightly different ways.
- Reboot your device, or use a different one. WHOOPPEE should cache your progress in the cloud, so you should be able to pick up where you left off.
All app-related questions can be directed to your teaching team. Please contact Wharton Computing if you notice any technical issues not resolved by performing the steps in the previous section.