WHOOPPEE (For Students)

In this article, you'll learn the basics of using the WHOOPPEE app. 

Minimum Requirements 

You will need the following before you can use WHOOPPEE: 

  • A laptop or desktop PC
    • Currently tablets and phones are not supported.
  • A modern web browser
    • Most browsers are supported; if Canvas works in your browser, WHOOPPEE should as well.
      • Firefox tends to function best of all browsers tested. Safari for Mac may exhibit some slightly resolution/scaling issues.
  • Software to generate PDFs for Canvas submission
    • Most word processors and PDF creators such as Adobe Acrobat should all have PDF creation functionality. Usually this can be found under something like File > Print to PDF.

What is WHOOPPEE?

WHOOPPEE is the Wharton Online Ordinal Peer Performance Evaluation Engine which allows students to anonymously submit, rank, and review assignments from their peers. The app operates through 4 phases: Confirm Submission, Rank & Review, Respond, and Results. Its basic workflow is:

  1. Submit an assignment through Canvas. This assignment must be stripped of all personally identifiable information (hereafter referred to as "PII") and completely anonymous.
  2. WHOOPPEE automatically checks to see if step 1 has been completed anonymously.
  3. WHOOPPEE then distributes back random batches of other submissions from your peers back to you.
  4. Once you have the batch of submissions, you can review them, comment on them, and rank them in order of strength.
  5. After ranking and reviewing, you will then be able to view your peers' comments on your own submission, and decide if the comments are helpful or unhelpful.
  6. The results of the assignment (including your grade) are displayed.
    • WHOOPPEE also provides some helpful context in addition to the grade of your submission itself; examples include the strength of the batch your submission was in, as well as the strength of your grading of your peers (and/or a Gold Standard of ranks and reviews). All of this combined helps determine your grade for the assignment.

Step-by-Step Guide

Here is an example of a student going through an assignment with WHOOPPEE.

Submission Phase

  1. Once your submission has been uploaded through Canvas, log into the WHOOPPEE app through the link provided in the assignment.
    • Note: The file must be in PDF format. This allows for the PII check to run successfully. 
  2. After logging in, select the course you're currently enrolled in to advance to the Confirm Submission phase:Test course example
    When you see your relevant course, click it to advance.
  3. If your submission is determined to be completely anonymized, you will see this screen that allows you to confirm your upload or go back to Canvas to resubmit if you wish. (You can skip step 4 and just click Confirm Upload to advance to the next phase):Submission confirmation pageThis is the confirmation page. You can also refresh by clicking the Paper not showing? Click here button.Otherwise, if your submission is deemed to have information that could identify you, you'll be notified on this screen. You (as well as the teaching team) will also receive an e-mail.
    • You can attempt to remove the data automatically and resubmit, or click the link to view instructions on how to remove metadata manually using Adobe Acrobat.
      • If you're using Microsoft Word, there are instructions for removing the metadata below.
        • Word for Windows:
          1. Click on the File menu and click Info from the left sidebar.
          2. Click Check for Issues and select Inspect Document.
          3. Click Inspect.
          4. Click Remove All in the section for Document Properties and Personal Information, then click Close.
          5. Save document again in PDF format.
        • Word for Mac:
          1. From the Review menu, select Protect Document.
          2. Ensure that "Remove personal information" is checked.
          3. Save document again in PDF format.
    • NOTE: When you return from WHOOPPEE after resubmitting, click "Paper Not Showing? Click Here" and the page should refresh with your newest submission.

    • If you're still getting this screen and know for a fact that you've anonymized your paper, reach out to your teaching team and they will investigate the issue.Submission confirmation page with PII errorIf the automatic PII removal method doesn't work, you can try to remove it manually. If you're still having trouble, please contact your teaching team.
  4. Once your submission is confirmed, you will see a confirmation page. In the interim, just sit back for the time being and wait for other students to finish these previous steps, and the Rank & Review phase will automatically advance.Submission Acceptance pageThis page indicates your submission was accepted. (In this example, we've redacted the title.)

Rank and Review Phase

Once the Rank and Review phase begins the WHOOPPEE app will automatically display the Rank and Review section when you're logged in.  This is where you will review the assignments in the Not Ranked section (the lower field) and then drag them to the Ranked section (the upper field) in the order of your preference.

Your page should automatically advance to this Rank & Review page once all submissions have been accepted.

  1. Click the three dots under the Actions column on the assignment you'd like to review. Click Review to begin the review process.
    • You can also download the assignment for later viewing, or report a concern (for example, if you think you know the identity of the person that submitted this assignment).Example of the action menu represented by three dots, with review being selected.Click the action menu (represented by the three dots) on the right next to the assignment, and then click "Review" to start reviewing it.
  2. The download link will appear on the left, and when you're done reading, you can submit your comments on the right.
    • A sample of the review page, with the download link on the left and submit review button on the bottom right.Click the Download Link button on the left to download the submission. When you're finished reading, leave your review and click Submit Review.
  3. When you're finished reviewing the assignment, click Submit Review and you'll be taken back to the landing page, with the assignment status changed to Reviewed.  Continue until you have reviewed all assignments.Example of a completed review, with the status updated accordingly.
  4. When you've completed review of all the assignments, click and drag the six dots on the left side of the assignment to move the assignments into the Ranked section in the order you like. (You'll see the cursor turn from an arrow to a hand when you're on the right spot.)Example of dragging assignments from unranked section to ranked section, and then ordering them in order of preference.Drag the assignments from the Ranked section to the Unranked section, and then order them from top rank to bottom rank.
  5. When you're satisfied with the order of your reviews, click Finalize My Rankings and Reviews in the top right corner.

Respond Phase

In the respond phase, you will be able to read the reviews of your assignment, and mark down whether the comments were helpful, unhelpful, or neither.

  1. Read the reviews of your submission.
  2. Click the face that corresponds with your opinion (you'll see the color change).Here's an example of some of the reviews from an assignment a student submitted. You can choose individually if the reviews of your work were helpful, unhelpful, or neither.
  3. When you're done, click the I'm all set, now take me to the results! button below the final review box.

Results Phase

Finally, you'll land on the Results page, which allows you to see your score. This page is divided into three sections: Class Distribution, My Paper, and My Rankings

  • Class Distribution: This is your score visualized in a class grade distribution graph as shown below (you can view it in either a histogram or slope graph):Final results, displayed as a slope graph.This is the Results page, which displays your final score and the score of your peers.Final results, displayed as a histogram.This is the same data as above, but displayed as a histogram by clicking the Viewing button in the top right corner.
  • My Paper: This is a table that includes all of the ranks that your paper received. This is broken down by each student's class rank as a ranker, the strength of the batch your paper was in, and your paper's rank within that batch.A table indicating the submissions' ranking in batches and batch scores.The second tab, "My Paper," will display attributes about the students that graded your submission, as well as the strength of their batches and where your paper ranked in the batches.
  • My Rankings: This is a table that includes all of the papers you reviewed and ranked. This is broken down by each paper's overall paper rank within the class, the strength of the batch of papers it was in, and the rank you gave it.A table indicating the overall paper ranks for the submissions the student reviewed.The My Rankings section contains the papers you ranked, their class rank, and the strength of the batch they were in.

And that's it! Any further questions about your results and how they were calculated can be directed to your teaching team.

Questions?

All app-related questions can be directed to your teaching team. Please contact Wharton Computing if you notice a technical issue.