Create a UNSW Workshop Activity in Moodle

Create a UNSW Workshop activity

Creating a UNSW Workshop activity is a multi-part process, consisting of a workshop setup phase, then an assignment submission phase:

Setup phase

• A. Set the workshop introduction, and provide submission and assessment instructions
• B. Edit assessment form
• C. Prepare and assess example submissions

Submission phase

• D. Switch to Submission phase and accept submissions
• E. Allocate submissions

Assessment phase

• F. Assess submissions
• G. Set grading evaluation settings

Other instructions you might find useful are:

• Protect student anonymity during peer review—set up blind marking
• Organise for students to review topics different from their own
• Allowing students to resubmit
• Set up and use Team Evaluation with a group based UNSW Workshop activity
• Flag reviews as unfair

A. Set the workshop introduction

1. On the Moodle course home page, on the top right corner, click Turn editing on . In the relevant section, click Add an activity or resource, scroll down and select Workshop (UNSW) from the pop-up activity chooser.
   
2. On the Adding a new Workshop (UNSW) page, click Expand all at the upper right, then in the General section enter a Workshop (UNSW) name, and a Description if necessary. This is a useful place to tell students how the workshop process will work—for example, "The workshop is divided into different phases..."
3. Under Grading settings, select:
   • the Grading strategy. For example, use Comments for a non-assessable, process-oriented item (click Help for further information)
   • the Grade for submission—the maximum grade a student can earn for submitting the assessment, if applicable (and a grading category, if applicable)
   • the Grade for assessment—a maximum grade for assessment of the submission, if applicable (and a grading category, if applicable)
   • the number of Decimal places to which you want the grades to display.

4. On the Adding a new Workshop (UNSW) page, under Submission settings:
   • Enter Instructions for submission.
   • Select the Maximum number of submission attachments.
   • Select the Maximum submission attachment size.
   • Select the Late submissions check box. (Note that late submissions can become complicated if your workshop activity has several phases.)
   • Select Team mode to allow students to submit work as a group rather than as individuals.
5. Under Assessment settings:
   • Enter Instructions for assessment. This is a useful place to provide information about the peer review process and give students some general guidelines and tips for providing constructive feedback. For help in wording these guidelines, this page on student peer assessment may be useful.
   • Select Use self-assessment if you want to allow students to self-assess

6. Under Feedback:
   • Select your preferred form of Overall feedback mode—or disable this mode if you wish.
   • If you are allowing reviewers to attach feedback files, set a Maximum number of overall feedback attachments, and a maximum size for them.
   • Provide Conclusion text—this displays to participants at the end of the Workshop activity.
7. Will you be giving the students sample submissions to assess (e.g. of poor / fair / good / excellent submissions)?
   • If no, continue to the next step.
   • If yes:
     • Under Example submissions select the Use examples check box.

• Select a Mode of examples assessment. If you are using self-assessment, you have a choice whether to have students assess their own submission before or after the examples. It's useful to specify that Examples must be assessed before own submission to get students thinking about what constitutes a good submission before they consider their own work.
• Select the Number of examples you want your students to be presented with. Set this if you want to provide a greater number of examples than each student will see. Moodle will present to each student a roughly even spread of poor to good submissions. This is useful to prevent cheating when using the calibrated evaluation method. We advise you to select at least 3 into this field if you are using calibration. If you leave this set to All, all the example submissions you provide will be shown to your students.
• Select whether you wish to allow Example comparison and/or Example reassessment as desired.

8. Under Availability:
   • Select dates and times to open and close the submission period.
   • We advise you not to select to automatically Switch to the next phase after the submissions deadline, as the scheduled allocation function is currently disabled.
   • Set the assessment period dates and times. This determines when the assessment phase will automatically close off—however, you must still manually move on to the next phase.
9. Complete the remainder of the page as relevant and click Save and display.
10. In the workshop page, under Setup phase, click Edit assessment form.

B. Edit assessment form

11. The assessment form that displays will depend on the selection you made at Grading strategy in step 4 above:

On the Accumulative grading page:

1. For each aspect:
   • Enter a Description, including guidelines for commenting and questions reviewers should consider before they comment.
   • Select the Best possible grade/Scale to use for the aspect, from the drop-down list.
   • In the Weight field, select a number between 0 and 16 to indicate how important this aspect is in relation to other aspects.
2. Click Blanks for 2 more aspects to enter additional aspects and repeat step a for each aspect. Regularly click Save and continue editing if the aspects list is lengthy.
3. When all aspects are created, click Save and preview or Save and close.

On the Comments page:

1. For each aspect, enter a Description, including guidelines for commenting and questions reviewers should consider before they comment.
2. Repeat step a for Aspect 2 and Aspect 3 as necessary.
3. Click Blanks for 2 more aspects to enter additional aspects and repeat step a for each aspect. Regularly click Save and continue editing if the aspects list is lengthy.
4. When all aspects are created, click Save and preview or Save and close.

On the Number of errors page, for each assertion:

1. Enter the assertion in the Description box.
2. Change the Word for the error and the Word for the success, if you wish.
3. In the Weight field, select a number between 0 and 16 to indicate how important this aspect is in relation to other aspects.
4. Repeat steps a to c for Assertion 2 and Assertion 3 as necessary.
5. Click Blanks for 2 more assertions to enter additional assertions and repeat steps a to b for each assertion. Regularly click Save and continue editing if the assertions list is lengthy.
6. When you have created and weighted all the assertions, click Save and continue editing. The updated Grade mapping table will display at the bottom of the page.
7. Make any necessary changes to the grade mapping table and click Save and preview or Save and close.

On the Rubric page, for each criterion:

1. Enter the criterion in the Description box.
2. Enter the Level grade and definition for each level of achievement within the criterion, using descriptions, guidelines and/or questions for the reviewers.
3. Repeat steps a to b for Criterion 2 and Criterion 3 as necessary.
4. Click Blanks for 2 more criteria to enter additional criteria and repeat steps a to b for each criterion. Click Save and continue editing after entering each criterion.
5. Under Rubric configuration, select whether you want the rubric to display in List or Grid (table) format.
6. Click Save and preview.
7. On the Assessment form page, check the rubric and click Back to editing form.
8. On the Rubric page, make any necessary changes, then click Save and preview or Save and close.

C. Prepare and assess example submissions

If you are including example submissions:

12. On the workshop page, under Example submissions below the table, select Add example submission.
    
    
     
13. On the Submission page:
    • Enter a Title for the example. Include the word "example" or "exemplar" in the title to avoid confusing students.
    • Enter instructions, or the example Submission content, into the text editor.
    • Add an attachment if required.
    • Click Save changes.
      
      

14. On the Example submission page, click Continue (or Assess example submission if you are providing reference assessments).
    
    
     
15. On the Assessed example submission page, provide an assessment of the sample submission for your students to refer to in their peer assessment or self-assessment, and click Save and close.
16. To add additional examples, repeat steps 12 to 15.

D. Switch to Submission phase and accept submissions

Students can submit their work when the Workshop is in Submission phase and the opening date for submission is reached. To switch to the Submission phase:

17. On the course home page, click the workshop's link.
18. On the workshop page, under Submission phase, click the Switch phase icon .
19. On the confirmation page, click Continue. 
20. The workshop page displays again, with the Submission phase highlighted. Now students may submit their work (within the submission access control dates), and instructors can allocate submissions for peer assessment.

E. Allocate submissions

Complete this part when students have begun submitting their assessments.

You can allocate submissions manually, or allow Moodle to allocate them randomly. For a useful description of how random allocation works, view the PDF in the first post to this moodle.org discussion board.

21. On the course home page, click the workshop's link, and on the workshop page, click Allocate submissions. The Submission allocation > Manual allocation page displays.

22. Do you want to manually or randomly allocate submissions to reviewers? If you have a large class, random allocation makes sense. You can randomly allocate, then adjust the allocation using manual allocation if you need to. Whichever method you use, it is good practice to have at least 3 people reviewing each submitted piece.
    • If allocating manually, continue to the next step.
    • If allocating randomly, click the Random allocation tab and go to step 23.
23. On the Manual allocation page:
    • In the Participant is reviewed by column, select the name(s) of the user(s) you want to review this student’s work.
    • In the Participant is reviewer of column, select the name(s) of the user(s) whose submitted work you want this student to review. (You can also upload a CSV file containing your allocations. See screen instructions for explanation and format.)
    • As you enter your allocations, they will be saved automatically. If you need to delete an allocation, click the Delete icon . A confirmation message will display; click Yes, I am sure.
    • Go to step 24.
24. On the Random allocation page, select:
    • any relevant Group mode
    • the desired Number of reviews reviewers are to perform
    • whether this number is per submission or per reviewer. This choice will depend on your teaching strategy.
    • whether to Remove current allocations
    • whether students can assess their peers’ submissions without having submitted work themselves
    • whether to include self-assessments in the Number of reviews, if applicable.
25. Click Save changes. A confirmation page will display, summarising the allocations and deallocations you have made. Click Continue. The Submission allocation page will re-display.

F. Assess submissions

Complete this part after the submissions deadline.

On the course home page, click the workshop's link, and on the workshop page, under Assessment phase click the Switch phase icon . The submissions that have been assigned for you to assess are listed under Assigned submissions to assess.

G. Set grading evaluation settings

Complete this part when students have completed submitting their assessments.

26. Open the workshop page. If you have chosen automatic phase-switching, the Grading evaluation phase section will be highlighted. (If you have opted for manual phase-switching, under Grading evaluation phase click the Switch phase icon .)
27. Choose your preferred grading evaluation method (Comparison with best assessment or Calibrated against example assessments).  When using calibrated grading:
    • Comparison against example assessments—This setting specifies how accurate the reviewer must be when marking the example assignments. The strictness scale specifies how strict the comparison should be. The stricter the comparison, the closer the marker's assessment should be to the provided example assessment.
    • Consistency of assessment accuracy—This setting specifies how consistent the reviewer must be. A stricter consistency means that all the marker's assessments must more closely match the example assessments.
28. Click Re-calculate grades.

Protect student anonymity during peer review—set up blind marking

You can set up single blind or double blind marking in the Workshop (UNSW) activity’s Permissions page, using the permissions View author names and View reviewer names. By default, students have permission to view author however not reviewer names. To create blind marking, you will remove these permissions from students.

Single blind marking

You can set up single blind marking in two ways:

• View author names controls whether a student reviewer can see the names of the students whose work she or he is to review. If you remove this permission only, a student reviewing her peers' work will not be able to see the names of students whose submissions she reviews, either during the Assessment phase or once the activity is Closed. However, once the activity is Closed, the names of the reviewers of her own submission will display with her final grade.
• View reviewer names controls whether a student can see who reviewed her or his own submission. If you remove this permission only, the names of students whose work a student is marking will display to her, but when she views her final grades, the names of students who reviewed her own submission will not be visible.

Double blind marking

If you remove both View author names and View reviewer names permissions, once the workshop is Closed neither the names of the students whose work she reviewed nor the names of the students who reviewed her own work will display to a student reviewer.

To set up blind marking:

1. On the Moodle course home page, click the link for the Workshop (UNSW) activity.
   
   
    
2. In the Settings block, select Workshop administration > Permissions.
   
   
    
3. On the Permissions page, scroll down to the View author names or the View reviewer names permission. (See the notes above for what these permissions control.)
   
   
    
4. For the relevant permission, in the Roles with permission column, next to Student, click the Prevent icon .
   
   
    
5. On the confirmation page, check that you have selected the correct permission to prevent and click Continue. The Permissions page re-displays, and Student no longer displays in the Roles with permission list.

What students will see in the Grade page:

If you prevent View author names permission only, when the workshop has closed the student will see who reviewed their own work (in the Grades received column), but not whose work they themselves reviewed (under Assigned submissions to assess).

If you prevent View reviewer names permission only, when the workshop has closed the student will not see who reviewed their own work (in the Grades received column), but they will see whose work they themselves reviewed (under Assigned submissions to assess).

If you prevent both View author names and View reviewer names permission, when the workshop has closed the student will see neither who reviewed their own work (in the Grades received column) nor whose work they themselves reviewed (under Assigned submissions to assess).

To restore permission to a student:

1. On the Permissions page, in the Roles with permission column for the relevant permission, click the Allow icon .  (Take care not to click the Prohibit icon in the next column.)
   
   
    
2. On the Allow role page, at the Select role drop-down, select Student and click Allow. The Permissions page will re-display, and Student will display in the Roles with permission list.
   
   
    

Organise for students to review topics different from their own

So you want to make sure that students review topics or questions that are different from the topic or question they've completed themselves?

You can do this in both the standard and the UNSW Workshop tool, by following the steps below:

1. Before you add the Workshop in Moodle, create a grouping, and within it create different groups for each different question or topic you will assign. 
2. At part A, step 9, on the Adding a new Workshop page, under Common module settings:
   • at Group mode select Visible groups
   • at Grouping select the appropriate grouping, and
   • select the check box Available for group members only.
     
     
3. At part E, step 23, on the Random allocation page, select the Prevent reviews by peers from the same groups check box.

Allowing students to re-submit

Once the submission date has been reached (if set) and the phase progressed, students will not be able to re-submit or edit their previous submission. To allow one or more students to do this, you will need to:

1. Switch to the Submission Phase
2. If enabled and restricting submission, extend the submission deadline
   1. On the left hand side, within the Settings block, click 'Workshop (UNSW) administration' and then 'Edit Settings'
      
      
       
   2. Under the Availability heading, extend or disable the 'Submissions deadline', and then scroll down and click 'Save and display'
      
      

Flagging reviews as unfair

During the closed phase of the UNSW workshop the 'Allow submitters to flag reviews as unfair' checkbox can be checked to allow students to potentially unfair flag reviews for staff to check.

Instructors will see flagged reviews with a yellow highlight within the workshop activity and can review them by:

1. Select the “Review assessments that users have flagged as unfair’ button
2. Select ‘This assessment is fair’ or 'This assessment is unfair and should be discounted' for each flagged review
3. Click 'Re-calculate grades' at the bottom of the page

This will set unfair reviews to have a weighting of 0 and take it out of the calculations for the student's submission mark. Marks will be updated once the 'Re-calculate grades' button is pressed.