Assessing with Role Play and Simulation

Overview

Role play and simulations are forms of experiential learning (Russell & Shepherd, 2010). Learners take on different roles, assuming a profile of a character or personality, and interact and participate in diverse and complex learning settings.

The terms "role play" and "simulation" are sometimes used inconsistently or interchangeably. However, "simulations" often involve a familiar or realistic situation in which a participant’s role may not be as prominent or distinctive as it would be in a role play. Frequently simulations incorporate role play, leading to the term "role-playing simulation". The difference is generally one of degree rather than kind.

Role plays and simulations function as learning tools for teams and groups or individuals as they "play" online or face to face. They alter the power ratios in teaching and learning relationships between students and educators, as students learn through their explorations and the viewpoints of the character or personality they are articulating in the environment. This student-centered space can enable learner-oriented assessment, where the design of the task is created for active student learning. Students are actively involved in both self and peer assessment and obtain sustainable formative feedback.

Video Series - Using Role Plays in Formative Assessment - Ben Barry & Gail Trapp

When to use

Good-quality learning design provides opportunities for situated and authentic learning. As Figure 1 indicates, high quality learning is situated in a real-life context and simulates the learning activities of the graduate's employment area.

Figure 1: eLearning opportunities to address principles of high-quality learning design (Wills, Leigh & Ip, 2011).

Boud & Prosser: Summary of Principles
Engage learners	Consider learners' prior knowledge and desires and build on their expectations.
Acknowledge learning context	Consider how the implementation of the learning design is positioned within the broader program of study for the learner.
Challenge learners	Seek the active participation of learners, encouraging them to be self-critical.
Provide practice	Encourage learners to articulate and demonstrate to themselves and their peers what they are learning.
Siemens & Tittenberger: Summary of Principles
Social	Learning is a social process and knowledge is an emergent property of interactions between networks of learners.
Situated	Learning occurs within particular situations or contexts, raising the importance of educational activities mirroring actual situations of use.
Reflective	Learners require time to assimilate new information.
Multi-faceted	Learning incorporates a range of theory, engagement, "tinkering" or bricolage, and active construction.

Authentic learning and task design provides students with:

opportunities to reflect on the way knowledge will be gained in real life
activities that are authentic in nature
modelling of expert performances and processes
the opportunity to learn about new perspectives and roles in life
opportunities to reflect on learning.
opportunities to see how tacit knowledge can be made explicit
scaffolding and coaching at critical times in the learning and assessment process
assessment that is aligned with learning objectives within the task.

Role plays and simulations significantly contribute to students' learning and assessment when they allow students to view multiple perspectives on their responses in a safe but challenging environment.

Benefits

Widespread evidence suggests that educators and students experience satisfaction with assessment-as-learning through role play, games and simulation (Russell & Shepherd, 2010). Simulated learning environments (SLEs) provide a safe, supportive environment where students can develop their clinical skills, competency and agency.

SLEs are also flexible and controllable, and educators find they can design suitable and varied education events within them.

The blended learning environment can provide face-to-face students with a virtual classroom where students and educators can deliver content and interact in a simulated learning environment. This benefits both staff and students; it has, however, meant a shift in practice for the educator as the room changes to accommodate the required infrastructure and seating arrangements.

The benefits listed below are attributed to either role plays or simulations; however, some benefits can be attributable to both forms to some extent.

Benefits of assessing by role play

Role play is an excellent means of evaluating decision-making and interpersonal communication skills.
Role play is particularly useful to students who will operate in a tense professional environment (e.g. diplomacy, acute or sensitive medical care settings, psychology and counselling) or requires complex decision-making.
Scenarios can be scaffolded, gradually increasing in complexity to ensure that students reach a sufficient level of competence.
Role plays help you evaluate students' ability to work under pressure and with others, including providing opportunities for inter-professional learning.
With online SLEs, students can role play anonymously.

Benefits of assessing by simulation

Simulation is a form of authentic assessment. When exposed to active, experiential, reflective and contextual learning approaches such as simulated environments, students can see the direct relevance of their educational experience to their future practice.
Educators can assess a student's preparedness for the practical placement component of their degree.
Technology-based forms of simulation can enable instant feedback to students.
Simulations are effective means of evaluating students' competencies, such as their professionalism, as well as their content knowledge.
In the medical and allied health literature, SLEs are consistently found to have 3 significant benefits. They:
- promote an increase in self-efficacy in clinical decision making,
- improve clinical communication skills, and
- promote greater awareness in students of the role they play in a collaborative care setting.

Challenges

As with the benefits, the issues are listed here as characteristic of either role plays or simulations; however, some apply to both forms.

Challenges of assessing by role play

Role plays are resource intensive and both costs and available time will constrain them. Constraints can be reduced by developing a bank of role play scenarios and sharing role play resources. Many universities utilise shared training and evaluation centres in the fields of medicine and allied health.You can reduce the setup cost of role plays by utilising the university platform. You can also use many learning designs that are distributed online at no cost.
A new platform for learning, requiring students to learn new skills just to participate in the learning, can distract them from the conceptual learning the role play was intended to promote.
Institutional Learning Management Systems that require students to tick a box to ensure that each of their posts is anonymous can compromise student anonymity.

Challenges of assessing by simulation

For students who struggle with public speaking or group participation, simulation assessments can create so much anxiety that it affects their performance or participation.
It is impossible to genuinely recreate authenticity in a simulated environment. The most you can do is use different aspects of simulation to cater to the assessment needs of the students.
Students need to be guided throughout a simulation, and learning must be scaffolded.
It is advisable to flag the timing of simulations with your colleagues, as preparing for a simulation can prevent students from completing other learning and assessment work.
Setup costs can be significant.
Staff and students' accessibility can present challenges during simulation setup.
Facilitators sometimes need to invest significant time learning the tools required to develop a simulation, to track and structure activity and to monitor and communicate with students during simulations. This investment can only benefit their teaching, but expect time-pressed teachers to be resistant at first.

Strategies

When assessing role plays and simulations, we recommend that you:

align the task with the learning outcomes and structure it accordingly
provide clear and explicit information as to what is expected of students
ensure that the task is authentic and real-world based
scaffold the learning experience, breaking tasks down to manageable size
use both formative feedback and summative assessment.

Once debriefing sessions have been held, evaluate the learning design. Gredler (1996) suggests using a 3-step evaluative procedure to redesign a role play or simulation:

Document the design validity of the innovation.
Verify the cognitive strategy and the social interactions using formative feedback and redesign them where needed.
Conduct follow-up evaluation and research on specific processes and effects of the learning and assessment.

Role play

Online role play is described by Project EnROLE as having the following characteristics (Wills, Leign & Ip, 2009):

It is designed to increase understanding of real-life human interactions and dynamics.
Participants assume someone else's role or place themselves in someone else's situation.
Participants undertake authentic tasks in an authentic context.
The task involves substantial in-role interaction with other roles for collaboration, negotiation and debate.
Interaction between roles takes place substantially online.
Learning outcomes are assessable and generate opportunities for student reflection.

These characteristics can also aid in setting up the face-to-face learning environment of a role play or simulation.

As Figure 2 demonstrates, adaptive learning in role plays includes modelling and input from students that can alter the learning outcomes. These disciplines can utilise this type of active and adaptive learning and can film it for evaluation (including peer- and self-evaluation). Actors can be used to perform the role of a patient or client, so that students' communication and clinical decision-making skills can be explored. Actors are usually trained in the details of a case, in the array of issues and behaviours a patient or client is likely to present to the health professional, and to replicate the performance from student to student to ensure standardisation of assessment . Studies have shown the level of standardisation achieved is usually very high.

In the following table, Siemens and Tittenberger (2009) outline the ways in which you can use role play with other experiential learning. They enumerate the opportunities that role-based eLearning provides for high-quality learning design and the tools that can be integrated into the learning experiences. The elements of design for authentic eLearning specified by Lombardi (2007) include basing a learning task on real-life problem solving, within a meaningful context. Online role plays embody Lombardi's suggestion that a learning task provide long-term student engagement with learning, involve a variety of resources and perspectives over a sustained period of time, and entail collaborations to promote engaging, open conversation.

Figure 2: Role play in comparison with other experiential learning activities (Wills, Leigh & Ip, 2011, modified from Siemens & Tittenberger, 2009).

Type of learning activity	What is it?	Media forms	Technique	Technologies	Tools
Assimilative	Processing narrative media—managing and structuring information	Lectures, DVDs or reading texts	Concept mapping, brainstorming, buzzwords, crosswords, defining, mind maps, web search	Word processing software, presentation software, text, image, audio, video	CMAP, Hot Potato, Google, MS Office products, social bookmarking, blogs, wikis, pageflakes, Google reader
Adaptive	An environment that changes according to learner input	Simulations, games	Modelling	Virtual worlds, models, simulations, games	Second Life, MMORPG
Communicative	Discussing	Asynchronous or synchronous discussions, chats, text messages	Reasoning, arguing, coaching, debate, discussion, negotiation, performance	Electronic whiteboards, email, discussion boards, chat, instant messaging, VOIP, videoconference, web conferencing, blogs, wikis	Online bulletin board, Skype, IM, Facebook, social bookmarking, blogs, wikis
Productive	Learners producing something	Creating, producing, writing, drawing, composing, synthesising, remixing, mashups	Artefact, book report, thesis, essay, exercise, journalling, literature review, multiple choice questions, puzzles, voting portfolio, product, test	Creative applications (image editing, CAD, design software) computer-aided assessment tools, electronic learning environments	Indesign, Photoshop, YouTube, Google Video, Office software, Sketch
Experiential	Interactive activities that focus on problem solving	Practising, applying, mimicking, experiencing, exploring, investigating, performing	Case study, experiment, laboratory, field trip, game, role playing, scavenger hunt	Virtual lab, 3D immersive environment	Google Earth, MMORPG, Second Life

Online role play/simulation

Important steps in the process:

Design the problem.
Design the rules and roles.
Set up the scenario.
Assign student roles.
Consider the practical limitations of playing out the simulation, and make adjustments to the task design as necessary.
Develop moderation and other necessary skills.
Assess the technological requirements and develop them as necessary.
Assist in developing students' skills in asynchronous posting online.
Develop students' understanding of the "story" that accumulates as they post.
Develop their skills in reflective practice; use a blog, journal or wiki space during the role play or simulation, and afterwards to assess students' participation and to evaluate the role play or simulation as a learning experience.

Games

You can use role play and simulation within a game, or use a game within a role play or simulation. Games are engaging, can be highly authentic and can incorporate a competitive element, up to and including advancement to the next stage or problem, or winning a prize at the conclusion of the game. Game feedback is generally immediate, reinforcing the student’s application of subject matter knowledge.

The success of using games in learning relies on the application of strict rules. In the academic setting they should meet 2 requirements (Gredler, 1996):

Random factors should not contribute to winning.
Winning should depend on the application of knowledge of the subject matter.

Games have 4 general purposes in learning and assessment, says Gredler:

for practice and refinement of skills
to aid in identifying gaps or weaknesses in knowledge
for review or evaluation
to learn new ways of investigating concepts and principles in the learning of problem-solving skills.

Crookall and Saunders (1989) view academic games as a simulation—a representation of an authentic real-world system that can itself take on some aspects of reality for participants or users. Games are useful tools as feedback responses for students; a key characteristic of game learning is that one cannot progress to the next stage of a game without gaining the knowledge to accomplish the requisite task. Groups as diverse as the American military and the National Association of Home Builders in the United States invest in games that represent and instruct their particular content and views (Squire, 2006). "Serious games" such as the US Army's America's Army are designed to impart their content by immersion of the players in game-playing activities.

Simulations

Simulations have the "potential to develop students' mental modes of complex situations as well as their problem solving strategies" (Gredler, 1996). You can use experiential simulations in a number of strategic ways for groups or individual students, and assess them using various techniques. Some examples of simulations are:

data management—often team-oriented and containing variables that are manipulated
diagnostic and crisis management—cause and effect contingencies are drawn from real cases; experts aid in the working through of the task.
social-process—These simulations require the learners to personally interact with the situations, and can have unexpected outcomes.

Experiential learning that focuses on the interactive activities of problem solving fits with Kolb’s experiential learning cycle (Kolb (1984). Kolb differentiates learners according to which feature of the experiential learning cycle they prefer: concrete experience, active experimentation, reflective observation or abstract conceptualisation. Kolb (1984) developed this concept of the learning process to "ensure that teaching and tutoring activities give full value to each stage of the process. This may mean that for the tutor or mentor, a major task is to 'chase' the learner round the cycle, asking questions which encourage reflection, conceptualization, and ways of testing the ideas" (Atherton, 2010).

Assessing simulations

In the course outline, give students adequate warning of the workload requirement for the simulation. This gives them a chance to opt out if their current load is already heavy.
Arm students with the required content.
Have them work on a position paper and an objective sheet from the initial stages.
Conduct surveys before and after the simulation, and implement a debrief questionnaire.
Dedicate the final class to debriefing the students about the process and evaluating the learning within the simulated environment.

Virtual reality and other online learning tools

Virtual reality and other online tools can play an important role in both online teaching and simulated environments. They provide what Russell and Shepherd (2010) referred to as optimal elements of learning design, a complex social learning space and reflective practice.

Educators commonly combine these simulated environments for assessment. For example, combining a part-task trainer within a role play scenario, or a DVD can create a high-fidelity simulation environment.

Low-fidelity SLEs such as case studies and role-plays are being overlooked "despite an established base of research to support their effectiveness". Published research does indicate that higher fidelity is better; however, the effectiveness of any simulation technology depends on how it is used (Beaubien & Baker, 2004, 55).

High- and medium-fidelity mannequins

Mannequins are commonly used by medical and allied health disciplines to assess clinical competency, such as self-confidence, clinical judgment, interpersonal communication and inter-professional teamwork. More common to the medical and allied health disciplines are part-task trainers, which utilise anatomical models or computer software models that replicate a specific physical intervention; for instance, a spinal simulator to assess physiotherapy students' ability to perform passive oscillatory movements.

Case studies

Video series - Role Plays: Case Studies from UNSW

Additional information

External resources

Online simulations: Harvard Business for Educators

Acknowledgments

The contributions of staff who engaged with the preparation of this topic are gratefully acknowledged.