Part of the main critique concerns the issues of validity and reliability of the method, which are seen as the difficult problems for practice to handle. The validity issue concerns the usability and correctness of the facilitated perspectives; the reliability issue whether we can expect the learning process to take place at all.
Validity Issues
The main concern of validity is about the orientation towards teleos; can role-play be used to facilitate knowledge and perspectives of relevance outside the learning practice, and can this benefit become available outside the game context?
Learning how to manipulate one’s way around a problem, or simply moving the problem by basing the interaction on a cardboard gun might not be the most appropriate benefit. It might be valid entertainment in the game, but not in the real world. Though problematic, due to the use of the simplifying simulation, the role-play must be able to develop perspectives valid in the outside world. If this is not achieved, the role played by the medium is reduced to mere entertainment.
This also stresses the need for the actual presence of an educational payload, of something to learn inside the role-play (such as perspectives on business economics), and the acknowledgement of the fact that just because something is present in a practice, the participant does not necessarily learn any of it. Seeing role-play from the perspective of the interpretative model, limitations are set on what topics can be effectively learned through role-play. Due to the interpretative element, the method seems awful for teaching hard knowledge (information) to the participant (Henriksen 2003), but relevant in presenting complex problems, such as social or humanistic problems.
The other part of the validity issue is the cross-contextual use of the benefit. As previously mentioned, the benefit of the inductively-based learning process must be decontextualised, in order to make it transferable and consciously applicable to other contexts. As the non-decontextualised perspectives only have limited effect in other contexts, actions must be taken in order to decontextualise the benefit.
An important tool for both qualifying the use, and facilitating the cross-contextual use of the facilitated perspectives, is by using the role-play as a supplementary in a 1-2-1 model, beginning with a theoretical exploration, followed by the practical experiences facilitated through role-play, and then returning to the theoretical exploration of the points and shared experience of the role-play. By giving the participant theoretical knowledge about the problem and the problem context prior to the role-play, the participation is qualified. By returning to the classroom in order to address the points of the role-play afterwards, an important step towards decontextualisation is taken. By addressing the points from a theoretical perspective, cross-contextual use is facilitated;
by addressing them in (inductive) practice, a thorough understanding of the perspective applied to a specific object is facilitated. Such use of a combination of inductive and deductive learning processes has proved successful within the Danish health sector (see Akre & Ludvigsen 1999). In spite of the use of the 1-2-1 model, as education cannot build on relations alone, firm points are needed as well, making the validity issues important to consider in the game design.
Reliability Issues
The main concern about reliability is the ability to attain a developmental goal; in order
system requires the availability of stable methods, enabling the teacher to plan and meet criteria for the learning process. The use of role-play must accept the evaluation criteria for this system, in order to become a supplementary part of it.
From the point of view of the critical issues of the circle model, role-play generally shows very little reliability, due to interactive and interpretative perspectives, allowing the participant to enact only the perspectives relevant to him. This issue can to a certain degree be countered by the use of a good game design and game mechanics, which unfortunately puts the validity at risk. It can be argued that this is the case for almost any learning process, which deprives the teacher of the ability to directly and consistently dictate the perspective of the participant (Davydov 1989), but without undermining the need for addressing the issue of reliability. This problem arises both as a part of the role-creating process, in the interpretation of the object, and during the interaction between participants, as this alters the course of the action. By increasing the degree of linearity at the expense of the degree of freedom, the reliability of the method is improved, pointing out a need for balancing the two.
Another critical part of the issue is the use of conflict in order to construct action in the role-play. By placing the participants in conflicting perspectives, different stimuli are offered to the different perspectives, creating grounds for different experiences and different benefits. This presents a problem within the established educational system, as the sheer number of students forces the system to focus collectively, especially during primary education.
The issue of different perspectives can in some respects be turned into an advantage to use, as it holds a potential for a subsequent shared reflection on different views on a case or object, thereby facilitating a reflective stance to the difference of perspectives. On further reflection on balancing the degree of linearity vs. the degrees of freedom, see Montola (2004).
Other Problems to be Addressed
The validity and reliability issues both represent significant problems in using role-play in order to facilitate a specific benefit for the participant, but there are other problems associated with use as well.
The general attitude in Denmark towards role-play is clouded by certain unfortunate events, which occurred between 1960 and 1980. Long before role-play became a means of entertainment it was a widely used educational tool, especially within the popular T-training and sensitivity training (see Høyrup 1975). These programmes were designed in order to evolve the participant, especially towards a higher level of social, cooperative, motivational, attentional and emotional behaviour.
At the beginning of the 1960s the practice was well founded theoretically, but as the popularity of its use increased, the practice outran the theoretical exploration, leaving the practice undermined in the 1970s. The popularity had created a large demand for the practice, encouraging a very diversely skilled group to undertake it. Slowly the practice deteriorated, modularising itself from the theoretical foundation. Though Høyrup (in
1972 and 1975) expressed the need for the licensing of practitioners, the warning was ignored until 1980, when a group of sceptical journalists forced the balloon to collapse (Haslebo & Nielsen 1998). The market died, and the acknowledgement of role-play as an educational tool died with it.
This widespread understanding and attitude towards the educational use of role-play causes the work of getting the practice acknowledged as a serious educational tool to be an uphill effort. The project is somewhat hindered, as part of the current Danish practice refuses to consider the theoretical foundation in order to quality their practice, thereby reproducing the conditions for the unfortunate events of the 1960s to 70s.
One major obstacle to the availability of educational role-play the historic events, another is the difference of interests in the projects initiated to design such material.
There is an increased interest in initiating design projects, but unfortunately only a few of them seem to survive beyond the first customer, often failing to publish any material useful to others. Observing such initiatives during the last five years, a major problem seems to be a lack of project co-ordination. This conclusion is based on the tendency of such initiatives to die out after using up all the resources of the project group on the initial project, leaving the group without the strength to go any further.
Another tendency is the lack of clarity between project partners about the goal of the project; a commonly seen example of this is the widespread joint-venture between a social or educational institution and a group of role-players. While the institution merely seeks something that can serve as a break from their regular activities, and which doesn’t reach further than the initiating project, the role-player often sees this as an opportunity for both career and big business, leaving the partners with quite incompatible goals.
Some attention to the diversity of goals might save practice some trouble, and thereby free resources for a reflective attitude towards the use of role-play.
Though including non-role-players in the design process is sometimes problematic, the validity of the role-play requires the design team to include both people skilled in role-play and people educated in the craft the teleos is orientated towards.