7.1 Introduction
This chapter will discuss this study’s findings, presented in the previous two chapters, integrating the results of this explanatory sequential mixed methods study of peer assessed medicines management [MM], conducted in the simulated practice learning environment (see figures 9 and 10). This study investigated the topic from two perspectives:
1. The reliability of the peer assessed medicines management objective structured clinical examination [PAMMO], as an assessment strategy
2. How peer students make judgements of safe medicines management This enabled the following research question to be answered
Do pe e r assessed m edicines management Objective S tructured C linical Exam inations (OSCE), conducted in a sim ulated practice environment, o ffe r a reliable assessm ent to o l fo r student nurse peers to make Judgements about pre-registration student nurses’ safety in m edicines m anagement?
Results from phase one of the study suggested that the PAMMO demonstrated relatively good inter-rater reliability across most assessment criteria and the final pass/ refer scores (see chapter 4). Higher levels of disagreement between two groups of pre-registration nurses were evident in the global scores of safe MM practice using the assessment criteria “safe/ unsafe/ borderline”. Phase two of the study used focus group interviews [FGIs’j to explore with a sample of students from phase one of the study how their judgements of the peer student’s MM safety, in the video vignettes were formed. The focus group interview findings indicated that students’ judgements and decisions of peers’ safe MM were founded upon the following factors:
• The students’ interpretations of the PAMMO assessment proforma.
• The students’ understanding of the standards and policies supporting safe practice.
• Students’ experiences of MM gained during practice placements.
• Students’ experiences of MM OSCEs undertaken in the simulated practice learning environment.
Findings from phase one and two will be presented as two inter-related theories. The findings will be framed within the interconnected theory of social constructivism (Burr, 2003; Pallincsar, 1998), and the cognitive continuum of decision and judgement making (Standing, 2010), which were initially presented in the literature review (see 2.52). To illustrate this, a conceptual map of the peer students judgement and decision making has been developed (see figure 21). This figure illustrates social constructivism (indicated by a red line) as the outer theory that inter-relates to the inner theory of the cognitive continuum of judgement and decision making represented by the blue dashed line. The interrelationship between the two theories is not uni-directional as the students’ applied both theories throughout their assessment of the peer student’s safe MM practice. The PAMMO students had socially constructed their individual views of safe MM prior to participating in the study, however during the study they adapted and changed their worldview of medicines management. By adapting their worldview the students were drawing upon their prior knowledge to form a deeper understanding of this new (for the PAMMO students) approach to their learning (Gillani, 2003). For the PAMMO students developing their understanding of their new learning and making judgements of safe MM practice included using the PAMMO proforma. In terms of Vygotsky’s (1978) socio-cultural theory their understanding was mediated by this assessment tool which helped them
to co-construct their knowledge of safe practice as well as internalising this to aid their knowledge development as an individual.
Learning from a social constructivists’ stance requires peer interaction, student ownership of the curriculum and authentic educational experiences (Azzarito, Ennis, 2003). This study utilises a peer approach to assessing the peer student’s safe medicine management practice using video vignettes. Similar to a social constructivist approach to learning the PAMMO students were informed of the process and rationale for the PAMMO assessment as part of the informed consent process, hence they had a degree of ownership. Additionally the peer students’ in the video vignettes demonstrated their MM in a simulated practice learning environment which was intended to reflect an authentic practice environment. From a social constructivist’s perspective learners are responsible for their own learning (Bay, Bagceci, Cetin, 2012). Having responsibility for ones own learning links to Knowles (1984) principles of andragogy, explored earlier in this thesis. As a consequence learners, or in this case the PAMMO students, were able to self-monitor and manage their own learning and practice (Bay, Bagceci, Cetin, 2012; Gruba, Sondegaard, 2001). The self monitoring and management of learning requires the development of problem solving, analysis, synthesis, critical thinking skills where deeper understanding of a phenomenon can be developed (Steffe, Nesher, 1996; Koc, Demeril, 2007). Hence the development of these skills requires self-insight and metacognition however without self-insight and metacognition it could be argued that students will be unable to effectively acquire knowledge, comprehend it, recall it and be able to apply it across and within learning situations (Bay, Bagceci, Cetin, 2012; Hartmann, 1998).
The inter-relationship between social constructivism and the cognitive continuum, represented by the blue dashed line in figure 22, indicates that students move
between and within both these theoretical models and along the cognitive continuum of decision and judgement making when constructing their assessment decisions. MM is a complex task and therefore is high on the continuum of task structure represented by the first column on the left hand side of the figure. The far right hand column identifies the possibility of manipulating variables. Levels of manipulation of variables for the PAMMO, such as the simulated practice learning environment [SPLE], the PAMMO proforma and the timescales for the assessment, were low as all were controlled.
Analysis of the focus group interviews suggested that students constructed their decisions and judgements of their peer’s MM safety at the lower end of the cognitive continuum. Consequently they used intuition, reflective, peer, systems and critical review of experiential and research aided judgement and decision making (see figure 22 bold black writing and boxes). The higher end of the cognitive continuum: action research, qualitative, survey and experimental research aided judgement and decision making (boxes with light grey shading) was not represented in the phase one and two data, hence it falls outside social constructivism (red line). The PAMMO students have developed their own and collective understanding of safe medicines management however this has been developed over time and in conjunction with their peer groups, their mentors in practice and their nurses educators, hence they have socially constructed their understanding of safe medicines management. Ethically students’ decisions and judgements did not involve ethical committee approval, which Standing (2010) illustrates at the higher end of the ethical continuum. This study’s students’ ethical judgements were driven by their personal and professional accountability. These were, in part, founded upon their code of conduct and ethics (NMC, 2008) - this will be explored in greater detail later in this chapter.
— re re 0) .2 « S -g o> 00 re OÛ C/Î 1 1 1 2 f i l l S m CO £L is: ÜJ a c I § ^ 8
HllIlL
e g g » #CD CO7.2 Peer Assessed Medicines Management OSCEs and social constructivism
Chapter two (see section 2.42) introduced the notion that the cognitive theories of learning and the principles of adult learning have social constructivism at their base. Critics of the existing strategies for MM learning and assessment highlight the problems students have with assimilating and transferring their learning, between different aspects of the university curriculum and practice (Grandell-Niemi et al., 2005; Manias, Bullock, 2002; Meechan, Jones, Valler-Jones, 2010). This suggests that students find constructing new knowledge and reconstructing this with their existing knowledge and skills set problematic, despite knowledge assimilation being basic tenet of social constructivism (Burr, 2003). As Evans et al. (2010) articulate this may be more challenging for student nurses due to the frequency of their placement changes. Hence workplace (placement) re-contextualisation of subjects like medicines management and pharmacology may be problematic.
Studies presented in chapter two exploring healthcare students’ perceptions of MM concur with this view (Grandell- Niemi et a!., 2005; Meechan, Jones, Valler-Jones, 2011; Wheeler et a!., 2008). Indeed the problems that healthcare students have with applying and transferring their MM knowledge and skills from different learning environments and making sense of this learning is not unique to students and may include registered nurses is problematic for both student nurses and registered nurses (Manias, Bullock, 2002). Findings from this PAMMO study’s focus group interviews concur with Grandell Niemi et a/s.’ (2005) and Unver et a/s.’ (2012) views that students find knowing what they need to know about MM is difficult. Similarly students find understanding the expectations of their mentors in practice and their HEI challenging. This was illustrated by the students in the focus group interviews stating they did not just want to memorise facts as this did not help them to be “good” nurses. Hence there was a tension between their perceptions that the university required them to memorise MM information for examinations but safe MM needs the nurse to
be able to apply this information to the patient situation. Consequently students in this PAMMO study felt they needed more guidance from their nurse educators about how to apply their learning to the practice situation. Potentially they could become passive recipients of knowledge rather than adult learners who actively engage with their learning (Knowles, 1968), in terms of social constructivism actively construct their own reality of their learning. This may be counterintuitive as the NMC (2005; 2010b) require registrants to be able to think critically, use active learning processes such as reflection, re-contextualise their knowledge and be self-directed. Hence they are required to socially construct their knowledge and reconstruct this within the world of healthcare and nursing (Evans et al., 2010). If students cannot socially construct their learning they will be unable to articulate the rationale for their decisions and actions (Brandon, All, 2010).
Results of the Simulation Effectiveness Tool [SET] presented in chapter five identified that the PAMMO video assessments permitted students to develop their MM learning and decision making confidence (see 5.3), thereby developing their confidence in their MM learning and decision making . This was also reflected in each of the Focus Group Interviews [FGI]. Analysis showed that using the PAMMO assessment proforma and viewing the video vignettes of peer students MM facilitated deeper learning through reflecting on and in practice, peer discussion and feedback. These findings concur with studies of peer assessment by Bucknall et al. (2008), Larsen, Jeppe-Jensen (2008) and Papinczak, Young, Groves (2006).
Increased confidence in clinical skills is important for students, as confidence, competence and proficiency are intrinsically linked (Yuan, Williams, Fang, 2011). However the development of students’ clinical skills needs to be facilitated by nurse educators, thereby enabling students to socially construct their learning to enhance their competence, proficiency and confidence (Fisher, King, 2013; Houghton, et al.,
2012). Although Arnold et a/s' (2008) study of student confidence and Simulated Practice Learning [SPL] concurs with this view both Kessler and Burton (2011) and Fisher, King, (2013) advocate educators moving beyond evaluating simulated practice learning [SPL] to researching the outcomes of educational strategies conducted in the simulated practice learning environment [SPLE], such as the PAMMO. This was in part achieved in this PAMMO study through the collation of the SET data, where the concept of satisfaction with the simulation experience was explored.
Although analysis of each of the focus group interviews suggested that video data recordings were useful for assessment and learning the students also identified they had some limitations. These limitations arose from difficulties in viewing some aspects of the video recordings, such as calculations and drug trolley supervision which made assessment of the related PAMMO criteria and sub-criteria problematic. Despite these difficulties students identified various development strategies for future use of the PAMMO and video recordings. Suggestions were: using a camera with a wider angled lens, having a further peer assessor in the room and being able to view the MM calculation after the assessment. Indeed students in this PAMMO study felt they had ownership over the assessment and the learning process due to the short introduction to peer assessment they received as part of the informed consent process. This introduction included the rationale for peer assessment, ground rules, the PAMMO proforma and the PAMMO assessment process. This was a new experience for the students'; as they felt elements of their own nurse curriculum such as OSCE assessment criteria were hidden (see 6.34). It could therefore be argued that having ownership over the assessment process enabled the student to engage with the MM OSCE from the perspective of the assessment as well as their learning. Hence the students were able to assimilate their knowledge and experiences of safe medicines management and use them in conjunction with the PAMMO proforma to construct their assessment judgements.
The notion of students owning their learning and assessment fits the ethos of social constructivism (Burr, 2003). Ownership of the PAMMO through enhanced understanding of its processes and procedures empowered the students to develop their learning. Indeed at a basic level the relationship between knowledge and power can be explained by knowledge enhancing an individual's power (Burr, 2003). From Foucault's perspective (1972) the relationship between knowledge and power is more complex, but it is recognised to be most effective when it produces knowledge (in this case knowledge of what constitutes safe medicines management). For the students in this PAMMO study their knowledge of safe MM practice was balanced by their views of the nurse educator's role in transforming their learning and being an “expert” (see 6.34). Although the students' did not explicitly articulate that they perceived this as a power relationship they perceived that nurse educators had the power to make them, as students, more knowledgeable by directing their learning. Conceivably the students changed from absolute learners, requiring concrete knowledge directed by the “expert” to get their MM learning right, to learners able to contextualise their learning. Hence they understood that knowledge was not always concrete, it required adaptation according to the learning and assessment situation to connect it to the reality of practice (Yilmaz, 2008). Knowledge is therefore adaptive (Glaserfield, 1995). The view of the nurse educator as the “expert” however conflicts with the espoused constructivist viewpoint of the educator as a facilitator, assisting students to construct their knowledge in negotiation (Brandon, All, 2010). Students perceived the role of the mentor as different to that of the nurse educator due to the variations in MM role modelling, teaching and supervision they (the student) received in practice. This variability in practice learning left some students feeling vulnerable (see 6.32) and reflects findings from Reid-Searle et al. (2008) and Reid-Searle, Moxham, Happell, (2010) studies of student nurses MM supervisory experiences.
The PAMMO enabled students to form their, initial, judgements of peer proficiency on the PAMMO assessment criteria and sub criteria (see chapter 6, section 6.34 and appendix 6). Good to very good agreement levels, between the two student groups for each of the 3 video vignettes (see chapter 5 section 5.2) were identified for most criteria and sub-criteria and “pass/ refer” scores. The global assessments of safe practice demonstrated the most disagreement, between groups and in groups (see section 5.28). The FGIs illustrated that the self and peer reflective processes used by students to make safe judgements of MM practice included collaborative enquiry and feedback. Indeed when the students were completing their PAMMO assessments students conferred with their peers and expressed their views on the student peers’ performance in the video vignettes. Although this may have created bias in the PAMMO data this peer learning was felt to be important to the data collection and this study’s ethos of peer assessment and social constructivism.
Consequently social interaction was important to this study as it provoked further thinking and therefore critical reasoning by evaluating whether the practice was relevant and practical to the task (see section 6.31). In terms of educational practice this has clear links to assessment strategies and the validity and reliability of assessment. Berkenstadt et als. ’ (2006) study, amongst others presented in chapter 2, evaluated the face, criterion, construct and content validity of OSCE assessments for anaesthetists however it did not evaluate the reliability of these assessments. Hence the relevance and practicality of the assessment criteria for each of the OSCE stations were evaluated but not the reliability of the assessment judgements. Conversely this PAMMO study evaluated the reliability of the PAMMO from the perspective of the inter-rater reliability of peer assessors but not its validity as the tool had been developed using the Delphi technique and blue printing. Developing OSCE assessment proformas is a complex task. As studies by Berkenstadt et al. (2006) and Feeley et a/s.' (2003) indicate development of OSCE assessment proformas must use
blue printing and/ or the Delphi technique; additionally they must refer to the evidence based guidelines and involve students and expert practitioners. Following such processes will assure valid assessment tools, processes and outcomes are utilised in OSCE development. The PAMMO assessment proforma was designed in conjunction with experts in practice and the evidence base, which is a common approach to OSCE development (Berkenstadt et al., 2006; Bucknall et al., 2008; Dannefer et al., 2005; Goff et al., 2002). However students were not involved, this may have impacted upon the students’ ability to socially construct their learning to the assessment situation.
Learning and assessment theory supports the idea that learning is important and must be combined with understanding the topic area (Bransford et al., 2000). From the FGIs it was evident that students’ assessment decisions were facilitated by the PAMMO and their own experiences of practice (see section 6.31 and figure 39). Being able to articulate the rationale for one’s actions is a professional requirement of the NMC “The Code” (2008). This requires a deeper understanding of MM to be communicated, rather than the surface knowledge and recall required in a simple “pass: refer” assessment. The students highlighted their expectation that the peer students, in the video vignettes, should have verbalised a more detailed account of the pharmacokinetics and pharmacodynamics of the medication to both the assessor and the simulated patient. They associated this with patient dignity and safe MM practice (see section 6.311).
Although OSCEs are viewed as a tool for learning and assessment (Pope, 2005; Wilkinson et al., 2003) gaining consensus from a panel of experts may be problematic (Feely et al., 2003). Similarly designing an assessment document for use across practice areas, or geographical locations, may also be challenging. Students from one
HEI may attend placements within different healthcare organisations and each may have a different assessment tools, prescription charts and policies.
Despite the non-involvement of students in the initial design of the PAMMO it had construct, face and content validity as consensus was gained from a panel of experts during its development. Additionally evidence based guidelines for MM were referred to and ongoing student feedback was utilised following its use in the SPL, these all have contributed to its validity as an assessment tool. Hence its empirical accuracy, and consequently the rationality of the assessment judgements made by the students were potentially enhanced (Hammond, 2007).
Phase one of this study did not permit the rationality of these MM safety judgements