The initial plan was to design the assessment system Step One, in a similar way to the UK driving knowledge and hazard test. In other words, to start with knowledge questions followed by assessment of the hazard perception clips as in the driving test structure. This plan was changed after consulting the available cognitive literature discussed in Chapter Two.
Kahneman (54) described the possibility of training System One to detect hazard clues and warn System Two. Such training would be best facilitated by providing a familiar environment with a realistic sequence. Despite the lack of standardised steps to perform surgical procedures, they usually follow certain phases. To translate Kahneman’s theory into surgical practice we needed to train System One to detect clues for specific possible hazards in each phase within the surgical operation. Such a sequence would help the brain to look for certain clues at certain times within the operation. This would reduce and focus brain training by providing a certain order.
Taxing the brain to answer knowledge questions about various parts of the
operations, then presenting the hazard clips about those parts again, would create an artificial split and would not reflect the real experience within the operation. It would force candidates to flip back and forth between operation steps causing tiredness and shifting the attention away from the intended hazard-spotting training.
48
As a result the new design organised the cognitive hazard training module in a merged fashion. I also divided the module into four main sections and divided the sections into various parts to signpost the shift from one topic to the next. This is to highlight the expected risks in each step and reduce the mental shifting between various tasks.
The UK driving test part two (Practical Test) relies on direct observation by an examiner armed with a check list. The examiner has to test the driver and tick the check list while maintaining safety on the road. This is possible in driving exams as the test follows certain known routes and the driver has been trained to full
competence or very close to full competence by their driving instructor. In other words the practical test is a summative test of competency for a trained driver with limited risky behaviour, allowing examiners to focus mainly on the test, while maintaining safety, with no training component.
Surgical training and assessment is far from the driving test scenario described above. Consultant surgeons are busy assisting and teaching trainees and they are scrubbed for sterility so they cannot touch normal pen and paper. They have to train trainees with variable degrees of competency and frequently assess their progress using the PBA forms. Current recommendations advise that each trainee should carry out 40 WBAs per year. Consultant surgeons are more like driving instructors
carrying out training and frequent assessments simultaneously while keeping an eye on the operation’s progress and the patient’s safety. They cannot tick the check list during the procedure as well as carrying out all those tasks even if they manage a
49
way to overcome sterility issues. As a result PBA forms are usually completed after the operation list on one of the consultant’s admin/free days relying on recall which is subject to decay over time. To overcome this problem I used the video-review session of trainees’ operations to provide objective performance evidence and improve trainers’ feedback.
Video-review facilitates trainees’ reflective practice as discussed in Chapter Two. They review their own performance in a stress-free environment and reflect on it (66). Multiple studies (in relation to open surgery, anaesthesia, music and sport) have demonstrated the benefit of this practice in improving skills and improving trainees’ ability to self-assess as described in Chapter Two. Reviewing one’s video of practice facilitates internal feedback and helps to focus training on individual trainee needs. It shortens training time by making the best of the available training opportunity and enhances safety by allowing improvement beyond traditional training. The benefit of such practice was studied previously in various fields, as mentioned in Chapter Two, but was technologically difficult to conduct on a large scale in the theatre
environment. Technology advancement has recently overcome this difficulty.
As the majority of surgical consultants will scrub to assist their trainee during surgical procedures, video-review will give them the chance to reflect on their teaching styles. In this sense consultants are evaluating their teaching style and giving trainees direct performance feedback. This overcomes assessors’ time wasting as the result of reviewing blinded videos which was the only disadvantage of video assessment argued by Aggarwal (98). Video-review will also enhance the validity
50
and reliability of the surgical skills assessment by providing objective performance evidence.
Video-review will also allow trainers to assess trainees’ non-technical skills, if they have the training to do so. When consultants review the operation videos with their trainees they will not be occupied by mentoring and observing their trainees’ operative action. This will leave them free to focus on assessment and feedback of technical and non-technical skills, thus avoiding competing duals roles (101). To compensate for supervisors’ involvement in theatre Crossley et al used non-surgeon assessors to mark the NOTSS form. Such a step deprived trainees of their trainers’ feedback and casts some doubts about the assessment value.
As discussed in earlier chapters, the surgical environment has a special character, distinguishing it from other disciplines. Such distinction presents a different set of skills in communication and coordination. Grote et al. established the unique implicit coordination and higher leadership required in the theatre environment (34). This will affect leadership and situation awareness interpretation by non-surgical assessors, limiting their ability to judge those domains as was the case in other studies (101, 102).
I opted not to include NOTSS in my research due to the lack of widely available training for my target group. However I am arguing that my structure will facilitate the use of this form and overcome the obstacles described in the literature once such NOTSS training becomes available.
51
3.4 Chapter summary
This chapter described the new cognitive hazard training and reflective formative assessment design principles and the reason to modify the design from the current UK driving test model. The next chapter will detail the challenges encountered and the hurdles that I had to overcome in creating the practical example for laparoscopic cholecystectomy assessment. I will describe the steps taken to create each part and the logic behind using the selected clips and materials along with a detailed
52
Chapter Four: Putting the design into
production
4.1 Introduction
In the previous chapter I laid the foundation for my new design starting with the UK driving test as a model and finishing with the new cognitive hazard training and reflective formative assessment design. As design-based research, establishing the principle foundation for the intended formative assessment was the first step in the process. This step was followed by creating a practical example of the system and putting it to the test by conducting a feasibility study.
In this chapter I will describe my journey in creating a practical example of my design for laparoscopic cholecystectomy. I will also mention the difficulties faced throughout the process in finding the suitable materials, getting the legal permissions to use them, solving the technical and design challenges in processing the material to serve the intended value within the system and the ethical challenges encountered in planning and conducting the feasibility study.