Video- or text-based e-learning when teaching clinical procedures? A randomized controlled trial

(1)

Advances in Medical Education and Practice

Dove

press

O r i g i n A l r E s E A r c h

open access to scientific and medical research

Open Access Full Text Article

Video- or text-based e-learning when teaching

clinical procedures? A randomized controlled trial

steen Vigh Buch1

Frederik Philip Treschow2

Jesper Brink svendsen3

Bjarne skjødt Worm4

1_{Department of Vascular surgery,}

rigshospitalet, copenhagen, Denmark;

2_{Department of Anesthesia and}

intensive care, herlev hospital, copenhagen, Denmark; 3_{Faculty of}

health and Medical sciences,University of copenhagen, copenhagen, Denmark;

4_{Department of Anesthesia and}

intensive care, Bispebjerg hospital, copenhagen, Denmark

correspondence: steen Vigh Buch Department of Vascular surgery, rigshospitalet, 9 Blegdamsvej, København Ø, copenhagen 2100, Denmark

Tel +45 26 880 445 Email [email protected]

Background and aims: This study investigated the effectiveness of two different levels of e-learning when teaching clinical skills to medical students.

Materials and methods: Sixty medical students were included and randomized into two comparable groups. The groups were given either a video- or text/picture-based e-learning module and subsequently underwent both theoretical and practical examination. A follow-up test was performed 1 month later.

Results: The students in the video group performed better than the illustrated text-based group in the practical examination, both in the primary test (P,0.001) and in the follow-up test (P,0.01). Regarding theoretical knowledge, no differences were found between the groups on the primary test, though the video group performed better on the follow-up test (P=0.04).

Conclusion: Video-based e-learning is superior to illustrated text-based e-learning when teaching certain practical clinical skills.

Keywords: e-learning, video versus text, medicine, clinical skills

Background

E-learning and web-based learning is rapidly becoming an integrated part of the teach-ing methods used in medical schools and continuteach-ing medical education throughout the world.1,2_{Of the many reasons, the primary one is availability. With an increasing} number of smartphones, tablets, and laptops, e-learning is easily accessible whenever and wherever needed.

Further e-learning benefits include increased student satisfaction, especially when e-learning is used as an augmentation to traditional teaching methods.3–6_{Cost is both} an advantage and a disadvantage. Upfront development costs can be prohibitively high and require considerable technical skills. Once developed, however, e-learning offers the advantages of scale: it can be used countless times at minimal marginal cost.2

Previously published studies have shown little or no difference in outcome when comparing web-based e-learning modules to paper-based modules of the same curriculum.3,7,8_{Similarly, studies comparing based e-learning to traditional} case-based classroom teaching show no significant differences between groups.3,7,8

As for acquisition of practical clinical skills mediated by e-learning, the existing literature is scarce. Most available sources deal with e-learning in addition to traditional class-based learning. Results in these studies vary though, with both better and poorer results from e-learning-augmented teaching, compared to classroom teaching alone, having been described.4–6,9,10_{This lack of evidence-based knowledge raises the question} of whether or not a clinical skill can be mediated by e-learning alone.

Advances in Medical Education and Practice downloaded from https://www.dovepress.com/ by 118.70.13.36 on 19-Aug-2020

For personal use only.

Number of times this article has been viewed

This article was published in the following Dove Press journal: Advances in Medical Education and Practice

(2)

Dovepress Buch et al

E-learning in general is a rapidly growing market, reach-ing US$35.6 billion worldwide in 2011, and with a 5-year compound annual growth rate of 7.6% expected to reach $51.5 billion by 2016.11_{With this rapidly developing market,} there is an urgent need of evidence-based guidance on how to develop and implement the most effective e-learning in the future.12_{Therefore, it is relevant to examine whether} some e-learning methods are more effective than others on a stand-alone basis, in particular for the acquisition of practical clinical skills.

This study investigates the effectiveness of two dif-ferent e-learning methods on a population of fourth- and fifth-year Danish medical students. While previous studies have focused primarily on the use of video or illustrated text incorporated into the traditional method of teach-ing as augmentation to certain topics, our aim was to examine which of the two modalities was best suited for acquiring both theoretical and practical knowledge of a new subject.

A previous study, dealing with assessment of patients with head trauma according to the Glasgow Coma Scale, showed improved outcomes associated with higher levels of e-learning.13_{Using this “level division”, the two groups of} the current study would have been divided into multimedia level 1 (illustrated text) and level 2 (video). However, this prior study was focused solely on the acquisition of theo-retical knowledge. We thus designed our study to determine whether the results would also apply for the acquisition of practical skills.

For this study, we chose the Dix–Hallpike test as a teaching subject. This test is used by clinicians to examine patients with symptoms of dizziness for benign paroxysmal positional vertigo. Practical skill as well as considerable theoretical knowledge is required, in order for both the test to be performed and the results interpreted correctly.

Aim of the study

This study aimed to test if video-based e-learning was better than illustrated text-based e-learning when teaching clinical procedures. Subsequently, the study also aimed to test if there was a difference between theoretical knowledge and practical skills.

Materials and methods

setup

This was a randomized controlled study of two groups learning the Dix–Hallpike test using either video-based or illustrated text-based e-learning. Test subjects were medical

students from the University of Copenhagen. Students were recruited from participants at a voluntary course in emer-gency room surgery where the Dix–Hallpike test was not part of the expected curriculum (Danish Medical Associa-tion, Copenhagen). All interested students were invited to join. Three students who had previous clinical experience with or knowledge of the Dix–Hallpike test were excluded from this study.

Before students could access the e-learning modules, they were required to register and specify their sex and age. At the same time, they were assigned to one of two groups using simple computerized randomization. Allocation con-cealment was maintained throughout the study. The students were randomly assigned to their groups and kept unaware of the presence of other groups and the teaching methods applied. After using the e-learning module for a maximum of 15 minutes, the students were asked to take a test (multiple-choice questionnaire [MCQ]) and observed structured clini-cal examination (OSCE), and subsequently a follow-up test (MCQ + OSCE) 1 month later. Group 1 was the video-based group while group 2 was the illustrated text-based one (as illustrated in Figure 1).

illustrated text-based e-learning

This simple e-learning module was purely an illustrated text-book with a series of six pictures showing the different steps of actions during the procedure. It also described evaluation of the test as well as basics about vertigo (Figure 2).

Video-based e-learning

This module was almost the same as the illustrated text one. The only difference was a video-sequence of about 2.2 minutes showing the procedure itself. We chose a maximum time of 15 minutes within the modules based on pilot studies indicating about 10–15 minutes was needed.

Development of the test instruments

McQ

In order to avoid questions being seen before the test and follow-up-test, both consisted of 50 true/false questions, randomly chosen from a pool of 100 questions. These were validated using two groups of 15 students in pilot studies. An index of reliability was computed as the difference between the proportions of high and low scores answered correctly. All questions had a reliability index above 0.10, which was why no changes were made. The questions were divided into procedural questions (25 points) and questions regarding theory (25 points).

(3)

Dovepress Video versus text for e-learning

Inclusion and registration

Group 1 video

Test

MCQ

n=21 n=22

n=30

n=30 n=30

n=3

n=30

Follow-up test OSCE

OSCE Questionnaire

Group 2 illustrated text

Figure 1 Flowchart describing the sequence of this study.

Abbreviations: McQ, multiple-choice questionnaire; OscE, observed structured clinical examination.

OscE

The OSCE was evaluated using a mark sheet divided into procedural skills (a total of 25 points regarding ten predefined actions), and the students were scored in order to evaluate if the actions were performed safely and accurately. The stu-dents were also verbally asked ten questions regarding theory, and had the ability to gain a total of 25 points (all or none

for each question). An index of reliability was computed as the difference between the proportions of high and low scores answered correctly. All questions and actions had a reliability index above 0.10, and no changes were made. The examiner in charge of supervising the OSCE was blinded, and thus had no knowledge of whether the student had been in the illustrated text group or video group.

Module development

The test, follow-up test, questionnaire, and e-learning training and test-module were all designed using Moodle (Moodle Pty Ltd, Perth, WA, Australia), a freeware (GPLv3-licensed) PHP web application for producing modular Internet-based courses integrated into a free Danish website for medical education. All parts of the study were closed and required a password for admittance. The program was carried out in Danish. The video and a rewritten combination of the two modules were available online.

Outcomes

The primary outcome was the difference between the two groups’ scores obtained on the validated knowledge test. Secondary outcomes were the difference in self-evaluated skills and satisfaction with the teaching material.

statistical tests and study size

The Mann–Whitney U test was used to compare the two groups. Cohen’s d was calculated as a standardized mea-sure of effect. Based on experiences from a pilot study, the difference was anticipated to be 5%. The anticipated range for this difference would be 10%, thereby a standard devia-tion applied for all groups of 5%. We anticipated that both groups would only deviate slightly from each other, and a 6% difference was chosen as a minimum relevant difference. The significance level was set at 5%, and statistical power at 80%. This yielded a total requirement of 22 subjects in each group.

Ethics

The study was purely educational, and the Danish National Committee on Health Research Ethics (DNVK), Denmark was consulted. Their conclusion was that the study did not require ethical approval (h-4-2013-fsp 41). All recruited students were asked to give electronic consent before entering the system.

Results

Table 1 illustrates the characteristics of the 60 students who completed the test and the 43 students who also completed

(4)

Dovepress Buch et al

Table 1 Demographic characteristics of participating students

Video % Text/ pictures

% P-value

Primary test and questionnaire

n 30 30

Mean age (sD) 24.7 (1.3) 24.6 (1.2) 0.75

Sex

Male 13 43 14 47 0.83

Female 17 57 16 53

Follow-up test

n=106 22 21

Mean age (sD) 24.7 (1.4) 24.6 (1.6) 0.94

Sex

Male 8 36 10 48 0.54

Female 14 64 11 52

Note:P-values are for differences between the two randomized groups.

Abbreviation: sD, standard deviation.

Table 2 Differences in mean numbers of correct responses, time, and efficiency

Video Text/ picture

Difference P-value Cohen’s d Primary test

n 30 30

McQ (sD) 44.5 (2.7) 44.6 (3.5) −0.1 0.7 0.04 Procedure

(sD)

20.9 (2.0) 20.7 (2.5) 0.2 0.5 0.10

Theory (sD) 23.5 (1.8) 23.8 (1.6) −0.3 0.5 0.18 OscE (sD) 44.2 (1.3) 40.2 (3.8) 4.0 0.001 1.39 Procedure

(sD)

23.1 (1.0) 19.2 (3.5) 3.9 ,0.001 1.50

Theory (sD) 21.0 (1.0) 20.9 (1.1) 0.1 0.9 0.10 Follow-up test

n 22 21

McQ (sD) 40.8 (1.6) 39.5 (3.0) 1.3 0.15 0.55

Procedure (sD)

19.8 (1.0) 18.6 (1.9) 1.2 0.04 0.79

Theory (sD) 21.0 (1.4) 20.9 (1.5) 0.1 1.0 0.07 OscE (sD) 40.5 (1.7) 38.0 (3.9) 2.5 0.01 0.80 Procedure

(sD)

21.1 (1.4) 19.0 (2.9) 2.1 0.005 0.96

Theory (sD) 19.3 (1.8) 19.0 (2.0) 0.3 0.6 0.14

Notes: Significance calculated using Mann–Whitney U test. cohen’s effect-size value (d): high (.0.8), moderate (0.5–0.8), or small (0.2–0.5) practical significance.

Abbreviations: sD, standard deviation; McQ, multiple-choice questionnaire; OscE, observed structured clinical examination.

Figure 2 Example of pictures taken from the illustrated text material.

Note: reproducedwith permission from Medviden.dk.

the follow-up test. There were no statistically significant differences between the groups’ demographic parameters, and no students were excluded due to prior knowledge of the test.

Table 2 illustrates the differences in test scores (knowledge tests) between the two groups. For the MCQ, there were no statistically significant differences between the two groups in the primary test, but the video group performed statistically better in procedural questions (P=0.04) for the follow-up test. The OSCE showed significant differences between the two groups both in the primary test (P=0.001) and follow-up test (P=0.01).

Further analysis of the OSCE results revealed, that the result was primarily due to the differences in the procedural scores (P,0.001 in primary test; P=0.005 in follow-up test), while there were no significant differences in scores for the theoretical part (P=0.9; P=0.6). The follow-up questionnaire (Table 3) revealed that the video group was more satisfied with the teaching material than the illustrated text-based group (P,0.001). However, the two groups evaluated their clinical skills as being equally good.

Discussion

In this study, we found that video-based e-learning was a more effective method than illustrated text-based e-learning for the procedural part of the OSCE in teaching the Dix– Hallpike test. This was in line with our expectations, as video can present a smoother and more exact sequence of steps in a given clinical procedure compared to an image sequence. In contrast, a less practically oriented teaching session with more factual points might be better suited for illustrated text-based learning, but this is beyond the scope of this study. In line with previous research,3_{our results show} that higher levels of e-learning yield both better results and higher student satisfaction.

(5)

Dovepress Video versus text for e-learning

Table 3 Questionnaire

Video Text/ picture

Difference P-value Cohen’s d

n 30 30

satisfaction (sD) 7.3 (1.1) 5.9 (1.0) 1.4 .0.001 1.37 self-evaluated

clinical skill (sD)

6.1 (1.3) 6.3 (1.2) −0.2 0.45 0.200

Notes: Questionnaire rated 0–10, where 10 was highest. Significance calculated using Mann–Whitney U test. cohen’s effect-size value (d): high (.0.8), moderate (0.5–0.8), or small (0.2–0.5) practical significance.

Abbreviation: sD, standard deviation.

Conclusion

Based on our findings, we conclude that video-based e-learning is superior to illustrated text-based e-learning when teaching certain practical clinical skills, such as the Dix–Hallpike test. We suggest further studies in order to obtain a more general conclusion.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis.

JAMA. 2008;300(10):1181–1196.

2. Cook DA. Web-based learning: pros, cons and controversies. Clin Med. 2007;7(1):37–42.

3. Cook DA, Dupras DM, Thompson WG, Pankratz VS. Web-based learn-ing in residents’ continuity clinics: a randomized, controlled trial. Acad

Med. 2005;80(1):90–97.

4. Bloomfield JG, Jones A. Using e-learning to support clinical skills acquisition: exploring the experiences and perceptions of graduate first-year pre-registration nursing students – a mixed method study.

Nurse Educ Today. 2013;33(12):1605–1611.

5. Preston E, Ada L, Dean CM, Stanton R, Waddington G, Canning C. The Physiotherapy eSkills Training Online resource improves performance of practical skills: a controlled trial. BMC Med Educ. 2012;12:119. 6. Hibbert EJ, Lambert T, Carter JN, Learoyd DL, Twigg S, Clarke S.

A randomized controlled pilot trial comparing the impact of access to clinical endocrinology video demonstrations with access to usual revi-sion resources on medical student performance of clinical endocrinology skills. BMC Med Educ. 2013;13:135.

7. Worm BS. Learning from simple ebooks, online cases or classroom teaching when acquiring complex knowledge. A randomized controlled trial in respiratory physiology and pulmonology. PLoS One. 2013;8(9): e73336.

8. Maloney S, Storr M, Paynter S, Morgan P, Ilic D. Investigating the effi-cacy of practical skill teaching: a pilot-study comparing three educational methods. Adv Health Sci Educ Theory Pract. 2013;18(1):71–80. 9. Perkins GD, Kimani PK, Bullock I, et al. Improving the efficiency

of advanced life support training: a randomized, controlled trial. Ann

Intern Med. 2012;157(1):19–28.

10. Worm BS, Buch SV. Does competition work as a motivating factor in e-learning? A randomized controlled trial. PLoS One. 2014;9(1): e85434.

11. Adkins SS. The Worldwide Market for Self-Paced eLearning Products

and Services: 2011–2016 Forecast and Analysis. Monroe (WA):

Ambient Insight; 2013.

12. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad

Med. 2010;85(5):909–922.

13. Worm BS, Jensen K. Does peer learning or higher levels of e-learning improve learning abilities? A randomized controlled trial. Med Educ

Online. 2013;18:21877.

A potential weakness in this type of study is whether or not the two sets of teaching materials were equivalent with regard to quality. As we found that both groups of students performed equally well in both the MCQ and in the theoretical questions of the OSCE, we concluded that the factual parts of our materials were comparable.

In their study of video sequences, in addition to regular tuition of medical students, Hibbert et al6_{found no effect with} regard to the procedure of thyroid examination. They argued that the expected “gain” of the intervention group might have been undermined due to the fact that a number of instructional videos on the topic are already available online, and are thus available to both the control and intervention groups.

To eliminate this effect, the primary test was admin-istered immediately after the teaching session. However, as the subsequent follow-up test was conducted 1 month after the primary test, the performance of both groups could potentially have been biased by other instructional videos or acquisition of knowledge elsewhere. Even so, the video group still performed better than the illustrated text-based group in both MCQ and OSCE procedural questions on the follow-up test. This could be due to two factors: either our instructional video was of a higher quality than the ones available else-where online, or the students in the illustrated text-based group had not sought additional knowledge online.

The strengths of this study are comparable groups, simple setup, and relevant end points. The sample size was sufficient to detect statistical difference with regard to the procedural part of our end points, but a larger study might have shown difference in the theoretical part as well. This study supports the need for differentiating and individualizing e-learning into different modalities depending on the subject, and that further research is warranted in order to continuously develop e-learning on an evidence-based scale.

(6)

Advances in Medical Education and Practice

Publish your work in this journal

Submit your manuscript here: http://www.dovepress.com/advances-in-medical-education-and-practice-journal

Advances in Medical Education and Practice is an international, peer-reviewed, open access journal that aims to present and publish research on Medical Education covering medical, dental, nursing and allied health care professional education. The journal covers undergraduate education, postgraduate training and continuing medical education

including emerging trends and innovative models linking education, research, and health care services. The manuscript management system is completely online and includes a very quick and fair peer-review system. Visit http://www.dovepress.com/testimonials.php to read real quotes from published authors.

Dovepress

Dove

press

Buch et al