Using Clicker Technology in a Clinically-based Occupational Therapy Course: Comparing Student Perceptions of Participation and Learning

(1)

Using Clicker Technology in a Clinically-based Occupational Therapy Course: Comparing Student Perceptions of

Participation and Learning

Thomas Mernar, PhD, OTR

(2)

Acknowledgements

I would like to thank

• The Office of the Provost, the Teaching, Learning, Technology Center, and Drs. Michael Vigorito and Theresa Bartolotta for the support of this program assessment proposal

• Research assistants Danielle Dorsi, OTS; Michelle Merciadez, OTS,

Jennifer Ostaszewski, OTS, Janine Petito, and OTS; Rachel Sellers,

OTS

(3)

Background

• Student–teacher interaction is ranked highly among the factors influencing learning performance (Bullock et al., 2002; Hake, 1998)

• Clicker technology has been changing how students and their instructor interact within the classroom by

providing new opportunities to enhance in-class

participation and learning in traditional, lecture hall

classroom environments

(Blasco-Arcas, et al., 2013).

(4)

Purpose

• Less is known about how clicker technology improves in-class participation and learning in more clinically content based courses

• The purpose of this program assessment was to examine how

learning and in-class participation is effected by the use of

clicker technology within a clinically based occupational

therapy course

(5)

Research Design

• Nonequivalent pretest posttest control group design

(6)

Study Sample and Group Assignment

• Study sample

• 32 first year, first semester occupational therapy students

• 15 control group

• 17 experimental group

• All students enrolled into Occupational Therapy Practice Skills course; 2 sections (Monday and Thursday)

• Group assignment:

• Monday= Experimental group

• Thursday= Control group

(7)

Measurements

Pretest and Posttest ordinal data

• Student Assessment of Learning Goals (SALG)

• Self Consciousness Scale revised (SCSR)

Posttest only ordinal data

• Perceived Importance of Clicker Technology (PICT)

• Only given to experimental group at end of semester

(8)

Student Assessment of Learning Goals

• SALG instrument focuses exclusively on the degree to which a course has enabled student learning

• SALG asks students to assess and report on their own learning, and on the degree to which specific aspects of the course have contributed to that learning

• Instructor can customize questions

• The following 32 questions have been used in this program

assessment: Understanding (6), Participation (5), Proficiency (8),

Feelings about learning (4), Willingness to seek help (6), and Habits (3)

• Students rate questions as:

• “strongly disagree” (-2)

• “disagree” (-1)

• “agree” (1)

• “strongly agree” (2)

(9)

Self Consciousness Scale Revised

• SCSR is a 23- item questionnaire developed for the general public, which measures individual private self-consciousness, public self-consciousness, and social anxiety

• Students rate questions as:

• 3 = a lot like me

• 2= somewhat like me

• 1= a little like me

• 0= not like me at all

(10)

Perceived Importance of Clicker Technology

• Utilized SALG and reformatted all 32 questions to ask about how clickers have enhanced their understanding, participation, proficiency, feelings about

learning, willingness to seek help, and habits of the content and skills learned throughout the course to create the PICT

• For example:

• Under the SALG, a question would read as:

• Presently, I am… Able to demonstrate proficiency in exercising correct infection control techniques

• Under PICT, the same question would read as:

• Since using the clickers over the past few weeks, the use of clickers increased my…

Proficiency in exercising correct infection control techniques

• Students rate questions as:

• “strongly disagree” (-2)

• “disagree” (-1)

• “agree” (1)

• “strongly agree” (2)

(11)

Application of Independent Variable

• IV= use of clicker technology during class over 5 weeks during Fall 2013 semester to obtain polling of questions related to:

• Assigned readings

• Content of lecture

• Past experiences related to weekly topic

• Confidence in performing lab skills

• Feelings about the class

• Classes included:

• Transfer training

• Infection control

• Vital signs

• ADL Topics: Dressing, Toileting, and Bowel & Bladder

• ADL Topics: Eating & Feeding, Personal

Hygiene, Bathing, and Sexuality

(12)

(13)

Which is the most common mode of infectious disease contamination?

1 2 3 4

58%

5%

32%

5%

For example

1. Contact 2. Droplet 3. Airborne

4. Common vehicle

(14)

Statistical Analyses

• All pretest and posttest questionnaires scored twice for accuracy

• Independent samples t-test used to distinguish differences in outcome scores at baseline

• Independent samples t-test used to distinguish differences

in outcome scores from posttest to pretest (change scores)

(15)

Dependent Variables

Dependent Variable Learning Gains Total

Learning Gains in Understanding Learning Gains in Participation Learning Gains in Proficiency

Learning Gains in Feelings Towards Learning Learning Gains in Willingness to Seek Help Learning in Habits

Self-consciousness Total Private self-consciousness Public self-consciousness Social Anxiety

Total Perceived importance of clicker technology *

DVs have pretest, posttest, and change score values, except for Total

PICT*, as it was given only to experimental group at posttest

(16)

Descriptive Comparison of Groups at Baseline

Learning Gains

Total

Learning Gains in Understan

ding

Learning Gains in Participation

Learning Gains in Proficiency

Learning Gains in Feelings

Learning Gains in Willingness

Learning Gains in Habits

Self- conscious ness Total

Private Self- conscious

ness

Public Self- conscious

ness

Social Anxiety Mean 31.1333 10.1333 6.8667 -6.0000 6.0667 10.4667 3.6000 39.0667 19.2667 13.3333 6.4667

N 15 15 15 15 15 15 15 15 15 15 15

Std.

Deviation

8.45042 1.24595 1.50555 4.86973 2.25093 1.95911 1.29835 7.48777 1.98086 3.22195 4.98378

Mean 30.2353 9.9412 6.9412 -6.8824 6.5882 9.8824 3.7647 39.1176 18.2941 12.7647 8.0588

N 17 17 17 17 17 17 17 17 17 17 17

Std.

Deviation

9.25695 1.67595 1.81902 4.99853 1.62245 2.28808 1.75105 6.33327 3.53137 2.79574 3.94447

Mean 30.6563 10.0313 6.9063 -6.4688 6.3438 10.1563 3.6875 39.0938 18.7500 13.0313 7.3125

N 32 32 32 32 32 32 32 32 32 32 32

Std.

Deviation

8.75697 1.46979 1.65314 4.87908 1.92788 2.12678 1.53323 6.78404 2.90717 2.96740 4.46085 Descriptive Statistics of Control and Experimental Group

type of group

control group

experimental group

Total

(17)

Inferential Comparison of Groups at Baseline

.

Lower Upper

Equal variances as s um ed

1.231 .276 .285 30 .777 .89804 3.14913 -5.53335 7.32943

Equal variances not as s um ed

.287 29.956 .776 .89804 3.13070 -5.49610 7.29218

1.830 .186 .364 30 .719 .19216 .52811 -.88639 1.27070

.371 29.220 .714 .19216 .51838 -.86770 1.25202

1.190 .284 -.125 30 .901 -.07451 .59514 -1.28995 1.14093

-.127 29.895 .900 -.07451 .58800 -1.27555 1.12653

.002 .966 .504 30 .618 .88235 1.74956 -2.69073 4.45544

.505 29.683 .617 .88235 1.74662 -2.68632 4.45102

1.235 .275 -.758 30 .454 -.52157 .68767 -1.92598 .88285

-.743 25.153 .464 -.52157 .70187 -1.96665 .92352

1.104 .302 .770 30 .447 .58431 .75839 -.96453 2.13315

.778 29.980 .443 .58431 .75089 -.94925 2.11787

Independent Samples Test Levene's Tes t for

Equality of Variances t-tes t for Equality of Means

F Sig. t df

Sig. (2- tailed)

Mean Difference

Std. Error Difference

95% Confidence Interval of the

Learning Gains Total

Learning Gains in Understanding

Learning Gains in Participation

Learning Gains in Proficiency

Learning Gains in Feelings

Learning Gains in Willingness

(18)

Inferential Comparison of Groups at Baseline

Equal variances assumed

.393 .535 -.299 30 .767 -.16471 .55130 -1.29061 .96120

Equal variances not assumed

-.304 29.196 .763 -.16471 .54106 -1.27097 .94156

.442 .511 -.021 30 .983 -.05098 2.44293 -5.04010 4.93814

-.021 27.622 .984 -.05098 2.46925 -5.11213 5.01017

4.916 .034 .943 30 .353 .97255 1.03171 -1.13448 3.07958

.975 25.709 .339 .97255 .99757 -1.07912 3.02422

1.344 .255 .535 30 .597 .56863 1.06351 -1.60335 2.74060

.530 27.977 .600 .56863 1.07324 -1.62988 2.76714

1.468 .235 -1.008 30 .322 -1.59216 1.57984 -4.81862 1.63431

-.993 26.634 .330 -1.59216 1.60346 -4.88431 1.70000 Learning Gains

in Habits

Self- consciousness

Total

Private Self- consciousness

Public Self- consciousness

Social Anxiety

(19)

Discussion of Groups at Baseline

• A statistically significant difference was found between control group and experimental group at baseline for one dependent variable

• Private self-consciousness (p=0.034)

(20)

Inferential Comparison of Groups from Posttest - Pretest

.

(21)

Inferential Comparison of Groups from Posttest - Pretest

3.164 .085 1.316 30 .198 .86667 .65872 -.47863 2.21196

1.355 26.968 .187 .86667 .63977 -.44610 2.17944

.680 .416 -1.070 30 .293 -2.21569 2.07075 -6.44472 2.01334

-1.060 27.860 .298 -2.21569 2.09082 -6.49950 2.06813

1.273 .268 .429 30 .671 .44314 1.03361 -1.66778 2.55406

.420 25.329 .678 .44314 1.05428 -1.72675 2.61303

1.398 .246 -1.325 30 .195 -.92549 .69861 -2.35223 .50125

-1.306 26.705 .203 -.92549 .70885 -2.38069 .52971

.005 .944 -1.520 30 .139 -1.73333 1.14045 -4.06245 .59579

-1.492 25.607 .148 -1.73333 1.16206 -4.12377 .65710 SCSRprivateSC

change

Public Self- consciousness

Social Anxiety Learning Gains

in Habits

Self- consciousness

Total

(22)

Discussion of DV Group Comparisons

• A marginally statistically significant difference was found between control group and experimental group in change scores for one dependent variable

• Learning gains in habits (p=0.085)

Presently, I am in the habit of…

Connecting key ideas I learn in class with other topics covered in this class

Strongly Disagree

Disagree Agree Strongly Agree Connecting key skills I learn in class with

other skills covered in this class

Strongly Disagree

Disagree Agree Strongly Agree Connecting key ideas I learn in class with

other classes in the curriculum

Strongly Disagree

Disagree Agree Strongly

Agree

(23)

Descriptive Statistics for Perceived Importance of Clicker Technology

17 participants in experimental group scored the PICT at the end of the semester

32 questions were scored on an ordinal scale of

• “strongly disagree” (-2)

• “disagree” (-1)

• “agree” (1)

• “strongly agree” (2)

Descriptive Statistics

N Minimum Maximum Mean Std. Deviation

Perceived Importance of Clicker Technology Total Score

17 -14.00 48.00 28.4118 17.12841

Valid N (listwise) 17

(24)

Discussion of Perceived Importance of Clicker Technology

• A mean score of 28.41 was obtained from this DV

• A mean score of 31 would suggest average agreement

(1=agree) on items, conversely, a mean score of -31 would

suggest average disagreement (-1=disagree)

(25)

Limitations

• Effect of IV was 5 weeks, a longer duration may have yielded different results

• Some classes were out of classroom, such as the community where clickers could not be used

• Participants themselves were not randomized into groups but class section instead

• Small sample size

(26)

Conclusion

• Clicker technology was trialed in a clinical (lab) occupational therapy course

• Although statistical significance (p≤0.05) was not obtained for DVs between control and experimental groups, the

experimental group rated clicker technology has having agreeable importance

• Clicker technology may be better suited for lecture style instruction compared to laboratory style instruction

• Qualitative inquiry may be useful

• Utilization of clickers was reliable due to radio frequency

• Students using clickers typically display enthusiasm during in-

class polling

(27)

(28)