Internet and Higher Education

Does teaching presence transfer between MBA teaching environments?

A comparative investigation of instructional design practices associated

with teaching presence

John E. Wisneski

⁎

, Gamze Ozogul

, Barbara A. Bichelmeyer

Indiana University, School of Education, 201 N. Rose Avenue, Room 2222, Bloomington, IN 47405-1006, United States c

Indiana University, School of Education, 201 N. Rose Avenue, Room 2226, Bloomington, IN 47405-1006, United States

a b s t r a c t

a r t i c l e i n f o

Article history:

Accepted 20 November 2014 Available online 26 November 2014 Keywords:

Teaching presence Transfer Online teaching Community of inquiry

As demand for online course delivery in management education continues to grow, it is imperative that individ-ual instructors transition their teaching skills from traditional classroom settings to this new teaching environ-ment. This study examined the ability of the instructor to achieve consistent levels of teaching presence in both the classroom and online, and investigated the extent to which instructors engaged in transfer activities related to teaching practices associated with this phenomenon. Threefindings are of interest. First, instructors who emphasized the tailoring of teaching practices from the classroom achieved greater levels of teaching presence online than in the classroom. Second, instructors spent more effort tailoring teaching practices for use online than in integrating practices used online back in the classroom. Finally, providing words of encourage-ment, affirmation, or validation of student contributions had a positive impact on teaching presence. Practical implications for improving the instructional design of online courses are discussed.

© 2014 Elsevier Inc. All rights reserved.

1. Introduction

Advances in technology have made online education an increasingly common occurrence across most collegiate programs (Zhang, Zhao, Zhou, & Nunamaker, 2004), including business schools where internet-based courses are seen as vital alternatives or supplements to classroom-based courses (Rossin, Ro, Klein, & Guo, 2009). Online courses afford the student greater convenience andflexibility through anytime, anywhere learning (Bocchi, Eastman, & Swift, 2004), thus allowing individuals participating in Masters of Business Administra-tion (MBA) programs the ability to remain in the workforce while completing their degree requirements. In 2013, more than 7 million col-lege students enrolled in online courses in the U.S. (Allen & Seaman, 2014). This represents a 6.1% increase over the previous year. Given the popularity of online U.S. MBA courses, it is no longer a question of whether this delivery model is a viable solution, but rather how can MBA educators adapt their teaching practices to be effective in both teaching environments.

2. Problem statement

In a 2011 survey conducted by the Online Learning Consortium, 58% of the 4564 participating faculty members at institutions of higher education in the U.S. indicated the growth of online learning at their in-stitutionfilled them with“more fear than excitement”(Allen & Seaman, 2012). Despite numerous studies (Arbaugh, Godfrey, Johnson, Pollack, Niendorf, & Wresch, 2009; Brownstein, Brownstein, & Gerlowski, 2008; Topper, 2007), nearly two-thirds of these faculty members believed the learning outcomes for an online course were‘inferior’or ‘somewhat inferior’to learning outcomes for a comparable classroom-based course, and only 40% agreed that online education could be as effective in helping students learn.

Despite this skepticism among faculty, the Online Learning Consor-tium also cited that two-thirds of the 591 chief academic officers surveyed described online learning as a critical component of their insti-tutions' long-term strategy. As online education becomes more promi-nent, administrators must learn to organize and leverage existing assets to offer programs to students in multiple teaching environments. This includes asking more and more faculty to teach either exclusively online, or a mix of both online and classroom instruction simultaneous-ly. However, as recently as 2010, nearly 20% of institutions did not pro-vide any training (not even informal mentoring) for faculty teaching online courses (Allen & Seaman, 2010). Due to this lack of training, faculty members need to apply their own teaching practices from the

⁎ Corresponding author. Tel.: +1 812 855 2705.

E-mail addresses:[email protected](J.E. Wisneski),[email protected] (G. Ozogul),[email protected](B.A. Bichelmeyer).

1

Tel.: +1 812 856 8281. 2

Tel.: +1 812 856 8239.

http://dx.doi.org/10.1016/j.iheduc.2014.11.001 1096-7516/© 2014 Elsevier Inc. All rights reserved.

Contents lists available atScienceDirect

Internet and Higher Education

classroom to the online environment, andfind ways to successfully teach in new environments.

3. Purpose of the study

Given the situation above, the purpose of this study is to explore an instructor's ability to transfer teaching skills between teaching environ-ments. The research questions are as follows:

1. How do instructors' teaching presence levels differ across classroom and online teaching environments?

2. How are the instructional design practices associated with teaching presence transferred between classroom and online teaching environments?

The_findings of this study are intended to help instructors better an-ticipate the challenges in adapting teaching skills across multiple teach-ing environments, and may serve as a basis for helpteach-ing administrators prioritize specific training opportunities to assist in the process. 4. Literature review

4.1. Transferring skills and knowledge

In the broadest sense, the term transfer refers to a simple pattern of learn-it-here, apply-it-there (Perkins & Salomon, 2012). Most instruc-tors seeking to transfer their instructional strategies from the classroom to an online teaching environment will undoubtedly elect for simple routine transfer. Transfer situations can also be opportunities for inven-tion and reorganizainven-tion, however, and are not confined to simply carry-ing forward and applycarry-ing the same knowledge (Lobato, 2012). Adaptive transfer involves not just routine application of knowledge and skills in a new context, but rather suggests adapting and revising prior knowl-edge in the context of the transfer (Schwartz, Chase, & Bransford, 2012). In much the same way that adjustments may be made in the des-tination context, backward transfer refers to the phenomenon where dealing with the new situation may in fact lead to revisions in a prior conception. An example of backward transfer might be the revision of a teaching practice in an online environment that is subsequently applied back in the classroom environment.

The process of transfer can also be the result of the movement of knowledge within the organization that yields an exemplar perfor-mance. In most organizations, a best practice is a relevant example that produces results better than any known alternative. To the extent that stand-out online instructors are willing to share and promote their instructional strategies with their peers, best practices may be de-fined and incorporated into an institutions' online learning platform and serve as a standard template to help new instructors become proficient in the environment. The success of best practice transfer is never guar-anteed. Several barriers to best practice transfer, collectively referred to as the“stickiness”of knowledge transfer, include the recipient's level of prior knowledge, how well the transferred practice is under-stood within the organization, and the recipient's receptiveness to adopting the best practice (Szulanski, 2003). In higher education, where autonomy often overrides collective knowledge sharing, the de-gree to which best practices are shared is unclear.

4.2. Transitioning from classroom to online instruction

Given the more recent emergence of online management education, it is common for instructors to teach in the classroom before being asked to transition to the online teaching environment.Sammons (2003)argued that transitioning from the classroom to online requires the instructor to move from a teacher centered conception of their role to a learner-centered conception.“Becoming an effective teacher in a learner-centered environment will require effort to develop the needed

skills”(p. 396), and asKim and Bonk (2006)stated,“such new roles for online instructors require training and support”(p. 22).Rovai and Downey (2010)found that being attentive to online course design and pedagogy is critical to a successful instructor transition, yet very few insti-tutions provide faculty development for online teaching responsibilities. In an effort to clarify the responsibilities of online instructors,

Goodyear, Salmon, Spector, Steeples, and Tickner (2001)facilitated a workshop involving individuals with experiences in online teaching to construct a competence framework at a conference co-sponsored by the International Board of Standards for Training (ISPTI), the Center for Studies in Advanced Learning Technology (CSALT), Lancaster University, and the Joint Information Systems Committee of the UK universities funding councils. The resulting framework included eight specific roles for online instructors: researcher, assessor, advisor/counselor, process facilitator, content facilitator, designer, manager/administrator, and technologist. Using this framework,Briggs (2005)added co-learner, e-tutor, knowledge expert, and mentor, and sought tofind differences in these competences between classroom and online instructors. In Briggs' sampling of 52 instructors teaching in both environments, a majority suggested that 10 of the 11 roles (technologist excepting) were common in both classroom and online environments, but the responses did indicate that it was the ongoing demands of student interaction in online teaching that differentiated it from the classroom. Difficulties in managing the time commitment of teaching online ap-pear to manifest themselves not only in terms of being available“24/7” to respond to student inquiries, but also in terms of design and organization of the course.Smith, Ferguson, and Caris (2002)

interviewed 22 college instructors teaching both in the classroom and online to compare differences in their experiences. A recurring theme from these interviews suggest greater emphasis needs to be placed in being explicit online with the course procedures and assignments, which may lead to more up-front effort in setting up the course as com-pared to traditional classroom environments.

In addition to these general concerns related to online instruction, there are also a number of challenges specific to MBA education in the literature. For example, while the case method continues to be a com-mon pedagogical technique used to teach the fundamentals of business in the classroom setting, technical barriers often present challenges online for instructors seeking to simulate synchronous face-to-face discussions (Lee, Lee, Liu, Bonk, & Magjuka, 2009; Rollag, 2010). The use of time delimited discussion boards and multi-media case materials may mitigate these challenges, but require additional design and development effort for instructors teaching online. Another common pedagogical approach in MBA education is the use of group work for student assignments. AsKim, Liu, and Bonk (2005) point out, supporting and facilitating group work in online MBA courses requires instructors spend time teaching students how to develop virtual team skills. Related to teamwork, many students are drawn to MBA Programs because of the potential to increase their professional network through interactions with other students. However, asMcGorry (2002)found, online MBA students often identify a lack of networking and interaction as one of the greatest disadvantages of taking online courses. Finally,

Bocchi et al. (2004)found that online faculty's use of frequent feedback promoted MBA students' satisfaction and helped them with application of their newly acquired skills more quickly in a work setting. Given the importance of feedback,Ice, Curtis, Phillips, and Wells (2007)suggest greater emphasis be placed on supplementing text-based instructor feedback with audio feedback which may afford students the ability to understand nuances not provided in written feedback.

Although there is no one recipe for transitioning successfully,Pelz (2004)found that exemplary educators must“strive for presence”in order to be effective (p. 34). In a traditional classroom, presence is observed simply by an instructor's physical attendance. In an online classroom, however, presence is more than just being responsive to forum posts or personal email interactions, and extends to being aware of student's individualized learning needs and responding in an

empathetic and supportive manner (Mandernach, Gonzales, & Garrett, 2006). The online instructor must be an active participant in the course to avoid the perception of being absent (Picciano, 2002).

4.3. Defining teaching presence

The most widely accepted definition of teaching presence comes from the work ofGarrison, Anderson, and Archer (2000), via the Community of Inquiry (CoI) framework. Teaching presence is the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes (Anderson, Rourke, Garrison, & Archer, 2001). Teaching presence is composed of three elements: design and organization, facilitating discourse, and direct instruction.

The design and organization element of teaching presence refers to the planning and design of the structure, process, interaction, and eval-uation of the course (Arbaugh & Hwang, 2006).Swan (2003)found that clear and consistent course structure along with engaged instructors and dynamic discussions have been found to be the most consistent predictors of successful online courses.

The facilitating discourse element is the means by which students are engaged in interaction related to building knowledge from the infor-mation provided in the course instructional materials. This role is asso-ciated with not only facilitating discussion, but also with sharing meaning, identifying areas of agreement or disagreement, and building towards consensus for a common understanding.

The direct instruction element is the most common referent of teaching presence, and considers the intellectual and scholarly leader-ship of the instructor based on their subject matter expertise. It includes the assessment and feedback of student performance, and requires the instructor to recognize student misunderstandings, often directing them to additional materials and/or relevant course interventions.

Teaching presence, as a social phenomenon, wasfirst observed in a computer conferencing context in early forms of online learning (Anderson et al., 2001). Online teaching environments evolved from basic asynchronous communication between student and instructor, to more media rich environments that often include synchronous com-munication and multi-media. Additionally, a hybrid form of learning subsequently materialized that blended virtual teaching environments with elements of residential learning. Across these teaching environ-ment variations, although teaching presence has proven to be highly correlated with both student satisfaction and perceived learning (Garrison & Arbaugh, 2007; Garrison & Cleveland-Innes, 2005),

Rourke and Kanuka (2009)have called for research that moves beyond these self-report measures of perceived learning. In recent years, studies of teaching presence have tended to focus on either validating one or more of its elements (Arbaugh, 2007; Garrison, Anderson, & Archer, 2010; Shea, Hayes, & Vickers, 2010) or on the impact of teaching pres-ence in a multitude of different teaching settings. These settings include formal classroom and online teaching environments (Shea, Li, & Pickett, 2006), coaching (Stenbom, Hrastinski, & Cleveland-Innes, 2012), collab-orative wikis (Daspit & D'Souza, 2012), and onsite facilitation (de la Varre, Keane, & Irvin, 2011).

5. Methods 5.1. Research design

In order to investigate the research questions, a mixed methods approach was implemented. The central premise of mixed methods research is that the use of both quantitative and qualitative approaches, in combination, provides a better understanding of research problems than either approach alone (Creswell & Clark, 2007). This design approach allows the researcher to triangulate the methods by directly comparing statistical results with qualitativefindings for“corroboration and valida-tion purposes” (Creswell & Clark, 2011). In this study, quantitative

assessment of both teaching presence and instructor transfer activities was collected and analyzed concurrently with qualitative data obtained from classroom observations and online course repositories. After analyz-ing the two data sets independently, we sought to interpret the ways in which the two sets of results combined to create a better understanding of teaching presence in response to the purpose of this study.

5.2. Participants

The participants of this study were six instructors from a large Midwestern U.S. business school. This sample was selected from a pool of instructors who taught at least one course in both the face-to-face and online MBA programs during the 2013_–14 academic year. Invi-tations to participate were sent through campus email, and contained details on the purpose of the study, and the approach to data collection. Faculty members who agreed to participate were granted access to all data collected pertaining to their courses, while remaining anonymous and unidentifiable in the subsequent analysis and publication of results. Six instructors volunteered to participate in this study. Each instruc-tor agreed to contribute at least one face-to-face course, and at least one online course for which they were the lead instructor. Each instructor indicated similarities in the learning goals between the online and face-to-face courses included in the study.Arbaugh (2005)studied the effects of subject matter in online learning, and found course conduct variables such as instructor experience and student use of the course website had a much more profound effect on course outcomes than the subject matter of the course. He concluded that“while subject mat-ter may matmat-ter, course conduct probably matmat-ters more”(p. 66). All but one of the courses included in the study had been previously taught by the instructor. The disciplines represented by these six instructors in-cluded Economics (1), Finance (2), Marketing (2), and Management (1). A summary of the demographic profiles, participation level, and teaching experience of the six instructors is provided inTable 1.

After soliciting instructor participants, a subsequent invitation outlining the purpose of the study was sent via campus mail to the students enrolled in the courses included in the study during thefirst week of instruction. The email made clear that the student's participa-tion was voluntary and anonymous, and would not affect their standing in the course in any way. Across the 20 courses included in the survey, 165 students enrolled in online courses, and 186 students enrolled in face-to-face courses agreed to participate. Of the 165 students enrolled in online courses, 119 were male, and 46 were female. Over 69% of the online participants were between 25–34 years old. While all online participants had completed at least one online course, 71 of the 165 on-line students had completed more than ten courses in the onon-line MBA program prior to participating, and 92 of the 165 students reported part-time enrollment status. Of the 186 students enrolled in face-to-face courses, 128 were male, and 58 were female. Over 85% of the class-room participants were between 25–34 years old. One hundred and twentyfive of the 186 classroom students had completed more than ten courses in the in-residence MBA classroom prior to participating, while only 20 participants reported this being their_first in-residence MBA experience. All 186 students reported full-time enrollment status. A summary of the participation rates for students by instructor is pro-vided below inTable 2.

Face-to-face courses in this study met twice a week for seven weeks and each class meeting was ninety minutes in duration. The average class size for the courses was 52, with a range of 24 to 69 students. Face-to-face classrooms were equipped with the following technology: a desktop PC capable of accessing the internet, a DVD player with audio system, an overhead projector, and a video projector capable of displaying images from the desktop PC or instructor laptop. Each face-to-face class was supported by a common learning management system, which each instructor configured to meet the specific learning needs of their class. Typically, the instructor used the learning manage-ment system to share electronic resources, track and manage student

assignment submissions, and broadcast announcements to students outside of the classroom meeting times.

Online courses met at the discretion of the instructor and were con-ducted over a twelve week period. The average class size for courses in-cluded in this study was 41, with a range of 15 to 69 students. Online courses utilized an online learning management system, which served as the primary method of communication between instructor and stu-dent. Instructors conducted synchronous discussions with students that were recorded for the benefit of students who could not attend the pre-determined online meeting time. During these synchronous discussions, although students could see and hear the instructor, the in-structor did not have the ability to visually see students who attended the session. Instructors' interaction with students during these sessions was limited to a text-based chat window provided through the learning management system. The learning management system was also used to promote asynchronous forum discussions, share electronic resources, track and manage student assignments, and broadcast announcements to students enrolled in the online course.

5.3. Procedure

The specific data collection instruments and procedures used in this study are described below.

5.3.1. Teaching presence scale

To measure teaching presence, the Teaching Presence Scale (TPS) was developed by Shea, Li, and Pickett to assess effective instructional design and organization, facilitation of productive discourse, and direct instruction as described in the original Community of Inquiry frame-work (Shea et al., 2006). Reliability analysis was applied to examine the TPS by the original authors of the survey, and revealed satisfactory measures for Cronbach's alpha across all three components of teaching presence; design and organization (.98), facilitated discourse (.97), and direct instruction (.93). A detailed factor analysis was also performed, and two factors were extracted from the model.“Directed Facilitation”, which was a combination of facilitation of discourse and direct instruc-tion in the original model fromGarrison et al. (2000), and“Instructional Design & Organization”accounted for 78.8% of the variability of the teaching presence construct.

In this study, the TPS served as the instrument to capture students' perceptions regarding the instructor's teaching presence. In order to

maintain symmetry of Likert-scale responses, the response choices were modified from the original Likert choices of“Strongly Agree”, “Agree”,“Neutral”,“Strongly Disagree”, and“I choose not to answer this question”to“Strongly Agree”,“Agree”,“Undecided”,“Disagree”, and“Strongly Disagree”. In addition to responses to the TPS, the follow-ing demographic data were collected from student respondents: gender, age, registration status, and number of courses completed to date.

Two weeks prior to the end of each course, the TPS was sent to each participant, and a reminder was sent one week prior to the end of the course. After the data collection was completed, a factor analysis was done, and it confirmed that the same factors found byShea et al. (2006)“Instructional Design & Organization”and“Directed Facilitation” were extracted from the model. These two factors accounted for 88% of the variability of the teaching presence construct.

5.3.2. Teaching practice transfer survey

In order to measure the transfer of teaching practices between the classroom and online teaching environments, an instructor survey was developed asking instructors to characterize their transfer activities for each of the 17-items in the TPS. For each item in the TPS, instructors were asked a simple“yes/no”question regarding whether they applied practices“As-Is”from the classroom in the online environment. If instructors did not apply a strict routine transfer of a given practice, they were further asked to characterize the amount of tailoring, or adap-tation of their practices, on a 5-point Likert-scale ranging from“1—not at all”, to“5—a great deal”. Instructors were also asked whether or not they employed principles of backward transfer by utilizing practices first acquired online back in the traditional classroom. Following their responses to these questions, the survey provided open-ended ques-tions to allow instructors to provide examples of three types of transfer between their online and classroom experiences. The survey is included inAppendix A.

5.3.3. Course repositories

The face-to-face electronic repositories allowed instructors to configure the following features based on course need: calendar, roster, announcements, syllabus, assignment descriptions, lecture notes, grade performance, mail, discussion forums, survey, and electronic reading materials. The online repositories, also configurable by instructor, included all of the above features, as well as recorded synchronous video lessons, synchronous chat, and a report feature that allowed an instructor to observe the frequency and duration of student logins to the site.

The research team inspected the following online repository compo-nents: mail, announcements, discussion forums, lessons, and recorded synchronous video lessons. The unit of analysis was a spoken or written sentence or sentences that when combined, formed a single undivided meaningful entity. Using a rubric aligned with the TPS, the research team coded each reference unit to each of the elements confirmed by a factor analysis of the TPS (Instructional Design & Organization and Facilitated Instruction). The rubric is included inAppendix B. Two researchers each coded the same course repository, and then met to discuss their results. The Pearson correlation for the average of the Table 1

Instructor demographic profile.

Instructor discipline Gender Age range Faculty rank Range of years at institution

Number of classroom coursesa

Number of online coursesa

Economics Male 40–49 Clinical Associate Professor 15+ 1(5) 1 (1)

Finance Male 50–59 Clinical Professor 15+ 1 (10) 2 (10,10)

Finance Male 50–59 Clinical Associate Professor 5–9 2 (12,12) 3 (7,6)

Management Male 50–59 Lecturer 0–4 2 (0,1) 1 (4)

Marketing Male 60+ Clinical Associate Professor 5–9 2 (23) 1 (5)

Marketing Male 50–59 Professor 15+ 2 (10) 2 (3,2)

a

Number of previous conducts of each course included the study in parentheses ().

Table 2

Student participation summary by instructor. Instructor

discipline

Online Classroom

Participants Participation rate Participants Participation rate

Economics 21 .39 51 .25 Finance 1 11 .16 12 .16 Finance 2 28 .33 22 .30 Management 34 .55 33 .36 Marketing 1 25 .42 22 .34 Marketing 2 46 .63 46 .31 Totals 165 .41 186 .28

three elements of teaching presence and all 17 measures in the coding rubric was .99, indicating a high degree of inter-rater reliability for this course. After calibrating on the coding rubric with the results obtained from this course, the researchers completed reviews of all repositories.

5.3.4. Classroom observations

Two of the six instructors invited the lead researcher to observe a face-to-face class session. The unit of analysis was a spoken sentence or sentences that when combined, formed a single undivided meaning-ful entity. During each of the observations, the researcher used the same coding rubric used for online repository review to count the instances of Instructional Design & Organization and Facilitated Instruction that occurred throughout the session. The researcher also documented ex-amples of each instance of observed teaching presence, and reviewed these examples with the rest of the research team.

5.3.5. Data analysis

In order to answer thefirst research question, an independent-samples t-test was conducted to compare the teaching presence scores for all instructors in both the online and classroom courses. To under-stand the possible reasons for any observed differences between the two groups, comparative analysis of the online repositories and class-room observations was performed.

To answer the second research question, mean score variance analysis of instructor's perceptions of adaptive, and backward transfer was conducted for all instructors. Adaptive transfer scores measured the extent to which instructors reported tailoring classroom activities for use online, while backward transfer scores indicated the extent to which instructors apply teaching practicesfirst adopted online back in the classroom. To understand better the nature of their transfer activi-ties, a comparative analysis of the transfer examples that the instructors provided was performed.

6. Results

The next sections present the results of the analyses that were conducted to answer each of our research questions.

6.1. Research question 1: how do instructors' teaching presence levels differ across classroom and online teaching environments?

Table 3shows the mean scores and corresponding standard devia-tions for the six instructors on measures of Teaching Presence, Instruc-tional Design, and Facilitated Instruction ratings. Overall, there was significant difference in Teaching Presence scores, showing higher scores for online courses (M= 4.33,SD= 0.63) than classroom courses, M= 4.04,SD= 0.63;t(349) = 4.23,p= 0.00. Instructional Design

scores revealed significant difference, with online courses being scored higher (M= 4.45,SD= 0.60) than classroom courses,M= 4.13,SD= 0.67;t(349) = 4.68,p= 0.00. Facilitated Instruction scores also re-vealed significant difference, with online courses scoring higher (M= 4.26,SD = .70) than classroom courses, M= 3.99,SD = 0.68;t (349) = 3.67,p= 0.00. When examining Teaching Presence and its el-ements at the instructor level, there was a significant difference in scores for three of the six instructors between their online and class-room courses (seeTable 3).

In order to analyze the online course repositories, individual instruc-tors were grouped according to their student reported teaching pres-ence scores. Group 1 represented those instructors whose teaching presence scores were not statistically different between the online and classroom teaching environments (Economics, Finance 1, and Market-ing 1), while Group 2 represented those instructors whose teachMarket-ing presence scores were statistically higher online than in the classroom (Finance 2, Management, and Marketing 2).

Analysis revealed that instructors from Group 1 took fewer opportu-nities to invoke practices associated with Instructional Design (11% of total observations) than the instructors from Group 2 (19% of total observations). Group 1 relied primarily upon online mail to support their Instructional Design practices (66% of Instructional Design obser-vations), while recorded synchronous lessons were the most common vehicle used to promote practices associated with Facilitated Instruction (65% of total observations). Group 1 did not make use of broadcast an-nouncements, and reserved discussion forums for practices associated with Facilitated Instruction.

Group 2 instructors also relied primarily on the use of online mail to support their Instructional Design practices (65% of Instructional Design observations), but employed a more balance usage of the online repos-itory features to support practices associated with Facilitated Instruc-tion. Group 2 relied upon recorded synchronous lessons (41%), online mail (30%), discussion forums (28%), as well as broadcast announce-ments (1%) to support practices associated with Facilitated Instruction. In order to better understand the nature of online recorded synchro-nous lessons, which served as the primary feature to support teaching practices associated with Facilitated Instruction across all instructors, the research team sampled one recorded lesson from the fourth or fifth week of each of the ten online courses included in the study. Re-sults show that instructors from Group 1 relied heavily upon presenting course material in the observed recordings (65% of total Group 1 obser-vations). Group 2, however, spent considerably more time in the observed recordings providing words of encouragement, affirmation, or validation of student contributions (23% of total Group 2 observa-tions) and providing opportunities for students to“think out loud”or explore new ideas (25% of total Group 2 observations).

Finally, in order to compare thefindings from the analysis of the online course repository with face-to-face performance, the research

Table 3

Mean scores and corresponding standard deviations on teaching presence, instructional design, and facilitated instruction for classroom and online courses by instructor. Type of measure

Teaching presence Instructional design Facilitated instruction

Classroom Online Classroom Online Classroom Online

Instructor M SD M SD M SD M SD M SD M SD Economics 4.21 0.58 4.47 0.56 4.25 0.63 4.60 0.49 4.19 0.60 4.37 0.64 Finance 1 3.71 0.63 3.33 0.92 3.67 0.69 3.71 0.90 3.73 0.66 3.14 1.01 Finance 2 3.65b 0.58 4.34b 0.56 3.82b 0.49 4.46b 0.49 3.57b 0.72 4.28b 0.66 Management 3.79b 0.58 4.27b 0.45 3.80b 0.62 4.44b 0.47 3.79a 0.65 4.18a 0.48 Marketing 1 4.06 0.48 4.08 0.72 4.26 0.55 4.15 0.72 3.95 0.51 4.04 0.75 Marketing 2 4.29b 0.66 4.67b 0.31 4.44a 0.67 4.69a 0.38 4.21b 0.69 4.66b 0.33 All Instructors 4.04b 0.63 4.33b 0.63 4.13b 0.67 4.45b 0.60 3.99b 0.68 4.26b 0.70 a

Significant at alphab.05. b

team observed two class sessions from“Marketing 1”, an instructor from Group 1, and“Finance 2”, an instructor from Group 2. Results show the instructor from Group 1 took more opportunities to provide words of encouragement, affirmation, or validation of student contribu-tions (24% of total observacontribu-tions) compared to the instructor from Group 2 (9% of total observations).

6.2. Research question 2: how are the practices associated with teaching presence transferred between classroom and online teaching environments?

Overall, there was evidence that instructors did engage in both adaptive and backward transfer activities of those practices associated with teaching presence. Adaptive transfer scores for both Instructional Design (M= 3.13) and Facilitated Instruction (M= 3.08) were higher than the backward transfer scores for Instructional Design (M= 1.80) and Facilitated Instruction (M= 2.05), indicating that instructors reported engaging in higher levels of adaptation of practices for the online environment than in adopting online practices back in the classroom.

To further the analysis, the same instructor grouping structure was used as described inSection 6.1. Comparative analysis revealed that Group 2 reported higher levels of adaptive and backward transfer activ-ities than Group 1 for both Instructional Design and Facilitated Instruc-tion. A summary of instructor transfer activities for practices associated with teaching presence by group and overall is provided inTable 4.

When instructors were asked for examples with respect to routine transfer, instructors from Group 1 referenced both Instructional Design and Facilitated Instruction practices as candidates for strict reuse with-out tailoring. Both instructor“Economics”and“Finance 1”stated using the same PowerPoint slides for lecture purposes, as well as practice problems and multiple-choice examinations. Instructor“Marketing 1” referred to Facilitated Instruction practices when asked about routine transfer. He stated,“I pose the same specific questions for case discus-sions that lead the class to logical concludiscus-sions and the same learning op-portunity.”With instructors from Group 2, however, reported routine transfer was limited to Instructional Design practices communicated through their course syllabus. Two of the three instructors from this group mentioned the course description, course learning outcomes, and course schedules as elements for reuse between environments. Instructor“Management”indicated, “The way I communicate key dates is becoming consistent between the two formats, with the way I drive explicit dates and detailed calendars for students.”Instructor “Finance 2”was the only instructor from this group who moved beyond the syllabus, and reported reuse opportunities with“some of the read-ings from the textbook”.

When asked to describe adaptive transfer activities, instructors from both groups all commonly referred to teaching practices that lead to Facilitated Instruction. Two instructors from Group 1 directly commented on the difficulty with leading group discussions online, and how they had to tailor approaches that they relied upon in the classroom. Instruc-tor“Finance 1”expressed the differences between environments as:

“I can't have a case discussion online involving the same number of people that you have in the classroom. I have just not found a way to control theflow of comments online. In class, you call on people

with their hand up, but for me this doesn't work as well online if there are lot participants.”

Instructor“Marketing 1”expressed a similar sentiment:

“I spend less time trying to stimulate class discussion online because generating a stimulating discussion online is extremely time con-suming, particularly with a large class.”

Instructors from Group 2 reported similar difficulties, but offered several adaptive solutions. For example, instructor“Management” shared the following tailored approach:

“In the classroom, we have the ability to have small group discussion on a regular basis. Online, I have adapted this to involve the forum tool_—driving discussion prior to a live session. Additionally, while I use video in the classroom to provide examples, I can pre-assign these videos online. The goal is to minimize one way communication during online sessions so that more discussion can occur._”

Instructors from Group 2 also expressed awareness that students participating in online courses were not confined to standard meeting times as prescribed in face-to-face classes. For example“Finance 2” expressed a need to tailor the interaction with online students:

“I try to respond very quickly to any emails from the online students, even over the weekend and in the evenings because I realize this is the time they tend to be working on course materials.”

Finally, instructors from Group 2 also discussed the need to tailor practices based on the limitations of the online teaching environment. For example, instructor“Finance 2”reported:

“I do try to recognize each of the comments made and repeat them for the rest of the class, as some students do not always have good connections that enable them to see all the typed comments in the Chat window.”

Instructor“Marketing 2”reported limiting the use of supplemental analytic tools, and reserved the use of such tools for classroom instruc-tion only due to access restricinstruc-tions online.

When asked about backward transfer, or the opportunity to adopt teaching practicesfirst employed online back in the classroom environ-ment, instructors from Group 1 described the use of Instructional Design assetsfirst developed online as supplemental aids back in the class-room. For example, instructor_“Finance 1_”shared recorded online lec-tures with classroom participants as an optional exercise. Similarly, instructor“Marketing 1”provided classroom participants with written case explanations originally developed online, to allow for added struc-ture when summarizing a class session.

Instructors in Group 2 also used recorded online sessions back in the classroom. Instructor“Management”was the only respondent to refer-ence attempts to transfer practices that lead to Facilitated Instruction first conceived online back in the classroom. Instructor“Management” used forums to pose questions to students in advance of an online ses-sion and subsequently replicated this practice in face-to-face classes to

Table 4

Adaptive and backward transfer of practices associated with Teaching Presence between classroom and online teaching environments. Instructor teaching presence performance groups Instructional design

adaptive transfer Facilitated instruction adaptive transfer Instructional design backward transfer Facilitated instruction backward transfer

Group 1: Consistent between environments 2.67 1.80 1.07 1.50

Group 2: Higher teaching presence online 3.60 4.37 2.53 2.60

stimulate thinking prior to the class meeting. Instructor“Management” also used a procedure to build a sense of teaming among class participants:

“I_first instituted a team charter policy as part of several team development modules online. The course requires exceptional team commitment, and I do not collect or review these documents unless a team is having difficulty, at which point we use the charter as a starting point to resolve issues. It has worked so well, that I have begun to institute the same policy in my face-to-face classes.”

7. Discussion 7.1. Overall results

This study explored the transfer of teaching presence between classroom and online teaching environments, and whether instructors were able to achieve consistent presence in these environments. Three of the six instructors included in this study achieved statistically signif-icant higher levels of teaching presence online than in the classroom. The fact that these three instructors came from different business school disciplines (Management, Marketing, and Finance) supports the earlier findings ofArbaugh (2005)who found that regardless of subject matter, instructors have the ability to achieve high levels of teaching presence in the online environment, as long as they focus on instructional design and course conduct. Arbaugh found instructor experience with online courses and student use of the course site to be the two most consistent-ly significant course conduct variables. In this study, several observa-tions from a review of the instructor's online course repositories and classroom sessions may provide further explanations for the reported difference among these three instructors. First, instructors who achieved higher levels of teaching presence online employed a more balanced use of the features of the online software to support teaching practices associated with Facilitated Instruction than instructors who showed no statistical variance across the teaching environments. By en-gaging students in dialog across mail, synchronous class discussions, and asynchronous forums, it is possible that students felt these instruc-tors were more interactive in the course than instrucinstruc-tors who relied more heavily on the use of synchronous class discussions alone. Second-ly, thefindings indicate that instructors were able to achieve higher levels of teaching presence online when taking more opportunities to prompt students to“think aloud”, followed by acknowledgement of their contributions through words of encouragement, affirmation, or validation. In contrast, the instructors who did not achieve higher levels of teaching presence online compared to their face-to-face courses, tended to leverage synchronous discussions online more for presenta-tion of course material than for engaging students in dialog.

It is also important to note thatfive of the six instructors included in this survey achieved higher reported teaching presence scores online than in the classroom. There might be several contributing factors related to student demographics that may provide an explanation for thisfinding. First, 55% of the online students in this study reported part-time enrollment status, indicating that a majority of these students were working to complete their coursework while also maintaining full-time employment status. In comparison, 100% of the classroom students included in this study reported full-time enrollment status. Full-time students have chosen to remove themselves from the work-force in pursuit of their MBA. As a result, full-time students may place a higher priority on the educational opportunity, and as a result may have higher expectations regarding the quality of instruction they receive while in the program. This may lead classroom survey respon-dents to be less forgiving of instructors who fall short of their expecta-tions, and all else being equal, may drive more pessimistic responses to the TPS than those provided by online students. Additionally, online

students in this survey were slightly older than classroom students, and as a result, may have more years of work experience than their classroom counterparts. AsAnderson et al. (2001)pointed out, it is both the instructor and student participants who contribute to a teach-ing environment with strong teachteach-ing presence. Online students in this study may have benefited from the additional work experience of their classmates, and may have been able to rely more upon the contributions of their peers than classroom students in achieving higher levels of teaching presence.

This study also examined the extent to which instructors employ transfer activities of those teaching practices that are associated with teaching presence. Thefindings revealed that emphasizing the adap-tation of classroom practices associated with both Instructional De-sign and Facilitated Instruction might have led to achieving higher levels of teaching presence online. Routine transfer of instructional design assets such as the course description and learning outcomes was common among this study's participants. Instructors expressed similarities in both the content of these assets across environments, and the ways they chose to communicate with students, which was mostly through the use of broadcast messaging. However, instruc-tors did not report focusing on backward transfer. Instrucinstruc-tors who did make attempts to transfer teaching practicesfirst conceived on-line back in the classroom did not achieve higher levels of teaching presence than they did online.

7.2. Implications for MBA instructional design

This study has four important implications for management educa-tors who teach in both traditional classrooms and online. First, instruc-tors should not think of instructional design as being specific to the teaching environment the course will be delivered. Results show that actively engaging in the adaptation of teaching practices, as opposed to either strictly copying or“starting from scratch”is associated with achieving higher levels of teaching presence. This includes thinking not only about how one might tailor practices from the classroom for use online, but also how practicesfirst implemented online may enhance the quality of instruction back in the classroom.

Second, results show that instructors who engage students in the communicative process of learning achieved higher levels of teaching presence online. Whether it is framing questions to start the discussion, or directly prompting students to engage in dialog along the way, hav-ing students contributhav-ing to the conversation online was a common characteristic of instructors who achieved statistically significant higher teaching presence scores. Simply presenting course material, and turn-ing the online class meetturn-ing time into an instructor-led lecture, does not provide a teaching environment that contributes to strong teaching presence. Therefore, instructors would be well advised to make instruc-tional design choices that minimize lecture time and maximize the time spent engaging students during synchronous discussions. This can be done by providing lecture materials in advance of the online meeting, or referring to lecture materials only when prompted by discussion from the student group.

Third, while facilitating instruction is certainly part of the instructor's role in the online environment, the presentedfindings high-light the importance of communicating the instructional design to on-line participants. Using synchronous meeting time to review the schedule for the course, answer questions pertaining to student assign-ments, and communicate key components of the structure of the course contribute to building student perceptions of teaching presence. Relying solely upon direct mail to communicate components of the course design might leave the impression that the instructor is treating the learning opportunity as more of a correspondence course, rather than the truly collaborative environment online students seek.

Finally, irrespective of the type of teaching environment, thefindings show strong evidence that instructors who take the time to acknowl-edge the contributions of students through words of encouragement,

affirmation, or validation can achieve high levels of teaching presence. Facilitating instruction both in the classroom and online requires the in-structor to foster an environment where student contributions are not just received, but rather are celebrated and recognized for the collabora-tive learning potential they provide. In fact, enabling a collaboracollabora-tive learning environment not only increases teaching presence, but also supports the social and cognitive presence required to build a true Com-munity of Inquiry. Acknowledgement of student contributions can be provided immediately in the moment with simple words of encourage-ment, or more formally in a subsequent direct message to individual stu-dents who made strong contributions.

7.3. Limitations

While this study contributed important _findings regarding an instructor's ability to achieve consistent levels of teaching presence across face-to-face and online teaching environments, and the extent to which the practices associated with teaching presence are transferred across these environments, there are limitations that should be noted. First, all of the data were gathered from MBA students at a single institution. Second, because our instructor candidate pool was limited to those instructors teaching simultaneously online and in the class-room, the sample size for instructor participants (N = 6) prevents our ability to generalize thesefindings beyond the scope of this study. We recommend that future research explores these phenomena in other university contexts, with a greater pool of participant instructors to assess the generalizability of ourfindings. Finally, use of self-report data to assess instructor transfer activities may not be a reliable measure of transfer. We recommend further investigations of transfer activities that include instructor interviews, course artifact analysis and observa-tions during all phases of the instructional design process, which may

provide an even deeper understanding of the teaching presence phenomena.

7.4. Future research

Future research on teaching presence and the transfer of practices associated with the phenomena could take several possible directions. It would be appropriate to conduct follow-on studies to explore the re-lationship between an instructor's transfer activities and achieved levels of teaching presence. Further correlational studies may reveal several patterns of adaptation for practices that lead to increased levels of teaching presence based on the instructor's experience level. Similarly, research that explores the impediments to transfer of practices associat-ed with teaching presence may help administrators target specific practices in training materials designed to help instructors be successful in a new teaching environment.

8. Conclusion

This study investigated the extent to which instructors' teaching presence levels differed across the classroom and online environments, and whether or not the practices associated with teaching presence were transferred between these environments. Instructors who achieved statistically significant higher levels of teaching presence on-line actively sought opportunities to adapt teaching practices from their classroom experiences for their online instruction, used the full complement of online course repository software features to facilitate instruction, and spent time communicating their instructional design to online participants via announcements and direct messages. These findings provide important implications for instructors seeking to broaden their teaching skills across an increasingly complex teaching environment landscape.

Appendix A. Instructor survey

Element Use As-Is from the

classroom online

Tailor from the classroom before using online

Utilize practices from online back in the classroom

Yes No Not at all Little Somewhat Much A great deal Not at all Little Somewhat Much A great deal Instructional design and organization

Communicating important course goals Communicating important course topics Providing clear instructions on how to participate in course learning activities

Communicating important due dates/time frames for learning objectives

Helping students take advantage of the learning environment to assist their learning

Helping students understand and practice the kinds of behavior acceptable in the learning environment Facilitating discourse

Helping students identify areas of agreement and disagreement on course topics

Guiding students towards understanding course topics

Acknowledging student participation in the course Encouraging students to explore new concepts in the course

Helping keep students engaged and participating in productive dialog

Helping students keep on task Direct instruction

Presenting content or questions to help students learn

Helping to focus discussion on relevant issues Providing explanatory feedback

Revising students thinking to help them learn Providing useful information from a variety of sources

In order for us to get a better description of your responses above, to the extent that you utilized teaching practices from the classroom“As-Is” online, could you please provide 1 or 2 examples of either artifacts or techniques that you were able to apply without modification?

In order for us to get a better description of your responses above, to the extent that you tailored teaching practices for use online, could you please provide 1 or 2 examples of either artifacts or techniques and how you tailored them?

In order for us to get a better description of your responses above, to the extent that you utilized teaching practicesfirst applied online back in the classroom, could you please provide 1 or 2 examples of either artifacts or techniques and how you utilized them?

Appendix B. Teaching presence coding rubric

Evidence of instructional design & organization Number of occurrences Spoken or written examples Referencing course learning outcomes

Referencing course learning topics, reading assignments, and/or deliverables Referencing instructions for course assignments

Referencing the course schedule

Referencing instructions on how to participate in course discussions

Referencing instructions on course participation expectations, including guidance on quality, quantity, and/or tone of contributions

Evidence of facilitated instruction Number of occurrences Spoken or written examples

Consensus building between divergent views of course topics Exposing areas of agreement on course topics among participants Framing questions to start course discussion

Providing words of encouragement, affirmation, or validation of student contributions Providing opportunities for students to“think out loud”or explore new ideas Prompting students to engage and participate in dialog

Presenting course material

Focusing course discussion on relevant topics Providing explanatory feedback

Correcting misunderstandings of course material

References

Allen, I. E., & Seaman, J. (2010).Class differences: Online education in the United States, 2010. Needham, MA: Online Learning Consortium.

Allen, I. E., & Seaman, J. (2012).Conflicted: Faculty and online education, 2012.Needham, MA: Online Learning Consortium.

Allen, I. E., & Seaman, J. (2014).Grade change: Tracking online education in the United States, 2013.Needham, MA: Online Learning Consortium.

Anderson, T., Rourke, L., Garrison, D., & Archer, W. (2001).Assessing teaching presence in a computer conferencing context.Journal of Asynchronous Learning Networks,5(2), 1–17. Arbaugh, J. B. (2005).How much does“subject matter”matter? A study of disciplinary effects in on-line MBA courses.The Academy of Management Learning and Education, 4(1), 57–73.

Arbaugh, J. B. (2007).An empirical verification of the community of inquiry framework. Journal of Asynchronous Learning Networks,11(1), 73–85.

Arbaugh, J. B., & Hwang, A. (2006).Does“teaching presence”exist in online MBA courses? The Internet and Higher Education,9, 9–21.

Arbaugh, J. B., Godfrey, M. R., Johnson, M., Pollack, B. L., Niendorf, B., & Wresch, W. (2009). Research in online and blended learning in the business disciplines: Keyfindings and possible future directions.The Internet and Higher Education,12(2), 71–87. Bocchi, J., Eastman, J. K., & Swift, C. O. (2004).Retaining the online learner: Profile of

students in an online MBA program and implications for teaching them.Journal of Education for Business,79(4), 245–253.

Briggs, S. (2005).Changing roles and competencies of academics.Active Learning in Higher Education,6(3), 256–268.

Brownstein, B., Brownstein, D., & Gerlowski, D. (2008).Web-based vs. face-to-face MBA classes: A comparative study.Journal of College Teaching and Learning,5(11), 41–48. Creswell, J. W., & Clark, V. L. P. (2007).Designing and conducting mixed methods research.

Thousand Oaks, CA: Sage publications.

Creswell, J. W., & Clark, V. L. P. (2011).Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage publications.

Daspit, J. J., & D'Souza, D. E. (2012).Using the community of inquiry framework to introduce wiki environments in blended-learning pedagogies: Evidence from a business capstone course.The Academy of Management Learning and Education,11(4), 666–683. de la Varre, C., Keane, J., & Irvin, M. J. (2011).Dual perspectives on the contribution of

on-site facilitators to teaching presence in a blended learning environment.International Journal of E-Learning and Distance Education,25(3).

Garrison, D., Anderson, T., & Archer, W. (2000).Critical inquiry in a text-based environment: Computer conferencing in higher education.The Internet and Higher Education,2, 87–105. Garrison, D. R., Anderson, T., & Archer, W. (2010).Thefirst decade of the community of inquiry framework: A retrospective.The Internet and Higher Education,13(1), 5–9. Garrison, D. R., & Arbaugh, J. B. (2007).Researching the community of inquiry framework:

Review, issues, and future directions.The Internet and Higher Education,10(3), 157–172. Garrison, D. R., & Cleveland-Innes, M. (2005).Facilitating cognitive presence in online learning: Interaction is not enough.American Journal of Distance Education,19(3), 133–148.

Goodyear, P., Salmon, G., Spector, J. M., Steeples, C., & Tickner, S. (2001).Competences for online teaching: A special report.Educational Technology Research and Development, 49(1), 65–72.

Ice, P., Curtis, R., Phillips, P., & Wells, J. (2007).Using asynchronous audio feedback to en-hance teaching presence and students' sense of community.Journal of Asynchronous Learning Networks,11(2), 3–25.

Kim, K., & Bonk, C. J. (2006).The future of online teaching and learning in higher education: The survey says.Educause Quarterly,29(4), 22.

Kim, K. J., Liu, S., & Bonk, C. J. (2005).Online MBA students' perceptions of online learning: Benefits, challenges, and suggestions.The Internet and Higher Education,8(4), 335–344.

Lee, S., Lee, J., Liu, X., Bonk, C. J., & Magjuka, R. J. (2009).A review of case-based learning practices in an online MBA program: A program-level case study.Educational Technology and Society,12(3), 178–190.

Lobato, J. (2012).The actor-oriented transfer perspective and its contributions to educational research and practice.Educational Psychologist,47(3), 232–247. Mandernach, B. J., Gonzales, R. M., & Garrett, A. L. (2006).An examination of online

instructor presence via threaded discussion participation.MERLOT Journal of Online Learning and Teaching,2(4), 248–260.

McGorry, S. Y. (2002).Online, but on target? Internet-based MBA courses: A case study. The Internet and Higher Education,5(2), 167–175.

Pelz, B. (2004).Three principles of effective online pedagogy.Journal of Asynchronous Learning Network,8(3), 33–46.

Perkins, D., & Salomon, G. (2012).Knowledge to go: A motivational and dispositional view of transfer.Educational Psychologist,47(3), 248–258.

Picciano, A. G. (2002).Beyond student perceptions: Issues of interaction, presence, and performance in an online course.Journal of Asynchronous Learning Networks,6(1), 21–40.

Rollag, K. (2010).Teaching business cases online through discussion boards: Strategies and best practices.Journal of Management Education,34(4), 499–526.

Rossin, D., Ro, Y. K., Klein, B. D., & Guo, Y. M. (2009).The effects offlow on learning outcomes in an online information management course.Journal of Information Systems Education,20(1), 87–98.

Rourke, L., & Kanuka, H. (2009).Learning in communities of inquiry: A review of the literature (Winner 2009 Best Research Article Award).International Journal of E-Learning and Distance Education,23(1), 19–48.

Rovai, A. P., & Downey, J. R. (2010).Why some distance education programs fail while others succeed in a global environment.The Internet and Higher Education,13(3), 141–147.

Sammons, M. (2003).Exploring the new conception of teaching and learning in distance education. In M. G. Moore, & W. G. Anderson (Eds.),Handbook of Distance Education (pp. 387–400). Mahwah, N.J.: Lawrence Erlbaum Associates.

Schwartz, D. L., Chase, C. C., & Bransford, J. D. (2012).Resisting overzealous transfer: Coordinating previously successful routines with needs for new learning.Educational Psychologist,47(3), 204–214.

Shea, P., Hayes, S., & Vickers, J. (2010).Online instructional effort measured through the lens of teaching presence in the community of inquiry framework: A re-examination of measures and approach.The International Review of Research in Open and Distance Learning,11(3), 127–154.

Shea, P., Li, C., & Pickett, A. (2006).A study of teaching presence and student sense of learning community in fully online and web-enhanced college courses.The Internet and Higher Education,9(3), 175–190.

Smith, G. G., Ferguson, D., & Caris, M. (2002).Teaching on-line versus face-to-face.Journal of Educational Technology Systems,30(4), 337–364.

Stenbom, S., Hrastinski, S., & Cleveland-Innes, M. (2012).Student–student online coaching as a relationship of inquiry: An exploratory study from the coach perspec-tive.Journal of Asynchronous Learning Networks,16(5), 37–48.

Swan, K. (2003).Learning effectiveness online: What the research tells us.Elements of Quality Online Education, Practice and Direction,4, 13–47.

Szulanski, G. (2003).Sticky knowledge: Barriers to knowing in thefirm.Thousand Oaks, CA: Sage Publications.

Topper, A. (2007).Are they the same? Comparing the instructional quality of online and face‐to‐face graduate education courses.Assessment and Evaluation in Higher Education,32(6), 681–691.

Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F., Jr. (2004).Can e-learning replace classroom learning?Communications of the ACM,47(5), 75–79.