Effects Of Web-Based Interactive Modules On Engineering Students’ Learning Motivations

Effects Of Web-Based Interactive

Modules On Engineering Students’

Learning Motivations

Yunjun Xu, University of Central Florida, USA Nina Orlovskaya, University of Central Florida, USA

Mingming Zhou, University of Central Florida, USA

ABSTRACT

The purpose of this study is to assess the impact of a newly developed modules, Interactive Web-Based Visualization Tools for Gluing Undergraduate Fuel Cell Systems Courses system (IGLU), on learning motivations of engineering students using two samples (n1=144 and n2=135) from senior engineering classes. The multivariate analysis results

revealed that the participants had a significant increase in their learning motivation after the treatment with the IGLU modules. This result was cross-validated with the two samples, in which the motivation mean posttest scores are significantly higher than the mean pretest scores, systematically (Sample 1: the mean score is increased by 2.09 [.32, 3.87]points, p = .021; Sample 2: the mean score is increased by 1.38 [.14, 2.61] points, p = .029). With the use of instructional technology prevailing in current university courses, the education initiative of the IGLUsystem and the assessment of its impact on student learning motivation provide us information to improve the modules to serve a more diverse student body. It will greatly help the development of engineering educational curriculum. With regards to the statistical inference, it is desirable to conduct further studies with a quasi-experiment control group design to assess the program effect focusing on student learning and its associations with student learning motivations and learning styles.

Keywords: Motivation; Technology; Instruction: Engineering Education

INTRODUCTION

quasi-experimental research study was conducted to assess the effectiveness of a newly developed web-based modules on learning motivations and outcomes of undergraduate engineering students. The participants were from the college of engineering at a large, urban research university in the southeastern United States. To meet urgent industry needs for engineers who are educated in renewables energy technologies with system level thinking, the researchers of this project created a program, named Interactive Web-Based Visualization Tools for Gluing Undergraduate Fuel Cell Systems Courses (IGLU). The interactive and interconnected visualization modules in the software were created to enhance undergraduate students’ motivations toward learning engineering courses. One of the project objectives is to assess the impact of IGLU program on learning motivations of engineering students.

Following the introduction, we have included sections on the background of study, supporting literature on instructional technology as the means to promote student learning motivation, summary of the development of the overall study including the research design, instrumentation, data analysis results, discussion, and conclusions of the study.

BACKGROUND

To answer the call of the National Science Foundation (NSF) for transforming undergraduate education in Science, Technology, Engineering, and Mathematics (STEM) program to improve the quality of STEM education for all undergraduate students, the research team created the IGLU program under the NSF grant support that started in 2013. The focus of the IGLU program was on transforming undergraduate engineering education through bringing about potential widespread adoption of classroom practices using instructional technology.

The program supports efforts to create, adapt, and disseminate new learning materials and teaching strategies to reflect the advances both in STEM disciplines, and in what is known about teaching and learning practices. The project involved implementation of the educational innovation, and evaluation of the innovation through quasi-experimental design to study student motivation and learning.

Specifically, in response to promoting undergraduate education in engineering courses, the topic on Fuel Cells is selected as the ideal course for designing the IGLU project. As we know, due to limited petroleum resources and environmental considerations, research investigation and development of different alternative energy sources have been a hot topic in recent years. The fuel cell is an electrochemical conversion device that could use hydrogen or other hydrocarbon fuels to produce electricity, with water as the byproduct (if hydrogen is used as the fuel), which has been increasingly used in automobiles, hospitals, hotels, power plants, aircrafts, and shuttles, just to name a few. However, the progress in fuel cell systems as well as their efficiency and robustness depend not only on the current quality of science and engineering research and development, but also on the quality and preparedness of our next generation students, who may be involved in fuel cells research and development in the future. Currently, there are two major challenges facing the educators in Engineering: the decreasing number of student enrollment and graduation rates (Yarmolenko, et al., 2009).

Figure 1. A few screenshots of animations and movies in the educational software. (A) “How does a fuel cell work?” part of the introduction module; (B) screenshot of the “Electro-osmotic drag” animation; (C) design of tubular fuel cell stacks; (D) hybrid system of batteries and fuel cells in unmanned aerial vehicles; (E) equation used in reaction kinetics science of fuel cells; and (F) picture and video showing the fuel cell production process.

B

A C

The trend of the student enrollment and graduation rate is not only related to the level of complexity associated with science and engineering subjects, but also the lack of proper tools to improve students’ motivations to learn and make students aware about the possible direction of their future careers. Therefore, there is an urgent need for engineering educators to develop innovative teaching approaches to better suit the learning styles of the new generation of technologically savvy engineering students, to arouse their interest in the engineering courses.

At the same time, for those students who are already in engineering schools across the US, there is a lack of understanding of the “bigger picture” for system concepts. NSB (2015) stated, “U.S. engineers need to excel at high-level design, systems integration, innovation, and leadership” (p.14). However, engineering courses such as fuel cell related courses are sometimes technically too intensive for the undergraduate level students and could be too focused on particular aspects of the fuel cell system rather than giving them the broad picture which helps them develop system-level thinking. This significantly impairs student learning motivation for engineering courses and their retention in the engineering programs.

The key question that the IGLU addresses is “How to motivate our next generation engineers in learning techniques used in fuel cell systems and grow them with system level thinking?” Our answer is: to achieve the goal through the interactive and technically interconnected visualization tools augmented with hardware demonstrations, which will have sufficient level of technical sophistication for undergraduate students, yet are clear and precise enough to keep them interested in the subject and be actively involved in learning. We believe that such education is “learning based” instead of “teaching based”, and it will help motivate the students to stay in the program and excel.

Figure 2. From left to right, the figure shows the main interface of the software (A), followed by the components of the Fuel Cell System module (B) and finally the Fuel Cell Processing Sub-system (C).

Furthermore, recent studies have shown that interactive visualization tools can improve the educational experience for engineering and computer science students (e.g., Messner & Horman, 2003; Mohler, 2011; Nguyen & Khoo, 2009). Therefore, web-based interactive instruments are expected to play a significant role in attracting students to learn and to complete their courses successfully. However, to date, there are very limited resources available specifically for engineering education, such as tools that are suitable for fuel cell related undergraduate courses. Therefore, IGLU specifically aims at building an essential link that is currently missing: the link between understanding fuel cell science and technology to understanding of the overall fuel cell system.

Taking into consideration all these points in the IGLU project (Aman, Xu, Bai, Orlovskaya, 2015), we focused the learning modules to cover all aspects of a fuel cell system and broke it down into five main modules: “introduction”, “applications”, “fuel cell system”, “cell level”, and “fuel cell science”. These modules are highly interactive and interconnected for improving college engineering students’ understanding of fuel cells knowledge at various levels. Based on Felder and Silverman (1988), “Most people of college age and older are visual [learners]” (p. 676), we designed the modules with active animations to transform the abstract concepts into a composition of small parts

that not only build on top of previous concepts linearly but are also inter-connected in a way to give students the freedom the navigate the topics as they see fit.

We also made the modules in a way that they directly map the student learning outcomes to Bloom’s taxonomy (Anderson & Krathwohl, 2001). We used ARCS model (Keller, 1987) to guide us to design the visualized and interactive modules for increasing student motivation in learning fuel cell knowledge through the interactive diagrams such as presented in Figure 1 depicting the sub-system and system-level components of the fuel cell system (Bai, Xu, Aman & Orlovskaya, 2014). As mentioned earlier, the entire IGLU software is comprised of five main modules, each module addressing the concepts related to fuel cell systems at different levels. Each module in turn is composed of various sub-modules. For example, the Introduction module includes topics concerning the definition of a fuel cell, working principles, types and comparisons of fuel cell systems to other energy conversion technologies; that gives the student a quick overview of what they need to know about fuel cells. Applications module goes into details about how and where fuel cell systems are currently being used and the advantages of doing so. The third module, the Fuel Cell Systems module, talks about the different sub-systems that make up a complete fuel cell system. Figure 2 shows the one of the components of the Fuel Cell System module along with the main software interface. The Cell Level module discusses the components of a single cell, how they are produced and other key concepts. Finally the Fuel Cell Science module deals with all the fundamental science behind the working of a fuel cell such as thermodynamics, reaction kinetics, charge and mass transport in the cell. The user interface is simple in design and is made of clickable links connected to different parts of the software. Links throughout the software enable the user to jump from one topic to another based on their interest or learning pattern. The software can be accessed via any web browser through a specific server or a website. In addition to animations, there are also videos and some of the animations are accompanied by audio narratives. The dynamic components of the software are designed to keep the students engaged and motivated to learn, at the same time keeping each animation/video short enough will demand less attention span from the student which could help in higher retention. The intention is that the software is used as a supplementary tool to enhance student learning and as such could be used in the classroom to further illustrate concepts or as a bonus outside-the-classroom learning tool or even the basis of a flipped classroom method.

The purpose of this study is to assess the effects of a newly developed web-based IGLU system on learning motivations of engineering students. With the online learning prevailing in current university courses, we expect that the education initiative of the IGLUsystem and assessment of its impact on student learning motivation will provide information for improving the modules to serve a more diverse student body. It will greatly help curriculum development for engineering education.

FRAMEWORK AND SUPPORTING LITERATURE

Motivation and Learning Interest

How to motivate student to learn has long been recognized an important question for educators and researchers. In the vast literature of motivation theory, it is well known that intrinsic and extrinsic motivation play an essential role in developmental and educational practices (Rayan & Deci, 2000). Intrinsic motivation refers to the motivation derived from personal interest which generates from inside, while extrinsic motivation refers to doing things for specific outcomes which arises from outside of an individual (Deci & Ryan, 1975). Students’ intrinsic motivation usually leads them to engage in learning in different ways and persists in solving problems, while students with extrinsic motivation may attempt to finish the tasks as a goal rather than enjoying the process of learning; as a result, students may begin a task actively but give-up when they come across difficulties (Deci & Ryan, 1975). However, the important aspect of the relationship between the two distinguished types of motivations is that extrinsic motivation can reinforce intrinsically motivated behaviors, which is mediated through stimulating personal interests (Deci, Koestner, & Ryan, 1999). The mediation approach has promoted the educational practices for educators to explore strategies to motivate students to learn through triggering their learning interests.

and leading theory orientating researchers and educators in this field for about 50 years until 1980s. The major approach of Dewey (1913) is that if teaching can secure interest in their given set of factors, such as format and materials, the students are sure to direct their motivations towards mastering them, while if the teaching factors are not guaranteed interest, educators may not be able to ensure students willingly to learn in any given cases, or in other words, students will not learn well when they go at it without putting their hearts in it.

Following Dewey’s argument on the leading role of student interest in motivating them in learning, researchers started to further detail the relationships between interest and learning motivations since 1980s. Kintsch’s (1980) study was one of the most influential approaches to address the association between interest and learning. Kintsch (1980) theorized the concept of situational interest and its relationship with other factors related to learning. Situational interest is defined as provisional interest that arises spontaneously due to environmental factors such as task directions or an appealing text (Schraw, Flowerday, & Lehman, 2001). According to Kintsch (1980), there are two types of situational interest as emotional and cognitive interest. Emotional interest was likely to be aroused by information with strong affective response such as delightfulness or irritation, while cognitive interest arises when learners get involved in event or activities.

Subramaniam (2010) pointed out that situational interest played the key role as a motivator in enhancing student engagement in learning process. Educational studies in this area focus more on the effect of situational interest on motivating student learning because it is the factor that can be managed and manipulated by educators. Researchers (e.g., Hidi & Baird, 1986; Mitchell, 1993) emphasized that situational interest appeared to be especially important in catching students’ attention, and hereafter to initiate personal interest in holding their motivation in learning. Hidi (1990) proposed that situational interest increased learning when learning tasks or instructional materials were innovative. Other researchers (e.g., Schraw & Dennison, 1994; Shirey, 1992) also empathize that student learning interest is likely motivated by information pertinent to a task or learning goal.

With the theoretical guidance of learning interest and motivation, empirical studies have been flourishing in the past decades. As presented in Potvin and Hasni (2014), there were 228 studies at k-12 levels since 2000, and in another search by the authors of the current study in April 2016, there were 1362 research articles related to the learning interest and motivation topics, and 353 of them are at college level studies during 2000-2016 indexed in the ERIC database. Among them, there were 295 articles related to the use of technology to enhance student learning interest and motivate them to learn better. The large amount of studies on student learning interest and motivation related to technology use in the most recent years reflect the current research focuses on student learning and the shift of the educational instructions to e-learning environments to promote student learning.

Learning Motivation and Instructional Technology

In the e-age, student interest is more diverse than ever before. To motivate student interest in learning academic courses has become more difficult. In the past decade, the problem of decreasing of students majoring in science and technology in higher education has become even more acute (Osborne, Simon, & Collins, 2003; Potvin & Hasni, 2014). In the current situation, the question of how to improve student learning motivation through initiating their learning interest is imperative. The motivation and interest theory as discussed above has provided guidance for us to find answers to the above question. According to the literature, student learning interest can be motivated when the instructional materials (1) match their learning styles (Felder & Silverman, 1988), (2) leverage the difficulty levels (Cordova & Lepper,1996), (3) have feasible missions or tasks (Flowerday & Schraw, 2000), and (4) arouse their curiosity (Kohn, 1993; Lowenstein, 1994). Instructional technology brings more opportunities for teachers to manage student situational interest through incorporating the above factors influencing student interest in course learning.

styles. It is clear that digital tools can help with visual learners through presenting live pictures or illustrating the interactive procedures for some scientific process which may not be available to be seen in either a classical classroom or lab.

Second, technology can leverage the learning difficulties on learning some abstract concepts (Mishra & Koehler, 2006). The use of technology can make students learn interactively and easily through digital demonstrations or interactive visualization tools. For example we can design learning modules to make student easily navigate through interactive visualization tool, find the learning materials that they are interested in at that moment, input their own answers, provide their suggestions/comments, learn an interactive forum. Furthermore, hardware demonstrations and experiments can be incorporated as movies in the interactive visualization tools for online learning to help student learn difficult contents (Heafner, 2004).

Third, technology can segment an ambitious learning goal into interconnected feasible specific objectives to maintain student learning interest (Raman, Ryan, & Olfman, 2005). While the technology can be used to fit well with the learning style of students (Dede, 2004), it can help to make learning objectives achievable, such as through interactive procedures with interconnected single missions, and therefore to improve student learning efficiency. Technology can also give immediate and interactive feedbacks to students’ online activities; hence, it could increase students’ confidence level (Mun & Hwang, 2003) for them to maintain their motivation in achieving their learning goal.

Last but not the least, technology can arouse student curiosity in learning. The use of technology can provide a non-threatening and fun learning environment for all students and incorporate self-regulated learning (SRL) strategies (Zimmerman & Schunk, 1989) that usually prompt learners’ curiosity and interest to think and reflect critically about how they learn.

With regards to the opportunities using technology to help with students learning through enhancing their learning motivations, we would like to explore the answers to the following research questions in this study:

• Are there any significant differences in learning motivation changes of engineering students over the IGLU intervention period controlling for their demographic variables (e.g. gender and ethnicity)? • What are the significant changes in student course learning motivation after the students attending the

IGLU intervention if any?

Method

The quasi-experimental pre and posttest research design is used to assess the effect of interactive learning module with technology assistance on engineering student learning motivation. The repeated measure multivariate analysis of covariance (MANCOVA) are used to answer our research questions. The strength of the current study is that we were able to use two empirical samples to cross-validate the statistical analysis results.

Participants

Instrumentation

The Course Motivation Scale (CMS, Kebritchi, Hirumi, & Bai, 2010) was used to collect student learning motivation data, and the researcher-created Fuel Cell Knowledge Test was used to measure student learning outcomes before and after the IGLU intervention period.

Student Learning Motivation Survey

The Course Motivation Scale (CMS, Kebritchi, Hirumi, & Bai, 2010) was created based on Keller’s ARCS Model (1987). The questionnaire is a 5-point Likert scale (from Not true =1 to Very true =5) with 20 items measuring student course motivation on four dimensions, including attention, relevance, confidence and satisfaction based on Kebritchi, et al. (2010). Attention measures students’ interest arising and preserving during educational activities, and sample questions include, “There is something interesting about this course that will capture my attention,” and “I believe that I will enjoy this course so much that I would like to know more about this topic.” Relevance denotes whether a student sees the activity pertinent to personal interest, and a sample questions is, “The course is relevant to my interests.” Confidence represents whether a student presumes a feasible goal, and sample questions include, “I believe that I will be confident in my ability to successfully complete all class assignments and requirements after working on this course for a while.” Satisfaction refers to the value that the students see from the course work, and sample questions contain “I believe that completing this course will give me a feeling of satisfaction,” and “It is apparent to me how people use the information in this course.”

The content validity of the Motivation survey was examined and confirmed by the experts well understand Keller's ARCS model. Exploratory factor analysis using the pretest data from Dataset 1 for this study found clear four-factor structures with 62.88% of the total variance explained which confirmed the construct validity. The Cronbach's alpha reliability of ARCS measured by the current data was 0.92 for Data 1 with a sample of 122, and .91 for Data 2 with a sample of 135.

Fuel Cell Knowledge Test

The Fuel Cell Knowledge Test consisted 10 multiple choice questions with the total score of 30. The content validity was confirmed by two experts in the Fuel Cell and related engineering subjects. The instrument was also reviewed by an expert in measurement and evaluation in education to validate the quality of measure.

Data Collection Procedure

The Institutional Review Board (IRB) approval at the authors’ university was obtained before collecting data. The IRB procedures were strictly followed during data collections. All participants were secure confidentiality but not anonymity because their pretest and posttest data needed to be matched. The participation of the project was on the volunteering bases.

For the first round of data collection, paper-based surveys were given to the students in a normal classroom setting. Before the class instruction, the researchers distributed the test papers and questionnaires to the students in one large classroom along with the consent form approved by IRB as the cover page of the survey package. One of the researchers read the consent form to the students and made students aware that they were invited to participate in the research on a volunteer basis. Students were asked to complete the Student Learning Motivation survey and the Fuel Cell Knowledge Test as the pretests before class. Then one of the researchers taught the entire class with the software modules as the intervention for 25 minutes, and the whole class duration was about 50 minutes. After the intervention session, students were asked to complete the Student Learning Motivation survey and the Fuel Cell Knowledge Test again as the post-tests. Students’ demographic data, such as gender and ethnicity, were also collected.

instruction the researchers gave 15 minutes for those who had not taken the pretests to complete the pretests online using personal laptop computers, iPads or smart phones. The posttests were available 24 hours for students to complete online right after the treatment.

DATA ANALYSIS RESULTS

Before conducting the multivariate data analysis, both data sets were examined for assumptions. The dependent variables, pre and posttest scores of student motivation and knowledge tests were all normally distributed for both datasets for this study. Homoscedasticity was also tested, and the equal variance and covariance among the dependent variables were assumed for both data sets with Box’s M = 44.51 (F(30, 1350) = 1.20, p = .21) for Data 1 and Box’s M=66.17 (F(50, 2844) = 1.10, p = .29) for Data 2 (See Table 1).

The correlations among the dependent variables were examined. Pretest and posttest motivation scores were highly correlated with r = .737 (p < .001) for Data 1 and r = .343 (p < .001) for Data 2. Pretest and posttest knowledge scores were also significantly correlated with r = .351 (p < .001) for Data 1 and with r = .852 (p < .001) for Data 2. (See Table 2). With repeated measures, multicollinearity is not a concern for testing within subject effect.

Table 1. Box's Test of Equality of Covariance Matricesa

Data 1 Data 2

Box's M 44.509 66.174

F 1.197 1.102

df1 30 50

df2 1350.506 2844.004

p .214 .290

Note. Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups. a. Design: Intercept + Gender + Ethnicity + Gender * Ethnicity;

Within Subjects Design: Time

Table 2. Pearson correlations for student motivation scores and knowledge test scores

Data Source CMS_pre_T _(p-value) CMS_post_T _(p-value) Q_pre_T _(p-value) Q_post_T _(p-value)

Dataset 1 (N = 122)

CMS_pre_T 1 _(<.001) .737** _(.771) .027 _(.007) .242**

CMS_post_T 1 .041 .369

**

(.654) (<.001)

Q_pre_T 1 .351**

(<.001)

Q_post_T 1

Dataset 2 (N=135)

CMS_pre_T 1 .852** _{.064 .067}

(<.001) (.462) (.443)

CMS_post_T 1 .206* .110

(.016) (.206)

Q_pre_T 1 .343**

(<.001)

Q_post_T 1

Note. CMS = Motivation test; Q = Knowledge test.

removing the gender and ethnicity from the model and ran repeated measure MANOVA to only examine the within-subject effect for motivation and knowledge test scores presented in Table 3 for both Data 1 and Data 2. The descriptive statistics for both data sets are shown in Table 4.

Table 3. Within-Subjects Factors

Measure Time Dependent Variable

Motivation 1 _{2 CMS_post_T} CMS_pre_T

Knowledge 1 Q_pre_T

2 Q_post_T

Note. CMS = motivation test; Q = knowledge test

Table 4. Descriptive Statistics

Data Source Mean SD Na

Data 1

Motivation_pre 67.01 13.006 122

Motivation_post 69.10 14.128 122

Knowledge_pre 8.221 4.2456 122

Knowledge_post 22.689 4.3467 122

Data 2

Motivation_pre 70.5407 13.150 135

Motivation_post 71.9185 13.511 135

Knowledge_pre 11.8000 5.068 135

Knowledge_post 21.578 4.855 135

Note.a _{listwise deletion}

Multivariate analysis results for both Data 1 and Data 2 revealed the significant effect of the combined variable on the effect of pre and posttest with Wilks' λ = .101 (F(2, 120) = 533.33, p < .001, η2 = .899) for Data 1, and Wilks' λ =.237 (F(2, 133), p < . 001, η2= .765) for Data 2 (See Table 5). The results indicated that there were statistically significant differences between pre and posttest scores on either motivation scores or knowledge tests or both of the dependent variables.

Table 5. Within subjects effect of Multivariate test results a,b

Data source Wilks' λ F Hypothesis df Error df p η2

Data 1 .101 533.330c _{2 120 <.001 .899}

Data 2 .237 213.631c 2 133 <.001 .763

Note. a. Design: Intercept; Within Subjects Design: time; b. Tests are based on averaged variables; c. Exact statistic

Table 6. Univariate Tests of Within-Subject Effects

Data

Source Source Measure time

Type III Sum of Squares

df _Square Mean F p

Data 1

time Motivation Linear 266.496 1 266.496 5.436 .021

Knowledge Linear 12767.316 1 12767.316 1066.010 <.001 Error(time) motivation Linear 5932.004 121 49.025

Knowledge Linear 1449.184 121 11.977

Data 2

time Motivation Linear 128.133 1 128.133 4.864 .029

Knowledge Linear 6453.333 1 6453.333 398.746 <.001

Error(time) motivation Linear 3529.867 134 26.342

With regards to the multivariate results, we further conducted univariate tests to examine the specific pre and posttest differences on each dependent variable. As we can see from Table 6, univariate test of within-subject effects on both Data 1 and Data 2 revealed that there were significant pretest and posttest differences for both motivation and knowledge test scores. The changes over time of motivation scores were significant at α = .05 level with F(1, 121) = 5.436 (p = .021) for Data 1 and F(1, 134) = 4.864 (p = . 029) for Data 2. The changes of knowledge tests were also significant with F(1, 121) = 1066.01 (p < .001) for Data 1 and F(1, 134) = 398.746 (p < .001) for Data 2. (See Table 6)

Considering the analysis results from univariate tests, we further examine the specific mean differences of post-test of motivation scores for both Data 1 and Data 2 on both dependent variables. From Table 7 we can see that the mean of motivation posttest (M = 69.10; SD = 14.13) (see Table 4) was at 2.09 [.32, 3.87] points significantly higher (See Table 7) than the mean of pre-test score (M = 67.01, SD = 13.01) (see Table 4) in Data 1. The mean of motivation posttest (M = 71.92; SD = 13.51) (see Table 4) was at 1.38 [.14, 2.61] points significantly higher (See Table 7) than the mean of pre-test score (M = 70.54, SD = 13.51) (see Table 4) in Data 2.

Table 7. Pairwise Comparisons for the motivation and knowledge test score changes from time 1 to 2

Data

source Measure time (I) time (J)

Mean Difference

(I-J)

Std.

Error pb

95% Confidence Interval for Differenceb

Lower Bound Upper Bound

Data 1 Motivation 1 2 -2.090* _{.896 .021 -3.865 -.315}

2 1 2.090* _{.896 .021 .315 3.865}

Knowledge 1 2 -14.467* _{.443 <.001 -15.344 -13.590}

2 1 14.467* .443 <.001 13.590 15.344

Data 2 Motivation 1 2 -1.378* .625 .029 -2.613 -.142

2 1 1.378* _{.625 .029 .142 2.613}

Knowledge 1 2 -9.778* .490 <.001 -10.746 -8.809

2 1 9.778* .490 <.001 8.809 10.746

Note. Based on estimated marginal means*. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments).

We also further examine the specific mean differences of post-test of knowledge scores for both Data 1 and Data 2. From Table 7 we can see that the mean of the knowledge posttest (M = 22.69; SD = 4.35) (see Table 4) was at 14.47 [13.59, 15.34] points significantly higher (See Table 7) than the mean of pre-test score (M = 8.22, SD = 4.25) (see Table 4) in Data 1. The mean of knowledge posttest (M = 21.58; SD = 4.86) (see Table 4) was at 9.78 [8.81, 10.75] points significantly higher (See Table 7) than the mean of pre-test score (M = 11.80, SD = 5.07) (see Table 4) in Data 2.

DISCUSSION AND CONCLUSION

The quantitative data analyses revealed that the students had significant increase in their course motivation after the instructions on the fuel cell content knowledge with the assistance of IGLU modules. The study results have been cross-validated by two data sets collected from two groups of engineering students at two separated treatment periods using measures in two forms, paper and online based instruments for data collections. We also found significant increase in student learning outcomes measured by the fuel knowledge tests; however, we want to be cautious to draw the causal inference at the current stage because of the absence of a control group study. We argue for the causality of student learning motivation changes due to the intervention which is our current study focus, but not for the effect on the increase of knowledge scores with the current data because participants are expected to acquire more content knowledge after any instructions; however, the change of learning motivations really depends on many course related factors.

The design of the IGLU modules followed the factors defined in Keller’s ARCS Model (1987) to make a better learning environment to increase student situational interest in learning. Researchers of the project followed the ARCS model to make the content knowledge delivery (1) to attract student attention using movies, live pictures, and animations, (2) to be relevant to student interest by displaying the fancy technology uses to segment the process of fuel cell productions into interconnect subsystems, (3) build student confidence by partitioning the big concepts into manageable parts and lively depicting the abstract ones using animations for students to easily understand, and (4) satisfy digital youth’s learning needs though accommodating their learning styles with their familiar means, the use of instructional technologies, for content knowledge learning.

There are many factors that affect student motivation, while the four important focuses of the IGLU modules based on ARCS model play a major role to improve student course learning motivations. This goal was achieved through managing student’s situational interest in learning by the instructor using technology. This study results are consistent with previous research findings in the same area. Chang (2007) stated that with the rapid development of technology and science, instructional technology was increasingly used over time. There are some valuable similar studies (e.g., Shellnut, Knowltion, & Savage, 1999) on the positive effects using interactive modules through technology use in engineering instructional design. Many researchers and designers have applied games and animations to their curriculum and achieved positive results in terms of motivating student learning. Vogel et al. (2006) argued that the consistent findings about the positive effect on the cognitive gains and improvement of student motivations in learning were “extremely unlikely to be due to change” (p.235) based on their meta-analytic study of the literature.

Based on practical experiences about our interactive modules and student motivation, we would have the same conclusion with Michau, Gentil, and Barrault (2001) that there is a promising view about web-based modules or the use of instructional technology for increasing student learning interest so that to motivate the digital generation to learn better. Our study results support the claims about the positive impact of popularity to use instructional technology, specifically for engineering students. It improved student learning motivation through addressing their learning habits or styles and using their favorite means to deliver content knowledge. While, we agree with some researchers (e.g., Walddeck, & Dougherty, 2012) that web-based or technology-based learning could be a double-edged sword for students’ motivation because the result is dependent on whether the technologies are used effectively. Admittedly, not all of the technologies will be beneficial to students’ learning motivations if not used appropriately. Some studies (e.g., Tai & Ting, 2015) found that some technology use may satisfy student interest, because students may take technology use, such as videos or games, as a way of entertainment rather than learning, and the leisure environment may result in an unacceptable achievement of students (Behnke & Greenan, 2011); therefore, the instructional technology use has to appropriately map the content knowledge with a right amount of usage.

Limitations and Future Study

There are some limitations in the current study. Even though we confidently claim the positive impact of the IGLU modules as an intervention on student motivation in learning, we agree that the treatment length was relatively short. However, with the current promising results from the relatively short period of treatment, we can reasonable assume more positive effect if we could have longer or more sessions to implement the interventions. Another concern could be that we used quasi-experimental pre and posttest designs other than the control group design. We understand that the quasi-experimental control group design is more rigor for us to make causal inferences from the statistical analysis results; however, our current study focuses on the assessment of the improvement of student learning motivation other than the student learning outcome. With regards to student learning motivation as the specific construct we studied, it is justifiable for our claims on causality of the positive effect of the program because student learning motivation can be both positively and negatively influenced by the instructions.

We also expect that such study will add knowledge to curriculum design through assessing the effectiveness of IGLU instructional modules for engineering student learning outcomes.

With the online learning prevailing in the current university courses, the significance of the current study is that the education initiative of IGLU system and the assessment of its impact on student learning motivation provide important information for improving the modules to serve a diverse student body. It will greatly help curriculum development for engineering education.

ACKNOWLEDGMENTS

The researchers of this project thanks for the support from NSF grant #1245747.

AUTHOR BIOGRAPHIES

Haiyan Bai, Ph.D., is an Associate Professor in the College of Education and Human Performance (CEHP) at the University of Central Florida. Her research areas include issues that revolve around quantitative methods and the applications of statistical methods in educational research and behavioral sciences. Her publications include books, book chapters, articles in refereed national and international journals, and refereed international conference proceedings. She has also given lectures, talks, and many refereed professional presentations at the international/national and the regional levels and won grants from National Science Foundation’s (NSF) and U.S. Department of Education.

Amjad Aman is a Ph.D. candidate at the Mechanical and Aerospace Engineering department at the University of Central Florida. Amjad's research has been dedicated to solid oxide fuel cells: electrochemical modeling of single fuel cells, structural modeling of electrolytes, material characteristic studies of the cathode material and the development of fuel cell educational software.

Yunjun Xu, Ph.D. (U. of Florida, 2003) is an Associate Professor in the Department of Mechanical and Aerospace Engineering at the University of Central Florida. He teaches courses like Aircraft Design, Flight Mechanics, Orbital Mechanics, Feedback Control, and Guidance, Navigation, and Control. His research interests are robots, control theory, guidance, and engineering education.

Nina Orlovskaya, Ph.D. (National Academy of Sciences of Ukraine, 1993) is an Associate Professor in the Department of Mechanical and Aerospace Engineering at the University of Central Florida. She teaches courses like Solid Mechanics, Thermodynamics, Senior Design, Science and Technology of Fuel Cells. Her research interests are energy conversion devices, electrochemically active ceramics, and mechanical behavior of ceramics, boron rich solids and engineering education.

Mingming Zhou is a Ph.D. student in the Methodology, Measurement, and Analysis Program in the College of Education and Human Performance (CEDHP) at the University of Central Florida. She is a research assistant in the IGLU project and has also been working in the Office of Innovative Analytics and Application in CEDHP.

REFERENCES

Aman, A., Xu, Y., Bai, H., Orlovskaya, N., (2015, June). Interconnected software modules to aid the learning of fuel cell courses. Paper presented in the ASEE’s 122nd Annual conference and Exposition, Seattle, WA.

Anderson, L.W. & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.

Bai, H., Xu, Y., & Orlovskaya, N. (2014, April). Web-based interactive modules for engineering college undergraduate students. Paper presented at the International Conference of the Chinese American Educational Research and Development Association 2014, Philadelphia, PA.

Behnke, C. & Greenan, J. P. (2011). The relationship between emotional intelligence and attitudes toward computer-based instruction of postsecondary hospitality students. Journal of Career and Technical Education 26(1): 62-84.

Cordova, D. I., and Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88, 715–730.

Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin, 125(6), 627.

Deci, E. L., & Ryan, R. M. (1975). Intrinsic motivation. John Wiley & Sons, Inc.

Dede, C. (2004). Planning for “neomillennial” learning styles: Implications for investments in technology and faculty. Harvard Graduate School of Education.

Dewey, J. (1913). Interest and Effort in Education. Riverside Press, Boston, MA

Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in engineering education. Engineering education, 78(7), 674-681.

Flowerday, T., & Schraw, G. (2000). Teacher beliefs about instructional choice. Journal of Educational Psychology, 92, 634– 645.

Heafner, T. (2004). Using technology to motivate students to learn social studies. Contemporary Issues in Technology and Teacher Education, 4(1), 42-53.

Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational research, 60(4), 549-571. Hidi, S., & Baird, W. (1986). Interestingness—A neglected variable in discourse processing. Cognitive Science, 10(2), 179-194. Hull, G. A., & Schultz, K. (Eds.). (2002). School's out: Bridging out-of-school literacies with classroom practice (Vol. 60).

Teachers College Press.

Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & education, 55(2), 427-443.

Keller, J. M. (1987). Development and use of the ARCS model of motivational design. Journal of Instructional Development, 10(3), 2-10.

Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why anyone would read a story anyway. Poetics, 9(1), 87-98.

Messner, J. I., & Horman, M. (2003, September). Using advanced visualization tools to improve construction education. In Proceedings of CONVR 2003 Conference (pp. 145-155).

Michau, F., Gentil, S., & Barrault, M. (2001). Expected benefits of web-based learning for engineering education: examples in control engineering. European Journal of Engineering Education, 26, 151-168. doi:10.1080/03043790110034410 Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. The

Teachers College Record, 108(6), 1017-1054.

Mohler, J. L. (2001). Using interactive multimedia technologies to improve student understanding of spatially-dependent engineering concepts. In Proceeding of the International Conference on Computer Geometry and Graphics (Graphicon 2001) (pp. 292-300).

Mun, Y. Y., & Hwang, Y. (2003). Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. International journal of human-computer studies, 59(4), 431-449. National Science Board (NSB). (2015). Moving forward to improve engineering education. Retrieved from:

http://www.nsf.gov/pubs/2007/nsb07122/nsb07122.pdf.

Nguyen, T., & Khoo, I. (2009). Learning and teaching engineering courses with visualizations. Proceedings of the 2009 World Congress on Engineering and Computer Science (WCECS), Vol. I, San Francisco, CA, Oct. 20-22.

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25, 1049–1079.

Potvin, P., & Hasni, A. (2014). Interest, motivation and attitude towards science and technology at K-12 levels: a systematic review of 12 years of educational research. Studies in Science Education, 50(1), 85-129.

Raman, M., Ryan, T., & Olfman, L. (2005). Designing knowledge management systems for teaching and learning with wiki technology. Journal of Information Systems Education, 16(3), 311.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist, 55(1), 68.

Schraw, G., & Dennison, R. S. (1994). The effect of reader purpose on interest and recall. Journal of Literacy Research, 26(1), 1-18.

Schraw, G., Flowerday, T., & Lehman, S. (2001). Increasing situational interest in the classroom. Educational Psychology Review, 13(3), 211-224.

Shellnut, B., Knowltion, A., & Savage, T. (1999). Applying the ARCS model to the design and development of computer-based modules for manufacturing engineering courses. Educational technology research and development, 47(2), 100-110. Shirey, L. L. (1992). Importance, interest, and selective attention. The role of interest in learning and development, 281-296. Subramaniam, P. R. (2010). Unlocking the power of situational interest in physical education. Journal of Physical Education,

Recreation & Dance, 81(7), 38-49.

Tai, Y., & Ting, Y.-L. (2015). Effects of commercial web videos on students' attitude toward learning technology. International Journal of Information and Communication Technology Education,11(3), 20-29.

Waldeck, J. H., & K. Dougherty (2012). Collaborative communication technologies and learning in college courses: Which are used, for what purposes, and to what ends? Learning, Media and Technology, 37(4), 355-378.

Yarmolenko, S., Sankar, J., Bernier, N., Klimov, M., Kapat, J., & Orlovskaya, N. (2009). Phase stability and sintering behavior of 10mol% Sc2O3–1mol% CeO2–ZrO2 ceramics. Journal of fuel cell science and technology, 6(2), 021007. Zimmerman, B. J., & Schunk, D. H. (Eds.) (1989). Self-regulated learning and academic achievement: Theory, research, and