Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 48 Sameena Falleiro
Department of Computer Science, Parvatibai Chowgule College of Arts and Science (Autonomous), Gogol, Margao Goa 403 602, India.
ABSTRACT
With the expansion of internet and distance technologies, Learning Management System (LMS) has provided an integrated platform for content, delivery and management of learning, as well as accessibility to a range of users. The LMS acts as the central component of any e-learning implementation. Notwithstanding the same, one pressing requirement with regards to LMS is the understanding of how it may be used to enhance overall student learning in higher education. This is imperative since more the students are engaged and motivated to learn; the more likely they will be successful in their efforts; which then gets reflected in their academic performance. Modular Object Oriented Dynamic Learning Environment (MOODLE) is open source software LMS, designed to help educators and researchers create effective online learning communities. This paper examines the effectiveness of LMS-MOODLE on the key facets of student learning namely: Engagement, Motivation and Academic Performance with the help of tools and scales prepared by the researcher to assess the same. The two sample ‘post-test only’ experimental research design was used for this study. The study found a significant difference in the engagement, motivation and performance of students, with students using LMS-MOODLE performing better. The findings are useful to conclude that LMS-MOODLE does contribute positively towards improving the outcomes in a classroom-like situation it could likewise be an efficient as well as cost-effective tool in higher education.
Keywords:ICT, MOODLE, LMS, Engagement, Motivation and Academic Performance
INTRODUCTION
We are living in time of a converging world and deep global changes. For the first time we are preparing students for a future we cannot clearly describe (Warlick, 2013). New challenges for teaching and learning for higher education continue. The growing importance and new dimensions of the face-to-face
learning are opened by the rapid development of educational ICT technology. The effective integration of ICTs into the educational system is a complex multifaceted process that involves not just technology but also curriculum and pedagogy, institutional readiness, teacher competencies, long-term financing etc. Learning Management Systems (LMS) is changing the way students are
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 49
getting educated nowadays. Emergence of the Internet has facilitated growth of LMS. This new paradigm has become a critical component in the mission of academic institutions, which includes imparting best education to their students, extending the learning process beyond the classroom, improving measurable student success, attracting talented students to the institute and improving its overall image.
Student engagement is generally
understood to be the primary mechanism that enables motivational processes to contribute to learning and development (Furrer, & Skinner, 2003). From students perspective an LMS assists students in their study, which leads to better academic performance. However, it has been found that both educators and students often underutilize the system features, i.e. ineffective use of system features. Hence, although the system has many possibilities that can be used to augment traditional face-to-face teaching in order to achieve
better academic performance, the
underutilization often hinders to reap the
possible benefits (Islam & Mantymaki,
2012). The importance of interpersonal contacts is rising. Blended learning by
using an LMS like MOODLE (Modular
Object Oriented Dynamic Learning
Environment) is a very effective tool for
university education. Student learning in the MOODLE e-learning environment is one of the very perspective solutions for lifelong learning. The courses on which the online and face-to-face delivery is blended will have the predominant role in the future.
This paper discusses the effectiveness of
the LMS-MOODLE on student
engagement, motivation and academic performance, with the help of tools and scales prepared by the researcher to assess
the same. Understanding this relationship
will guide stakeholders in HE to structure their course resources, activities and use this environment to enhance the learning
experience in a technology enhanced
global classroom environment.
REVIEW OF LITERATURE
LMS like MOODLE provides
collaborative tools like email, chat and discussion forums that assist students as they construct knowledge. The aim is to create learning environments centered on students as learners and to offer new learning opportunities all at one place.
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 50 MOODLE was designed as an LMS to
support and promote users interested in developing constructivist, student-centered learning environments. It is based on the
constructivist pedagogy wherein
knowledge is actively constructed by the learner, not passively received from the environment (Dougiamas, & Taylor, 2003). It has three main functions: authoring, administrative and a means of communication (Peter, 2004). Open source LMSs were analyzed and it was observed that MOODLE among other LMSs offers a clear advantage over other LMSs, besides having many needed tools, included features that not only improved pedagogical quality, but also scored high with regards to modular design, interface, and authentication issues (Aydin & Tirkes, 2010).
Student engagement results provide
educators across a variety of campus programs and departments, information to consider in their efforts to understand the student experience and to collaborate in the design of educationally productive activities and programs (NSSE, 2011). MOODLE-LMS provides an environment that enables learning, as learners work
individually on assigned materials and decide when they would engage. Learners can assess the outcomes of their engagement by how successful they were progressing through the materials with the help of assessment features like Quiz, Assignment, etc. all built into the LMS (Peter, 2004). Several studies (Chapman, 2003; Ethan, 2006; Kuh, 2007) have highlighted the significant role that such factors can play in the learning process, laying particular emphasis on those associated with student engagement levels.
The plethora of features offered in MOODLE-LMS along with the embedded cognitive strategies like a feedback feature makes it an LMS that provides students with a learning advantage.
OBJECTIVES
The objectives of the study are:
1) To study the effectiveness of
LMS-MOODLE for student engagement,
motivation and academic performance. 2) To find which features available in the
LMS-MOODLE affect student
engagement, motivation and academic performance in the experimental group.
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 51
RESEARCH
DESIGN
AND
METHODOLOGY
The Research Design used for this study was the ‘Experimental method’ with ‘post-test only’ design. The students in the two groups were enrolled in a compulsory course titled Software Engineering during the last (sixth) semester (in a six semester B.Sc. Computer Science, Undergraduate degree programme offered by Goa University, India) for the academic year 2012–2013. Both groups were taught the course in the conventional environment, with the only difference being that only students from the experimental group had access to MOODLE-LMS. Engagement implies the use of three interrelated criteria
namely, Cognitive, Affective and
Behavioral to assess student engagement
levels (Kuh, 2001; Chapman, 2003).
Keeping the above studies in mind, the researcher developed an Engagement Scale using a four-point rating scale with 21 items, with an objective to study the cognitive, affective and behavioral aspects of engagement. A Motivation Scale was developed by the researcher by suitably modifying and adapting Keller ARCS motivation scale (Keller, 1987) to this study. The scale was developed using a
five-point rating scale with 24 items, with an objective to study the attention, relevance, confidence and satisfaction aspects of motivation. The Performance Test was an achievement test with a maximum score of 25 marks. All the above tools were constructed by the researcher and validated by experts before administering them as post-tests to both the groups. Besides these tools an
Evaluation Rubric (ER) was also
constructed to help determine the extent to which MOODLE-LMS features namely
Chat, Discussion Forum, Glossary,
Assignment and Quiz actively promote student engagement and participation. The scores obtained from the rubric helped determine whether such activities appeal to the learners, thereby increasing their engagement, motivation and interaction. The criteria in the ER were designed keeping in mind the objectives of the study with an additional focus being to understand the quality of interactions that was taking place through the various
activities and resources that the
MOODLE-LMS offers. An attempt was made to know a student’s willingness, need, desire and compulsion to participate in, and be successful in the learning
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 52 process by offering the student various
LMS features which in turn are expected to affect their engagement, motivation and
performance. The faculty for the
treatment group made available resources like lecture slides and website links to the students. Activities related to their
coursework like Assignments (for
submitting), Forums (for participating and sharing views on topics/content related to the module), Quiz (to test the extent of the
matter taught), Chat sessions (to discuss
topics and difficulties), etc were all made available through MOODLE-LMS. The participants were encouraged to build a Glossary for the modules. All the activities were related to the topics covered in the modules. The students were expected to access and participate in all the activities that the faculty made available to them via the MOODLE-LMS from time-to-time. At all times, for all their activities students were able to regulate their time-on-task between the start of the task and the deadlines (if any) for submission, as these were displayed in their work area, through
the Calendar utility provided in
MOODLE.
FINDINGS AND DISCUSSIONS
In order to test the first objective of the study regarding significance of the difference between post-test scores of the learners in the experimental group and control group, the t-ratio was computed. A Comparison of the means of post-tests of the Experimental and Control group post-tests on Engagement, Motivation and Academic Performance shows that the students from the experimental group reported a higher mean score on all parameters of the study as compared to students from the control group. The finding that students who use MOODLE-LMS have significantly higher degree of
academic performance vis-à-vis the
control group students, is in congruence with studies done by Bulger, Mayer, Almeroth & Blau (2008), wherein a positive correlation was found between MOODLE use and academic performance for high-support courses. The t-values
(0.029 for engagement; 0.007 for
motivation and 0.000 for academic
performance at 0.05 and 0.01 level of
significance) indicate a significant difference in engagement, motivation and academic performance of the students in the experimental group over the control
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 53 group. Thus indicating that the
LMS-MOODLE plays a significant role in engagement, motivation and academic
performance of students in the
experimental group. Therefore there is a need to study, what features available in the LMS are responsible for affecting these three parameters, which is also the second objective of the study.
An analysis of the ER indicates the extent to which the features in MOODLE-LMS were used by the students. The results of the correlation between LMS features like Chat, Forum, Glossary, etc. used vis-à-vis Engagement, Motivation and Academic Performance in the Experimental Group shows that there is a correlation between overall LMS features and student engagement (0.578), motivation (0.668) and academic performance (0.913) at the 0.05 and 0.01 level of significance. However, among all the mentioned features it is the Forum and Assignment that are very significant to all three parameters namely Engagement, Motivation and Academic Performance. There is a positive relationship between features like Chat, Forum, Assignment, Quiz and student academic performance. There is also a positive relationship between features like Forum, Assignment, Quiz and student motivation.
The results show that the perceived richness of online discussion forum has significant positive effect on student participation, interaction and learning, when used along with traditional classroom lecture. Interaction and quality of interaction among the students and instructor are considered as an essential part of the academic progress. It is also worth noting that the type of interaction taking place through the LMS plays a very significant role on Engagement, Motivation and Academic performance of students in the experimental group.
CONCLUSIONS
AND
SUGGESTIONS
Previous research shows that LMS influence engagement and research into their effect on engagement is still in its infancy (Coates and Baldwin, 2005; Lopes, 2008; Beer, Clark & Jones, 2010). The current study has established that there is a significant difference between the Engagement and Academic Performance levels between students of the experimental group and control group. In the Experimental group, of all the activities that the LMS-MOODLE provides, the Discussion Forum and Assignment activities report a significant degree of co-relation on Engagement and Academic Performance. This shows that Forums and Assignment are important features of
LMS-Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 54
MOODLE. This implies that these features of the LMS act as contributory factors that help students in the learning process and hence perform academically better. The findings of the study can be applied to a technology enhanced global classroom environment wherein students are expected to use quality time both for their individual and collaborative activities without any face-to-face contact with faculty. Learning in an environment like MOODLE which is an open source LMS that provides collaborative tools like email, chat, discussion forums, virtual classrooms and blog features, all available free will definitely help educators and researchers to create effective online learning communities. These results are encouraging signs that Internet and Web-based learning technologies continue to have a positive impact on student learning in higher education.
ACKNOWLEDGEMENT
The author would like to thank:
University Grants Commission, New Delhi and Prof. Vasudha Kamat (Vice Chancellor, S.N.D.T Women’s University, Mumbai) for the support and guidance in conduct of the research.
REFERENCES
1. Aydin, C.C., & Tirkes, G. (2010). Open source learning management systems in distance
learning. The Turkish Online Journal of Educational Technology, Vol. 9(2). 175 - 184. Retrieved January 15, 2012, from http://www.tojet.net/volumes/v9i2.pdf
2. Bakharia, A., & Dawson, S. (2011). SNAPP: a bird's-eye view of temporal participant interaction. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 168-173).
3. Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement. Proceedings ascilite Sydney. Retrieved March 30, 2013, from http://cms.ascilite.org.au/
conferences/sydney10/procs/Beer-full.pdf
4. Bulger, M.E., Mayer, R, E., Almeroth, K.C., & Blau, S.D. (2008). Measuring learner engagement in computer-equipped college classrooms. Journal of Educational Multimedia and Hypermedia, 17(2), 129-143. Retrieved March 30, 2013, from http://www.editlib.org/p/23524
5. Chapman, E. (2003). Alternative approaches to assessing student engagement rates. Practical assessment, research and evaluation, 13(8).
6. Chen, P. S. D., Gonyea, R., & Kuh, G. (2008). Learning at a distance. Journal of online education, 4(3). Retrieved January 15, 2012, from http://innovateonline.info/index.php? view=article&id= 438&action=login
7. Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary education and management, 1192005, 19-36
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 55
8. Dawson, S. P., Macfadyen, L., & Lockyer, L. (2009). Learning or performance: Predicting drivers of student motivation.
9. Dawson, S. & McWilliam, E. (2008). Investigating the application of IT generated data as an indicator of learning and teaching performance. A report produced for the Australian Learning and Teaching Council. Retrieved January from http://olt.ubc.ca/learning_tools/research_1/rese arch/
10. Dougiamas, M. & Taylor, P. (2003).MOODLE: Using learning communities to create an open source course management system. Proceedings of the EDMEDIA 2003 Conference, Honolulu, Hawaii. Retrieved May 9, 2010, from: http://dougiamas.com/writing/ edmedia2003 11. Elias, T. (2011). Learning Analytics. Learning.
Retrieved July 15, 2013 from http://www.learninganalytics.net/LearningAna lyticsDefinitionsProcessesPotential.pdf 12. Furrer, C. & Skinner, E. (2003). Sense of
relatedness as a factor in children’s academic engagement and performance. Journal of Educational Psychology. Vol. 95, No. 1.
13. Hoskins, S. L., & Van Hooff, J. C. (2005). Motivation and ability: which students use online learning and what influence does it have on their achievement?.British Journal of Educational Technology, 36(2), 177-192. 14. Hollander, E. L., Saltmarsh, J., & Zlotkowski,
E. (2002). Indicators of engagement. In Learning to Serve (pp. 31-49). Springer US.
15. Islam, A. K. M. N., & Mantymaki, M. (2012). Understanding e-learning system
users’ post-adoption usage behavior and its outcomes. A study with a learning management system (Doctoral dissertation,
Doctoral Dissertation, University of Turku, Turku School of Economics).
16. Krause, K.L., & Coates, H. (2008). Students’
engagement in first-year university. Assessment & Evaluation in Higher Education, 33(5), 493-505.
17. Kuh, G.D. (2007). What student engagement data tell us about college readiness. Peer Review, Association of American Colleges and Universities, 9(1), 4-8.
18. Kuh, G. D. (2003). What we’re learning about student engagement from NSSE. Change 35(2): 24-32.
19. Lopes, V. M. (2008). Course Management Systems and Campus-Based Learning. Seneca College.
20. Margaret, H., Katherine McMillan Culp & Robert S. (2005). Critical Issue: Using Technology to Improve Student Achievement. Center for Children and Technology, North Central Regional Educational Laboratory. Retrieved May 30, 2010, from: http://www.ncrel.org/sdrs/areas/issues/method s/ technlgy/ te800.htm
21. MOODLE Online. (2013). Retrieved February 24, 2014, from http://www.MOODLE.org. 22. National Survey of Student Engagement.
(2011). Fostering student engagement campus wide—annual results 2011. Bloomington, IN: Indiana University Center for Postsecondary Research. Retrieved July 29, 2013 from http://nsse.iub.edu/NSSE_2011_Results/pdf/N SSE_2011_AnnualResults.pdf
Parvatibai Chowgule College of Arts & Science, Autonomous, Margao-Goa. Page 56
23. Peter, Ruthven-Stuart. (2004). An LMS from the ground up. Bulletin of Hokuriku University. Vol. 28, pp 94-130.
24. Piaget, J. (1975). Equilibration of cognitive structures, Chicago: University of Chicago Press.
25. Richards, I. (2009). Effective use of online discussion forums: The importance of assessment. Unpublished article. Retrieved
July 29, 2013 from
https://www.zotero.org/charlesdavis/items/ite mKey/2XKQZT9C
26. Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press. 27. Warlick, D. (2013). Keynote: Literacy &
Learning in the 21st Century. Retrieved on
July 24, 2013, from
http://premierespeakers.com/david_warlick/sp eech_topics
28. Yazzie-Mintz, E. (2007). Voices of Students on Engagement: A Report on the 2006 High School Survey of Student Engagement. Center for Evaluation and Education Policy, Indiana University.
*****
AUTHOR’S CONTACT Sameena Falleiro