Sunandan Chakraborty, Devshri Roy, Anupam Basu ∗
3. ARCHITECTURE OF THE ITS
3.3 Teaching Model
3.3.3 Result Analyzer
The third task of the Control Engine is to analyze feedbacks obtained from the student’s test results and perform some operations accordingly. This task is undertaken by the Result Analyzer (RA). After a session of learning through study materials the students must undergo a test. These tests are a reflection of the student’s understanding in the topic. Using this feedback, the ITS gets an updated knowledge of the student’s performance and uses this knowledge to take various decisions.
Preliminary task of the RA is to provide the results to the student model and update the various fields of the student profile. It supplies the test results to the student model so that it can compute the next state of the student’s performance. Apart from these, RA scrutinizes the test results to obtain some analysis on the student’s performance. This analysis finds out the various faults in the student’ learning and some decisions are taken, so that the student can overcome her flaws. RA also checks whether a change in the teaching plan is necessary for a better outcome from the student. Thus, RA helps the system to adapt dynamically with the changing condition of the student. An earlier plan may not work for a student and she may get stuck at some point and cannot advance from there. Such a condition can be only detected from the feedbacks (test results) from the student. Thus, RA identifies such situations from the test results and works out methodologies to get out from those situations i.e. revise the teaching plan computed earlier.
IF Performance is Excellent AND Pre-equisite_Performance is Excellent AND Interest is VeryHigh AND Hardness is Simple AND Importance is VeryHigh AND Scope is Excellent
4. CASE STUDY
The system was used in a real teaching and learning scenario. We organized a workshop for school students to learn basic ‘Programming in C’ using the ITS, Shikshak. Thirty three students from the different schools in the campus of IIT, Kharagpur participated in this workshop. The participants varied from 12 to 16 years in age. No students had any prior exposure to programming. The students studied various topics of C programming. First step was to configure the course in the ITS. Course Tree Editor was used to configure the Course Tree (CT) for this course. The configured CT is shown in Figure 7. As this course was meant for school students, only four basic topics were chosen. The topics were,
• Variables, data type and operators
• Loops
• Branching
• Array
The attributes and their values of these topics are shown in Table 2.
Table 2. Topics and the attributes
Topics Hardness Importance Prerequisite
topics
Concepts
Variables,Data types, Operators (V)
0.5 High None Variables, Arithmetic and logical operators, data types, printf, scanf
Loops (L) 0.8 High V For loops, while loops
Branching (B) 0.6 Moderate V If-else, switch-case, continue, break
Array (A) 1.0 Low L Linear array, string
Figure 7. Screenshot of the Course Tree Editor showing Course Tree constructed for the experiment
All the topics were furnished with the required details, such as concepts, prerequisites, difficulty etc.
Accordingly the system drew the TDG from the prerequisite information. The drawn TDG is shown in Figure 8.
Figure 8. Screenshot of TDG drawn by the system from the CT in Figure7
The next step involved preparation of the repository. For each topic various materials were created or collected and incorporated into the repository. Care was taken to make sure the materials were heterogeneous in nature. Moreover, files with different presentation styles like text-intensive, animation based or presentation type of documents were also incorporated. Some materials, particularly Macromedia flash based animation based documents were built specially for this purpose.
Screenshot for such a document is shown in Figure 8. Variety of documents enriched the repository and helped the Material Selection Module to select the appropriate type of materials from the different variety available. This served the basic purpose of the ITS, where unlike a classroom, students had the opportunity of studying from materials which suited them best. This was followed with the annotation of the materials. Annotation was done very carefully, as incorrectly annotated materials would lead to inconsistent retrieval from the repository and affect the system’s performance. In order to test the students’ knowledge various questions were also frames. The questions varied in terms of difficulty and focus (comprehension-ability or problem-solving skills)
Apart from domain model organization, some other preparations were required in the student model and the control engine. Student model was configured to accommodate the participating students. This initialization of the student model involves furnishing of some initial information about the students.
These data were obtained from their school records and fed into the student model. In the control engine some rules were framed, using the Rule Editor. These rules defined the strategies to be taken by the Topic Planner to sequence the course for each student.
Figure 8. A sample learning material
The Topic Planner computes separate topic sequences for different students. In Figure we show the different sequences can be drawn for this course.
Figure 9. All possible sequences drawn from the TDG (Figure) used in the workshop
We present the percentage of students following each sequence in Figure 10. The “others” column collectively represents all the cases were sequences had to be re-planned for some students.
Sequence followed by students
0 5 10 15 20 25 30 35 40 45
VLBA VBLA VLAB others
V:variable ; L:Loops; B:Branching; A:Array
Figure 10. Percentage of each Sequence followed by students
We claimed that an important aspect of an ITS is that the students can learn and progress along a course at their own pace. The duration of the course was 15 days but many fast learning students completed the course before that. However, some of the students could not finish the course in 15 days.
Perhaps 15 days were not sufficient and they required some more time. The detailed observations are shown in Figure11. The red (last three) columns signify that those students were unable to complete the course in 15 days.
Figure 11. Days taken by each student to complete the course (the last three columns represent the students who attended the course for 15 days but could not finish)
5. CONCLUSION
In this chapter, we have discussed about the development of an Intelligent Tutoring System (ITS). We started the chapter by discussing about ITS and some its general features. In the following section we presented some previous works related to ITSs. Then, we described the complete system with a full
description of it’s each module. Finally, we presented some results obtained during a field trial of the system.
The main objective of our work was to develop a system that can teach and deliver contents in a customized and adaptive manner. The system considers the cognitive ability of each student and devises teaching plans which would suit them best individually and maximize their performance. In summary, we can claim that the system exhibits moderately good performance. The main contribution of our works are summarized below:
• A generic and domain independent structure to store and represent the courses. Using this representation scheme various courses can be configured in the system. Simple courses for primary education to more complex courses of higher education can be configured using this scheme. The configuration of courses using this scheme is also simple and requires very little experience of computer usage from the teachers.
• A set of tags through which learning materials can be annotated. Most of the tags have been used from the IEEE LOM set of standard metadata for annotating learning materials.
However, we have added some tags locally to make the annotation scheme compatible with our system.
• A fuzzy state transition based student model which is an abstract representation of the students using the system. This model also uses two important parameters to evaluate the cognitive ability of a student. They are comprehension ability which tests a student’s ability to understand a concept or a topic. The other is problem-solving skills, which checks a student’s ability to apply a concept in solving problems. The student model provides the basis of adaptation offered by the system.
• Another important contribution of this work is a fuzzy rule based teaching agent which controls the adaptive teaching process of this system. This agent communicates with the other modules of the system and produces customized teaching plans for the individual students.
This module also keeps track of a studen’s changing requirements along a course and adapts itself accordingly.
However, there are some limitations in the present system. One of the limitation is that there are no facilities to provide hints to students. This might help a student immensely in problem solving and avoid getting stuck into a problem. The present student model is capable of capturing two aspects of a student. These two parameters are not enough to capture a complete picture of a student’s cognitive ability. The current system has been evaluated on the basis of the users who came from good urban schools. However, this system is to deploy in rural areas where the system can be quite relevant and might play an important role in improving the existing infrastructure in education. It is quite important to check how the system performs when users from those areas actually use the system. Therefore, a major future work is to deploy the system in various rural educational centers and evaluate its performance.
References
Aleven, V., McLaren, B. M., Sewall, J., & Koedinger, K. 2006. The Cognitive Tutor Authoring Tools (CTAT): Preliminary evaluation of efficiency gains. In M. Ikeda, K. D. Ashley, & T. W. Chan (Eds.), in 8th International Conference on Intelligent Tutoring Systems (ITS 2006), Berlin: Springer Verlag, pp. 61-70.
Ali, M.S., Ghatol, A.A. 2004, A Neuro Fuzzy Inference System For Student Modeling In Web-Based Intelligent Tutoring Systems, International Conference on Cognitive Systems, New Delhi, December 2004.
Baffes, P., and Mooney, R. 1996. Refinement-Based Student Modeling and Automated Bug Library Construction. In Journal of Artificial Intelligence in Education, 7, 1 (1996), pp. 75-116,
Beck, J.E. 1997. Modeling the student with reinforcement learning. Machine Learning for User Modeling Workshop at the Sixth International Conference on User Modeling, 1997.
Beck, J.E., Woolf, B.P. 2000, High-Level Student Modeling with Machine Learning, in 5th International Conference on Intelligent Tutoring Systems, G. Gauthier, C. Frasson, K. VanLehn (Eds.):
ITS 2000, LNCS 1839, pp. 584–593, Montreal, Canada, 2000.
Bloom, B.S. 1984. The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educational Researcher 13 (6):4–16.
Brown, J., Burton, R., deKleer, J. 1981, Pedagogical Natural Language and Knowledge Engineering Techniques in SOPHIE I, 11, and IIIH, Tutoring Systems, Sleeman et a1 (eds), Academic Press.
Brusilovsky, P., Vassileva, J. 2003. Course sequencing techniques for large-scale web-based education. In International Journal of Content Engineering and Lifelong Learning, Vol. 13, Nos. 1/2, 2003.
Bull, S., Pain, H. 1995. "Did I say what I think I said, and do you agree with me?”: Inspecting and questioning the student model, Proceedings of AI-ED'95 - 7th World Conference on Artificial Intelligence in Education, AACE, pp. 501-508.
Chou, C., Chan, T., & Lin, C. 2003. Redefining the learning companion: the past, present, and future of educational agents, Computers & Education, v.40 n.3, pp.255-269, April 2003.
Clancey, W. J. 1982, GUIDON, In Barr and Feigenbaum (editors), The Handbook of ArtificiaI Intelligence, chapter Applications-oriented Al research: Education. William Kaufmann, Inc., Los Altos, 1982.
Collins, J. A., Greer, J. E., Huang, S. X. 1996. Adaptive Assessment using Granularity Hierarchies and Bayesian Nets. In Frason, C., C., Gauthier, G. and Lesgold, A., (eds.), Proceedings of Intelligent Tutoring Systems ITS'96. Berlin: Springer, pp. 569-577.
Conati, C., Gertner, A., VanLehn, K., & Druzdzel, M. 2002. On-line student modeling for coached problem solving using Bayesian networks. In Proc. Sixth International Conference on User Modeling, Vienna , 1997, pp. 231-242.
D’Mello, S. K., Craig, S. D., Gholson, B., Franklin, S., Picard, R. W., & Graesser, A. C. 2005.
Integrating affect sensors in an intelligent tutoring system. In Affective Interactions: The Computer in the Affective Loop Workshop at International Conference on Intelligent User Interfaces 2005, San Diego, 2005, pp. 7-13.
Evens, M. W., Brandle, S., Chang, R., Freedman, R., Glass, M., Lee, Y. H., Shim, L. S., Woo, C. W., Zhang, Y., Zhou, Y., Michael, J. A., & Rovick, A. A. 2001. CIRCSIM-Tutor: An intelligent tutoring
system using natural language dialogue, in Proc. Twelfth Midwest AI and Cognitive Science Conference, Oxford, 2001, pp. 16-23.
Ferguson, K., Arroyo, I., Mahadevan, S., Woolf, B., & Barto, A. 2006. Improving intelligent tutoring systems: Using expectation maximization to learn student skill levels. In Proc. 8th International Conference on Intelligent Tutoring Systems, Taiwan, June, 2006, Lecture Notes in Computer Science, No 4053, pp. 453-462.
Freedman, R. 2000a, What is an Intelligent Tutoring System?, Published in Intelligence 11(3): pp. 15–
16.
Freedman, R. 2000b. Plan-based dialogue management in a physics tutor. Proceedings of the Sixth Applied Natural Language Processing Conference. Seattle, WA, pp. 52-59.
Gertner, A., & VanLehn, K. 2000. Andes: A coached problem solving environment for physics. In Proc. 5th International Conference. Intelligent Tutoring Systems, Berlin, 2000, pp. 133-142.
Horvitz, E., Breese, J. S., Heckerman, D., Hovel, D., Rommelse, K. 1998. The Lumiere Project:
Bayesian user modeling for inferring the goals and needs of soft-ware users. Fourteenth Conference on Uncertainty in Artificial Intelligence. San Francisco: Morgan Kauf-mann, pp. 256-265.
Kavcic A., Pedraza-Jimenez R., Molina-Bulla H., Valverde-Albacete F.J., Cid-Sueiro J., Navia-Vazquez A. 2003. Student modeling based on fuzzy inference mechanisms, EUROCON 2003.
Computer as a Tool. The IEEE Region 8, vol.2, no.pp. 379- 383 vol.2, 22-24 Sept. 2003
Khuwaja, Ramzan, A., Evens, M. W., Michael, J. A., & Rovick, A. A. 1994. Architecture of CIRCSIM-Tutor (v.3): A smart cardiovascular physiology tutor. In Proceedings of the 7th Annual IEEE Computer-Based Medical Systems Symposium, Winston-Salem, NC, 1994, pp. 158-163. IEEE Computer Society Press
Kimball, R. 1982, A self-improving tutor for symbolic integration. In D. Sleeman & J.S. Brown (Eds.), Intelligent Tutoring Systems, New York: Academic Press, pp. 283-308.
Kodaganallur, V., Weitz, R., & Rosenthal, D. 2005. A Comparison of Model-Tracing and Constraint-Based Intelligent Tutoring Paradigms. International Journal of Artificial Intelligence in Education, Vol.
15, pp. 117-144.
Martin, B., Koedinger, K., Mitrovic, T., & Mathan, S. 2005. On Using Learning Curves to Evaluate ITS. Twelfth International Conference on Artificial Intelligence in Education, pp. 419-426, Amsterdam.
McDonald, J. 1981, The Excheck CAI System, in University-Level Computer-Assisted instruction at Stanford: 1968-1980, P. Suppes, ed., Inst. for Math. Studies in the Social Sciences, Stanford Univ., Palo Alto, Calif., 1981.
Mitrovic, A. 2003. An intelligent SQL tutor on the web. International Journal of Artificial Intelligence in Education, 13(2-4), 2003, pp. 171-195.
Murray, T. 1999. Authoring Intelligent Tutoring Systems: An Analysis of the State of the Art.
International Journal of Artificial Intelligence in Education (1999), 10, pp. 98-129.
Ong, J., & Ramachandran, S. 2000. Intelligent Tutoring Systems: The What and the How.
http://www.learningcircuits.org/2000/feb2000/ong.htm
Reye, J. 1996. A Belief Net Backbone for Student Modeling, Proceedings of third International Conference on Intelligent Tutoring Systems, June 12-14, 1996, pp. 596-604.
Reye, J. 1998. Two phase updating of Student Models based on Dynamic Belief Networks, Proceedings of fourth International Conference on Intelligent Tutoring Systems, Aug 16-19, 1998, pp.
274-283.
Ross, T.J. 1997. Fuzzy Logic with Engineering Applications. McGraw-Hill, 1997.
Sleeman, D., & Brown, J. S. 1982. Introduction: Intelligent Tutoring Systems. Intelligent Tutoring Systems, D. Sleeman, J. S. Brown, Ed. Academic Press, 1982, pp. 1-11.
Thompson, J.E. 1996. Student Modeling in an Intelligent Tutoring System. MS Thesis.
VanLehn, K., Martin, J. 1997. Evaluation on an assessment system based on Bayesian student modeling. International Journal of Artificial Intelligence in Education, Vol. 8. pp. 179-221.
Villano M. 1992. Probabilistic Students Models, Bayesian Belief Networks and Knowledge Space Theory, Proceedings of the second International Conference on Intelligent Tutoring Systems, June 10-12, 1992, pp. 491-498.
Vomlel, J. 2004. Bayesian networks in educational testing. Int. J. Uncertain. Fuzz. Knowl. Based Syst.
12(Suppl. 1), 83–100 (2004)
Wang, T., & Mitrovic, A. 2002. Using neural networks to predict student's performance. In Proc. of International Conference on Computers in Education, 2002, pp. 969-973.
Webb, G. I., B. C. Chiu, and M. Kuzmycz 1997. Comparative Evaluation of Alternative Induction Engines for Feature Based Modelling. International Journal of Artificial Intelligence in Education 8.
Amsterdam: IOS Press, pages 97-115.
Wenger, E. 1987. Artificial Intelligence and Tutoring Systems. Los Altos, CA: Morgan Kaufmann.
Weon, S., & Kim J. 2001. Learning achievement evaluation strategy using fuzzy membership function.
In Proceedings of the 31st ASEE/IEEE Frontiers in Education Conference, Reno, NV, 2001, October pp. 10–13.
Xu, D., Wang, H., Su, K. 2002. Intelligent Student Profiling with Fuzzy Models, In Proceedings of the 35th Hawaii International Conference on System Sciences, Hawaii, USA (2002), p. 81b.
Zapata-Rivera, D., Greer, J. 2004. Interacting with Inspectable Bayesian Student Models. International Journal of Artificial Intelligence in Education, Vol 14. pg., 127 – 168
Zhou, Y., Evens, M. W. 1999a. A Practical Student Model in an Intelligent Tutoring System, ICTAI.
(Eds. Sajja & Akerkar), Vol. 1, pp 101 – 125, 2010