Experiment 2 to Test Hypothesis H2

The main purpose of the system evaluation was to assess the quality of the produced material.

Choosing the best teaching material that could suit the learning purposes is always a

challenging task for teachers (Ellis, 1997). Predictive and retrospective evaluation can be

conducted by teachers to evaluate available learning material. Predictive evaluation is carried

out by expert reviewers prior to delivering the course based on specific criteria, represented by

Module Title :Fundamental Programming Module Code: CF201

Aims of the Module

The aim of this module is to introduce the student of fundamental programming concepts and enhance their problem-solving. Students will learn the basics of scalar types (Integers, Strings, Booleans) and fundamental control structures in procedural programming (loops, assignment statements, conditional expressions). The module uses the C++ programming language as the implementation environment. This course also will allow the students how to implement file I/O, functions and recursion for solving a problem.

Learning Outcomes

1.Identify and describe uses of primitive data types. [Familiarity] 2.Write programs that use primitive data types. [Usage] 3.Write programs that use standard conditional [Usage] 4.Write program that use iterative control [Usage] 5.Write program that use functions. [Usage] 6.Write a program that uses file I/O. [Usage]

7.Choose appropriate conditional and iteration constructs for a given programming task. [Assessment] 8.Describe the concept of recursion and give examples of its use. [Familiarity]

Program Structure

Topic name Recommended Link Learning Hours Exercise Evaluation

Conditional Link 2 Link Is this useful ?

Loops Link 2 Link Is this useful ?

Variables Link 2 Link Is this useful ?

Functions Link 2 Link Is this useful ?

139

a checklist on how to achieve the course outcome (Ellis, 1997). On the other hand, retrospective

evaluation is carried out after the material has been used in a teaching context. After that, a

decision is made on whether or not the material has worked for learners. Despite the limitations

of predictive evaluation represented by the lack of well-defined formula and a subjective nature

(Sheldon, 1988), this type of evaluation was employed in this research due to time constrains.

Predictive evaluation was performed in this research by involving ten instructors experienced

in the field of computer science; they evaluated the quality of the material extracted by APELS

on whether it would satisfy the targeted learning outcome as defined by standard curricula.

Before this phase took place, a greater level of understanding was provided to the experts in

relation to the learning outcomes, as were defined in the ACM/IEEE Computing curriculum

(Sahami et al., 2013). In current research, the learning outcomes in ACM/IEEE are defined in

terms of three tasks: Familiarity, Usage and Assessment as depicted in Figure 6-6 and explained

in more details in Appendix B. The experts are requested to respond to the question “would the

content produced by APELS form a good learning material for the learners; a material that

could meet one of the targeted learning outcomes, namely familiarity, usage or assessment”.

140

Although the learning module was designed and returned to the expert, it was not possible to

evaluate all the information provided in the module specification page as it would take very

long time, therefore a controlled experiment was conducted to test three selected topics

(Recursion, Variable and Loop). This also helped in allowing a consistent view on these three

topics.

Therefore, two websites were used that the system returned for a specific module for a

particular learner; one with a high score (very suitable) and the other with a low score (not

suitable) according to the APELS ranking system. The experts were then asked to state whether

or not these websites satisfy the learning outcomes and whether or not the teaching material is

good for the learners.

The participating experts took their time to read the produced learning material and to evaluate

the content before answering the questions. They were asked to answer the following three

questions with simple “yes” and “no” answers.

Q1: Would you agree that this content satisfies the learning outcome, Familiarity?

This question is designed to elicit the experts’ opinion on the quality of the content delivered; whether or not it is associated with the Familiarity task. The experts would verify whether the

content provided definitions of the topic as well as important terminology and explanation that

help learners to fully comprehend the concept.

Q2: Would you agree that this content satisfies the learning outcome, Usage?

The second question is associated with the Usage task; it attempts to evaluate the content

produced by APELS. The domain experts assess whether the content provides example, block

of code, or flowchart that assist the learner to understand how to use or apply the concept

141

Q3: Would you agree that this content satisfies the learning outcome, Assessment?

The third question is designed to explore the experts’ opinion regarding the content quality

delivered: whether or not it is related to the Assessment task or not. The experts check whether

the content includes a simple introduction and some examples that clarify each concept in order

to enable the learner to select the method appropriate for a specific problem.

6.6 Analysis of the Experiments Results