Vol. 28, No. 8s, (2019), pp. 819-823
Evaluation of Pre-final Test on Improving Students’ Performance in Engineering Mathematics
N. Lohgheswary
Centre of Engineering Education Research,Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia,Bangi, 43600 Selangor, Malaysia.
S. Salmaliza
Centre of Engineering Education Research,Faculty of Engineering and Built Environment,SEGi University, Kota Damanasara, 47810 Selangor, Malaysia.
A. Wei Lun
Centre for Sustainable Process Technology (CESPRO),Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia,Bangi, 43600 Selangor, Malaysia.
Abstract—Engineering Mathematics expecially Vector Calculus, Linear Algebra and Ordinary Differential Equations has been vital courses in all engineering curriculum. Lack of fundamental knowledge of Engineering Mathematics will reflect badly on the students‟ performance when they are unable to grasp higher level of other engineering subjects. Thus the objective of this study is to cross examine the students‟ performance in Linear Algebra subject in Electrical Engineering department in a public university in Malaysia.
Students were given pre-final test based on the course outcomes of the Linear Algebra course. The results of the pre-final test were transferred to excel and then analyzed using Rasch model. A total of one hundred students from the Electrical Engineering department participated in the pre-final test. There were a total of five questions in the pre-final test which were evaluated by two lecturers. Each question was develop based on one course outcome from the Linear Algebra course. Rasch model able to categorize the exam questions according to the level of difficulties of the questions. Two questions were identified as difficult for the students in Linear Algebra course.
The course outcome related to the questions are vector space and power series. With this early identification, the existing teaching method need to be re-examined. Theories on Linear Algebra need to be reinforced to the students after several classes. Furthermore, more practical application questions should be discuss with students to bridge the theory of Engineering Mathematics among the engineering undergraduates.
Keywords—Pre-final test; Engineering Mathematics; Linear Algebra; performance;Rasch model
I. INTRODUCTION
Engineering Mathematics is one of the pillar or foundation subjects for all engineering courses. Therefore engineering students should have good grasp of fundamental Mathematics knowledge in their earlier years of engineering course. This foundational knowledge later will be applied in other engineering subjects as well as in solving real world engineering problem.
Linear Algebra is one of the compulsory subjects across the departments of chemical, civil, mechanical and electrical
engineering in a particular public university of Malaysia.
Linear Algebra course requires students to be able to understand, analyse and apply the concepts of matrices, vector space and series in solving practical engineering problem.
However, previous results for other batch of students revealed that students fail to understand the fundamental concept of Linear Algebra.
Many researchers have conducted research on examining the Programme outcome and Course Outcome. A study used an exit strategy to assess students‟ Programme Outcome [1].
An exit strategy consists of the exit survey, exit test and exit interview. All twelve Programme Outcomes are achieved with this exit strategy. In this study, only the exit survey and exit test were used to assess the Programme Outcomes. Both results from the exit survey and exit test agree with each other by representing the students‟ achievement of all Programme Outcomes.
Students‟ achievement of the Programme Outcomes for the Reinforced Concrete Design Course for civil engineering students was analysed [2]. The project which was given to the students was related to the real structural design project.
Programme Outcomes achievement is measured based on the final examination, mid semester examination, tutorials and group project. There was an increment of 9 – 39% on all Programme Outcomes on students‟ achievement between the two sessions. Moreover all Programme Outcomes have achieved more than 50% marks in the later session which made the study successful.
A study was conducted to evaluate the Course Outcomes for the Civil Engineering Design II course [3]. The Course Outcomes from the peer assessment contributes to the highest percentage in this study. The study concludes that students still have some difficulties in understanding design projects.
Rasch model is able to associate the pattern between students and the performance level of each Course Outcome whereby this cannot be achieved using the standard
Vol. 28, No. 8s, (2019), pp. 819-823 measurement method [4]. An assessment model was
developed based on students‟ mark register with the Rasch model [5]. This model measured students‟ performances in terms of the course Outcomes for the Civil Engineering Design II course. Rasch model results were compared with the conventional distribution marks. Rasch model results have similar patterns with the conventional method.
Students‟ achievement which based on the test, quizzes and assignment provides an overall indicator of students‟
cognitive ability [6]. Yet it is lacking in measuring achievement of individual. Thus stochastic Rasch model gives an insight view of individual performance. A study was conducted to analyse the performance of matriculation students in Physics examinations. The measure of questions gives a good fit measurement where else the measure of persons gives a fair fit.
Students‟ performance in Computing II subject was evaluated using Rasch model [7]. Every final examination question is mapped to the number of students who answered correctly. Based on the findings of 75 students from the Foundation of Engineering, students can answer fairly question from various level of difficulty of final examination questions.
II. METHODOLOGY
All the engineering students in a particular public university in Malaysia are subjected to take Vector Calculus, Linear Algebra and Ordinary Differential Equations subjects in their first, second and third semester of engineering undergraduate course. All the subjects are 4 credit hour subjects.
A pre-final test for Linear Algebra was conducted in semester II 2015 / 2016. A total of 100 students from Electrical Engineering department participated in the test.
Exam questions were constructed based on Course Outcome and Programme Outcome and the questions were validated by two experts. The duration of the test was 2 hours and 5 subjective questions were prepared. In constructing the pre- final questions, Bloom Taxonomy (Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation) were considered.
Table 1 list the Course Outcome for the Linear Algebra subject.
TABLE 1 Course Outcome for Linear Algebra subject Course
Outcome Description
1 Understand the fundamental concepts on the matrix and its basic operations and applications.
2
Able to use the concepts of vector space, linear independent in the space dimension and matrix transformation.
3 Able to apply the eigenvector and eigenvalue in engineering problems.
4 Able to use the diagonalization and quadratic forms in the matrix solution for engineering problems.
5 Able to understand the concepts of Power Series.
Table 2 shows the Programme Outcome for Linear Algebra subject.
TABLE 2 Programme Outcome for Linear Algebra subject Programme
Outcome Description
1 Engineering knowledge
2 Problem analysis
3 Design / development of solutions
4 Investigation
5 Modern tool usage 6 The engineer and society 7 Environment and sustainability
8 Ethics
9 Communication
10 Individual and team work 11 Lifelong learning
12 Project management and finance
Table 3illustrates the distribution of pre-final questions together with the marks.
TABLE 3Pre-final questions
Q Description Marks
1
The accompanying figure shows a network of one-way streets with traffic flowing in the directions indicated.
The flow rates along the streets are measured as the average number of vehicles per hour.
x4
x1
650
180 150 650
50 300
100 650 275
x2
x3
(i) Setup a linear system which solution provides the unknown flow rates.
4
1 (ii) Determine whether or not the system has a solution.
Justify your answer
6
1
(iii) Explain how you might use the least square methods
to estimate the flow rates on each street. Show all your work.
10
2
Determine whether or not the set of vectors under addition and scalar multiplication defined by
1 3 1 3 1 1 3 3
1 3 1 3
, 0, , 0, , 0,
, 0, , 0,
a a b b a b a b
k a a ka a
is a vector space. Use all 10 axioms.
15
3
Suppose that the temperature at a point ( , )x y on a metal plate is
2 2
( , ) 4 4 T x y x xyy .
A lady bug walks on the plate along a circle of radius 5 13
Vol. 28, No. 8s, (2019), pp. 819-823
Q Description Marks
centered at the origin.
(i) Use the Quadratic Forms to determine the highest and
lowest temperature encountered by the ladybug and state the point where it attains the highest and the lowest temperature.
3 (ii) Give the location and classification of the critical ( , )
T x y points .
7
4
Given the following matrix:
13 60 60 10 42 40 5 20 18 A
(i) Show that the matrix A is diagonalizable.
5
4 (ii) Find a matrix P that diagonalizes A. 5
5
Use the geometric series
2 3
1 1 ...
1 x x x
x
to find the series representation of ln 1 x .
5
The details entry of each pre-final question is given in Table 4.
TABLE 4Entry number for pre-final questions
Q Course Outcome
Programme Outcome
Bloom Taxonomy
Level
Description
1(i) 1 1 1 Knowledge
1(ii) 1 1 2 Comprehension
1(iii) 1 1 3 Application
2 2 1 2 Comprehension
3(i) 3 2 3 Application
3(ii) 3 2 2 Comprehension
4(i) 4 2 3 Application
Q Course Outcome
Programme Outcome
Bloom Taxonomy
Level
Description
4(ii) 4 2 2 Comprehension
5 5 1 2 Comprehension
III. RESULTSANDDISCUSSION
All the marks are entered in the Excel *prn format. This file was transferred using Bond & Fox Steps [8], which is a customized WINSTEPS. The WINSTEPS provides detail description on Summary Statistics and Person-Item Distribution Map.
Summary Statistics gives detail description of the items (questions) and the persons (students).Figure 1 shows the summary statistics for items.
---
| TOTAL MODEL INFIT OUTFIT |
| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD |
|---|
| MEAN 225.4 79.0 .00 .09 1.05 .0 1.11 .1 |
| S.D. 92.3 .0 .54 .02 .23 1.0 .44 1.0 |
| MAX. 351.0 79.0 .82 .13 1.40 1.4 2.02 1.6 |
| MIN. 106.0 79.0 -.73 .07 .78 -1.2 .60 -1.3 |
|---|
| REAL RMSE .10 TRUE SD .53 SEPARATION 5.37 Item RELIABILITY .97 |
|MODEL RMSE .09 TRUE SD .53 SEPARATION 5.85 Item RELIABILITY .97 |
| S.E. OF Item MEAN = .22 | ---
Figure 1 Summary statistics for items
The summary statistics reveals an excellent spread of item difficulty with item reliability of 0.97. Higher value of item reliability increases the possibility of the item having similar attempt when given similar test [9]. This means among this pre-final questions there are some which is difficult and some which is easy. The spread of the items are from minimum logits of 10.73 to maximum logit ar 0.82.
Item mean (measure) is 0. The item separation indicates
number of the groups the question can be divided. The summary statistics shows that the item separation is 5.37.
Figure 2 is the Person Item Distribution Map where Rasch model provides the location of all the person on the vertical logit ruler, indicated by the dashed vertical line. The higher the person located on the map, it indicate the most competent the student is. On the contrast, the item is like the hurdle to the students. The higher the item is on the ruler, the more difficult the question is.
Vol. 28, No. 8s, (2019), pp. 819-823 The discussion aims at the performance of the item of all
the 9 questions spread on the logit scale. On the right hand side of the dashed line, the questions are aligned from easy to difficult, starting from the bottom. The distribution of students is on the left side of the vertical dashed line is increasing the order of ability. Letter „M‟ describes the students and item mean, „S‟ is one standard deviation distance from the mean and „T‟ marks two standard deviation distances from the mean. Since the number of students who participated in the pre-test were 100, all their metric number are not shown in Figure 2. Instead that, „#‟
represents 5 students. Below than 5 students, it will be represented by „.‟.
From Figure 2, the questions can be divided into 3 groups. In order to establish the cut-off point of the 3 groupson the logit ruler, it starts from the mean item 0.00 logit. At mean 0.00, Rasch model established a virtual zero of the logit ruler [10]. At this point, given a task, a person has 50:50 chance of success in performing the task, before
he decides based on his experience that he may or may not be able to complete the task.
Questions above the mean line are defined as being in the „moderate‟ and „difficult‟ category. From a total of 9 questions, 2 questions fall into the „difficult‟ and 4 questions fall in „moderate‟ category. The questions in the „difficult‟
category are question 2 and question 5. Question 3 (i), question 4 (i), question 4 (ii) and question 1 (ii) belongs to
„moderate‟ group. On the other hand, questions below the mean line are easy for the students to answer during the pre- final test. Question 1 (i), question 1 (iii) and question 3 (ii) are the ones deemed easy to solve.
In terms of the Course Outcomes, Course Outcome 1 and Course Outcome 3 fall into the easy category. In Course
2 + 2 5 |
| | | | | | | | | | | | |T 1 + . | | | 3(i) ## | | T|
|S .# | # |
| 4(ii) S| 1(ii) 4(i) # |
.### | .### | 0 +M M|
.######### | | .######## | ## | 1(iii) .#### S|
.## | |S |
| 3(ii) .T| 1(i) | |
| -1 +
Sukar
Sederhana
Mudah
DifficultModerate
Easy
Figure 2 Person-Item Distribution Map
Vol. 28, No. 8s, (2019), pp. 819-823
Outcome 1, students perform well in questions related to knowledge and application. In addition, Course Outcome 3 related to comprehension was also easy for the students.
Therefore, the fundamental concepts on the matrix and its basic operation, eigenvector and eigenvalue are considered as easy chapters for the Linear Algebra subject. Other Course Outcomes for instance Course Outcome 2 and Course Outcome 5 were difficult for students to solve.
Students are not able to answer of the comprehension questions for both these Course Outcomes.
Course Outcome 4 was moderate for the students. This means the questions are not difficult or easy to be answer. In particular, for Course Outcome 4, students are average in the comprehension and application stages. It is noted that students are average problem solvers in the concepts of vector space, diagonalization and quadratic forms and power series.
There exists a big gap between questions in the difficult category. This gap is between question 3 (i) and question 2 and question 5. This gap is indicated by the line with both sides arrow in Figure 2. The difficulty level of both questions 2 and 5 is extremely high for the engineering students. Although both questions are comprehension level from Bloom Taxonomy, students do badly in answering these questions which are related to Course Outcome 2 and Course Outcome 5.
Table 5 summarizes the results of the person-item distribution map.
TABLE 5 Summary of the person-item distribution map
Question
Course Outcome
Level of Bloom’s Taxonomy
Difficulty Level
1(i) 1 Knowledge Easy
1(ii) 1 Comprehension Moderate
1(iii) 1 Application Easy
2 2 Comprehension Difficult
3(i) 3 Application Moderate
3(ii) 3 Comprehension Easy
4(i) 4 Application Moderate
4(ii) 4 Comprehension Moderate
5 5 Comprehension Difficult
From Table 5, two Course Outcomes are identified as difficult Course Outcomes for Linear Algebra subject. Table 6 summarizes the difficult Course Outcome for Linear Algebra subject.
TABLE 6 Difficult Course Outcome Course
Outcome Description
Level of Question 2 Able to use the concepts of vector space, Difficult
linear independent in the space dimension and matrix transformation.
5 Able to understand the concepts of Power Series.
Difficult
IV. CONCLUSION
Performance of students in Linear Algebra course was assess using Rasch model. Rasch model is capable of measuring the reliability of the intended questions and to group the students according to their level of understanding on the course. The correlation level between the performance of students from each department and the pre- final questions were identified.
The Person Item Distribution Map showed that the pre- final test questions can be classified into 3 groups. They are difficult, moderate and easy. The pre-final test questions need to be revised to reduce the gap between the difficult questions in the Person Item Distribution Map.
Acknowledgment
The authors wish to express gratitude towards SEGi University(SEGiIRF/2018-14/FoEBE-21/84) and Universiti Kebangsaan Malaysia(GUP-2018-151) for supporting the research.
References
[1] A. Ayob, H. F. A. Wahab, N. arshad, A. A. A. Bakar and H. Husain,
“Assessment of student Programme Outcomes through a comprehensive exit strategy”Procedia Social and Behavioral Sciences, vol 18, pp. 33-38,2011.
[2] S. A. Osman, W. H. W. Badaruzzaman, R. Hamid, K. A. Taib, and A.
K. Rashid, “The Programme oUtcomes achievement for basic Reinforced Concrete Design Course between two different sessions,”
Procedia Social and Behavioral Sciences, vol 18, pp. 260-285, 2011.
[3] S. A. Osman, O. Jaafar, W. H. W. Badaruzzaman and R. A. A. O. K.
Rahmat, “The Course Outcomes Evaluation for Civil Engineering Design II course,” Procedia Social and Behavioral Sciences, vol 60, pp. 103-111, 2012.
[4] M. Saifuddin, A. A. Azrilah, N. A. Rodzo‟an, M. Z. Omar, A.
Zaharim and H. Basri, “Easier learning outcomes analysis using Rasch model in engineering education research, Proceedings of the 7th WSEAS International Conference on Engineering Education, pp.
442-227, 2010.
[5] S. A. Osman, S. I. Naam, O. Jaafar, W. H. W. Badaruzzaman and R.
A. A. O. K Rahmat, “Application of Rasch model in measuring students‟ performance in Civil Engineering Design II course”, Procedia Social and Behavioral Sciences, vol 56, pp. 59-66, 2012.
[6] F. M. Ibrahim, A. A. Aziz, Y. Z. Zubairi and A. Zaharim, “Using the Rasch model to assess examination beyond students‟ scores,”Global Engineering Education Conference, 2012.
[7] R. F. M. Said, “Application of Rasch measurement model in evaluating student performance for foundation of Computing II”, 7th International Conference on University Learning and Teaching Proceedings, pp. 251-259, 2014.
[8] T. G. Bond and C. M. Fox, “Bonds & Fox steps (Customize version of WINSTEPS) Computer Software,” 2006.
[9] J. M. Linacre, “Sample size and item calibration stability”, rasch Measurement Transactions, vol. 7, pp. 328, 1994.
[10] A. A. Aziz, “Rasch model fundamentals: Scale construct and measurement structure”, 2009.
