Volume 3, Issue 3, March 2014
Page 214
ABSTRACT
Educational assessment is usually focused on determining how well students are learning and has become an integral part of the quest for improving education, and assessment method is also required to be simple, understandable, easily applied but objective and accurate. The purpose of this paper is to propose a new application of Rasch GSP which can evaluate the difficulty of questions in educational testing. This method promotes the advantages of Rasch model and Grey Student-Problem (GSP) chart to create the creative application of evaluation on the basis of statistics and arrangement in order of priority criteria. In order to identify the rationality of method, the experimental results of the method have been compared with results processed by BILOG-MG 3 software which implemented the task based on Item Response Theory (IRT). The research result shows that the advantage of the proposed method is simple, and easy to apply. Especially, it is not only best for handling small samples but also applicable for large sample. This new application is a more convenience way for teachers and education managers to carry out the evaluation process.
Keywords: assessment method, Rasch GSP, GSP, small sample, large sample.
1.
INTRODUCTION
Assessment plays an integral role in teaching and learning, it is used in classrooms to help students learn, so it must be transformed in two fundamental ways that the content and character of assessments must be significantly improved and the gathering and use of assessment information must become a part of the ongoing learning process [1].Whether within the scope of a class or within the wider scope of a school, evaluating academic achievement of students occurs frequently to ensure the provision of timely feedback results in service of teaching, it requires evaluation results to be obtained quickly but accurately and objectively. Therefore, the construction and design of models and assessment methods that have high effectiveness are interesting. An important requirement for assessment methods is that be accurate and easy to apply but also objective, valid, reliable, and practicable to meet the needs of teaching[2].
In the classroom space, the Student-Problem chart (S-P chart) is an evaluation method, which effectively evaluates the students’ results of learning through the tests, it has been in use for many years. In 1969, S-P chart was invented by Takahiro Sato, it can help teachers diagnose abnormal performance. The main purpose of the S-P chart is to get the diagnostic data of each student, and teachers can provide better advise for each student academically depending on this analyzed data[3].In 1982, Deng proposed Grey system theory wherein grey relational analysis is an effective mathematical tool. Grey relational analysis measures the degree of similarity or difference between two sequences based on the grade of relationship between them[4]. In order to overcome the weaknesses of S-P chart which only processed dichotomous data, Nagai proposed GSP chart in 2010, it is a combination of S-P chart and Grey system theory to analyze S-P chart data more specifically. With GSP chart analysis, the uncertainty factors in the study are analyzed clearly[5].
In 1960, Rasch introduced a model for analyzing the test data whose test broadly refers to an assessment of an examinee’s level of ability in a particular domain such as math or reading, or a survey of an examinee’s behaviors or attitudes toward something[6]. The aim of this model is to measure each examinee’s level of a latent trait that underlies his or her scores on items of a test, so it suitable for the analysis of large data. The concept of item response theory (IRT) was known during the 1950s and 1960s, but it did not become widely used until the late 1970s and 1980s by three of the pioneers, namely Lord, Rasch, and Lazarsfeld who pursued parallel research independently. The purpose of IRT is to provide a framework for evaluating how well assessments work, and how well individual items on assessments work. The most common application of IRT is in education, where psychometricians use it for developing and designing exams, building item banks for exams, and equating the difficulties of items for successive versions of exams [7]. In practical application, the parameters of IRT are estimated by computer programs because of the vast number of parameters that must be estimated. One of the most popular software used is BILOG-MG [8].
The view of the Rasch model was applied to GSP chart was a creativity suggestion by Nagai in 2010. This theory has become a method to judge uncertain factors [9]. Rasch GSP can make problem analysis more specific and clear, it shows the subject’s identity status completely and provides the best result which can then be used to provide digital curriculum design and aid development of a reference index [10].
An Applied Research of Rasch GSP for
Evaluating Difficulty of Test Questions
Tian-Wei Sheu1, Phung-Tuyen Nguyen2, Duc-Hieu Pham3, Phuoc-Hai Nguyen4and Nagai Masatake5
1,2,3,4,5
Volume 3, Issue 3, March 2014
Page 215
S-P chart analysis method has been used popularly for many years, however it was only suitable for small samples; on the other hand, it only evaluated students and questions at a general level with no specific arrangement of difficulty order of questions. Meanwhile, BILOG-MG software processed data based on IRT theory which gave the evaluation results with specific question difficulties, however, the samples which satisfied its assumptions are always large statistical data sets. It is hypothesized that it was possible to construct an assessment method which not only applied for small sample but could also handle large statistical data sets. The current study proposes a new application of Rasch GSP which can evaluate the difficulty of questions in the test, it could be applied for both small and large samples.The Rasch GSP is implemented as a method of non-parametric statistics for small samples, its success has been proven through some studies[10, 11], now this study would like to propose its new application. In order to show its features, the study applies it in educational testing with large statistical sample whose data collected from 46-question test with more than 1000 students. Evaluation results are compared with the results processed by BILOG-MG 3 software to determine the rationality.
2.
BASIC THEORY
This study proposes a new application of Rasch GSP method to evaluate difficulty of questions, and then its experimental results are compared with treatment results of BILOG-MG 3. So, the basic theories needed to introduce as follows:
2.1 Rasch model
The Rasch model was named after the Danish mathematician Georg Rasch. The model shows what should be expected in responses to items if measurement is to be achieved [12]. The model assumes that the probability of a given respondent affirming an item is a logistic function of the relative distance between the item location and the examinee location on a linear scale.
Georg Rasch first announced this model for analyzing the response of the answerers to obtain an objective interval scale that can measure the latent trait of an answerer [13]. In the Rasch model, the correct response probability of a student is a logistic function of the difference between that student’s ability and the item difficulty [14]. The relation between latent trait (theta) and correct response probability is described by an item characteristic curve (ICC). The Rasch model has descriptive function and predictive function. In descriptive function, this model can clearly explain the relationship between student’s ability and item difficulty, the difference between students and the difference between items. In predictive function, this model can predict the probability of a student who has a specified ability to answer a specified item correctly [13].
Figure 1 ICC for three different items in Rasch model
2.2 Student - Problem Chart
S-P chart is known as a method that can analyze, process, and arrange data in a defined order, it is very useful for diagnosing the learning state of student and question quality[15, 16].
Definition 1: S-P chart matrix Let
n m
ij x
X[ ] be the student response matrix and called S-P chart matrix, wherei1,2,,mis the order of student,
n
j1,2,, is the order of question,m,nN, and
j i j i ij x problem answer to correct gives student , 1 problem answer to wrong gives student ,
0 (1)
Caution Index for Student (CS):
) )( ( ) ( ) )( ( ) )( ( 1 1 1 x x x x x x x CS i l j j i n j j ij i
where
n j j x n x 1 1and
n j ij i x x l 1 (2)
Caution Index for Problem (
CP
):) ' )( ( ) ( ) ' )( ( ) )( ( 1 ' 1 1 x x x x x x x CP j l i i j m i i ij j
where
m i i x m x 1 1' and
m i ij j x x l 1
Volume 3, Issue 3, March 2014
Page 216
Based on the CS value and the rate of problems being answered correctly by student, the students would be classified and diagnosed, similarly, based on the CP value with the rate of students answering problem correctly, the problems would be also classified.2.3 Grey Relational Analysis (GRA)
In grey system theory, GRA is an effective mathematical tool to treat the uncertain, multiple, discrete and incomplete information. This study would like to refer the localized grey relational grade which is proposed by Nagai [17]. Its procedure as follows:
2.3.1 Establishment of original vectors Definition 2: Basic vectors in GRA
The reference vector
x
0 and inspected vectors of original datax
i are established as followed: )) ( , ), ( , ), 2 ( ), 1 ( ( 0 0 0 00 x x x j x n
x
N n m n j m i n x j x x x x n x j x x x x n x j x x x x n x j x x x x m m m m m i i i i i , ; , , 2 , 1 ; , , 2 , 1 )) ( , ) ( ), 2 ( ), 1 ( ( )) ( , ) ( ), 2 ( ), 1 ( ( )) ( , ) ( ), 2 ( ), 1 ( ( )) ( , ) ( ), 2 ( ), 1 ( ( 2 2 2 2 2 1 1 1 1 1 (4) There is n m ij x j i x X
[ ( )] [ ] representing original data in S-P chart, if Si is called a series of answered states by the i-th
student for all questions and Pj is a series of answered states by all students for the j-th question, then:
)) ( , ), ( , ), ( ), ( ( )) ( , ), ( , ), 2 ( ), 1 ( ( 2
1 j x j x j xm j
x P n x j x x x S i j i i i i i (5)
The data of S-P chart which have been shown in formula (5) will be applied for GRA. 2.3.2 Calculation of GRA
In original formula,
x
0is reference sequence of local grey relational grade, andx
iare inspected sequences. The established sequences have to satisfy three conditions: non-dimension, scaling, polarization. Grey relational generation has three ways: larger-the-better (the expected goal is bigger the better), smaller-the-better (the expected goal is smaller the better), and nominal-the-better (the expected goal is between maximum and minimum).Definition 3: Localized grey relational grade (LGRG) The localized grey relational grade is defined as follows:
min max
0 max 0
0 ( ( ), ( ))
i i
i x j x j (6)
wheremax and minare maximum value and minimum value of 0irespectively, 0i is the absolute distance between
0
x
andx
i, its formula as follows: 1 1 0 0
0 (( ( ) ( )) )
n
j i
i
i x x x j x j (7)
i
0
is called Minkowski distance. This study applies2, so 0iis also known as Euclidean distance.
When 0i is close to 1, it means that
x
0 andx
iare highly correlated, in contract, 0i is close to 0, the relationship betweenx
0andx
i is lower.Grey relational ordinal: The whole decision-making is made by the comparison of the grey relation0i. Through the ordinal, different causes can be identified, and the most important influence can be found, becoming the relational standard in the system.
2.4 GSP chart and Rasch GSP
GSP chart is the combination of GRA and S-P chart, it was developed in order to overcome the weaknesses of the S-P chart. GSP chart can make the analysis more concrete and accurate, and the uncertain factors in the studies can also be analyzed [18]. Its description is shown in Table 1.
Definition 4: Gamma value
In GSP chart, GSi is the localized grey relational grade of the i-th student, and GPjis the localized grey relational grade
Volume 3, Issue 3, March 2014
Page 217
mi
GS i
i
i , 1,2, ,
min max
0 max
0
; GPj j j ,j 1,2, ,n
min max
0 max
0
(8)
Table 1: GSP chart
Nagai applied the view of Rasch model in GSP chart to propose the Rasch GSP method, it used logistic regression to analyze the relationship between two sets of data which were sets of the order value of students (or problem) and the localized grey relational grades. The purpose was to find a function that represented the characteristics of the entire data [11]. This function is called Rasch GSP function, and its graph is called the Rasch GSP graph.
Definition 5: Rasch GSP function
Let ( )
1 1 )
(
x
e x
f
y
(9)
be the three-parameter logistic regression function, where ,,Rare regression coefficients. When x is the order of
student ability or the order of item difficulty and y is the localized grey relational grade, the above functiony f(x) is called Rasch GSP function.
If x is the order of student ability and y is LGRG-S then the above functiony f(x) is called Rasch GSP function for students, similarly, if x is the order of item difficulty and y is the LGRG-P then y f(x) is called Rasch GSP function for problems.
2.6 Item Response Theory with BILOG-MG software
IRT has become one of the most popular scoring frameworks for measurement data. IRT models are used frequently in computerized adaptive testing, cognitively diagnostic assessment, and test equating. One of the programs that exists for this purpose is BILOG-MG, that has proven particularly useful and reliable over recent decades for many applications. In order to fit into IRT models estimated with BILOG-MG, experimental data have to satisfy three assumptions, these are local independence, monotonicity, and uni-dimensionality[19].
For estimation of the IRT model parameters in BILOG-MG, the degree of bias and estimation error for parameter estimates depends on factors such as the number of parameters, number of examinees and test length. If any general guidelines can be given, it appears that for tests with between 15 and 50 items, approximately at least 250 examinees are required for the one-parameter logistic model, and two-one-parameter logistic model approximately at least 500, maybe even 1,000 examinees are required for the three-parameter logistic model [20].
3.
METHODOLOGY
In this paper, three terms are used in the same meaning: problem = question = item.
This study creatively applies Rasch GSP method to promote its effectiveness in assessing academic achievement, the process of applying is presented in the following diagram (Figure 2).
Figure 2 Diagram of new application of Rasch GSP method Raw data
Analyze data by GSP chart
Rearrange data by CS, CP
Apply regression method by using Rasch GSP curve
Volume 3, Issue 3, March 2014
Page 218
The main purpose of study is to evaluate the difficulty of questions, so the process of processing data is carried out through the algorithm which is presented as follows:Algorithm and process of processing data to evaluate the difficulty of questions in the test: Step 1: Calculate and arrange the data with GSP
The original data are processed and arranged according to the rule of GSP chart, the results are presented according to specifications in table 1 (in the basic theory section).
Step 2: Give priorities in order to arrange the data according to Gamma value
The questions are arranged in the order of their Gamma values (GP) from small to large. Step 3: Rearrange the order according to the caution index for problem (CP)
The questions, which have the same Gamma value, are rearranged in order of CP value from large tosmall. Step 4: Apply the logistic regression method for arranged Gamma values according to the above steps The Rasch GSP method is applied, Rasch GSP curve is plotted for the entire data set.
Step 5: Match the new Gamma values for all the questions
Each question corresponds to a point on the Rasch GSP curve, the value corresponding to each question on the y axis is its new Gamma value that needs to be determined. The difficulty of the questions is calculated according to new Gamma value.
4.
EXPERIMENT, RESULTS, AND DISCUSSION
4.1 Experimental designThe study has performed two experiments to test the Rasch GSP method about the ability to assess the question difficulty, those are:
Experiment 1 conducted a 22-question Math test for 37 students in Taichung, Taiwan, the obtained result had Cronbach’s Alpha reaching 0.884. Using the MATLAB toolbox for Rasch GSP to analyze the data and applying the method mentioned, the result would be obtained and compared with the result processed based on Classical Test Theory.
In order to demonstrate the strengths of the Rasch GSP method, the research continued to conduct an experiment with large statistical data. Experiment 2 conducted a 46-question English test for 1119 students also in Taichung, the test result also had high Cronbach’s Alpha, it reached 0.833. This satisfied the assumptions of IRT and estimation of IRT model parameters in BILOG-MG. Data were analyzed using the Rasch GSP method to give assessment results, then these data were also analyzed with the BILOG-MG 3 software, the two analytical results were compared with each other.
4.2 Results
The result of Rasch GSP analysis (as shown in Figure 3) is Rasch GSP curve for problem that is mentioned in step 4 of the methodology section.
Figure 3 Rasch GSP curve for problem in experiment 1
To show the result of evaluating difficulty of each question, a coordinate method was applied. The difficulty of questions were calculated and exported from a program written by MATLAB (Program for Rasch GSP) and presented in Table 2, they were arranged in order from small value to large value. This result was compared with the result calculated by Classical Test Theory (Table 3). The questions were highlighted in two tables to compare together to show that the order of their difficulties were arranged in the same position for the two methods.
Table 2: Evaluation result for questions difficulty by the proposed method in experiment 1
Question-number 9 11 8 1 6 19 2 5 21 7 16
Difficulty 0.153 0.176 0.201 0.229 0.261 0.295 0.331 0.370 0.412 0.445 0.449
Question-number 10 15 14 12 17 4 20 18 13 22 3
Volume 3, Issue 3, March 2014
Page 219
Table 3: Evaluation result for question difficulty by Classical Test Theory in experiment 1Question-number 9 11 8 1 6 19 2 5 21 7 16
j
D 0.865 0.838 0.784 0.757 0.757 0.730 0.730 0.730 0.676 0.649 0.649
Question-number 10 15 14 12 17 4 20 18 13 22 3
j
D 0.622 0.622 0.595 0.568 0.541 0.541 0.514 0.486 0.459 0.405 0.351
In Table 3, the difficulty Dj of the j-th question was calculated by the following formula:
N n
Dj j
Where njis the number of students who answered the j-th question correctly, N is the number of students taking this test.
The larger the Dj value is the smaller the question difficulty gets, so the questions are arranged by value of Dj from large
to small.
In experiment 2, Rasch GSP analysis was applied (Figure 4), the difficulty of questions were put out from Program for Rasch GSP and then arranged in order from small value to large value (Table 4).
Figure 4 Rasch GSP curve for problem in experiment 2
Figure 5 Evaluation results for questions processed by BILOG-MG 3 in experiment 2
Volume 3, Issue 3, March 2014
Page 220
represented by the parameter b were also arranged from small value to large value (Table 5) in order to compare with the results of the proposed method, question number 1 corresponds item 01 and question number 2 corresponds item 02, etc.Table 4: Evaluation results for question difficulty by the proposed method in experiment 2
Question-number 5 22 28 36 20 2 9 16 29 19 17 31
Difficulty 0.156 0.165 0.174 0.184 0.194 0.205 0.216 0.227 0.239 0.252 0.265 0.278
Question-number 6 24 23 21 18 33 25 40 30 41 26 39
Difficulty 0.292 0.307 0.322 0.338 0.354 0.370 0.388 0.405 0.423 0.442 0.461 0.480
Question-number 42 4 34 35 44 1 45 43 15 32 13 14
Difficulty 0.499 0.520 0.540 0.561 0.581 0.603 0.624 0.645 0.667 0.688 0.710 0.731
Question-number 10 3 27 38 12 7 11 46 37 8
Difficulty 0.753 0.774 0.796 0.817 0.837 0.858 0.878 0.898 0.918 0.937
Table 5: Evaluation results for question difficulty from BILOG-MG 3 in experiment 2
Question-number 5 22 28 36 20 2 9 16 29 19 17 31
Difficulty -4.141 -3.99 -2.934 -2.896 -2.813 -2.647 -2.626 -2.576 -2.536 -2.243 -1.992 -1.971
Question-number 6 24 23 21 18 33 25 40 30 41 26 39
Difficulty -1.936 -1.882 -1.796 -1.758 -1.665 -1.641 -1.558 -1.305 -1.305 -1.264 -1.088 -0.902
Question-number 42 4 34 35 44 1 45 43 15 32 13 14
Difficulty -0.879 -0.838 -0.793 -0.792 -0.729 -0.668 -0.663 -0.628 -0.615 -0.468 -0.186 -0.157
Question-number 10 3 27 38 12 7 11 46 37 8
Difficulty 0.113 0.130 0.201 0.247 0.310 0.784 1.427 1.450 1.782 2.070
4.3 Discussion
It was found that the proposed method has evaluated the question difficulty in the test and given the evaluated difficulty of each question. In experiment 1 with small sample, the sorting the order of question difficulty by the proposed method and the Classical Test Theory were similar, namely at highlighted position of 11 questions (50%). At these positions, the difficulties of the questions are completely different, so they are arranged in logical order. At the positions which are not highlighted, the proposed method distinguished the difficulty of questions obviously, so they are also arranged in logical order. Meanwhile, according to Classical Test Theory, these questions are not arranged in order clearly, namely at two questions number 1 and 6, three questions number 19, 2 and 5, two question number 7 and 16, two questions number 10 and 15, two questions number 17 and 4, because they have the same difficulty values. This weakness is due to the fact that the Classical Test Theory did not consider the priority criteria, the questions which have the same correctly answered rate, always have the same difficulty while the ability of students are different from each other. This shows the advantages of the proposed method which has already considered the quality and reliability of questions.
In experiment 2 with large sample, the results of arrangement of questions according to their difficulties were completely similar for the results processed by two methods. This confirms the rationality and reliability of the proposed method which can be used to handle large sample. The basic difference is that the difficulty distance estimated between two corresponding questions of two methods are not similar, the reason is the difficulty scale of question of two methods are not similar. This is also a limitation of the proposed method.
5.
CONCLUSION
With the assumption that it was possible to build an educational assessment method which can be applied to both small and large statistical samples, the present study proposed a new evaluation method based on a new application of Rasch GSP in evaluating question difficulty. The contributions of this study are as follows:
1. The study has developed a method of assessment that does not only well apply for small statistical samples, but also handle large statistical samples.
2. Evaluation results of the proposed method are compared with the evaluation results of the previous methods, the fit is relatively high and shows the advantages of the new method.
3. The advantage of the proposed method is that it is simple, easily understandable, and easy to apply; the processed results are accurate and objective.
Volume 3, Issue 3, March 2014
Page 221
References
[1] L. A. Shepard and L. A. Sheppard, "The role of classroom assessment in teaching and learning," 2000.
[2] D. Pulakos, "Selection assessment methods," SHRM Foundation’s Effective Practice Guidelines, Society for Human Resource Management, Alexandria, VA, p. 55, 2005.
[3] M. N. Yu, Educational Testing and Assessment, Third ed. Taiwan: Psychology Publisher, 2011.
[4] K. L. Wen, C. Chao, H. Chang, S. Chen, and H. Wen, Grey system theory and applications. Taipei: Wu-Nan Book Inc, 2009.
[5] T. W. Sheu, C. P. Tsai, J. W. Tzeng, D. H. Pham, H. J. Chiang, C. L. Chang, et al., "An Improved Teaching Strategies Proposal Based on Students' Learning Misconceptions," International Journal of Kansei Information, vol. 4, pp. 1-12, 2013.
[6] Karabatsos, "The Rasch model, additive conjoint measurement, and new models of probabilistic measurement theory," Journal of Applied Measurement, vol. 2, pp. 389-423, 2001.
[7] R. K. Hambleton, H. Swaminathan, and H. J. Rogers, Fundamentals of item response theory: Sage, 1991. [8] M. Du Toit, IRT from SSI: Bilog-MG, multilog, parscale, testfact: Scientific Software International, 2003.
[9] T. W. Sheu, J. W. Tzeng, J. C. Liang, B. T. Wang, and M. Nagai, "The Use of Rasch Model GSP Chart and Grey Structural Model Analysis for Vocational Education and Training Courses: Taking Enterprise Ethics and Professional Ethics Courses as an Example," Journal of Educational Research and Development, vol. 8(4), pp. 53-80, 2012.
[10]J. W. Tzeng, T. W. Sheu, J. C. Liang, B. T. Wang, and M. Nagai, "A new proposal based on raschmodel GSP chart and grey structural model withanalysis and discussion," International Journal of Advancements in Computing Technology, pp. 111-121, 2012.
[11]J. W. Tzeng, T. W. Sheu, J. C. Liang, B. T. Wang, and M. Nagai, "A New Proposal Based on Rasch Model GSP Chart and Grey Structural Model with Analysis and Discussion.," International Journal of Advancements in Computing Technology, vol. 4, pp. 111-121, 2012.
[12]Rasch, Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press, 1960. [13]W. C. Wang, "Rasch Measurement Theory and Application in Education and Psychology," Journal of Education &
Psychology, vol. 27, pp. 637-694, 2004.
[14]B. Baker, The basics of item response theory, Second ed. United States of America: ERIC Clearinghouse on Assessment and Evaluation., 2001.
[15]Y. C. Ho, "An Experimental Study of the Effects of Mastery Learning Combined with the Diagnosis of Microcomputerized S-P Chart Analysis on Students' Learning," Educational Psychology, vol. 22, pp. 191-214, 1989. [16]S. C. You and M. N. Yu, "The Relationships among Indices of Diagnostic Assessments Knowledge Structures and S-P
Chart Analysis," Education and Psychology, vol. 29, pp. 183-208, 2006.
[17]Yamaguchi, G.-D. Li, and M. Nagai, "Verification of effectiveness for grey relational analysis models," Journal of Grey System, vol. 10, pp. 169-181, 2007.
[18]T. W. Sheu, T. L. Chen, J. W. Tzeng, C. P. Tsai, H. J. Chiang, C. L. Chang, et al., "Applying Misconception Domain and Structural Analysis to Explore the Effects of the Remedial Teaching," Journal of Grey System, vol. 16, pp. 17-34, 2013.
[19]A. Rupp, "Item response modeling with BILOG-MG and MULTILOG for Windows," International Journal of Testing, vol. 3, pp. 365-384, 2003.
Volume 3, Issue 3, March 2014
Page 222
Appendices
Table 6: Response of students in experiment 1
Student Code Students’ response Number of correct
11201 1 1 0 1 0 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 13
11202 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 22
11203 1 1 1 1 1 1 0 0 0 0 1 1 0 0 1 0 1 0 0 0 0 0 10
11204 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 1 0 0 0 0 4
11205 1 0 1 0 1 0 0 1 1 1 1 0 0 0 0 0 0 0 1 0 0 1 9
11206 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20
11207 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 22
11208 1 1 1 0 0 1 0 1 1 0 0 0 0 0 1 1 1 0 0 0 1 0 10
11209 1 1 1 0 1 1 1 1 1 0 1 1 1 1 0 0 0 0 1 0 1 0 14
11210 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 0 0 1 1 1 0 17
11211 1 0 1 1 0 0 1 1 1 0 1 0 0 1 0 0 0 0 1 0 0 0 9
11212 0 0 0 1 1 0 0 1 1 0 0 0 1 1 0 0 0 0 0 0 1 1 8
11213 0 1 1 1 1 1 0 0 1 0 1 0 0 1 1 0 0 0 0 0 1 1 11
11214 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 18
11215 1 0 0 0 0 1 1 1 1 0 1 1 0 0 0 1 0 0 0 1 1 0 10
11216 0 1 0 0 1 1 1 1 1 1 1 0 1 1 1 0 0 1 1 1 1 1 16
11217 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 1 1 0 1 17
11218 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 19
11219 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 4
11220 1 1 0 0 0 1 0 1 1 0 1 1 1 0 1 1 1 1 1 0 1 0 14
11221 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 18
11222 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 0 1 0 17
11223 0 1 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 0 1 1 0 7
11224 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 19
11225 1 1 0 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 0 0 17
11226 0 0 0 0 1 0 1 0 1 0 1 0 0 0 0 1 1 0 1 0 1 0 8
11227 1 1 0 1 1 1 1 0 0 1 1 0 0 0 1 1 1 0 1 0 0 1 13
11228 1 0 0 1 0 0 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 11
11229 1 1 0 1 1 1 0 1 1 0 1 0 0 0 1 1 1 0 1 0 0 1 13
11230 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 20
11231 1 1 1 0 1 0 0 0 1 1 1 0 0 1 0 1 0 1 1 1 1 0 13
11232 0 1 0 0 1 0 0 1 1 1 1 1 0 1 1 1 0 0 0 1 0 0 11
11233 1 1 0 1 1 1 1 1 1 1 1 1 1 0 0 1 0 1 1 1 0 0 16
11234 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
11235 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 21
11236 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 21
Volume 3, Issue 3, March 2014
Page 223
AUTHORSTian-Wei Sheu received the Ph.D. degree in Mathematics from National Osaka University, Japan in 1990. He is the Dean of College of Education and a professor of Graduate Institute of Educational Measurement, National Taichung University, Taichung, Taiwan. His studies focus in IRT, Educational Measurement, and e-Learning, etc. He is the director of TKIA (Taiwan Kansei Information Association).
Phung-Tuyen Nguyen is currently a Ph.D. candidate in Graduate Institute of Educational Measurement and Statistics, National Taichung University, Taiwan. He received Master’s degree in Physics, in 2003 from Hanoi University of education, Vietnam. His research interests focus on item response theory, grey system theory, and educational measurement.
Duc-Hieu Pham received Master’s degree of education at Hanoi Pedagogical University No2 of Vietnam in in 2009. He works as a lecturer the Primary Education Faculty of Hanoi Pedagogical University No2, Vietnam. He is currently a Ph.D. candidate in Graduate Institute of Educational Measurement and Statistics, National Taichung University, Taiwan. His research interests include grey system theory, educational measurement and primary education.
Phuoc-Hai Nguyen received Master’s degree in Biology from Hanoi University of education of Vietnam in in 2006. He is currently a Ph.D. candidate in Graduate Institute of Educational Measurement and Statistics, Statistics, National Taichung University, Taiwan. His research interests include biology, item response theory, grey system theory, ordering theory and educational measurement.
