Reliability 6.2.1
Important matters in a measuring instrument are a number of suffi-cient items of different degrees of difficulty to be endorsed, and a variety of persons of different degrees of ability/willingness to agree. Two in-dexes of reliability are produced by the Rasch analysis.
The person reliability index “is an estimate of how well one can dif-ferentiate persons on the measured variable” (Bond & Fox, 2001, p.
207). This index is influenced by targeting items at the ability level and by the spread of the ability across the sample of persons. The person separation index, on the other hand, is used to define the number of abil-ity strata in the sample.
The item reliability index estimates the replicability of the items’ hi-erarchy when the same set of items is given to other persons with similar abilities. This is influenced by targeting persons at the difficulty level and by the spread of difficulty across the group of items (Linacre, 2011).
With the Albanian data from 2010, as explained in article three, both indicators were high for the sample of students in this study: person reli-ability was 0.81, which means that there is enough to distinguish be-tween students of different levels of preferences, and item reliability was 0.99. There is no need, according to Rasch analysis, to make decisions about a larger sample of students or a larger number of items. The person separation index was 2.07 and the item separation index 11.12.
Real-life contexts in mathematics and students’ interests. An Albanian study 83
Validity 6.2.2
For Bond and Fox (2001), an important aspect is the construct validity, which is based on the fact that “the recorded performances are reflec-tions of a single underlying construct: the theoretical construct as made explicit by the investigator’s attempt to represent it in items or observa-tions, and by the human ability inferred to be responsible for those per-formances” (p. 26). In the ROSME study presented in this thesis, as can be further read in the third article, a hierarchy of students’ preferred real-life situations that can be used in mathematics was constructed before-hand based on available data from experience, articles and books in the field. The latent trait’s validity, or the construct’s validity, was then dis-cussed as compared to the actual hierarchy obtained by analysing stu-dents’ answers with Rasch models.
According to Wright and Stone (1999), internal validity can be as-sessed by a mean-square test of fit to define to what extent data on each item fit the latent variable, and to what extent a person’s performances fit the model. This test was used with the data collected for this project and appropriate measures were taken to remedy large misfit problems (see Kacerja, article 3). The test of fit identifies items that may not be work-ing the way they were supposed to, and persons that may not have taken the questionnaire the way they were expected to. Once identified, items and persons showing misfit can either be excluded from the analysis, or reviewed for the next data collection, depending on the timing of the study and different objectives of it. In the case of this study, all items had appropriate fit statistics, while 23 students were excluded from the anal-ysis because of large misfit values.
Trustworthiness 6.2.3
One section of the study, as explained further in the thesis, was based on qualitative methods. Matters of reliability and validity on qualitative re-search can be translated into matters of trustworthiness, with its four cri-teria: credibility, transferability, dependability, and confirmability (Bry-man, 2004). Regarding credibility, students were continuously asked during the interviews in the study to confirm or deny if the interviewer’s perception of their opinions was correct. In order to ensure transferabil-ity, attempts were made to provide rich descriptions from the different time of interviews. To help in the dependability matter, descriptions about selection of research participants and data analysis decisions, as well as interview transcripts were offered. As explained in section 5.2.2., and related to confirmability issues, the researcher attempted to not allow personal values “to sway the conduct of the research and findings de-rived from it” (Bryman, 2004, p. 276).
To add to the first paragraph’s discussion, some practical matters are also discussed here. First, interviews were conducted during the data
col-84 Real-life contexts in mathematics and students’ interests. An Albanian study
lection with 24 students from grades 8 - 10 to assess students’ motives for their interest in some of the contexts that can be used for learning mathematics. The outcomes of the interview significantly depended on the communication between the interviewer, in this case the researcher, and the participant, in this case the student. There were some factors that helped in this direction, such as the interviewer’s experience in teaching mathematics to students in the target age group for 2 - 3 years. Further, the interviewer’s age was advantageous in allowing students to feel com-fortable in talking about things they like/dislike to do and sharing their opinions. Interviewing abilities were improving from one interview to the next as the number of interviews, 24, allowed more time and space for improvement. These interviews were also very helpful in preparing for the second interviews, the eight realistic interviews, where the expe-rience and ability of the interviewer is also crucial in directing the con-versation and completing the interviewing cycle, and to obtain answers for the primary questions raised in order to conduct the study. Another advantage that helped with interviewing, and also in interpreting the re-sults of the interviews, were the five teaching units conducted with the students from where the sample to be interviewed was chosen. During the lessons good communication was established, and the goals of the lessons and of the entire study were explained to the students. This made it easier afterwards to ask the students for voluntary participation in in-terviews, to interview them and rely on their openness and willingness to communicate.
Generalization 6.2.4
Matters of generalization are concerned with questions if “findings can be generalized beyond the confines of the particular context in which the research was conducted” (Bryman, 2004, p. 76). The data for this study were collected in only one district in Albania, the one from where the researcher comes, which was done for convenience reasons. When ad-dressing generalization, one can ask to what extent are the results gener-alizable to the Albanian population of grades 8 - 10 students. In 2009, there were a total of 9623 students in grades 8 -10 in Shkodra, and 825 of them participated in the study. This choice of sample was based on the researcher’s access to the schools. However, care was taken to include as many different schools, in terms of location, as possible from the ones where the researcher had access. A considerable number of both male and female students were included. Given these considerations, we con-clude with some caution that the results are generalizable to Albanian population of grades 8 -10 in Shkodra.
Real-life contexts in mathematics and students’ interests. An Albanian study 85
7 Results and discussion
In this section, results of the study will be presented as answers to the research questions posed in the beginning.