Questionnaire

This written response type instrument was designed to get objective answers from participants by generally providing them with structured answering patterns. It was aimed to be administered to all selected participants. For this reason, it was the major device to collect data from the predetermined sample.

It is composed of mainly two parts have to be completed by the participants. Part I represents both the demographic characteristics and computer game playing characteristics of the subjects. Part II basically has questions that aimed to investigate the subjects’ perceptions toward the use of computer games with educational features in education. There are totally 58 questions (Part I: 37 questions, Part II: 21 questions) in the questionnaire.

Because the language of instruction is in Turkish in three universities (Gazi, Hacettepe, and Ankara), the questionnaire was prepared in Turkish considering the subjects’ characteristics.The questionnaire is composed of representative questions regarding to the research questions as indicated in the Table 3.2.

As seen in the Table 3.3, the majority of the questions are in the form of four point Likert type which include agreement statements as: Strongly Disagree, Disagree, Agree, and

Strongly Agree. Neutral or Undecided options are not included in this likert scale due to the tendency of subjects to select these choices without deeply thinking about the question as in the case of pilot study.

Table 3.2 Content of the questions in the questionnaire, regarding to research questions Reference to

the Research Questions

Content of the Questions Question no. Question#

Demographic characteristics Part I: 1-7 7

Subjects’ access to computer games Part I: 8-12 5 Subjects’ previous experience with games Part I: 13-17 5 Subjects’ current experience of games in terms of time

spent with computer game playing among other leisure time activities

Part I: 18-23 6

Subjects’ general perceptions toward computer game playing

Part I: 24-35 12 Research

Question 1

Subjects’ preferences of games and game types Part I: 36,37 2 Subjects’ perceptions toward the use of computer

games with educational features, in the curricula.

Part II: 1-5 5

Perceptions of subjects in terms of capabilities of computer games with educational features, in helping students to fulfill the educational learning goals which are defined in the schools’ curricula.

Part II: 6-8 3

Perceptions of subjects in terms of the way computer games with educational features, should be used in education to be more effective in students’ learning.

Part II: 9-19 11 Research

Question 2

Perceptions of subjects in terms of how students and teachers think of the use of computer games with educational features in education.

Part II: 21 1

Research Question 3

The future plans of subjects regarding the use of computer games with educational features in their courses or in learning environments that they will design.

Part II: 20 1

Table 3.3. Types of questions in the questionnaire

Types of Questions Question no. Total #

Four point Likert type questions (Agreement Scale) PartI: 24-35, PartII: 1-19 31

Multiple Choice Questions PartI: 4,5,9-17 11

Short Answer Questions PartI: 1,3,6,7,36 5

Six Point - Rating scale Part I: 18-23 6

Dichotomous Questions Part I: 2, 8 2

Open ended Questions Part II: 20,21 2

Selecting and ordering type Part I: 37 1

Total: 58

This likert scale is used to investigate the questions that represent the perceptions toward computer games and the use of computer games in education. There are 11 multiple choice type questions that represent both categorical and continuous variables. The choices of some questions are determined with the help of the pilot study results. This type is used to investigate the subjects’ previous experience of games as well as their access to computer games. Short answer type questions are generally aimed to get demographic fact information from the subjects, such as age, cumulative grade point average, the year and the semester of studying at the university etc. Six point rating scale questions are used to provide students with choices that represents the extent of time they spend for some leisure time activities. Dichotomous questions that investigate the gender and possession of a computer can be encountered in the questionnaire, whose answers are in two-choice format as: female/male, no/yes. There are two open-ended questions in the last part of the questionnaire. The first one investigates the plans of subjects regarding the use of computer games with educational features in their future professional life. The second question is in a pictorial form that represents the classroom environment in which students are playing computer games and teacher helps them. Filling in the thought balloons is the response method. Due to its

answering method, the last type of question can be described as selecting and ordering type. It requests subjects to select their preferences of game types and want them to order these preferences and write the orders with numbers next to their game type preferences.

Development of the Questionnaire- Procedures to Ensure Validity of the Instrument Since the lack of existing questionnaires that aim to investigate the perceptions of teachers or university students who are studying at educational faculties, toward the use of computer games in education, the present questionnaire was developed by the researcher. Although some questions were inspired from an existing questionnaire developed by Squire and Jenkins (2003), the majority of the questions were developed regarding the literature about computer games in education.

However, since the content of this study is new to experts and also new to the subjects, pilot studies were conducted to support the content validity of the questionnaire. Besides, due to the awareness of the fact that, newly developed questionnaires have problems that will affect the internal validity of the research and the results, several tests were administered to ensure instrument validity of this questionnaire.

The procedure:

1. Development of the questionnaire.

2. First pilot study with 45 participants who meet the characteristics of subjects of this research.

3. Revision with the help of experts from CEIT department, Turkish Language

department and Academic Writing Center in the METU. Translating English version into Turkish.

4. Second pilot study with 12 participants who meet the characteristics of subjects of this research by using think-aloud protocol.

5. Third pilot study with 10 participants, 5 of them are research assistants of CEIT department in METU and 5 of them are master/doctorate students of CEIT departments of METU. Think aloud protocol was used.

6. Examination of the questionnaire by experts in the Computer Education and Instructional Technology field. Revisions were made.

1. Development of the Questionnaire

The first form of questionnaire was developed regarding to the literature review (Prensky, 2001; Malone, 1980) and existing questionnaires developed for different purposes (Squire & Jenkins, 2003; Media Analysis Laboratory, 1998; Tzeng, 1999). All the sources that the researcher inspired from were research studies conducted in different countries except Turkey. The language of the questionnaire was in English.

2. First Pilot Study

The first pilot study was in an experimental research design. For two weeks, 90 minutes for each week, 45 fourth year Computer Education and Instructional Technology students from the Middle East Technical University were exposed to different types of computer games. Two questionnaires were administered each week. During the administration period, subjects stated that some of the questions were not clear, they could not provide choices that they want to select and the questions were not easy to understand. Besides they asked meanings of some English words.

The results of the pilot study revealed that, students generally selected the “Neutral” option of the five point Likert scale. Moreover, some of the students left some of the questions without answering. For multiple choice questions some of the students selected the “other” option and wrote their own choices.

3. Revision with the help of experts from different expertise and Translation

After the pilot study, two parts of the questionnaire were combined and questions that didn’t match the research questions were excluded. With the help of the pilot study,

problematic questions were modified into more understandable wording, some words were replaced with their synonyms, and more options were added to the questions that the subjects in the pilot study selected “other” option. “Neutral” option was excluded from the five-point likert scale and replaced with four-point likert scale.

After all these revisions were completed, the English form of the questionnaire was examined by a faculty member of the CEIT department, METU. Revisions were made according to his advice.

Then, it was translated into Turkish by the researcher. In case of any wrong statement of the questions due to translation, the questionnaire was examined and some modifications were advised by the Academic Writing Center in METU. Moreover, due to the meaning deterioration caused by the translation, it was examined and some modifications were advised by a faculty member of the Turkish Language Department of the METU.

4. Second pilot study

The second pilot study was conducted with 12 participants who were fourth year Computer Education and Instructional Technology students, some of whom had participated in the first pilot study. They were invited to participate in this study. The new form of questionnaire was administered individually by using think-aloud method. The questions that were not understood, poorly and unclearly worded, and misleading were revealed and modified. While completing the questionnaire, they were also asked questions about what they understand from some of the questions, especially the questions that were designed to investigate the perceptions of the subjects. Having all the responses and determining the defected points in the questionnaire, some revisions were made.

5. Third pilot study

The third pilot study was administered to 5 research assistants and 5 MS/ Ph.D. students. They were asked to detect the questions that were not understood and misleading. According to their advices some modifications were made.

6. Examination of the questionnaire by experts

The improved form of questionnaire was handed in four faculties of the Computer Education and Instructional Department at METU. Three-point scale was provided for each question for them to determine whether the question should be removed from, modified, or remain same in the questionnaire. According to their advices some modifications were made.

After these steps in developing the questionnaire, it was administered with the pre- determined subjects from four universities.

Reliability of the Questionnaire

In the internal consistency estimates of reliability analysis of the questionnaire several methods were administered to confirm the reliability while holding the assumptions that “every item is assumed to be equivalent to every other item”, and “an item score is a sum of its true and its error scores” (Green, Salkind & Akey, 2000, pp. 305-306). Reliability analysis of different sections (See Table 3.4.) and combination of these sections were also calculated (See Table 3.5), because the questionnaire is not unidimensional and there are different types of questions in the questionnaire (Green, Salkind & Akey, 2000). Moreover, the sample size is not convenient to calculate Coefficient Alpha for the entire questionnaire.

For this reason, after some pre-analysis steps described in the ‘Data Analysis’ section like transformation and conversion of scores into z-scores, three sections are extracted to conduct these reliability analysis while excluding the demographic information section (questions 1 to 7 in the Part I), access to computer games section (questions 8 to 12 in the Part I), some of the leisure time activity questions (questions 21-23 in the Part I) and other different response type questions (questions 36, 37 in the Part I; 20 and 21 in the Part II).

Table 3.4. Descriptions and question numbers of sections that reliability analysis were

conducted upon.

Sections Description Question no. Total #

Section 1 Experience with games. Part I: 13-20 8

Section 2 General perceptions toward computer game playing Part I: 24-35 12 Section 3 Perceptions toward the use of computer games with

educational features in education.

Part II: 1-19 19

Total: 39

Table 3. 5. Reliability analysis results: Alpha Coefficients of different sections and

combinations

α coefficient % # of questions

Section 1 0,79 79 8

Section 2 0,64 64 12

Section 3 0,85 85 19

Section 1 & Section 2 0,77 77 20

Section 1 & Section 3 0,87 87 27

Section 2 & Section 3 0,84 84 31

Section 1 & Section 2 & Section 3 0,87 87 39

As reported in the Table 3.5., Coefficient Alpha is higher than 0,60 indicates that the scores are sufficiently reliable for the sample. Regarding the section about the students’ experience with games, about 79% of the “total score variance” is due to “true score variance” (Crocker & Algina, 1986, p. 139). However, in the section two which is about the general perceptions of subjects toward the computer game playing, only 64% of the total score variance can be explained by the true score variance. The alpha reliability coefficient is much higher (0,85) in section 3 that includes items related with the perceptions toward the use of games with educational features in education. Considering the combination of

different sections, the reliability coefficient is getting higher. This may be due to the fact that the test is not unidimentional although the scores were converted into z-scores.