In this section, I present description of the CS: BIPE survey, describe how the survey was administered and delivered to the two teacher groups, and explain how the data from the survey were analyzed.
Description of the CS: BIPE. The Computer Science: Best Instructional Practices for ELLs (CS: BIPE) survey was a 17-question, anonymous electronic survey designed by me. The CS-BPIE is based on the literature review findings combined with overlapping components of the constructivist theory (see Table 6). I separated the CS: BIPE, found in Appendix A, into three distinct sections: Part A and B focused on instructional strategies beliefs and frequency of use and Part C collected demographic information. I did not collect names, email or IP addresses in order to maintain
anonymity. The survey began with an informed consent screen which directed only participants who wished to consent to continue the survey.
Part A consisted of three multiple-choice questions, asking teacher to identify which of four presented strategies they believe would be most useful in order to increase positive academic outcomes. Each answer set for Questions 1-3 contained (a) one of the three overlapping strategies and at least one (b) EL instructional strategy and (c) one CS instructional strategy. In all, 12 strategies identified in the literature review were
presented as answer options. Eight strategies were overlapping, two were domain specific to EL instruction, and two were domain specific to CS instruction. In addition, Part A included the option for participants to see defined terms, by hovering over “Definitions here” listed at the end of each of question.
can have, a forced-choice survey has advantages in terms of construct and observed validity and eliminates the halo effect of Likert-scales (Bartram, 2007).
Part B consisted of three questions, investigating the frequency of use of the same 12 strategies presented in Questions 1-3. Question 4 asked teachers to rank their use of each strategy on a 0-5 Likert-type scale, from Never to Everyday. Questions 5 and 6 were optional, open-ended questions that allowed teachers to explain their choices, if they wished to provide more information. These answers were combined with the qualitative data.
Part C collected demographic information including number of years in the classroom, grade level(s) and subject (s) taught, and type of teaching credential. Part C employed skip-logic, so that if a teacher self-identified as a CS, EL, they only answered questions that pertained to that subject area.
Administration of the CS: BIPE. As indicated by Dillman et al. (2014), electronic surveys are attractive to participants and researchers alike because of speed, economy, and scale. I used the online program Qualtrics to deliver the survey. I electronically administered the survey for 32 days, between November 2018 and December 2018. Delivery via email was different for each teacher group, as outlined below.
EL-T. The Title III contact list obtained from the ODE website contained 411 email addresses. I eliminated all obvious non-teaching contacts, including secretaries and district office employees, reducing the number to 128 contacts. I emailed the remaining contacts, n=128 (Appendix B) requesting their participation in the study. Two emails came back undeliverable, reducing my population to 126. I sent a follow-up email (see
Appendix C) to remaining contacts three weeks later (n =126) referencing the first email and indicated the survey would remain open until mid-December.
CS-T. The current OCSTA president forwarded my invitation to the study to 120 current members (Appendix D). Because of organization membership bylaws, a follow- up email was prohibited. While most members of OCSTA self-define as “computer science teachers,” some work in the computer science industry in other areas. Thus, I was unable to ascertain how many of the 120 members were actual CS teachers. Two
members reached out and indicated that they were no longer CS teachers, so I subtracted them from my overall sample (n=118).
CS: BIPE data analysis. I used SPSS software to analyze the CS: BPIE data from questions 1 through 4 through descriptive statistics and four chi-square distribution tests (contingency tables).
Question responses were analyzed using a two-way contingency table to evaluate whether a statistical relationship existed between two variables (Green & Salkind, 2016). “A two-way contingency table consists of two or more rows and two or more columns. The rows represent the different levels of one variable, and the columns represent different levels of a second variable,” (p. 263) In this study, I used 2 x 4 contingency tables to evaluate Questions 1-3; two teacher groups by the four possible answers for results on each of the first three questions. Question 4 was evaluated similarly, analyzing strategy frequency by teacher group. Data from Questions 5 and 6, the optional, opened ended questions, were analyzed with the Phase II Qualitative interview transcriptions.
more cells is less than 5, the analysis is appropriately performed with the Fisher exact test” (Daya, 2002). SPSS allows the Crosstabs procedure to print the Fisher-Freeman- Halton exact test of independence, also known as Fisher’s Exact test, even when the contingency table tables were larger than 2x2 (SPSS manual). Thus, the reported p value reflects the Fisher-Freeman-Halton critical values of significance.