Validity and Reliability

Validity within a mixed-method study must consider the standards applied to both quantitative and qualitative research designs. The quantitative research used validated instruments that had been subject to critical peer review. The analysis o f study data was consistent with accepted statistical procedures for survey research. Establishing the validity of the qualitative data began with the identification of the study problem and proceeded to conclusion. Documenting the process o f inquiry is as important as the study outcomes. The research cohort worked diligently to keep the study on a tight

philosophical and procedural track. The opportunity to triangulate between quantitative and qualitative data sources produced a deeper understanding o f the problem, while contributing to the validity of the study conclusions.

Every effort has been made to document research and cultural bias and to build a logical pathway for reviewers leading to the final conclusions. Bazely (2004) writes:

approaches to research. As with any research, validity stems more from the appropriateness, thoroughness and effectiveness with which those methods are applied and the care given to thoughtful weighing of the evidence than from the application o f a particular set of rules or adherence to an established tradition, (p. 9)

3.14 Internal Reliability

Chronbach alpha (a) is a statistical measure of internal consistency or reliability that measures the degree of relationship between items in a group. Usually limited to a range between 0 and 1, the higher the value, the greater the relationship between items.

The Chronbach alpha calculated for the survey subscales are all within an acceptable range except the alpha for the “contrasting statements of teaching philosophy” index. Chronbach alpha calculations were evaluated using SPSS 19 by IBM.

Gliem and Gliem (2003) suggest the following legend to evaluate Cronbach alpha reliability coefficients: “a > .9 - Excellent, a > .8 - Good, a > .7 - Acceptable, a > .6 - Questionable, a > .5 - Poor, and a < .5 — Unacceptable.” (p. 87). The low alpha

value, (.537) for the issue of “contrasting statements on teacher philosophy” was not unexpected and is in itself an important element of the study data. While the survey questions are constructed to evaluate teaching philosophy, the questions were designed to identify inconsistencies between belief and action. In this particular case, the Chronbach alpha value identified the conflicts between what teachers believe about DLT and what teachers actually practice. Chronbach alpha values for each question are listed in Table

Table 13

Reliability Index fo r Teaching Styles and Technology Roles

Personal Use Chronbach Professional Chronbach Classroom Use Chronbach (TPU) Alpha(a) Practice (TPP) Alpha (a) (TCU) Alpha (a) Change Agent a = .805 Chang Agent a = .884 Change Agent a = .782

Connector/ Connector/ Connector/

Communicator a = .730 Communicator d II 00 Communicator a = .750 Contributor a = .732 Contributor a = .738 Contributor a = .956 Producer a = .708 Implementer a = .792 Implementer a = .801 Consumer a = .335 Producer a = .811 Producer a = .879 Consumer a = .766 Consumer a = .629 Teaching Style Indices (Becker & Anderson, 1998)

Vignettes Describing Contrasting Instructional Styles a = .830 Contrasting Statements of Teaching Philosophy a = .537 Opinion Statements on Pedagogy Q II 00

Note: Teaching style indices developed by Becker and Anderson (1998). Teacher roles by Lemke, (2009).

3.15 Response Rate

Using the formula defined by Rea & Parker, 2005 for small sample sizes that include both proportional and interval scale variables, a 95% confidence level was achieved with a +/- .08 margin of error. The relatively high margin of error (+/-.08) in Table 14 is due to the 40% survey response from survey one and will require a close watch over the hypothesis testing in the quantitative component of the study, especially with regard to the possibility of Type I and II errors. See Index C for a review of statistical procedures.

Table 14

Confidence Level fo r Population Sample

Survey Total Teacher 1:1 Population 1:1 Study Population Survey Response Response Rate Confidence Level Margin of Error 1 _{366 236 94 40% 95%} + 1 _{o 00} 2 _{366 236/94 32 14%/34% NC} NC

Triangulating between information derived from qualitative and quantitative data sources in a mixed-method study, while not reducing the margin of error, may contribute to increased confidence in the study conclusions. The response rate for the study was lower than anticipated during the design phase. While a number of conditions may have influenced the response rate, the length of the survey and test fatigue cannot be ruled out.

The follow-up survey, designed to collect more qualitative data from first survey respondents, garnered a response rate of only 34%. While this rate is lower than what may have been expected if the questions had been added to the primary survey, the responses to the second survey were more thoughtful. Charmaz (2006) suggests that data to support a grounded theory should “continue until gathering fresh data no longer sparks new theoretical insights, nor reveals new properties of your core theoretical categories” (p. 115). Dey, (as referenced by Charmaz, 2006) contends that data collection should continue until there is a “theoretical sufficiency” (p. 116). A review of the demographic information related to the second survey showed no variation between the initial and the second survey. Further, a demographic review of the respondents to the second survey reflected little variation from the respondents in the initial survey.

The placement of the open-ended questions at the end o f a lengthy survey increased the influence of test fatigue. Several superintendents indicated that teachers, continually barraged with surveys, were reluctant to participate in yet another.