Relationship B: Problem Solving and Teacher Noticing

A bivariate correlation was run to evaluate how teachers’ understandings of area, perimeter and volume influenced their noticing of higher levels of reasoning and understanding in student work samples. A scatterplot of Noticing Task against Problem Solving Task results was plotted as shown in Figure 4-18. A jitter effect was introduced to overcome the issue of over plotting in a study involving repeated measures (Chambers et al., 1983). Visual inspection of the scatterplot indicated a significant relationship between teachers’ responses to the Problem Solving Task and the Noticing Task. Pearson’s correlation coefficient [r = 0.674, n = 64, p = 0.01] confirmed a correlation worthy of further investigation.

Figure 4-18 Problem Solving and Teacher Noticing

Teachers’ understandings of area, perimeter and volume were significantly predictive of their noticing of higher levels of understanding and reasoning in student work samples on the same content. A regression was calculated to describe the proportion of variance in Noticing Task categories explained by Problem Solving results. A significant regression equation was found (F (1, 62) = 51.550, p<.000). The R² value of .454, suggested that Problem Solving scores accounted for 45.4% of the variation in Noticing Task Position categories. The adjusted value R² = 44.5% identified Problem Solving scores as a moderate effect on Noticing Task Position (Cohen, 1992).

A univariate ANOVA was conducted to explore differences between the Problem Solving Task means of responses in Noticing Task Position categories. The ANOVA was conducted using Problem Solving

categories to overcome the issue of error levels due to unequal group sizes in raw scores. The results of the ANOVA (F (4,59) = 13.264, p = .0001), illustrated in Figure 4-19, showed a statistically significant difference between the Problem Solving means of different Noticing Task Position categories.

Figure 4-19 Comparison of Noticing Task Position means in Problem Solving Categories

The Tukey post hoc test revealed the following differences in the Problem Solving means of Noticing Task Position Categories:

 The Problem Solving mean for the Limited Noticing Task Position (M=1.71) was significantly lower than the Problem Solving means for Thorough (p = .001) and Extensive (p = .000) Noticing Task Positions, but not significantly different to means in the Basic (p = 0.835) or Sound (p = .225) positions.

 The Problem Solving mean for the Basic Noticing Task position (M=2.20) was significantly lower than the Problem Solving means for the Thorough (p = .002) and Extensive (p = .000) Noticing Task positions, but not different to the mean for the Sound and Limited Noticing Task position.

 The Problem Solving mean for the Sound Noticing Task position (M=2.71) was significantly lower than the Problem Solving mean for the Extensive (p = .004) Noticing Task position, but not different to the mean in the Thorough (p = 0.084) Noticing Task Position.

The results showed that teachers’ responses in the Problem Solving Task were significantly predictive of their noticing of higher levels of student thinking. Yet, the Problem Solving scores of teachers in the

Extensive and Thorough categories were similar. Overall, teachers’ understandings of content, measured through the lens of Problem Solving, accounted for almost half (45%) of the variation in Noticing Task Position. While overall Problem Solving had a moderate effect on teacher noticing, correctly solving the item upon which work samples were based was associated with responses in the Thorough and Extensive Noticing Task Position categories. Fisher’s exact test was used to further examine the significance of this association by evaluating the contingency of teachers responding correctly or incorrectly to Item 9 and which work sample they identified as representing Extensive student thinking, as shown in Table 4-3. The exact test statistics value of <0.00001, significant at p < .05, confirmed that correctly solving Item 9, the problem that was the basis for the set of student work samples, was highly, significantly predictive of responses in the two higher Noticing Task Student categories. Incorrect solutions to Item 9 strongly predicted ranking lower levels of student achievement in the Extensive position.

Table 4-3 Fisher’s Exact Test for Noticing Task Position and Item 9

Higher Noticing Task Student Categories

Lower Noticing Task

Student Categories Marginal Row Totals

Item 9 Correct 24 3 27

Item 9 Incorrect 8 29 37

Marginal Column

Totals 32 32 64 (Grand Total)

The influence of Item 9 on teacher noticing was also examined visually. Figure 4-20 illustrates the extent to which teachers with any given Problem Solving score and Item 9 correct/incorrect ranked the same work sample in the Extensive position. Only one teacher with a Problem Solving score >4 and Item 9 correct did not identify a work sample demonstrating Thorough or Extensive mathematical thinking as representing Extensive achievement. By contrast, the majority of teachers with incorrect responses to Item 9 ranked the work sample demonstrating Sound achievement in the Extensive position. Teachers with correct responses to Item 9 were unlikely to achieve a Noticing Task Student category < 4. Teachers with Problem Solving scores > 5 were equally likely to rank Amelia’s work sample as Thorough or Extensive when Item 9 was correct.

Figure 4-20 Noticing Task Categories by Item 9 Correct/Incorrect

The following influences of Item 9 on Noticing Task Student Position, disregarding overall Problem Solving scores, were identified:

 No teacher ranked a work sample showing evidence of Limited achievement in the Extensive position, regardless of whether Item 9 was correct or incorrect.

 None of the teachers with correct responses to Item 9 ranked a work sample showing evidence of Basic achievement in the Extensive position in comparison to three teachers with Item 9 incorrect.

Problem Solving and Noticing Task Student Categories for correct

responses to Item 9.

Problem Solving and Noticing Task Student Categories for incorrect

 Three teachers with correct responses to Item 9 ranked a work sample showing evidence of Sound achievement in the Extensive position in comparison to 26 teachers with Item 9 incorrect.

 Twelve teachers with correct responses to Item 9 ranked a work sample showing evidence of Thorough achievement in the Extensive position in comparison to six teachers with Item 9 incorrect.

 Twelve teachers with correct responses to Item 9 ranked the work sample showing evidence of Extensive achievement in the Extensive position in comparison to just two teachers with Item 9 incorrect.

In summary, the analysis of Problem Solving and Noticing Task results identified a strong relationship between teachers’ understandings of area, perimeter and volume and their noticing of students’ mathematical thinking. Problem Solving scores were significantly predictive of noticing higher levels of student thinking in written work samples. Teachers’ understandings of content, measured through the lens of Problem Solving, accounted for almost half of the variation in Noticing Task categories. However, the correctness of teachers’ responses to the particular item on which work samples were based was the strongest predictor of teacher noticing. Teachers who did not correctly answer Item 9 were unlikely to identify Extensive mathematical thinking on related work samples and far more likely to rank less efficient, procedural thinking in the Extensive position, regardless of their overall Problem Solving score. Only teachers who did not solve Item 9 correctly ranked a work sample demonstrating Extensive achievement lower than work samples with incorrect solutions.

Results in the Extensive and Thorough Noticing Task categories were associated with higher scores on the Problem Solving Task, while results in the Limited and Basic categories were associated with lower scores on the Problem Solving Task. However, high Problem Solving scores did not necessitate discriminating among different levels of reasoning in work samples with correct solutions. Figure 2-21 illustrates the way in which results in the Sound category were linked to Problem Solving scores ranging from zero to nine. Two outliers were identified. *26 and *33 represent results where Amelia’s work sample was ranked lower than at least one work sample with an incorrect solution, even though approximately half or more of the items in the Problem Solving task were solved correctly.

Figure 4-21 Relationship B: Noticing Task Position and Problem Solving Scores