Assessment

There is a growing demand from employers and universities for school leavers to be able to apply their mathematical knowledge to problem solving in varied and unfamiliar contexts (Lesh & Zawojewski, 2007; English & Sriraman, 2010; OCED, 2014; Jones, Swan & Pollitt, 2014; ACME, 2016; English & Gainsburg, 2016). Assessment will impact on what is taught in the classroom and should be driven by mathematics that is valued and expected of a modern mathematics education (Suurtamm et al., 2016). Silver (2013, p. 273) reminds practitioners that “for students to become convinced of the importance of the sort of behaviors that a good problem-solving program promotes, it is necessary to use assessment techniques that reward such behaviors”. Viewed in this way, the assessment of problem solving is essential in order to ensure the effective learning and teaching of problem solving throughout primary and secondary education (ACME, 2016). Lesh & Zawojewski (2007, p. 794) posit that “there is a growing recognition that a series mismatch (and is growing) between the low- level skills emphasized in test-driven curriculum materials and the kind of understanding and abilities that are needed for success beyond school”. However, school mathematics

examination instruments are typically dominated by short, structured questions that fail to assess problem solving (Kilpatrick, 1992; Jones & Inglis, 2015).

In Scotland, the centrality of problem solving is recognised as an intrinsic feature within the learning and teaching of mathematics (Scottish Government, 2009) although, illogically, discharged from any form of assessment accountability. Ironically, this delineated position was implicitly bolstered during a recent report established to transform the status of mathematics in Scotland by not appearing in any of the ten recommendations highlighted for change (Scottish Government, 2016b). It is important to consider how to interpret the common theme to emerge from narratives emphasising the indispensable role of problem solving along with the current assessment arrangements that are integral to CfE. As a practising teacher, I am cognisant of the issues of bureaucracy and lack of clarity which undermines our national assessment system but refuse to supplement any rhetoric to this topic. Instead, I will focus my attention briefly on exploring how mathematical problem solving can be evaluated within a suitable framework.

Kilpatrick (1992) suggested that to assess mathematical problem solving effectively, the narrowing effects of current testing practice and the continued pressure for efficient measurement must be addressed. Since this proposition, multinational comparative assessments such as TIMMS and PISA have influenced policy makers throughout the world leading to political agendas fueled with neoliberal ideologies. Increasing operation is being made of external assessments to gauge mathematical knowledge and continue to serve different purposes to the design goals enshrined within the multidimensionality of classroom assessments (Suutamm et al., 2016). In Scotland, I believe the functionality of data from external assessments ultimately serves to encourage practitioners to ‘teach to the test’ to the detriment of assessment for learning (Hodgen & Wiliam, 2006). Still, this scenario would not exist if national assessments aligned with curriculum goals and ironically may be held as a positive practice (Swan & Burkhart, 2012).

Notwithstanding the nuances that arise from assessing complex processes involved in solving mathematical problems, Szetela & Nicol (1992) present four categories that teachers can use as a marking rubric; answers, answer statements, strategy selection and strategy

implementation. Though, it is argued that this method is unable to reliably capture the level of divergent thinking involved since thinking is not easily communicated to produce clearly formulated responses. Polya (1954, p. 154) highlights that: “The final form of the solution may be recorded, yet the changing plans and the arguments for and against them are mostly or entirely forgotten”. Since authentic problem solving tasks require an extended time period (since they are not suited to a timed examination) and observation to access evidence of process, the challenge is to design suitable mathematical problems that can be assessed within a controlled time.

Monaghan et al. (2009) argue that open-start mathematical problems offer a practical means to achieve this objective and encapsulate the type of problems involved:

 The mathematical knowledge needed to solve the problem must already be known securely: this is not about assessing curriculum content – it is about assessing the ability to deploy such knowledge.

 The problem-solver must not be familiar with a similar problem – the essence of ‘open-start’ is that it is not clear where to start and recall of a similar siltation would compromise this.  It would not be clear at the outset whether the strategy will work, and it will have to be

accepted by the problem-solver that further attempts may be needed (p. 26).

The authors suggest that much development work is required to implement this form of assessment. While no marking scheme can circumscribe all conceivable answers that examination candidates might offer, Monaghan et al. (2009) anticipate that this would not pose an issue for open-start problems. In my view, their contribution would have been more convincing if they had provided some empirical evidence.

In their study involving the design of a problem solving examination paper, Jones & Inglis (2015) administered a test to 750 English secondary pupils of varying mathematical ability. The participants work was assessed by experts using comparative judgement in addition to a specially designed resource intensive marking procedure. The construct of comparative judgment has an underlying theoretical basis grounded within a well-established psychological principle that people are more reliable when comparing outputs concurrently than when they are asked to judge something in isolation. In another English study, Jones, Swan & Pollitt (2014) demonstrated that comparative judgement was not a barrier to assessing mathematical problem solving. Results obtained from a review of a sample of examination scripts derived its validity from what is valued and expected by mathematics

professionals, rather than what can be precisely captured in scoring rubrics. Both Jones & Inglis (2015) and Jones, Swan & Pollitt (2014) found that comparative judgment was successful and raise the possibility of a richer diet of mathematical assessments anchored on holistic relatively unstructured tasks being available to future Scottish pupils. However, if the goal of developing proficiently in mathematical problem solving is to be realised, its importance must be communicated to pupils, teachers and the general public through the assessments that are offered (Silver & Kilpatrick, 1989). Moreover, the main summative assessment challenge for stakeholders in Scotland is not novel planning or peripheral methodology concerns but a deviation from traditional measurements fixated by the recall of facts and fluency of procedural knowledge.