Assessing thinking using the Structure of Observed Learning Outcomes (SOLO) The Structure of Observed Learning Outcomes (SOLO) modeldescribes development as

a combination of the two features of the mode and the response, where the mode refers to the abstractness of the task and the response is the level of sophistication of the person’s responses to the task (Pegg, 2003). Both the mode and the response are hierarchical.

The first key feature of SOLO is the modes of thinking, and the three modes pertinent to this study are, in order of increasing sophistication, the ikonic, the concrete-symbolic, and the formal. Individuals operating in the ikonic mode internalise actions by developing individual words and images. Individuals operating in the concrete- symbolic mode are still tied to their own concrete experiences where, for example, a few specific instances satisfy the reliability of the rule (Collis, 1975); this has implications for the study presented here where students examine the large population sample size model. Students working in the formal mode “consider more abstract concepts described as working with principles or theories where students are no longer restricted to concrete referents” (Pegg, 2003, p. 242).

Pegg (2003) noted that much of elementary and early secondary school teaching is adapted to suit students working at the concrete-symbolic level, and Panizzon, Callingham, Wright, and Pegg (2007) claimed that 14 and 15-old students in their study would work principally in the concrete-symbolic mode and display some elements in the formal mode. The modes do not preclude an individual working in a lower mode in any context, and individuals who respond in one mode may be unable or unwilling to respond in the same mode in a different context. An individual has the option of operating at a lower level; the formal mode, for example, does not necessarily subsume

75 operations in the concrete-symbolic mode (Pegg). All of the modes continue to develop throughout life. The three modes are presented in summary along with indicative age ranges in which the thinking first occurs (Pegg, p. 242).

1. Ikonic (from two years) where the action is internalised in the form of images and language. In adults, this leads to a form of knowledge referred to as intuitive thought.

2. Concrete symbolic mode (from 6 or 7 years) is demonstrated where a person thinks through written language and symbols to describe systems that have an internal logic. Transition to this mode from ikonic represents a major increase in abstraction.

3. Formal mode (from 15 or 16 years) is where individuals seek to understand the relationship between concepts. This represents a further increase in abstraction and a reduced reliance on concrete referents.

The second key feature of SOLO is the level of response, which is an individual’s ability to respond with increasing sophistication to the task. This study used SOLO’s five-tiered framework to assess students’ responses:

1. prestructural (P) responses where the task is not understood, or where irrelevant aspects provide a distraction;

2. unistructural (U) responses where only one aspect of the task is presented; 3. multistructural (M) responses where several disjoint relevant aspects are

processed but not integrated;

4. relational (R) responses demonstrating an integrated understanding with coherent structure and meaning; and

5. extended abstract (EA) responses, which go beyond relational to include information from outside the task.

An example illustrates the use of SOLO. When applied to an exploratory data analysis task examining marathon race times, a unistructural approach would be a single statistic such as “…the fastest time occurred in 2002…” A multistructural approach might be demonstrated by a graph of race-times against year, showing a progressive reduction in times. A relational analysis would provide an integrated and appropriate response selectively and appropriately using all the information within the task which might

76 include the change in race-times against year, the significance of gender, and the performance of specific athletes and country of origin.. An extended abstract response might explain a progressive reduction in race times on improved training methods or diet of the athletes.

The level of response is age related and the indicative ages at which levels of higher- order thinking occur are presented in Table 2.1. The subjects of the study were 14-15 years old so relational responses were anticipated.

Table 2.1.

SOLO Model. Summary Table for the Concrete-Symbolic Mode1

Level Acronym Indicative

age range (years)

Indicative characteristics in the students’ responses

prestructural P 4-6 did not understand question, did not complete the work, or where irrelevant aspects provide a distraction

unistructural U 7-9 used single features of task

multistructural M 10-12 used a range of evidence but not in an integrated way

relational R 13-15 combined salient features to the task at hand, provided evidence of multiple pathways, displayed a coherent and comprehensive understanding of all elements

extended abstract EA 16+ drew in features from outside task, showed understanding of underlying principles

1_{Adapted from Biggs and Collis (1982)}

A refinement of SOLO is where learning is seen as occurring in learning cycles (Campbell et al., 1992; Panizzon et al., 2007). This refinement was introduced in response to the observation that a single cycle analysis inadequately captured students’ learning (Pegg, 2003). Campbell et al. developed a two-cycle analytical framework where the relational response of the first stage became consolidated into the unistructural response of the second stage, i.e., the learning achievements of the first cycle provided a basis for learning in the second cycle. In essence the first and second cycle analysis attempts to describe the iterative nature of learning where initial

77 incomplete understandings are subsequently assembled into a more consolidated response. The researchers, denoting the first and second cycles by subscripts, presented this framework diagrammatically,

U1 M1 R1 = U2 M2 R2 (Campbell et al., p. 296).

Campbell et al. applied the framework in a longitudinal study of students’ development of understanding of volume from primary through to secondary school level. The first cycle referred to students’ conceptual development of understanding of volume and application of the formula for a simple rectangular prism, and the second cycle referred students’ application of the formulae to composite figures or rectangular and triangular prisms. Students’ responses to the classroom items presented here, where students apply the large population sample size formula to a contextual task, correspond to a second cycle analysis (Callingham, private communication May 27, 2011).In this study second cycle analysis is used for the one task tha will be known subsequently as the Mt. Wellington cable-car task.

Several education researchers have criticised SOLO. Shaughnessy (2007) thought the boundaries between the levels used in SOLO were blurred, and Chick (1998) and Chan, Tsui, Chan, and Hong (2002) noted that consistent assessment using SOLO, even amongst trained assessors, was difficult. Chan et al. also considered SOLO to be unstable, in that the one assessor may assess the same work differently at different times. Such characteristics are a significant disadvantage of SOLO. It would limit the potential to compare research studies, and it may limit the ability to compare an individual student’s performance amongst tasks.

Chan thought the ambiguity of SOLO levels could be reduced by the use of sub-scales. Otherwise known as transitional responses, sub-scales were first proposed by the developers of SOLO, Biggs and Collis, in 1982. Transitional responses occur when “the student is feeling for the next level, but doesn’t quite make it… [the responses] carry more information than would be expected in the level the student is emerging from but [the student] is forced to give up before reaching the complexity at the next SOLO level” (Biggs & Collis, 1982, p. 29). The initials P, U, M, R and EA (see Table 2.1) are used extensively in Chapter 4, and the subscript T is used to describe transitional responses. For example, MTR is a transitional response between multistructural and

relational levels. Such a refinement also provides an additional level of granularity in assessment of students’ responses in this study.

78 SOLO is used extensively in the statistics education research, and although it has limitations it was the formal assessment model used in this research study.. A short- coming noted by the researcher is that although SOLO was used to assess a student’s development at the conclusion of the task, it did not necessarily allow the student’s learning trajectory to be identified. This perceived short-coming is addressed in the subsequent sub-section.

The study was conducted as a normal teaching unit, so formal feedback to students in the same and familiar format used by the schools was offered to the two colleague teachers. Authentic assessment principles identified in the Teaching for Understanding framework (Blythe, 1998), the Essential Learnings program (Department of Education, Tasmania, 2003), and the school’s normal assessment protocols were to provide feedback.

2.6.5 Instrumental genesis, situated abstraction, and affordances and constraints