A UTHOR ' S PERSPECTIVES

3.2.1

Commitment to improving assessment practices

I bring to this study a strong bias towards CRA because of its emphasis on aligning assessment tasks with learning outcomes and ensuring expected standards are explicit, so that assessment practices are fair and valid. This bias arose from three seminal experiences. The first was the end of the year 12 physics exam that I sat in 1967 in Queensland, when all students who wished to attend university were assessed using exams set by university faculty. As year 12 students, we would spend the last six months practicing sitting for past papers. Physics was my last exam, and in the middle of the second three hour paper I was confronted with questions on unfamiliar material. The injustice of this rankled and still does nearly 50 years later. It has influenced all my work with academics as a developer as I championed transparency for students. The media revealed that the academic who set the paper had 'examined topics outside the syllabus' (Matters & Wyatt-Smith, 2008, p. 4). A government review led to radical changes, including a review of all secondary education; the

formation of a government body staffed by experienced teachers (the Board of Senior Secondary School Studies: the Board); and a move to CRA.

The second experience was the first university I attended, having been awarded a

Commonwealth scholarship, in the era of chalk and talk and a lot of laboratory work (I was studying pure science). These teachers kept you interested—no YouTube, no videos, just the blackboard and the occasional handout. They were enthusiastic and knowledgeable, and organised excellent

excursions and comprehensive laboratory sessions. However, what these great teachers did not do was let students in on the secrets of assessment, just as had happened in my final high school years. No curriculum documents (unit outlines) setting out learning outcomes, content, structured assessment tasks, lists of resources; and no student learning support—you relied on tutors if you could find them. Most of the academics at the time were collegial and approachable as the university was small and a college of a larger Australian university. They would tell you what to learn 'for the exam' but not let you see past papers. They also tended to teach content but not what to do with it. So for me, meeting the requirements of assessment in those days as a student was trial and error, and a source of

resentment that everything was still a secret. The seminal experiences outlined above meant that when I started secondary school teaching, I was on the students' side and I did not believe or practice 'secret business' in relation to assessment.

The third experience, in my first year of teaching in 1972 at a private school, involved me challenging the judgment of the state Chief Moderator that the final results for my year 10 science students were 'impossible'. I was instructed I could not have more than two students with As (the highest grade)—I had decided there were eight in my class of 15 mostly bright girls, and I was furious. The principal advised me to challenge the moderator who promptly demanded I send him all the students' assessment tasks with their task sheet descriptions in 15 folios, one per student and identified as rated by me as A, B, C etc. My original distribution was upheld based on the standard of evidence I submitted, rather than a predetermined distribution or norm. This experience is indelibly etched in my mind, and has underpinned my attitude to university norm-referencing as yet another unethical practice affecting students that I met later as an academic developer.

Meanwhile from the mid-1970s to 1980s, the new entity, the Board, was slowly

revolutionising teaching and assessment by changing the way syllabuses (the curriculum17 for each

17

There is no one definition of curriculum. In this thesis, I am referring to the 'official' or intended curriculum as specified in approved documents produced by academics. For single subjects these typically specify content, learning

objectives/outcomes, learning experiences, assessment tasks, and may include a list of resources. See for example: UNESCO. (n.d.). Curriculum Design. Retrieved December 15, 2014 from

subject) were written and implemented (no longer done by university academics). The Board termed the process 'criteria-referenced assessment' (or CRA) whereas teachers termed it CBA (criteria-based assessment) and some countries used the term OBA (outcomes-based assessment). Australian universities (starting in the mid-1990s) started to implement CRA. They added their 'own' term of 'constructive-alignment' (Biggs, 2003) to label the process of improving the quality and consistency of assessment processes by making expectations explicit to students. The three seminal experiences outlined above, together with my later experience at the Board, have given me a strong commitment to improving assessment practices in secondary and tertiary education. They have also strongly

influenced how I approached my academic developer role by considering the curriculum (and therefore assessment) needs of students and academics. Because of my bias in relation to the

importance of CRA, I conscientiously supported whatever institutional change model was being used in my careers in secondary and tertiary education. In terms of this study, I supported the

implementation of the DL model at DUU via my academic developer role because it was about CRA. CRA then became the vehicle for exploring this DL model.

3.2.2

Previous experience with DL

3.2.2.1 DL and the Board

After fifteen years teaching secondary school I joined the Board, which I now consider was using DL, although the term was not common parlance in 1995. I was one of eight permanent curriculum officers responsible for syllabus construction and revision involving committees of teachers and academics. We curriculum officers were meant to be mostly outsiders—given syllabuses outside our disciplines (I had all the arts, but my specialisations were the sciences and mathematics). The other group of distributive leaders was the 30 review officers—seconded for two years as insiders for their specialist discipline knowledge to assist teachers develop curriculum to a suitable standard. As officers of the Board we had no positional power (so power was not distributed, only influence), and reported to our managers who were curriculum and assessment specialists respectively. We had to rely on collegial and expert power to work effectively with our respective groups. As a curriculum officer, I trained the syllabus committees in syllabus design that adhered to CRA principles (a two- year process). This led to each member of a committee being able to operate as a distributed leader back in their respective contexts, spreading the skills of effective curriculum design.

http://www.ibe.unesco.org/fileadmin/user_upload/COPs/Pages_documents/Resource_Packs/TTCD/sitemap/Module_3/Modu le_3_1_concept.html

For one of my nine years I was simultaneously a curriculum and review officer (the latter in agricultural science). The review officer role also required training panels of teachers all over the state in making judgments about other folios of student work submitted to the panels. This process was to ensure comparability of assessment across the state. These trained panels formed another distributed network spreading the messages about how to implement CRA. As a biological science panellist before I joined the Board, I knew how effective training and advice from the panels could be and what an effective learning forum it was for me as Head of Science in my school. The Board's processes were unique at the time in Australia, and only Oregon in the USA had anything similar. My dual roles as curriculum and review officer prepared me for my PhD study by showing me the importance of regular curriculum review and participants' perspectives, and how these can and should inform future practice. My time at the Board prepared me for my later role as academic staff developer able to work with, and be supportive of, academics from any discipline, accept critique of my suggestions, be very adaptable but maintain high standards of curriculum development and evaluation. As a member of three DL networks (in my roles as curriculum officer, review officer and panellist), I saw how

effective they were in maintaining quality syllabus development, and consistent implementation across a vast number of secondary schools. DL at the Board was also successful because we officers or champions had managers, and were working full-time supporting teachers across Queensland. The Board was also well funded by the Queensland government.

Universities do not typically have the funds to take their teachers away from their students to work full time with other teachers in their faculty or school as distributive leaders. These academics also have to maintain a research profile and obtain grants which seconded school teachers do not have to do, further making the Board's DL model an impossible ideal for universities to implement.

Working at the Board has given me a unique perspective about its version of DL and prejudiced my views about whether DL of any description can be effective in implementing change in HE

institutions.

3.2.2.2 DL and a university

In mid-2005, I moved to a large Australian university as an academic staff developer to assist academic staff in implementing CRA. Implementing change in the tertiary sector had some differences to the secondary sector, so I had to adapt my repertoire of skills to ensure effectiveness. One of the first things I had to learn is that working with academics in their 'home' environment is not the same as when I worked with them at the Board (where they were out of their 'home' environment). I found that in the university context, some accepted my academic developer role as one of expertise in learning, teaching and curriculum areas and therefore an insider—a colleague to be respected. Others saw me as an outsider, not one of them at all, without positional power (Rowland, 2006; 2007) and therefore without credibility. Yet at the Board, academics on committees treated me as a valued and

knowledgeable colleague, possibly because they were outside the secondary education system and relied on my expert knowledge of that system.

At the large university referred to above, the model used to implement CRA could have been labelled DL but was not. It involved nine faculty champions referred to as learning and teaching consultants, who were supposed to spend 50% of their time providing CRA support to their faculties because they had been seconded and part-funded by the ADU. All except one (me) were not managed by the ADU. Two of us had knowledge of how to implement CRA (me and a colleague whom I had worked with at the Board) and two others were willing to learn. I was in the role for two years, but was refused permission by the head of the ADU to work with and upskill the consultants—the reason given was there was no time. Needless to say this DL model was ineffective because there were too few consultants for a large university and most had no CRA skills.

This experience illustrated that this DL model only worked in places where the consultants were motivated and had some CRA background, such as in law, creative arts, nursing and engineering. Its most useful feature was the secondment of the consultants or champions, which meant that they did not have to be a consultant on top of all their academic commitments. Except for me, they were all 'insiders' (not based in the ADU). Later at DUU (mid-2008 to end 2011), I was able to bring resources, strategies and experience from working at this Australian university to help DUU implement CRA with their model of DL. I knew from working at the Board that DL could work, but only under specific conditions, and from working at the other Australian university I knew that there needed to be more than a handful of change agents/champions or consultants, and these needed to have a shared understanding of CRA. I had severe doubts from the start that DUU's version of DL would work as well as intended, but I knew it had the potential to be more effective than what happened at the other Australian university. This proved to be the case but with much of its potential not realised.

3.2.3

Researcher-participant/insider-outsider: Balancing positions

As well as previous experience with DL, I brought to my research on the DUU project the following: four years' experience with hundreds of academics across all DUU faculties, and all except two schools; close involvement with many of the school champions over a protracted period; close involvement with one academic developer in the ADU; and existing relationships with a couple of the Associate Deans. As a researcher-participant/insider-outsider, I was involved closely in the

phenomenon as a participant as well as a data-gathering instrument, hence the use of first person in much of the thesis. There were various benefits and challenges involved in balancing these multiple positions as noted in the quote at the start of the chapter (Sherry, 2012).

3.2.3.1 Benefits

My experiences at DUU meant that I had 'longevity in the field' (Roulston, 2010, p. 217), was deeply embedded in the study and could save some time in interviews as I knew DUU-specific acronyms and policies. As a researcher I also had 'multiple insider and outsider positions' (Braun & Clarke, 2013, p. 10). According to Teddie and Tashakkori, 'a golden rule of making inferences in human research is to know thy participants' (2009, p. 289). My experience at DUU meant that my understanding of the cultures of the participants and the research context would be a valuable asset in making these inferences. As an insider—that is, someone who had a position in the organisation—I was an academic developer with expertise in assessment who was known by most of the interviewees, and had worked extensively with many of them over a protracted period of time. This gave me a distinct advantage in interviews as I had an insight into their busy world, did not have to establish rapport with most of them, and felt accepted as a peer by the majority.

3.2.3.2 Challenges

I was respectful of and grateful to the academics I interviewed and was aware, while writing this thesis, that I felt highly protective of them and DUU's reputation. I consciously attempted to mitigate these feelings by striving to maintain a critical and open stance to the data that emerged, which challenged me to remain a reflexive outsider during analysis. What many of the school champions said they did to support implementation of CRA, involved me. Therefore, much of their data had my influence on its production. Because of my researcher-participant position, I found it impossible to 'take a detached and neutral role' towards interviewees, preferring instead to 'to develop a collaborative relationship' to help me explore their perspectives on the DL project (Roulston, 2010, p. 224). During the CRA project I was heavily involved in supporting staff in implementing CRA and had not decided to commence a PhD on the project until towards the end of 2010. It made more sense to interview after the project had officially finished (December 2011), so that interviewees could reflect on the whole project. I also wanted a global view rather than minutiae, and this necessitated interviewees 'view[ing] the past through the lens of the present' (Silverman, 2010, p. 192).

I was offered a job in April 2012 at an interstate university before conducting the interviews. This, by happenstance, gave me the time to develop mental distance from the project (become more of an outsider) but still maintain some insider status in relation to the interviewees (Klein, 2004; Dwyer & Buckle, 2009). Ethical approval to carry out the research was given in December 2012. I

commenced interviewing after April 2012 and continued interviewing into early 2013. The majority of interviews were conducted by phone, and I do not think this situation had any impact on interviewees' responses as most knew me. I was fully aware that academic developers are viewed as outsiders by many academics because they are not faculty members (e.g. Bath & Smith, 2004; Brew & Peseta,

2008; Debowski, 2011; Manathunga, 2007; Rowland, 2006; Roxå & Mårtensson, 2008; Stefani, 2011). Yet I only experienced feelings of being treated as an outsider on six occasions while at DUU. In three of these, the feelings quickly passed as the academics and I became engrossed in the

curriculum task they had chosen, and they expressed positive comments about the process and the results. On the other occasions, I doubt whether the academics' treatment of me—one of superior indifference as well as 'power, authority, control and expertise' (Rathbun & Turner, 2012, p. 239)— would have been different had I been a faculty member, because that was how most of them treated each other, even in front of me.

I also was genuinely interested in the perspectives of all interviewees, however they chose to represent them, albeit retrospectively (Silverman, 1993). This caused me difficulties in terms of creating mental distance as an insider researcher by 'making the familiar [into] an unfamiliar object of analysis' (Bryman & Lilley, 2009, p. 343) in order to avoid the great danger of misunderstanding (Hammersley, 1990, p. 8). This is similar to Silverman's warning to researchers 'being so personally involved with people you are studying that it is difficult to be objective' (2010, p. 274), resulting in role confusion, in which the researcher responds to the participants or analyses the data from a perspective other than that of the researcher (Asselin, 2003).

3.2.3.3 Balancing empathy and alienation

Hellawell argues that this insider-outsider position (that is, partially an insider and an outsider) is an ideal one because it implies the researcher experiences both empathy and alienation—the latter meaning 'making strange' (2006, p. 487). Hesse-Biber identifies the challenge as involving 'taking on a multitude of different standpoints and negotiating these identities simultaneously' (2010, p. 74). Dwyer and Buckle suggest that one's insider or outsider status is not what is the core ingredient in qualitative research, but rather 'an ability to be open, authentic, honest, deeply interested in the experience of one's research participants, and committed to accurately and adequately representing their experience' (2009, p. 59). I found this messy state of affairs as a researcher-participant/inside- outsider initially exhausting at times, because I had to be constantly on my guard during interviews. I eventually decided to relax and enjoy the interactions and appreciate the generosity of these busy academics giving me an hour of their time.

3.3 S

In this chapter, I outlined the research setting by describing key features of DUU and the Australian state it is primarily based in. Next, I presented the implementation plan that showed the people involved, but unfortunately omitted the Heads. I then revealed the demographic data of the four groups of interviewees with the largest group, the school champions, as the main focus of my research.

Stark differences in the data about this group suggest some of them may have had much greater