Developing a qualitative approach to 360-degree feedback to aid
understanding and development of clinical expertise
ROBERT GARBETTM S c B N ( H o n s ) R N P G C e r t L L1, SALLY HARDYE d D M S c R G N R M N2, KIM MANLEYP h D M N B A R G N D i p N ( L o n d ) R C N T P G C E A C B E3, ANGIE TITCHEND P h i l ( O x o n ) M S c M C S P4and BRENDAN MCCORMACK D P h i l ( O x o n ) B S c ( H o n s ) P G C E A R G N R M N5
1Research Fellow, Nursing Development Centre, University of Ulster and Royal Hospitals Trust, Royal Victoria Hospital, Belfast,2Associate Research Fellow, Royal College of Nursing Institute, London,3Director of Practice Development (paper guarantor), Royal College of Nursing Institute, London,4Senior Research and Practice Development Fellow, Royal College of Nursing Institute, London and5Director of Nursing Research and Practice Development, Nursing Development Centre, University of Ulster and Royal Hospitals Trust, Royal Victoria Hospital, Belfast, UK
To optimize the usefulness of the process we recommend:
• Preparation is key; being clear with all involved about the purpose of the exercise and being explicit about the criteria used to identify role set members. • Approaches to gathering information need to reflect what is convenient and appropriate to colleagues.
• Anonymity – is not a necessary precondition to useful feedback.
Introduction
This paper describes a qualitative approach to 360-degree feedback developed as part of the Royal College Correspondence
Sally Hardy
Deputy Director of NAMRU Institute of Health
School of Nursing and Midwifery Edith Cavell Building
UEA Norwich NR4 7TJ UK E-mail: [email protected] G A R B E T T R , H A R D Y S , M A N L E Y K , T I T C H E N A , McC O R M A C K B . ( 2 0 0 7 ) Journal of Nursing Management15, 342–347
Developing a qualitative approach to 360-degree feedback to aid understanding and development of clinical expertise
AimThis paper presents one aspect of a 5-year multicentre action research study to develop an accreditation process for clinical nursing expertise. Part of the process consisted of the exploration, critique and refinement of qualitative 360-degree feedback as a tool for peer review.
BackgroundThree hundred and sixty-degree feedback is widely used as a personal and professional development strategy. This part of the overall study challenged assumptions about the necessity for anonymity and structured questionnaires to collect data.The study involved 32 experienced clinical nurses drawn from a range of clinical settings supported byÔcritical companionsÕ(colleagues from clinical practice, education, management and research, recruited to provide supervision and support).
Method(s)Study participants, facilitated by the project team (the authors), engaged in critiquing and refining 360-degree feedback as a process to help them examine and develop their practice.
Conclusion(s)On the basis of our findings this approach to gathering 360-degree feedback facilitates the collection of evidence that aids professional development. There are indications that it may also contribute to improved working relationships.
Keywords: 360-degree feedback, adult learning, expertise, practice development
of Nursing’s Expertise in Practice project (EPP) (Manleyet al.2005).
There is a global concern with monitoring the impact of health care. This has given rise to a range of gov-ernance structures in both commercial and social healthcare systems. Within this climate of increased accountability, nursing has developed professional-ization strategies that in turn aim to address chronic workforce shortages. For example, the emergence of new roles such as that of the consultant nurse (Manley 2000) has become a part of the far-reaching reform of career structure for allied health professions in the form ofAgenda for Change(Department of Health 2005).
The importance of accountability for performance over time is, in turn, apparent in the increasing importance of structures that help healthcare profes-sionals demonstrate lifelong learning throughout their careers (Department of Health 2003, Nursing and Midwifery Council 2004). Strategies such as maintain-ing a portfolio of evidence of professional and personal development are an important aspect of a trend (Royal College of Paediatrics and Child Health 2000, Depart-ment of Health 2003, Nursing and Midwifery Council 2004) that increasingly draws on concepts from adult educational theory that emphasize the importance of learning from experience(Higgs & Titchen 2001, Newman & Peile 2002).
The Expertise in Practiceproject emerged as a con-tribution to these agendas. In particular, through its study aims (see Box 1), it has helped to develop strat-egies by which practitioners can critically examine their practice in such a way as to address both governance and developmental agendas.
The project took the form of an emancipatory action research study (Grundy 1982). Over a 16-month period 32 nurses from six geographical areas across the UK recruited from a broad range of clinical specialities and settings worked with colleagues known as critical companions (Box 2) and in action learning sets meeting at monthly intervals in six centres across the UK. These mechanisms were used to help the nurses develop a portfolio of evidence to be accredited by the RCN based on plural sources of evidence:
• Self-assessment based on critical reflection using an evidence-based framework of nursing expertise (Manley & McCormack 1997) (Box 3);
• Feedback on observation of practice;
• Feedback from service users (this complex aspect of the project is reported elsewhere) (Manley et al.
2005, Hardyet al. 2006);
• Three hundred and sixty-degree feedback from col-leagues using qualitative data collection methods. It is this last aspect of evidence collection that is dealt with here. Responses from colleagues in a range of disciplines lead us to believe that the approach is potentially useful to all practice-based professionals as a means of critically examining practice and fostering deeper and more productive professional relationships. The concept of peer review is now a part of medical career development (Rodgers & Manifold 2002, Department of Health 2003). While in the business world, the utility of traditional paper-based approaches is being questioned and the value of face-to-face quali-tative approaches explored (Peiperl 2001). In this paper we report on the development of a qualitative approach to gathering feedback that contributes to these devel-opments.
Objectives
An expert panel in the field of accreditation critiqued the study during its development. They argued that corroboration was necessary so that practitioners could reflect on discrepancies between their own views and those of others and that such views could be used to affirm or challenge the practitioner’s own views. To this end, the concept of 360-degree feedback, used exten-sively in business and increasingly in health care (Alimo-Metcalfe 1998, Swain et al. 2004) as a devel-opmental tool, was examined and adapted.
Ward defines 360-degree feedback as,ÔThe systematic collection and feedback of performance data on an individual or group, derived from a number of the stakeholders in their performanceÕ (Ward 1997, p. 4). Data collected from stakeholders in an individual’s work provide feedback that the individual compares with his own self-assessment. The process consists of:
Box 1 Project aims
To identify, recognize and value expertise in UK nursing practice To develop a recognition process for expertise in practice
To develop further understanding of the concept of expertise in dif-ferent specialities
To explore the links between expertise and outcomes for service users and healthcare providers
Box 2
Critical companions
The termÔcritical companionÕhas been developed by Titchen (2001) to describe a form of helping relationship between colleagues in which one colleague with more expertise and/or experience in a particular aspect of practice uses a range of relationship and facilitative strategies to foster growth and learning in the other
• Role set selection (a group of colleagues selected to provide feedback on practice);
• Data collection – usually a validated psychometric tool;
• Score calculation of a feedback report.
By these means, the anonymity of feedback is assured. The literature asserts that this affords a degree ofÕsafetyÕ (Lepsinger & Lucia 1997, Tornow & London 1997, Ward 1997) for those involved. The literature suggests two principal benefits:
• Actively seeking feedback from a range of sources exemplifies responsiveness in an era when organi-zations change rapidly and service users expect action in response to their comments (Ward 1997); • Teams stand to benefit from developing cultures where there are frameworks that allow people to offer feedback on each othersÕ performance (Tor-now & London 1997).
The approaches traditionally used are predicated on two assumptions (Tornow & London 1997): anon-ymized feedback is more likely to result in honest
feedback; and, to ensure that feedback can be aggre-gated and anonymized; an adequate number of raters need to be invited to participate in the process (esti-mates recommended range from at least 12 to 30 or more role set members). The use of measurement scales facilitates the preservation of anonymity more readily than verbatim comments.
In this project, we questioned the assumption that randomization would deliver the kind of feedback that would be of use to our participants. Similarly we questioned whether anonymization was necessary for individuals to provide authentic feedback and whether overt provision of feedback could be ultimately more beneficial in promoting Ôimproved organizational com-munication and teamworkÕ (Tornow & London 1997, Day 2000). A traditional randomized and anonymized approach was hampered by the lack of relevant tools for this area of enquiry. Validated tools tend to address leadership and/or managerial behaviour. There are no relevant tools that look at practice expertise in nursing. On a practical level, models of 360 feedback appear to have been developed in organizations where workers
Box 3
Attributes of expertise – a concept analysis (Manley & McCormack 1997)
These attributes were developed using a research method known as concept analysis (Morse 1995). It is an approach that looks at the ways that a particular term (or assemblage of terms) is used. It enables the combination of evidence gathered from plural sources (e.g. various empirical approaches, policy, academic review, etc.) and has been used extensively in the development of theory in nursing
1.Holistic practice knowledgeis concerned with: Ongoing learning and evaluation from new situations
Using various forms of knowledge in practice (e.g. propositional, experiential, professional craft/phonetic, etc.)
Drawing from the range of knowledge bases (alongside experiential learning) to assess situations and inform appropriate action with consideration of consequences
Embedding new knowledge and accessing this in similar situations as they occur. 2.Saliencyis related to
Observation of non-verbal cues to understand the person's individual situation Listening and responding to verbal cues
Regarding the patient as a whole (i.e. recognizing their uniqueness) to inform treatment process Ability to recognize the needs of the patient, colleagues and others in the actions taken Picking up cues that can be missed or dismissed by others, to inform the situation 3.Knowing the patientis concerned with
Respect for people and their own view/understanding/way of being, of the world Respecting patientsÕunique perspective on their illnesses/situation
Willingness to promote and maintain a person's dignity at all times Conscious use of self to promote a helping relationship
Promoting the patient's own decision-making Willingness to relinquishÔcontrolÕto the patient Recognizing the person's own expertise 4.Moral agencyis concerned with
Providing information that will enhance people's ability to problem solve and make decisions for themselves Working at a level of consciousness that promotes another person's dignity, respect and individuality A conscientious awareness of integrity in one's own work
Working and living one's values and beliefs, whilst not enforcing them on others 5.Skilled know-howrefers to
Enabling others through a willingness to share knowledge and skills Adapting and responding with consideration to each individual situation Mobilizing and using all available resources
can anticipate contact with large numbers of coworkers. We could not assume that all nurses in the study would have contact with a large enough role set to mean that anonymization and randomization could be undertaken in any meaningful way.
Developing a qualitative approach to
360-degree feedback
We provided participants with guidelines for the col-lection of 360-degree feedback based on principles of qualitative data collection (see Box 4). The six groups then refined these processes in practice, an approach congruent with the project’s status as an action research study.
Participants provided an account of the approach that they used, together with their analysis of the evidence that they collected and its impact on their understand-ing of their own practice.
We supported the participants to work with their critical companions to identify a role set composed of a range of stakeholders in their practice, for example peers, junior and senior colleagues, colleagues from other disciplines and service users. Participants developed a Ôperson specificationÕ for a role set member. The attributes identified included: honesty and integrity; openness; a capacity to understand practice within the context of a bigger picture; fami-liarity with an aspect (or all) of the practitioner’s clinical practice; and people with both similar and different values and beliefs (see Table 1).
The role sets ranged in size from three to 10 with an average of 5.4 (and a median of 5). They broadly rep-resented theÔmixÕsuggested above. However, individual
selections varied. One participant asked for feedback from four colleagues who were of similar experience to herself (although she was relatively more ÔseniorÕ to them), stressing that, for her, the most important aspect in role set selection was finding people who had suffi-cient experience of working with her.
We provided the participants with a range of possible approaches to gathering feedback in terms of their potential advantages and drawbacks. The par-ticipants adapted to the methods suggested, some-times using the same approach with all role set members, sometimes using different approaches as dictated by the time and resources available to them. Broadly speaking, the approaches used fell into three categories (Box 5).
A variety of approaches was used to analyse the data collected; these can be summarized as:
• The critical companion analysing the themes in the feedback and presenting it to the expert participant; • The critical companion and expert participant
ana-lysing the data together.
For approaches where the anonymization of feedback was necessary, the first approach tended to be used as the process for developing themes from the data. This approach provided an opportunity to generalize the points made. However, this then made it difficult to present detailed and specific feedback.
Outcomes
The findings confirm that this form of data is useful to practitioners seeking to understand their practice. The data also provide some support to our belief that
ano-Box 4
How much feedback isÔenough'?
The guiding principle for role set selection emphasized identifying a group that would help the participants understand their practice expertise. To draw a parallel with qualitative research we suggested that participants identified aÔtheoretical sampleÕ(Silverman 2000) rather than a random sample. In other words, the sample is selected on the basis of relevance to the questions being posed and of peoplesÕcapacity to provide the information needed for the activity at hand (Mason 1996).
Table 1
A matrix for ensuring coverage of the scope of practice and different types of feedback
A role set member should have experience of my:
Client group Counselling skills Clinical knowledge Able to provide challenging feedback Will provide supportive feedback
Clinical nurse specialist A 4 4 4
Doctor B 4 4
Occupational therapist C 4
Nursing auxiliary D 4 4
nymity is not a prerequisite to the provision of open, honest feedback.
To ensure that feedback was sufficiently detailed and explicit, those collecting evidence from role set mem-bers had to actively probe with their questioning. One critical companion put it this way:
Ô(I have) done five interviews so far and got on with the analysis, people are being really positive but (we are) not getting the critique, not getting areas of development, there is some stuff, but its so subtle… so we have had to move to more interrogative interview style to get at the actual experiences.Õ
However, participants reported that even relatively gentle challenges were enough to stimulate reflection and self-critique.
Another general comment about the feedback was that it tended to be consistent despite the degree of detail being raised. For example, all members of a role set identified the nurse in question as being thorough and methodical.
There were nonetheless examples of role set members who spoke about the same person quite differently. For example, one role set member suggested that the par-ticipant did not see the person’s experience from all angles and did not fully appreciate the contribution of other approaches to treatment, while another respond-ent felt that their appreciation of the wider context was clear. This disagreement provided the opportunity for the participants to reflect on their practice more deeply. We set out to challenge the assumption that ano-nymity has an effect on the openness of the feedback offered. On the basis of this, admittedly small, set of examples we argue that the lack of anonymity does not mean that a role set member will not provide both supportive and challenging feedback, although it would seem that both candour and the capacity to provide detailed feedback require time and effort to develop. For example, in one case the opportunity to provide feedback was the catalyst for initiating difficult discus-sions about the extent to which the expert participant could be seen to dominate her colleagues through her own enthusiasm and dynamism. This situation had been hidden until the 360-degree feedback was gathered.
The activity provoked a degree of anxiety amongst the participants. However, they also reported feeling affirmed by the nature of the feedback that they received. One talked about how useful it was to know that she was valued as Ôthey had never made it explicit beforeÕ. Another said:
The most significant experience that I experienced …was the 360 feedback exercise. It was at that point in my journey that I experienced aÔdifficult phase of angst …Õ before my transformation. The feedback was in itself extremely positive. It was the analysis of the content of the feedback that really challenged my understanding and ability…Õ Although challenging, this participant found that the exercise helped her to explore her practice more deeply and ensure that skills evident in her practice with patients were applied equally to managerial and edu-cational aspects of her role.
Conclusions
Gathering evidence to demonstrate professional and personal effectiveness is increasingly important in health care. The qualitative approaches to 360-degree feedback developed as part of this project contributed corroboration and critique, as well helping participants to develop more open relationships with colleagues and service users. To optimize the usefulness of the process we would suggest the following recommendations:
• Thorough preparation. It is necessary to be clear with all involved about the purpose of the exercise and to be clear about the criteria used to identify role set members.
• Suitability. Approaches to gathering information need to reflect what is convenient and appropriate to the colleagues one is working with. They also need to be manageable for the person gathering the information; for example, interviews provide rich data but are time consuming to organize and pro-duce large quantities of data that needs to be proc-essed and analysed (which may mean the acquisition of new skills and knowledge). Producing and dis-tributing a questionnaire may be more manageable
Box 5
Approaches to gathering evidence from role set members
Feedback gathered in interviews by the critical companions – used most frequently (nine cases). The conceptual framework of expertise was used to explore responses to broad questions, for example,ÔHow would you describe X's work in this departmentÕ
Both critical companion and expert participant involved in gathering data – used in seven cases
and less threatening but the detail of feedback may be less.
• Anonymity. Within the project there were examples of both anonymized and open approaches to gath-ering feedback. Both were found to be useful. Open approaches provided more detailed information for the development of expertise and workforce effect-iveness.
The world of business has started challenging the impersonal approaches to gathering feedback suggesting that customized, qualitative feedback, though more dif-ficult and time consuming to generate, is more helpful in improving performance (Peiperl 2001). Similar logic is apparent in tools such as theÔMini Peer Assessment ToolÕ developed as part of the Modernising medical careers
framework (Department of Health 2003). On the basis of our research we would concur with this view and recommend the use and testing of this approach as part of a strategy for professional development.
Ethics approval
The London Multicentre Research Ethics Committee (Central Middlesex Hospital ref. MREC/OO/2/82) granted full ethical approval in March 2002.
Role of the funder in the research process
The study was funded through the Royal College of Nursing Institute. RG, SH, AT and KM were employees of the RCN Institute during the conduct of the study. Guidance from a wide range of stakeholder groups guided various phases of the project (Box 6).
References
Alimo-Metcalfe B. (1998) 360 degree feedback and leadership
development. International Journal of Selection and
Assess-ment6(1), 35–44.
Day D.V. (2000) Leadership development: a review in context. Leadership Quarterly11(4), 581–613.
Department of Health (2003)Choice and Opportunity
Modern-ising Medical Careers for Non-consultant Career Grade doctors. Department of Health, London.
Department of Health (2005) Agenda for Change: NHS Terms
and Conditions of Service Handbook. Department of Health, London, pp. 178.
Grundy S. (1982) Three modes of action research. Curriculum
Perspectives2(1), 23–34.
Hardy S.E., Titchen A. & Manley K. (2007) Patient narratives in the investigation and development of nursing practice
expertise: a potential for transformation.Nursing Inquiry 14
(1), 80–88.
Higgs J. & Titchen A. (2001) Practice Knowledge and
Exper-tise in the Health Professions. Butterworth Heinemann, Oxford.
Lepsinger R. & Lucia A.D. (1997) The Art and Science of
360-degree Feedback. Jossey-Bass Pfeiffer, San Francisco, CA.
Manley K. (2000) Organisational culture and consultant nurse
outcomes: part 2 – consultant nurse outcomes. Nursing in
Critical Care5(5), 240–248.
Manley K. & McCormack B. (1997)Exploring Expert Practice
(NUM65U). Royal College of Nursing, London.
Manley K., Hardy S., Titchen A., Garbett R. & McCormack B.
(2005)Changing PatientsÕWorlds Through Nursing Practice
Expertise: Exploring Nursing Practice Expertise Through Emancipatory Action Research and Fourth Generation Evalu-ation. RCN Institute, London.
Mason J. (1996)Qualitative Researching. Sage, London.
Morse J. (1995) Exploring the theoretical basis of nursing using
advanced techniques of concept analysis.Advances in Nursing
Science17(3), 31–46.
Newman P. & Peile E. (2002) Valuing learnersÕexperience and
supporting further growth: educational models to help
experi-enced adult learners in medicine.British Medical Journal325,
200–202.
Nursing and Midwifery Council (2004) The PREP Handbook.
NMC, London.
Peiperl M. (2001) Getting 360 degree feedback right. Harvard
Business Review79(1), 142–147.
Rodgers K.G. & Manifold C. (2002) 360-degree feedback: pos-sibilities for assessment of the ACGME core competencies for
emergency medicine residents… presented at the Council of
Emergency Medicine Residency Directors (CORD) Consensus
Conference on the ACGME Core Competencies: ÔGetting
Ahead of the Curve,ÕMarch 2002, Washington, DC.Academic
Emergency Medicine9(11), 1300–1304.
Royal College of Paediatrics and Child Health (2000)Portfolio of
Achievement for Senior House Officers. Royal College of Pae-diatrics and Child Health, London, pp. 68.
Silverman D. (2000) Doing Qualitative Research: A Practical
Handbook. Sage, London.
Swain G.R., Schubot D.B., Thomas V. et al.(2004) Three
hun-dred sixty degree feedback: program implementation in a local
health department.Journal of Public Health Management and
Practice10(3), 266–271.
Titchen A. (2001) Critical companionship: a conceptual
frame-work for developing expertise. In Practice Knowledge and
Expertise in the Health Professions(J. Higgs & A. Titchen eds), pp. 80–90. Butterworth-Heinemann, Oxford.
Tornow W.W. & London M. (1997) Maximizing the Value of
360-Degree Feedback. Jossey-Bass, San Francisco, CA.
Ward P. (1997)360 Degree Feedback. Institute of Personnel and
Development, London.
Box 6
For further information about copies of the full project report please contact:
RCN Institute Practice Development Team, 20 Cavendish Square, London WC1G ORN, UK. Tel.: + 44 (0) 20 7647 3673
Resources developed through the project can be found at: http:// www.rcn.org.uk/resources/practicedevelopment