'The art of decision-making: critical reflexivity when undertaking qualitative data analysis

'The art of decision-making: critical

reflexivity when undertaking qualitative

data analysis’

Symposium chair

Professor Faith Gibson

Clinical Professor of Children and Young People’s Cancer Care

Great Ormond Street Hospital for Children and London South Bank University.

Qualitative research and theoretical

frameworks: uncomfortable bed-fellows?

Professor Julie Taylor

Head of Strategy and Development, NSPCC (seconded from the University of Dundee)

Dr Caroline Bradbury-Jones

Research Fellow, School of Nursing and Midwifery, University of Dundee

Presentation aims

 Propose a typology regarding the

application of theoretical frameworks within qualitative research

 Explore the issues that occur when

undertaking qualitative research that is closely aligned to a theoretical model

 Examine strategies that may be developed to overcome any

Induction and qualitative research

 Induction and qualitative research are comfortable bed-fellows

 Induction is the process of drawing conclusions and building theory from data

 Induction makes generalisations from individual ‘cases’

 Induction stands in contrast to the theory testing of deduction, which is more

aligned to quantitative research

 Although these form a continuum, rather than being diametrically opposed,

tensions can arise when these two ontological positions merge

Qualitative research and theoretical

frameworks: a proposed typology



Retrospective



Prospective absent



Pseudo-prospective

An example: a qualitative study on

domestic abuse

 Theoretically, we drew upon Leventhal and colleagues’ (2003) Common-Sense Model (CSM) of self regulation of health and illness

 Central to this model is representations - or beliefs - about illness. CSM is typically used in disease/condition orientated research relating to how people’s beliefs alter their behaviour

 The five CSM domains are: identity, cause, time-line, cure/controllability and consequences

 Our application of this theoretical approach helped to impose structure and organisation throughout the research process.

Leventhal H, Brissette I & Leventhal EA (2003) The Common-Sense Model of self-regulation of health and illness. In: Cameron, L.D. & Leventhal, H. (Eds.) The Self-Regulation of Health and Illness Behaviour. Routledge: London, pp. 42-65.

Interpretation of CSM applied in the study

Domain Original understandings Interpretations in this study Identity The label given to a condition Recognition and naming of

domestic abuse ‘How it shows itself’

Cause Ideas about perceived causes The context in which domestic abuse occurs

Time-line Beliefs about how long the condition will last

Temporal aspects of domestic abuse, such as the ‘right time’ for routine enquiry and disclosure

Curability/ controllability

Beliefs about the extent to which a condition can be cured

Where the responsibility lies for disclosure and subsequent

response

Consequences Perceptions regarding the consequences and impact of a condition

Consequences of domestic abuse for women; their children and health professionals

The challenges



Close application of

theory in this particular

study, generated tensions

- particularly during data

generation and analysis



There was risk of

‘squeezing’ the data to fit

the model

Strategies employed

 Adaptation of the framework was necessary

 Inductive analysis, followed by mapping onto the

theoretical framework, was a key strategy

 Although helpful to overcome during analysis,

Discussion

 Theoretical frameworks are helpful – some would argue, necessary

 A counterargument is that nursing is hamstrung by application of theory

 Decisions need to be made regarding the degree to which a framework will drive the study, particularly the analysis

 Adhering rigidly to the framework may not always be helpful

 Any divergence from a framework (as originally intended) must be explained

Conclusion

Theoretical versus pragmatic design

challenges in qualitative research

Dr Joanna Smith

Lecturer in Children and Young People’s Nursing

School of Nursing, Midwifery and Social Work, University of Salford

Professor Francine Cheater

Director of the Institute for Applied Health Research Glasgow Caledonian University

Dr Hilary Bekker

Senior Lecturer in Behavioural Sciences, Institute of Health Sciences University of Leeds

Presentation aims

 Outline challenges when choosing a qualitative research design

 Explore the merits of undertaking qualitative research with or without being underpinned by a theoretical framework

 Outline potential benefits of adopting a generic design in a study that explored parents perspectives of living with a child with a long-term condition

Research designs

Design Paradigm Purpose and examples

Descriptive Qualitative / Quantitative

Describe situation or experiences to provide a picture of what is happening Cross sectional studies / survey designs

Theoretical Qualitative Develop theory to understand processes Typified by grounded theory

Analytical Quantitative Quantify relationship between two factors Experimental studies typified by RCT’s Observational studies typified by cohort and case control studies

Nursing research:

past and present debates

 Qualitative versus

quantitative debate

 Polarisation of

researchers whether

qualitative studies should be underpinning by

theoretical framework versus generic approach

Approaches to undertaking qualitative

research

 No recognised classification qualitative approaches  Range of approaches

can relate to:

 Use and meaning of language  Describing and interpretation participants’ views  Developing theory  Approaches can be  Underpinned by theoretical framework or epistemological position such as grounded theory or phenomenology  Independent of theoretical framework (generic approach)

Theoretical versus generic designs:

strengths

 Theoretical designs  Epistemology/ ontology

underpinning theoretical approaches shape/ provide meaning to data collection and interpretation

-methodological coherence

 Researchers position made explicit

 Invaluable in explaining organisational cultures, professional and patient behaviours

 Generic designs

 Data collection and analysis ‘fit’ research question rather than question ‘fitting’

particular theoretical position

 Accurately describes

participants’ experiences -stays close to the data

 Ensure researchers interpretations are transparency

Theoretical versus generic designs:

criticisms

 Theoretical designs

 Poor application of theories devalue qualitative research

 Theoretical perspectives dominate the research, rather than

answering the question - attempt to seek ‘epistemological

credibility’

 Analytical method rather than research design

 Researchers interpretations not transparent - findings may not accurately reflect participants’ accounts

Generic designs

 Absence of theory results in lack of coherence,

impacting on findings credibility

 Analytical method rather than research design

 Data analysis can be superficial

The art of decision-making

Consider whether aim of study is to:

 Develop theory or enhance theoretical understanding

The art of decision-making

Designing a study aimed at understanding parents’ experiences of living with a

child with hydrocephalus

Study aims Theory

Study context

 Parents are responsible for managing their child’s long-term condition, including recognising and responding to changes in their child’s condition

 Shunts, the main treatment for hydrocephalus, frequently malfunction which can be life threatening

 Detecting shunt malfunction is difficult:

 Symptoms unpredictable, variable, similar to general childhood illnesses

 Describing parents’ experiences could:

 Foster greater understanding between parents and professionals

 Improve parent-professional collaboration when determining whether illness symptoms are shunt related

Theoretical perspectives and concepts

Long-term conditions  Theoretical perspectives relating to  Shifting perspectives model of chronic illness  Disability-stress coping model Participation

 Patient and family-centred

 Shared decision-making

Study aims

Describe and understand parents’ experiences and perceptions of living with a child with shunted

hydrocephalus

Specific objectives:

 Explore parents’ understanding of hydrocephalus and its treatment

 Describe how parents learn about shunt management and associated complications

 Explore parents’ decisions when their child experiences illness symptoms and their subsequent health seeking behaviour when shunt malfunction is suspected

Choosing a design - influences

 Based on interrelated concepts:

interpretivism and reflexivity balanced with pragmatism and transparency

 Desire to:

 Describe parents experiences

 Apply analytical strategies systematically to accurately represent parents’ accounts

 Distinguish researchers’ interpretations from parents’ descriptions

 Be open to scrutiny to demonstrate transparency  Acknowledging personal values and potential bias

Study design

 Cross-sectional survey employing qualitative

methods

 Semi-structured interviews with 25 parents

Did a generic approach enable study

aims to be achieved?

 Data elicitation techniques generated rich data

 Links between the original data and final themes

were transparent not driven by theoretical framework

 Flexibility of framework approach and iterative

nature of analysis enable:

 Parents experiences to be described in depth

 Development of a conceptual framework that

explained the dynamic nature of living with a child with hydrocephalus

Findings: concepts and themes

Concepts Theme

UNCERTAINTY  Reactions to the child’s diagnosis

 Concerns about the shunt

 Receptiveness of professionals to interacting

with the family

 The child’s future DEVELOPING

EXPERTISE

 Making sense of hydrocephalus and treatment  Understanding support organisations

 Differentiating between childhood illness and

shunt malfunction A NORMAL

LIFE

 Barriers and facilitators to normal family life  Valuing normal life

UNCERTAINTITY DEVELOPING EXPERTISE Learning through experience Adapting to changes in the child Making decisions

Conceptual framework:

living with a child

with hydrocephalus

LIVING A NORMAL LIFE

Conclusion

 Choosing an appropriate qualitative research

designs can be challenging

 Decisions must be based on the specific

purpose of the study

 In clinically focussed research such as exploring

parents’ experiences, qualitative methods could stand alone rather than being theoretically

Application of framework approach across data

sets in a study that evaluated national

advanced communication skills training

programme

Professor Jane Coad,

Children and Family Nursing, Coventry University

Professor Faith Gibson

Clinical Professor of Children and Young People’s Cancer Care

Great Ormond Street Hospital for Children and London South Bank University

Dr Joanna Smith

Lecturer in Children and Young People’s Nursing, University of Salford

Professor David Pontin

Presentation aims

 Outline the application of

framework approach

across data sets in a study that evaluated an

advanced communication skills training programme

 Discuss the challenges in

applying the framework approach

Qualitative data analysis - 1

 Choice depends on study aims but process

similar:

 Transcribing interviews

 Immersion in the data

 Developing data coding system (categories)

Quantitative Analysis Qualitative Analysis Deductive Inductive Statistical Analysis Content Analysis Thematic Analysis Grounded Theory IPA DA

Qualitative data analysis - 2

Framework Approach

IPA (interpretive phenomenological analysis); DA (discourse analysis); CA (conversation analysis)

(Smith & Firth 2011)

Framework approach

 Gaining popularity because useful to

interpret complex data

 Approach is:

 Systematic

 Explicit

 Interconnected stages

Framework approach- overview

Data

management

 _{Become familiar with data}

 _{Identify initial categories / themes}  _{Develop a coding index}

 _{Assign data to categories}

Con

tinu

um

Descriptive accounts

 _{Synthesise range and diversity of coded data}

 _{Identify association between categories / themes} until the ‘whole picture’ emerges

 _{Develop more abstract concepts}

Explanatory accounts

 _{Develop associations/ patterns within concepts} and themes

 _{Reflect on original data/ analytical stages} -accurately represent participant accounts  _{Interpret concepts and themes}

Application framework approach

-evaluation of the advanced

communication skills training

programme

 Study aimed to identify whether the current

programme increased skills, knowledge,

competence and confidence of the participants when communicating with children, young

people and their families

 Make recommendations in relation to the future

Design and methods

 Evaluation based on appreciative inquiry

 Forty-five health professionals, primarily

doctors and nurses, participated

 Data included individual interviews, pre and

post course evaluations, e-mail survey, reflective work records

 Framework approach was used to examine and

Framework approach - processes

 Research team becoming familiar with the data

through a process of reading and re-reading participants’ narratives

 Narratives from pre and post course

questionnaires and two interviews (one recipient and one facilitator) were used to generate initial codes and preliminary categories

Identifying preliminary categories

Description (in-vivo codes) Preliminary thoughts Preliminary categories

Data: Interview 3 (JS) Recipient

Wanted to improve talking to families, particularly breaking bad news to families

Being effective

Being effective when engaging with

families

Data: Interview 2 (JC) Facilitator

it’s a surprise how little people know or the way they communicate.

There’s a perception that more experience

means you will have improved

communication but it’s not the case

Perceptions about communicating effectively

Data: pre-course questionnaire

I want to ensure I am giving most effective

information/advice/ support using most

appropriate skills I have

Being effective

Being effective

when engaging with families

Example of coding index

Concept: Pre-course perceptions

Preliminary themes Preliminary categories

Knowledge and skills deficits

Discussing difficult topics

Ensuring interactions are focused Communicating with children Managing conflicts Parents and child conflicts

Inter-professional conflicts

Family and professionals conflicts Building confidence Responding to emotions

Handling difficult topics Handling difficult situations Constraints to developing communication skills Workload pressures Time pressures Desired outcomes of effective engagement

Being effective when engaging with families

Developing effective parent-professional collaboration Ensuring family compliance

Descriptive accounts

 Labelled all interview data into categories in coding index but ongoing adaptation of coding index

 Crucial element within analysis is critical thinking in relation to:

 How participants’ descriptions are coded

 Links between data sets

 Links between codes and categories

 Links between categories and themes

 Reviewed data within each category  Ongoing refinement of categories

Framework approach:

developing themes and categories

 Iterative process

 Synthesising and summarising data

 Moving back and forth between initial themes and categories

 Constant refinement

Key findings: themes and categories

Pre-course preparation, preconceptions and expectations Course delivery, teaching and learning strategies Post-course evaluation and ongoing development needs  Accessing the course  Perceptions about communicating effectively  Expand knowledge and skills  To develop personally and professionally  Effectiveness of learning and teaching strategies  The learning environment  Evaluation of course structure  Evaluation of course content  Personal and professional development

44

Using framework approach :

pros and cons

Pros

 Systematic - particularly first stages

 Flexible - iterative process

 Stages grounded in the data  Comparison across data sets  Transparency can be demonstrated Cons  Grappling with terminology (themes, categories, concepts used interchangeably)  Managing the data

Conclusion

 Data analysis in qualitative research is a

complex ‘messy’ affair - more complex when there is a range of data sets

 Need to identify an approach to synthesising

and summarising the data that meets study aims

Team - Centre for Children and Families Applied Research (CCFAR)

Professor Jane Coad (Principal Investigator)

Reena Patel (Research Assistant); Wendy Higman (Research Assistant) Katie May (Project Assistant); Marion Lowe (Family Support Officer)

David Widdas (Nurse Consultant); Liz Herd (CCN); Joanne Holder (CCN) Rebecca Widdas (Health Practitioner)

Sean Graham (Technologist); Daisekh Ruparelia (Technologist); Prof. Lynn Clouder (CELE)

Application of thematic analysis across a national survey data set that evaluated an interactive

Presentation aims

 To discuss the value of qualitative data collection and the usefulness of thematic analysis within survey data.

 Using a Dept of Health funded project drawn from our CCFAR programme of work, this paper will explore ways of integrating thematic analysis with quantitative data

findings to enhance the evaluation of an interactive competency training programme

Background to project



Many carers are not adequately trained or competent in the care of Long Term Ventilated (LTV) children and

young people (Briscoe et al, 2010)



Evaluation of the impact of an interactive competency training programme on the competence and confidence of professionals and families when caring for children who have long term ventilation needs



The ‘Breathing Matters’ project focuses on improving training for carers, parents and associated professionals supporting families caring for LTV children/young people living at home

What is the project?

• Identify Training Needs of Staff/Carers • Developed Interactive Competency Framework The Coventry and Warwickshire Trust • Understand Training Needs of Staff in 5 nationals areas and 30 carers across England • Evaluate Interactive Competency Framework CCFAR in Coventry University Pilot ICF Survey Carers Assess Confidence/ Competence Train Carers

How was it implemented?

Phase 1

• E-survey Tool Development and Pilot

• Pre and Post training surveys developed in partnership with CELE

Phase 2

• Recruitment of staff carers • 5 nationwide sites

Phase 3

• Cascade training workshops hosted by Coventry University • Competency Training and survey undertaken

Phase 4

• Recruitment of parent and family member carers

• Family Support Officer and Clinical Researcher appointed

Phase 5

• Data Analysis and Evaluation

• Qualitative and Quantitative survey data retrieved

Data analysis framework

Theory of Change (TOC)

Quantitative data

Qualitative data

Data analysis challenge?

Quantitative Approaches

Used to measure levels of staff competency and confidence before and after training is undertaken

Qualitative Approaches

Used to explore lived experiences of staff and families caring for ventilated children

Mixed Method Approaches

How best to handle our data?

An analysis matrix was developed to ensure both quantitative and qualitative data was

systematically searched and categorised according to short and long term impacts

Decision around developing e-survey tools?

Survey questions were generated by mapping predicted outcomes from theory of change framework onto themed sub-scales

Conclusion

 Impact evaluation surveys are common but

ensuring integration of data from both

quantitative and qualitative data sets is essential in order to gain a rich understanding of

outcomes and key processes

 We hope that the usefulness of our solutions to

using thematic analysis in the matrix may have resonance for other researchers faced with such challenges in their data analysis

Professor Julie Taylor

Dr Caroline Bradbury-Jones Dr Joanna Smith

Professor Jane Coad Reena Patel

'The art of decision-making: critical reflexivity

when undertaking qualitative data analysis’

Leadership

Ethics and governance

Data extraction issues

‘Messiness’

Time issues

Team skills

'The art of decision-making: critical reflexivity

when undertaking qualitative data analysis’