Draft PRO questionnaire

In this section, the methods used to draft the PRO questionnaire are presented, including the planning of instrument usage, naming the questionnaire, generating verbatim items from CST users, constructing a draft version of the questionnaire, conducting semi-structured interviews to determine item selection, conducting a consensus meeting with CST practitioners to check item selection, undertake cognitive interviews with CST users to pre-test draft questionnaire and assess face and content validity.

107

4.9.1 Determined population

The population used to support the development and evaluation of a new PRO are CST users over the age of 16 (in practice, individuals age 16 years and over are considered adults).

4.9.2 Determined use

The new PRO is to be evaluative, assessing changes in health and wellbeing of those having CST. The PRO is intended for use in a clinical setting.

4.9.3 Naming the questionnaire

Factors accounted for when considering the name of the new questionnaire were as follows: the questionnaire may be applied in other CAM settings, therefore, caution was given not to limit the potential use of the questionnaire by putting ‘CST’ in the title. Adopting the name ‘Warwick’, utilising the reputation and the brand of the University, was discussed. In previous work (Brough, 2012) users attributed CST as having effects on aspects such as mind, body, and spirit, as well as being holistic. The researcher wanted to incorporate this into the title, resulting in the title Warwick Holistic Health Questionnaire (WHHQ). It is commonplace to use acronyms to shorten the name of a questionnaire. The newly created and named questionnaire will be referred to as the WHHQ throughout this report.

4.9.4 Construction of a draft version of the questionnaire

A draft version of the WHHQ including details such as the layout, instructions on completion, time frames, response options, and the wording of each item was prepared at the same time as the conceptual framework. The starting point for this thesis was to review the previous work by Brough (2012) based on interviews with a large number of CST clients. These data were used to create a list of potential items for a PRO. This was done by using verbatim statements to create items when possible. For example, when participants reported “feeling more relaxed and calm” (Brough et al, 2012, pg. 170) an item was generated “I’ve been feeling relaxed and calm” from the interview data. This process was repeated creating items related to

108

physical body (functioning, symptoms, daily activity), mental health (mind,

emotional affective, cognitive), spiritual wellbeing (awareness, relating to self and others), and holistic wellbeing. At that time the debate about the role of spirituality in healthcare and how to evaluate its influence on health was emerging and

questionnaires which covered outcomes of this domain were considered to aid the item generation for this domain including the World Health Organization’s Quality of Life - Spirituality, religion, personal beliefs instrument (WHQOL SRPB) (WHOQOL Group, 2006), JAREL spiritual wellbeing scale (Hungelmann et al., 1996), The Self- Perception and Relationships Tool (S-PRT) (Atkinson et al., 2004), (Atkinson et al., 2004; Group, 2006; Hungelmann et al., 1996; Ng et al., 2005; Peterman et al., 2002). An initial list of 35 items was created for the WHHQ.

When the conceptual framework had been evaluated and CST users and

practitioners had given their input and feedback on the content, further items were generated. The conceptual framework was used as a map and items were

generated for inclusion in the domain of social wellbeing and engaging in life; as were items covering the development of self-awareness, taking responsibility for self and everyday life, including life satisfaction and doing things I enjoy. This resulted in a list of 73 items for inclusion in the draft PRO. Attention was given to the way in which the items were phrased to keep the integrity of the participants’ input and to be able to evaluate how the development of awareness changed as individuals’ experience of CST deepened. Looking at the content of questionnaires in the field of mindfulness (Cardaciotto et al., 2008) and life satisfaction (Bussing et al., 2009; Bussing et al., 2007) assisted thinking about items linked to this domain, though the reliability and validity of these items were not checked. Items created to evaluate symptoms were reverse scored to ensure the flow of completion was smooth, but also as a means of checking that responders were paying attention to how they answered the items.

As described in the literature review (pg. 60) many questionnaires were studied and the design, layout, instructions and response options observed. The initial WHHQ draft was, therefore, based on these observations. These steps enabled content and face validity to be assessed later using cognitive interviews.

109

4.9.5 Evaluating the draft PRO

As the geographical spread of participants was broad, practical and financial

reasons led to the decision to undertake semi-structured interviews to facilitate the next phase of the research instead of the focus groups as initially intended.

Therefore, each face-to-face semi-structured interview was carried out at a place convenient to the participant, providing it was suitable for recording purposes. The semi-structured approach was adopted to explore areas where the participants perceived gaps and difficulties in the proposed questionnaire and to tailor the questions to the position and comments of the interviewee.

Recruitment was undertaken as previously explained (pg. 100) via the CSTA. Participants were contacted via email or by post at the request of the participant, and an introductory letter which outlined the aims and objective of this research was sent with a draft questionnaire containing 73 items (see appendix 5). An outline led the participants through a series of prompts (see below), providing space for them to record their thoughts and opinions, responses were used as reminders or prompts for discussion during the interview and handed in to the researcher for cross-referencing purposes during analysis.

Prompts used to assist in constructing the questionnaire consisted of: • Layout

1. What do you think about the design and layout of the questionnaire? 2. What do you like?

3. What don’t you like?

4. Does anything need changing? • Instructions

5. Are the instructions for completion clear? 6. Could you fill in this form without help? • Recall period

7. The questions focus on your symptoms over the last two weeks, is this a good time period?

110 • Response options

9. What do you think about the different response options? 10.Can you understand them?

11.Are there enough options?

12.If you don’t like them what would you like to see instead? • Questionnaire name

13.What do you think about the name: “Warwick Holistic Health Questionnaire”?

Participants were asked to select items from the list and to record the item number in one of six boxes, each box had a different heading: 1) items relevant to my experience, 2) I don’t understand the statement, 3) the response options are not suitable, 4) items are repeated, 5) item addresses more than one point, and 6) items you would like to remove, explain why? Participants were asked to make a note of anything else they wished to raise at the interview in relation to the topic. The data in relation to each item were documented under the relevant topic and items selected by each participant were recorded on a spreadsheet for later analysis.

4.9.6 Data analysis for semi-structured interviews

All interviews were audio recorded, the data were transcribed, anonymised and the transcripts were checked against the audios. Inductive thematic analysis (Braun & Clarke, 2006) was used to identify, analyse and report the themes found in the interview data. Concerning the 73-item WHHQ, the number of times an item had been either selected or not selected for inclusion on the questionnaire and the participant’s rationale for this, were considered as part of the analysis.

4.9.7 Consensus meeting with CST practitioners

An opportunity arose to consult with an established group of CST practitioners (n = 16), and it was decided that this would provide an opportunity to obtain

practitioner feedback on the draft questionnaire. All practitioners consented to taking part, to adhere to confidentiality and to having the session recorded for

111

transcription purposes. The group was separated into four, given a copy of the draft 73-item WHHQ and allocated a domain listing the proposed sub-domains and items for that domain. Practitioners were asked to work through the outline described in the previous section (see pg. 109) to evaluate the design, layout, instructions, time frames, response options and the clarity of each items in the domain allocated to them. Participants documented their discussions and feedback on the forms provided. Each group was allocated 20 minutes to undertake the tasks and each group fed back to the wider group and questions or concerns were worked through. The group discussion was audio-recorded and each group returned the completed feedback forms. Data were collated from the feedback forms allocated to each group, sorted and analysed based on the feedback of each small group and the consensus of the wider group. The audio recording was listened to but due to the competing group discussions it was not possible to capture data in this way. Comparisons were made between this group and the individuals who took part in the interviews. The results of both the semi-structured interviews and consensus meeting with practitioners was used to determine the content and items on the questionnaire. As a result, the 73-item WHHQ became a 52-item WHHQ.

4.9.8 Pre-testing the draft PRO (52-item WHHQ)

Cognitive interviews were undertaken to assess the content and face validity, that is, to find out if the questionnaire was user friendly, easy to understand, and to evaluate if the language, format and response options were acceptable to

participants. It also provided the opportunity to ascertain the average time it took participants to complete the 52-item WHHQ. Participants were interviewed at a convenient location for them. Consent was confirmed and an overview of the intention of the interview was given. The researcher followed an interview

schedule (see appendix 6). Interviews were audio-recorded and the length of time it took for each participant to complete the WHHQ was timed with a digital stop- watch. The researcher remained quiet until an issue arose and the respondent was then asked to think aloud questions such as ‘How did you go about answering that

112

was used on occasions when the interviewer noticed that the respondent hesitated before answering a question, revisited a question or did not complete a question. The audio-recordings were transcribed, and checked against the data.

4.9.9 Data storage

To ensure confidentiality, the Data Protection Act (1998) was adhered to and anonymisation or pseudonymisation of data was undertaken. Electronic data were backed up to an external hard drive and stored securely. Only the research team of the academic supervisors and researcher had access to the data. All data held on paper were kept under lock and key with only the researcher having access to it directly. Names and interviewee data were stored separately. As transcription of the focus group discussions was undertaken by a contractor, contracted

transcribers were asked to sign an agreement to observe confidentiality of the data.