Expert consultation

Experts were selected from a number of professions to reflect current understanding of widening participation policy and to ensure an array of views were included to limit the chance of potential bias. The 12 experts commented on the items as well as on the overall content of the scale.

University or HE. University was suggested as the more appropriate term to use within the measure, as university is a more widely known term than HE.

Family participation item. Consider breaking this item into parent/carer, siblings (brother/sister) and other family members, as this could be interesting for a multiple regression.

A number of the experts suggested adding in a sentence that stated there were no right or wrong answers, so respondents felt comfortable when answering the items.

Items were considered to cover the breadth of the topic area, with minimal rewording suggested to enhance quality. Considering comments, minor changes were made to the measure.

8.5.3C

I

Cognitive interviewing was an important stage of the scale construction as this ensured the scale could be used outside of the research context. This assessed the response process to items and identified any difficulties that required modifying. Furthermore to develop effective measures for adolescents the use of cognitive interviewing is recommended (Drennan, 2002). Cognitive interviewing was used to detect any errors in the question-answering process to then reword, delete, replace, or change the item order, so that the respondents could complete the scale with ease.

A number of approaches to cognitive interviewing can be used, for this project to improve the richness and quality of the data collected, the two approaches of

127 thinking aloud and verbal prompting were conducted simultaneously. Thinking aloud, involved the respondent speaking their thoughts about the question aloud, while the researcher either writes down or audio records the process. The personality of the respondent can impact on the richness of the data collected through the thinking aloud technique, with less extraverted people feeling uncomfortable with the situation and thinking aloud (Willis, 2005). Although, the method would allow the researcher to understand the cognitive process a respondent goes through when answering questions, this act of thinking aloud may not be appropriate for all respondents.

Therefore, this method was used alongside verbal probing to eliminate any bias from the thinking aloud process. This technique is becoming more favourable with cognitive researchers as the researcher plays a more proactive role in the interview (Willis, 1999). In verbal probing, the researcher reads out the question and the respondent then answers, the researcher then follows with probes to obtain additional information about the question/response. The types of probes used will depend on the research aims, the probes can be standardised and preset before the interview and the probes can be asked after each question or at the end of the questionnaire (Dietrich and Ehrlenspiel, 2010). Concurrent probing technique was used, as the information to reply is fresh in respondents mind reducing the possibility of the respondents fabricating an explanation, which is the risk of retrospective probing (Willis, 1999). Verbal probing allows the researcher to have control over the interview process and does not but extra strain on the respondent, as answering questions is what occurs in an interview situation.To ensure that the construction of the questionnaire eliminated any potential interpretation bias the two cognitive techniques were used simultaneously (Willis, 2005).

8.5.3.1METHOD

8.5.3.1.1 Sample

Nine young people (six girls and three boys) aged 13-18 (Years 9-11) from a school in Herefordshire were recruited.

8.5.3.1.2 Procedure

In each interview, the respondents were asked to complete an item, the researcher then asked the probe question and the respondent then answered the probe question. For each interview, this continued for each subsequent probe. Following Willis (1999) guidelines, the following scripted probes were used in the interviews:

128

1. Comprehension/interpretation probe- What does the term ‘research project’

mean to you?

2. Paraphrasing- Respondents were asked to repeat the item in their own words.

3. Recall probe- Can you remember how many of your family members have attended university?

4. General probes

a. Why did you choose to answer the item that way?

b. Was that an easy item to answer?

c. I noticed you seemed unsure there, could you tell me what you were thinking.

Informed parental consent was sought before the cognitive interview was conducted. Each respondent was interviewed in a quiet room on the school premise. Along with audio recording the interviews, a score sheet was used to document immediately any difficulties. Interviews were scheduled for half an hour, to reduce excessive demands on participants. Respondents were informed that the interview was not to collect survey data from them, but rather to test the measure and explore whether there were any items that were difficult to understand, hard to answer or made little sense, reassuring the young people to provide confidently their opinions.

8.5.3.1.3 Analysis

Content analysis was used to compute the frequencies of any difficulties, due to the flexible nature of the approach to understanding text data.

8.5.3.2RESULTS

Completion: All respondents reported that the instructions were clear and they understood how to complete the measure.

Items: Specific difficulties were noted for the following items.

‘Has a family member (for example parents/carers or siblings) attended or is going to attend HE?’ When answering the item respondents misunderstood the item and discussed younger and older siblings who intend to progress to HE and not family members who have been or were currently at university. The intention of the

129 item was to explore familial experience of HE, so was changed to -‘Has any of the following family members either gone to or is currently at university? Please tick as many options as applicable.’

‘I do not know enough about university to decide whether I want to go.’ Respondents acknowledged that they knew some information about university, but defined enough as knowing a wide range of information from courses to locations. The variations in the definition of ‘enough’ meant the understanding of the item was problematic, so the item was omitted.

‘Once I finish school, I would like to get a full time job rather than go to university.’ Respondents questioned whether school included sixth form. The item therefore needed to distinguish between compulsory and post-compulsory education. It was checked the respondents knew the difference between the two

educational stages. The item was rephrased to ‘Once I have finished compulsory

schooling, I would rather get a full time job than go to university’.

‘I know what grades are needed to get into university.’ The item was considered difficult to answer as respondents stated that they would only know the answer to this item once they were researching university options in sixth form, so the item was omitted.

‘I know I will achieve the grades needed to get into university.’ The phrasing of the item was considered emotionally loaded so respondents felt they could not agree, as they did not know whether they would achieve the grades, it was hoped they would. The suggestion was then to reword the item to ‘I think I will achieve the grades needed to go to university’.

‘My parents/carers encourage me to get good grades, so I can go to university.’ Respondents acknowledged that their parents encourage them to achieve at school, but not for the specific reason of attending university, as it was not an appropriate choice for all. The ‘so I can go to university’ part of the item was therefore omitted.

‘My school promotes progression to university’. Respondents questioned how this was evidenced in a school environment and were unsure whether their school

130

did. The suggested change was made to reflect this to ‘My school promotes

achievement and progression to further education’.

‘The career advice at school focuses on university’. Careers advice discussed by the respondents was to plan future goals, which may/may not include university, so

advice was not centered on university. The item was changed to reflect this ‘The

careers advice at my school promotes progression to university’.

Measure overall: Generally, respondents felt the measure to be of relevance and easy to complete, initiating at times discussions with the respondents about university and university life.

Cognitive interviewing was used to assess the interpretation of each item and to explore the response process to assess the measures applicability, few difficulties were detected in the respondent’s ability to complete the items. Those difficulties highlighted through item probing required minimal modifications, including rewording or redefining of what information the item was requesting from respondents. The Flesch-Kincaid reliability score for the final scale was 7.8 indicating that the items could be easily understood by a child aged 12 years old.

Mistakes in the early stages of scaledevelopment can lead to difficulties later on in the process. Therefore a considerable amount of time was dedicated to the development of the scale items. The three phases discussed ensured at the first stage of the measure development a concise valid measure was constructed. The next stage explored the psychometric properties of the measure.