Reliability and validity

3.8.1 Verbal report measures

Interview schedules and questionnaires employed in the Ceres Intervention Study were carefully developed to ensure reliability and validity, given the extent to which verbal report measures were relied upon for subject eligibility and for assessment of matching variables and treatment outcomes. The pre-test-post-test AUDIT scores and correlation methods used to compare answers between similar questions on different questionnaire schedules, compare adequately to studies in this field. The Project MATCH Research Group (1996) - Matching Alcoholism Treatments to Client Heterogeneity – a research project undertaken in the United States of America to match clients with three different intervention therapies - reported that urine drug screens were highly consistent with self-reported drug use at baseline and follow-up. When discrepancies were observed it was more likely that clients reported drug use when the urine screen was negative. Similarly, clients tended to report more use of drugs and alcohol than did their collateral informants. Collateral support persons were also used in the Ceres study to monitor self-reports on drinking. It was further reported by the Project MATCH researchers that self-reports of drinking were also examined in relation to gamma glutamyl transpeptidase (GGTP- blood samples used to analyse and monitor liver enzymes). The study indicated that self-reported alcohol use was consistently higher for clients with abnormal GGTP test results. The Project MATCH researchers concluded that after extensive research on biochemical measures, they found that the reliability and validity data indicate that a high degree of confidence can be placed in the accuracy of the verbal report data obtained in their study (Project MATCH Research Group, 1996). Following these and other reports on the confidence placed in the accuracy of verbal report data as well as the cost involved in using blood samples and urine drug screens in the community, it was decided to only use verbal self-report measures and collateral reports to collect alcohol use data in the Ceres Intervention Study. Written notes on observations made during interviews and field notes were utilised to clarify quantitative data.

3.8.2 Testing for reliability of responses

To test for the reliability of answers provided by pregnant women to questions on drinking recorded on the PAQ and the AUDIT, a model consisting of three drinking groups was defined at baseline, depicting a non-drinker group, an unconfirmed

68

drinker group and a confirmed drinker group (shown in Figure 3.3 below). For this, the alcohol responses from the PAQ about current drinking was used: “Do you currently drink alcohol?” to assess whether they are reported drinkers and the first question on the AUDIT “How often do you have a drink containing alcohol?” was used to confirm/not confirm their drinking. It was felt that pregnant women may have underreported their drinking and therefore the model was developed to compare with the AUDIT scores.

How the AUDIT questions and responses work, including the scoring of responses, for example:

Question 1: How often do you have a drink containing alcohol? 0 Never

1 Once a month or less 2 2-4 times a month 3 2-3 times a week 4 4 or more times a week

If they responded “yes” to the question “Do you currently drink alcohol?”, and confirmed this by responding 1 to 4 to the question “How often do you have a drink containing alcohol”, they were considered as “confirmed drinkers”. If they responded “no” to “Do you currently drink alcohol”, and they confirmed this by responding 1 to “How often do you have a drink containing alcohol?”, then they were considered as “non-drinkers” but included in the study as mentioned before because of reasons of underreporting and the possibility of drinking later on in the pregnancy.

If they responded “no” to “Do you currently drink alcohol?”, but responded 1 to 4 to “How often do you have a drink containing alcohol?”, then they were considered as “unconfirmed drinkers”.

69

Figure 3.3 Model of three drinking groups in the Ceres Intervention Study

The “unconfirmed drinkers” is a mixed group of women, either responding unreliably to “Do you currently drink alcohol?”, or women who stopped drinking when they became pregnant, or women who are truly past drinkers. According to the literature, past drinking is a strong indicator for drinking during pregnancy as they might start drinking again during pregnancy (Chang, et al, 2006:245-251). These three groups (1-3 above) correspond to the three groups discussed in Chapter 4. This indicator variable for the drinking groups was entered into the model as an interaction effect to assess whether the intervention effect (IE) is different for the three drinking groups. Results are reported in Chapter 4 and supported the decision to administer the full intervention (information, simple advice, brief counselling and continuous monitoring) to all the women in the intervention arm of the study, because any amount of alcohol could harm the foetus.

3.8.3 The interview schedules in the Ceres Intervention Study

A shortcoming of the study is that it was not planned to be a qualitative study and therefore qualitative observations are limited to ad hoc notes. Some of the qualitative

Question 1: Do you currently drink alcohol? (PAQ)

Question 2: How often do you have a drink containing alcohol?

1. Once a month or less. 2. 2-4 times a month. 3. 2-3 times a week. 4 or more times per week (AUDIT Q1:1-4)

Group 1:

Yes to question 1 + Yes to question 2 to (Q1:1 – 4) = reported drinkers

Confirmed drinkers

Group 2:

No to question 1 + Yes to question 2 to drink 1 × pm or less (Q1:1) = non- drinkers

Group 3:

No to question 1 + Yes to question 2 to (Q1:1 -4) = unconfirmed drinkers

Non-drinkers included in study

Unconfirmed drinkers

70

data was derived from answers on interview schedules, field notes made during the course of the study, and programme notes produced after the study.

It is important to point out the limitations and threats to the validity of the responses derived from interview and observation schedules. According to Babbie & Mouton, (2001:349) social desirability response sets where some respondents provide the interviewer with responses they believe are desirable or expected by the interviewer can influence the outcome of the study. In the Ceres Intervention Study, I correlated answers to previous answers and checked on information provided in earlier sessions. The trust relationship with the participant provided opportunities to probe for explanations to answers and to clarify confusing questions. My dual role as counsellor and researcher afforded an opportunity to probe deeper into the answers provided and the interactive interplay between counsellor and participants provided a platform for openness and honesty. I had to be constantly aware of interpretation bias and to seek objectivity in recording the information provided by participants. It was helpful to keep to the wording of the questions and to be trained in the contents and proper application of the screening questionnaires and interview schedules.

In the next chapter, results from both the quantitative and qualitative data are analysed and discussed.

71