Reliability and Validity

Reliability reflects the ability of a research instrument to yield the same results on repeated trials (Oppenheim, 2000). One method of determining the reliability of empirical measurements is the test-retest approach in which the same test is given to the same people at two points of time (Cohen et al., 2007). Reliability is then estimated by examining the consistency of responses between the two tests. Another way is to estimate the internal consistency (Cronbach α) of items within a construct. It reflects how well various items

complement each other in measuring different aspects of the same concept (Cohen et al., 2007).

Validity defines the degree to which a research instrument measures what it set out to measure (Oppenheim, 2000). There are several types. Construct validity (subdivided into convergent and discriminant validity), examines the degree to which the test measures the construct it was designed to measure (Neuman, 2005). Convergent validity occurs when measures of constructs that are expected to correlate do so whereas discriminant validity occurs when constructs that are expected not to correlate do not, such that it is possible to discriminate between them (Neuman, 2005). Content validity occurs when the research instrument provides adequate coverage of the subject being studied (Litwin, 1995). Its determination is principally judgemental and intuitive which cannot be expressed numerically. It can be determined by a panel of experts who can judge the extent of the measuring instruments’ standards (Litwin, 1995). Criterion validity examines the ability of a measure to predict a variable designated as a criterion (Neuman, 2005). Sub divided into concurrent (other criteria assessed simultaneously) and predictive validity (predicting future or past events), it is typically expressed as the coefficient of correlation between test scores (Litwin, 1995). External validity occurs when research results can be generalized to and across population, settings and times whereas Internal validity occurs when conclusions related to cause and effect can be made (Proctor and Capaldi, 2008).

The present study relied on self reports of students’ alcohol consumption. Much research about alcohol related issues in the college environment relies on self-reports of students’ alcohol use (Sobell and Sobell, 1995). The validity and reliability of self reported alcohol consumption has

been studied extensively. Consequently, a substantial body of empirical research shows that self reports are accurate proxies for objectively measured behavior both within (Midanik, 1988; Harrison and Hughes, 1997; Johnston, 2001; Brener et al., 2003; Lintonen et al., 2004) and out-with (Ramo et al., 2011) the alcohol consumption domain.

The present study utilized QF measures of self reported alcohol consumption the reliability (test-retest) and validity (content, construct and criterion) of which is evidenced in prior psychometric research (Romelsjo et al., 1995; Sobell and Sobell, 1995; Rehm et al., 1999; Room, 2000). The ecological validity of these measures is also established in similar (McAlaney and McMahon, 2007) and different settings (Perkins, 2007). The present study estimated the internal consistency reliability of personal consumption measures by examining the Cronbach α. The Cronbach α for the three items on personal consumption was found to be 0.735 which demonstrates good reliability.

The reliability and validity of SN measures (perceptions of descriptive and injunctive norms) is an important and complex issue in the field created in part because these measures tend to be tailored to the target population they are being used with. It also represents an under researched weakness of the field. However, in general several studies show participants’ estimates of the amount of alcohol consumed by their peers to be highly correlated with their own drinking (Wood et al., 1992; Perkins, 2002; Kypri and Langley, 2003; Linkenbach and Perkins, 2003a; Neighbors et al., 2006; McAlaney and McMahon, 2007; Perkins, 2007). In accordance with theory and past research, the current study also found high correlations between items of personal alcohol consumption and perceived norms (chapter 6, section 6.3) thus demonstrating construct and criterion validity. In addition, the present study found modest correlation

between descriptive and injunctive norms (r = 0.28 or less) indicating that these constructs are conceptually distinct. It adds to the evidence supporting the discriminant and convergent validity of descriptive and injunctive norms as has been noted in prior research (Grube et al., 1986; White et al., 1994; Sheeran and Orbell, 1999; Rivis and Sheeran, 2003). Hierarchical multiple regression procedure utilized in the present study revealed that perceived descriptive norms contributed a substantial proportion to the variance19 _{in personal alcohol consumption}

after several known confounding factors had been taken into account. This strengthens the evidence of concurrent validity of the descriptive norm measures used in the study. The high values of Cronbach α for both descriptive and injunctive norm measures (α = 0.772 and 0.756 respectively) provide confidence in the internal consistency of these measures.

The sample for the web survey was found to be representative of full time undergraduate student population at DIT in terms of gender and age. The information about student enrolments and demographics was obtained from the Higher Education Authority (HEA, 2012). Chi square goodness of fit analysis revealed no significant differences between the sample and population on these factors (p>0.01). Further, literature on issues related to non- response (Miller and Smith, 1983; Lindner et al., 2001) suggests that non-respondents tend to be similar to late respondents in responding to surveys. In keeping with this literature, the participants who responded to the first email invitation to the web survey within a week (responses received till 8th _{Nov, 2010) were labelled as early. Similarly, those who responded}

to the reminder invitation to the survey (responses received on or after 9th Nov, 2010) were

19 _{Perceived descriptive norms explained 20% of the variance in frequency of drinking in a typical month, 33.4%} of the variance in number of drinks consumed on a typical occasion and 20.9% of the variance in frequency of drunkenness in a typical month. Detailed results can be found in chapter 6, section 6.4.

labelled as late. Independent sample t tests were performed to compare early and late respondent groups on key variables (attached as appendix 6). No significant differences were found between early and late respondents indicating that the findings of the web survey can be reasonably generalized to the population. The aforementioned tests and subsequent findings strengthen the external validity of the present study.

The design of this study is cross sectional in nature which is why conclusions of cause and effect cannot be made with certainty. This limits the internal validity of the study (More on this in section 5.9.1.