Limits to the generalisation of the results

Given that this research seeks to improve understanding of UK domestic ICE appliance electricity consumption, and provide recommendations to inform policy, the small sample size raises the question of whether the findings from this research can be generalised to the wider UK population. The generalisation of results relates to the external validity of the research and is a standard aim for most quantitative research studies. This is usually achieved by statistical sampling procedures, which allows the degree of

representativeness of a sample to be checked and broader inferences about a larger population to be made (Robson, 2002). To achieve this, a large sample size is required to reduce bias and capture representative characteristics of a larger population (Silverman, 2006).

Both electricity monitoring studies and qualitative studies (irrespective of a study attempting to combine both research techniques) are faced with the difficulties in the generation of results that can be generalised to larger populations. This is because the

132

methods of data collection and analysis are difficult to apply to a large sample. Collecting household energy monitoring data “is a difficult, time-consuming and expensive process”

(Isaacs et al., 2006a p9) and “is usually not practical when a large sample size is targeted”

(Tso and Yau, 2003 p1680). Thus, most energy monitoring studies “can only involve small population samples that are usually drawn from the same locale” (Lopes et al., 1997 p2).

This inevitably produces results that can be difficult to extrapolate to a larger population (Lopes et al., 1997).

For example, despite the REMODECE project collecting electricity consumption

measurements from one hundred homes, in each of the twelve participating countries, the project concedes that the “sample used for the monitoring campaign should not be

regarded as representative and statistical inference cannot be directly applied to

households” (Larsonneur, 2006 p7). Therefore, REMODECE highlights that the study was more concerned with the identification of patterns of use, equipment efficiency and

behaviours that could reveal typical profiles and common electricity consumption

characteristics at the European level (Larsonneur, 2006). Similar issues are reflected in the standby power study by Ross and Meier (2000). The research concluded that their sample of ten homes:

...cannot provide definitive evidence of the magnitude of standby power consumption.

However, it can provide new insights to the scope of the problem and the opportunities for reducing it.

(Ross and Meier, 2000 p6)

The constraint to the sample size is mirrored in the qualitative component of this thesis.

Representative sampling procedures are typically unavailable to qualitative researchers, because larger sample sizes prevent the type of intensive analysis most commonly used in qualitative research (Silverman, 2006). Therefore, the generalisation of results has been the subject of much scientific debate between scholars from opposing philosophical positions across a quantitative and qualitative divide (Kvale, 1994). Similar to those

involved in energy monitoring studies, scholars who support the use of qualitative methods

133

also argue that smaller, detailed research studies can provide new and valid insights. For example, Gray (2004) contends that:

...just because a study does not find results that are capable of generalization does not mean they have no relevance. A small case study, for example, may produce findings that are interesting and possibly indicative of trends worthy of replication by further research.

And from a perspective-seeking view they may be seen as valid in their own right.

(Gray, 2004 p89)

Similarly, Crosbie (2006) cites Wilk and Wilhite (1986) who argue that qualitative methods:

...yield finely grained and detailed information that cannot be obtained through

questionnaires, and they often provide unexpected insights and lead to productive new lines of inquiry.

(Wilk and Wilhite, 1986 p52: cited by Crosbie, 2006 p740)

Thus, despite the small sample size it is believed that the results from this thesis can contribute to current understanding of ICE appliance electricity consumption in the UK.

Overall, the practical constraints, inherent to domestic electricity consumption monitoring and the use of qualitative research methods, directed this study to concentrate on a smaller sample size. This smaller sample size dictates that the results gained from this thesis cannot be generalised to the wider UK population. However, the quantitative data provides the opportunity (in a field significantly deficient in real world consumption data) to examine actual ICE appliance electricity consumption in the UK and compare real world measurements against current estimates and similar research. Likewise, the qualitative data has the potential to identify underlying motivations for new patterns of domestic electricity consumption. As a result, it is believed that the findings from this research could inform future research and in due course could help to inform policies to reduce domestic ICE appliance electricity consumption.

134 5.4.3.2 Sampling approach

A range of research methods literature assert that the most valid means to gain representative data is through the use of probability sampling and in particular random sampling (Creswell and Plano Clark, 2007; Silverman, 2006; Robson, 2002; Wheater and Cook, 2000; Tashakkori and Teddlie, 1998; Henry, 1990). Methods of probability sampling allow the probability that a participant will be included in a sample to be specified and thus the potential for sample bias to be assessed (Robson, 2002).

For this thesis, random sampling was considered to be unwarranted for a number of reasons. Although the sample size restricted representativeness, the research aimed to gain a variety of household types to explore a more diverse range of patterns of use. An issue with the random selection of a small sample is that very similar participants can be randomly selected (Robson, 2002).

Of particular significance to this study, was the consideration of practical issues. The monetary value of the monitoring equipment meant that it was essential to ensure that

“trustworthy” households were recruited. Also, it became evident during early trials that the monitoring equipment might require field adjustments. Thus, it was necessary to ensure that participants would be willing to provide repeated access, particular at the early stages of the monitoring phase.

As a result a more purposive approach was sought to help incorporate a range of household types into this study, and allow practical considerations to be addressed.

Purposive sampling can be described as a sampling approach where “the researcher deliberately selects the subjects against one or more trait to give what is believed to be a representative sample” (Gray, 2004 p87). The disadvantage of this approach is that the researcher may inadvertently neglect a significant population trait or characteristic, or may be subconsciously biased during the sample selection (Gray, 2004). Despite this

disadvantage purposive sampling can be found in many energy research studies. For example, a number of studies have used samples comprised of work colleagues or acquaintances (Meier et al., 2004; Røpke et al., 2010; Wall and Crosbie, 2009; Kofod, 2008).

135

Silverman (2006) argues that purposive sampling can be a useful method to improve the generalisation of research within a small sample size. However, he states that “this does not provide a simple approval of any case we happen to choose” (Silverman, 2006 p306).

Thus, it is necessary to think “critically about the parameters of the population we are interested in” (Silverman, 2006 p306). Therefore, this thesis faced the position of

attempting to generate a relatively diverse sample whilst balancing practical considerations and the need to reduce the potential for the researcher‟s subjectivity to bias the sample selection.

The sampling strategy chosen for this thesis was snowball sampling, which is considered to be a distinct form of purposive sampling (Henry, 1990; Robson, 2002). Snowball sampling relies on the researcher to select an initial participant /participants, who in turn identify other potential participants in the population. As previously identified participants name other participants, the sample develops like a snowball (Henry, 1990). Wall and Crosbie (2009) highlight that for “exploratory research with a qualitative component, snowball sampling offers practical advantages, not least the ability to quickly recruit participants at a low cost” (Wall and Crosbie, 2009 p2). Such advantages have led other energy studies to also use snowball sampling, such as the Hungarian constituent of the REMODECE project (Kofod, 2008).

For this research, snowball sampling offered a number of practical benefits. Firstly, due to the process of participant identification being, to an extent, out of the control of the

researcher, it was possible to remove a degree of the researcher‟s subjectivity. Secondly, it reduced the time and financial constraints associated with other sample selection

methods. Thirdly, by recruiting initial participants from within the researcher‟s

acquaintances, it was also possible to minimise potential dwelling access problems.

Issues of security (e.g. the monitoring equipment, the researcher‟s wellbeing) were also mitigated due to the good faith between participants throughout the participant chain.

136 5.4.3.3 Sampling procedures

A disadvantage of snowball sampling is that a degree of homogeny can be formed within the sample, due to participants often nominating members of the population from a similar social demographic or worldview. To reduce homogeny, participants were asked to nominate potential households that had different characteristics to their own. To aid diversity within the sample, the following criteria were used as a means to guide the

selection process: (i) household type (see Table 5-4); (ii) stage of life (e.g. retired, working, etc); (iii) gender; (iv) dwelling type (where possible). Household type was the primary parameter and households were distinguished from one another with reference to basic UK Office of National Statistics (ONS) classifications shown in Table 5-4 (ONS, 2009a).

Thus, the selection process aimed to gain a spread of these main household types.

Table 5-4 Household types Household types

Married / cohabiting couple Lone parent One

Households were selected through a number of additional parameters: (i) households must possess a relatively “typical” range of appliance types (e.g. at least one television);

(ii) the household must not be a secondary residence; (iii) the household must not be exceptionally large (i.e. over 6 householders). Unlike some other energy studies (such as the REMODECE project) households were not considered ineligible against other criteria, such as households with relatively unusual appliances (household server) and homes serving as offices. This decision was taken on reflection of arguments made by Isaacs et al. (2006b) who contend that households with more extreme values of energy consumption are “real cases that cannot be dismissed” (Isaacs et al., 2006b p10). In addition, it was also believed that such subjectivity from the researcher would introduce a significant degree of bias into the sampling process that would weaken the exploratory nature of the study by removing households that may provide particularly interesting findings. Table 5-5 summarises how participants were contacted.

137

Table 5-5 Summary of participant selection process Stage Description

Stage 1. Initial households were contacted directly by the researcher from his acquaintances.

Stage 2. The households were asked to identify other potential participant households at the appliance monitoring stage.

Stage 3. Potential households forwarded their contact details to the researcher via the previous participants.

Stage 4. The researcher contacted potential households to discuss the aims of the research, details of the monitoring and interview process and the general requirements of the study (e.g. the need to unplug appliances to fit the monitoring equipment). Participants were also asked about the general occupancy of their homes and their ICE appliance ownership.

Stage 5. Potential households were given the opportunity to consider whether to participate in the study and were contacted at a later date for confirmation and to arrange a suitable time for the monitoring (to ensure that no unusual occupancy patterns were likely to occur). All participants were also informed that they could withdraw from the study at any time.