Non-probability sampling

Non-probability samples are characterised by the following:

Some population units have no chance (zero probability) of being selected.

Units which can be selected have an unknown (non-zero) probability of selection.

Sampling errors cannot be quantified.

Non-probability samples are used in the absence of a sampling

frame.1 1_{Sampling frames (lists) were}

discussed at length in Chapter 2.

We shall consider three types of non-probability sampling:

Convenience sampling — individuals are selected from the members of the population which happen to be in the vicinity, for example those people in a shopping mall at the time of the survey, say a school day. This technique is convenient, hence the name! Of course, such an approach is unlikely to yield a

representative sample. (In the mall example, children and employed adults are unlikely to be in the mall at the time.)

Judgment sampling — this uses expert opinion to judge which individuals to choose. An example would be a research firm

selecting people to participate in afocus group2_{by telephoning} 2_{Focus groups are used for}

qualitative research by asking a group of individuals about their opinions toward a new product or advertising campaign, consumer preferences, etc.

people and identifying who matches the target profile through a series of questions developed by an expert.

Quota sampling — this attempts to obtain a representative sample by specifying quota controls on certain specified characteristics, such as age, gender, social class and any other variables relevant to the investigation being undertaken. The (approximate) distribution of such characteristics in the population is required in order to replicate it in the sample.

Of these, quota sampling is the most important type which we will consider now in greater depth.

Quota sampling

Probability sampling requires a sampling frame, however one might not always be available. Another reason for conducting a quota sample instead of a random sample might be speed. We may be in a hurry and not want to spend time organising interviewers for a random sample — much quicker to set target numbers (quotas) to interview.

As with any form of sampling, the objective is to obtain a sample which is representative of the population. Quota sampling is no exception and to do so the interviewer:

seeks out units which satisfy some control characteristics (known asquota controls), for example age, gender etc., and requires the distribution of these characteristics in the

population in order to replicate it in the sample.

Quota sampling is cheap, but it may be systematically biased by the choice of interviewee made by the interviewer and their willingness to reply. For instance, interviewers might avoid choosing anyone who looks threatening, or too busy, or too strange! Quota sampling also does not allow us to measure the sampling error — a

consequence of it being a non-probability sampling technique.

Each of the three techniques above has an appeal; however, in all cases we have no real guarantee that we have achieved an

adequately representative sample (of course we might do by chance, but this would be highly unlikely). For example, were the women we interviewed only those working in the local offices? Were the young adults all students?

Basically, since we do not know the probability that an individual will be selected for the survey, the basic rules of inference which we have been learning to use (hypothesis tests and confidence

intervals) do not apply. Specifically,standard errors (a key ingredient in inferential procedures) are not measurable. However, in the absence of a sampling frame then we will have to resort to non-probability sampling methods.

That said, non-random samples are also frequently used by market research organisations or companies when speed (if not accuracy) is important. They are rarely used by governments.

Activity

You would likely use a quota sample (the main non-probability sample considered in04a Statistics 1) in the following situations:

When speed is important. Clearly an interviewer with a target to reach a certain number (quota) of people on a given day is likely to be quicker than one which requires a specific person or household to be contacted (as determined by a random

sample). Typical quota controls for the interviewer to meet are: • age

• gender

Note the more controls the interviewer is given, the longer it will take to complete the required number of interviews (and hence it will take longer to complete your study).

No available sampling frame covering the target population. If you think obtaining a list is likely to be very complicated, then a sensible targeting of the population to take a quota sample might be helpful. You might, for example, wish to contact drivers of coaches and buses over a set of routes. There are a lot of bus companies involved, and some of them will not let you have their list of employees for data protection reasons, say. One of the things you could do in these circumstances is to carry out a quota sample at different times of the day.

There are often random alternatives though, using lists you may not have thought of. In the case above, you might be able to make a list of scheduled journeys on the routes you wish to study and take a random sample of routes, interviewing the relevant driver as he or she completes their journey.

When you need to reduce cost. Clearly, time-saving is an important element in cost-saving.

When accuracy is not important. You may not need to have an answer to your question to the high — and known — level of accuracy that is possible using a random sample; rather you merely require anidea about a subject. Perhaps you only need to know if people, on the whole, like your new flavour of ice cream in order to judge whether or not there is likely to be sufficient consumer demand to justify full-scale production and distribution. In this case, asking a representative group of people (quota) would be perfectly adequate for your needs.

Although there may be several reasons to justify the use of a quota sample, you should be aware of the problem caused by the

omission of non-respondents. Because you only count the

individuals who reply (unlike random sampling where your estimate has to allow for bias through non-response), the omission of

non-respondents3_{can lead to serious errors as your results would be} 3_{Non-response is discussed later in}

this chapter.

misleading. For this reason, members of the British Market Research Association have now agreed to list non-response as it occurs in their quota samples, and this is regarded as good practice.

One possible remedy to this problem is to introduce more detailed quota controls. For example, we might ask for age, gender, employment status, marital status and/or the particular age of the respondent’s children. However this can take away a lot of the cost advantages of using a quota, rather than a random sample. Imagine the time you would take locating the last woman for your sample aged 35–44, married with teenage children and a full-time job! There is the additional expense of paying interviewers more for a smaller number of interviews (on the basis of the time they spend on the job). If this is not done, the temptation to cheat, and therefore make results completely invalid, will be strong.