Sample Design

The sampling design for the interviews was developed through discussions with a number of individuals from Statistics Canada. The small size of the sample, the dispersed farm population and the lack of precise knowledge of the farm population in Canada were significant constraints. Attempting this multi-strategy methodology within a limited budget meant that the sample size for the quantitative portion of the study was limited to 618 respondents.

The respondents were distributed across Canada in proportion to the number of farms in each province according to the 1996 Census of Agriculture.⁶ The sample was stratified on the basis of gender and farm type. The small size of the sample was a barrier to further stratification and other variables were drawn out during the analysis. Half of the interviews included a woman, a man and a youth from the same family in order to allow me to explore how farm families handle farm and non-farm work as a household unit.

Control over the selection of respondents in the study was exercised through a two-step process:

1. A stratified random sample of 100 Canadian census divisions was drawn to select the census divisions in which the interviews are being conducted.

2. Respondent profiles were developed for each census division based on the dominant types of farming in each sampled census division. The respondent profiles determined the desired characteristics of each respondent in each census division.

The first step of the sampling procedure was to draw a stratified random sample of the census divisions in Canada in which a significant amount of agricultural activity was present.⁷ This sampling strategy ensured that the sample would represent the overall

6 It was necessary to slightly over-represent the Atlantic provinces in order to have a useable number of responses from that part of Canada. This is accommodated by under-representing the provinces with the largest number of farm families, Alberta, Saskatchewan and Ontario.

7 Canadian census divisions are determined by total population size and are not representative of

agricultural activity. As a result some census divisions included very few farms. In order to include those census divisions where there were large numbers of farms, the census divisions in each province were ranked in order of the number of farms in each. Those census divisions which contained 90% of the farms in the province were selected to form the basis of the random sample. The ranked 90% data was further broken down into groups containing 80% and 20% of the farms in eachprovince. Using random number

population of family farmers, each province would be represented in the sample and the selection of census divisions was not biased.

The second step in the sampling procedure was a non-proportional quota sample based on gender and farm type within the selected census divisions. Quota sampling was used for this step in the sample design because an inventory of the population does not exist for Canadian farm families, making it impossible to undertake a probability or random sample of the population. In addition, the research design involved interviewing the same respondents four times through the course of one year which would be impossible to achieve through traditional probability sampling techniques.

Quota sampling is criticized by statisticians because the selection of respondents is non-random and is left up to the interviewer, leaving the survey results open to potential bias.⁸ To minimize bias in this study, a stratified random sample of census divisions was taken and the interviewers were required to find their respondents in a very

tables, a stratified random sample was drawn in which 80% of the interviews were allocated to that proportion of the 90% sample which included 80% of the farms and 20% of the interviews were allocated to that proportion which included 20% of the farms in each province.

8 It is important to recognize that the ability of quota sampling methodologies to represent populations accurately is a subject of ongoing debate. Quota sampling is used extensively for market research and for polling. Recent studies have shown that quota sampling has compared favorably with random sampling in its ability to represent the population and to give results comparable to results generated with random sampling. A Swiss study which compared the two methodologies showed that differences between quota sampling and random sampling were negligible. It is also suggested that the decline in response rates for random samples introduces bias into the research as the probability of responding to a survey is not randomly distributed within a population (Schobi and Joye, 2001). Robert Worcester (1996) notes that British election polls conducted by random sampling were less accurate than those conducted by quota sampling. The problem appears to lie partially with poor compilation of registers, a problem common to many research projects in which there is no way to fully identify the population. Smith (1996) also notes that there is no evidence from previous elections, when both quota and probability sampling are used, that one method gave better results than the other. Deville (1991) states that for a small sample, the bias of a quota sample will be more tolerable than the lack of precision of a probabilistic survey. Smith (1983:402) outlines the conditions under which it is acceptable to draw inferences from quota samples, although he is cautious about this approach stating that “if wide acceptability is required, random sampling provides the most immediately acceptable sampling method”. The discussion over the ability of the results of a well selected quota sample to give comparable results to a random sample supports the use of quota sampling in situations like those of this study, in which the research design and economic factors make a

probabilistic sample impossible. As the methodology is non-probabilistic, calculations of sampling error are not possible. A non-probability sample limits our ability to generalize the findings of the study to the general population as we do not know if the sample is representative of the population.

structured manner. Although this strategy will never eliminate all bias, it removes a considerable amount of discretion from the interviewer in their choice of respondents.

Ideally, the sample would have included 206 women, 206 men and 206 youth, involved in five major types of agriculture in Canada. Half (309) of these respondents were to be members of families and half (309 respondents) were to be individuals. Respondent profiles were constructed based on the dominant types of agriculture in each census division. The farm types reported in the 1996 census were collapsed into 5 farm types based on the nature of work in each type. These categories are dairy; hogs and poultry;

cattle and livestock; wheat, oilseeds and field crops; and vegetables and fruits. The sample of 618 was allocated among farm types based on the proportion of each type of farming in the census division, with a slight over-sampling of the less common farm types. This resulted in 73 dairy farms, 50 hog and poultry farms, 199 cattle and livestock farms, 246 wheat, oilseeds and field crops farms, and 50 vegetable and fruit farms.

The interviewers assigned to each of the randomly selected census divisions selected two women, two men and two youth to be interviewed based on the respondent profiles.

One woman, one man and one youth were required to be members of one family. All of the respondents lived in a traditional nuclear family and all youth were aged 12 to 18 years at the beginning of the survey. A final requirement was that the respondents found were to come from farms in which the dominant activity was specified.⁹

Of necessity, the sampling frame was based on the 1996 census. However, the period 1996 to 2001 was a time of rapid change in some agricultural sectors and the most rapid decline in the number of farms since the 1980’s. As a result, some census divisions near urban areas lost considerable agricultural land to suburbanization. Census divisions with significant numbers of hog farmers in 1996 had lost many of those small operations

9 For example the interviewer in census division 10 in Alberta was instructed to find a family whose predominant agricultural activity was wheat, oilseeds and field crops, an individual farming dairy, an individual farming hogs or poultry and an individual farming cattle or livestock.

to large scale hog production facilities. In both situations, respondents were harder to find.