Method and Sampling 99

In meeting this research project's objectives it was decided that the research should be undertaken within a set market place rather than the country as a whole. In addition to budget limitations this decision reflected the fact that, in terms of the New Zealand radio industry, the country is de facto a collection of about 24 individual marketplaces. The number of stations broadcasting in each market place varies from five in the smaller markets to in excess of 30 stations in the largest market place. The selected marketplace was the author’s home district of the Manawatu, a regional marketplace with 20 radio stations regularly broadcasting on a 24 hour-seven-day-a week basis. The Manawatu was also thought to provide an adequate representation of the New Zealand adult population as measured by, by age, gender, ethnicity, household income and an urban/rural balance.

An initial sample of 6,000 randomly selected Manawatu residential telephone numbers was supplied by the local telecommunications company. All residents of the survey area aged 15 years and over with a telephone were potential survey respondents. People without telephones or resident in institutions, including rest homes and student hostels were excluded. However, retirement villages where residents had separate dwellings with individual phones were included. The sample telephone numbers were randomly divided into four groups – one for each week of the research period.

A team of six people comprising friends and family members of the researcher as well as local students was assembled to phone the potential respondents. Training was provided and the instruction sheets utilised for this preparation are attached as Appendix C.

The major study was undertaken over a four week period from Monday 17th October to Sunday 13th November 2005.

Phoning of potential respondents was carried out each week on Monday evenings (only), and on Tuesdays and Wednesdays during both the day and the evening. Up to three call backs were made to each unanswered number. If the phone was answered the interviewer introduced themselves, explained the purpose of the call, and asked to speak to the person in the household who was 15 years of age or older who had the next birthday.

This was designed to ensure the sample was as representative of the population as possible within the constraints provided by the requirement of telephone availability. It the required person was not available a call back time was arranged. If they were available they were then asked if they would be prepared to complete a one-week- long diary of their radio listening.

If the respondent agreed they were posted a radio diary package on either a Thursday or Friday in order to enable them to start recording their listening from the following Monday. Only one respondent was chosen from each household.

The radio diary package contained the following items:

• A seven-day radio listening diary, with instruction and information sheets

• A thank you letter designed to encourage completion of the diary

• A small bar of chocolate as a thank you gift

• A reply paid envelope enabling the respondent to return the diary upon completion.

As previously mentioned in Section 5.2, the radio diary’s format allowed each respondent to record the essential details of each radio listening occasion with the diary format breaking down each day into quarter hour time periods. Respondents were asked to indicate their radio listening in any given quarter hour where they had listened for eight minutes or more of that quarter hour. The definition of listening used was an important part of the survey design. ‘Listening’ was defined as ‘respondents being able to hear the spoken announcements being broadcast and so identify the station broadcasting’ – the same definition used by Research International, the “official” New Zealand radio research company.

In the front of the diary the respondents were asked to record:

• The number of radios in the household

• Where they mostly listened to the radio during certain times

• Radio stations of which they were aware, categorized by those they sometimes listened to and mostly listened to, and

In the diary itself:

• Whether they listened to the radio that day, as well as

• Their listening occasions – in 15 minute segments.

In the back of the diary the respondents were asked to provide their age, gender, ethnicity, number of people in the household, employment status, formal education and educational qualifications, as well as both their personal income and household income. At the back of the diary space was also provided for the respondents to write in comments about topics raised in the questionnaire and whether they would be prepared to take part in any follow up research. The return rates achieved over the next four weeks are shown as Table 13 which follows.

Table 13: Return Rates

Week One % Week Two % Week Three % Week Four % Total %

End of the first week 57 57 60 60 58 End of the second week 85 79 81 79 81

As shown by this table, the majority of respondents returned their diaries during the week following the week in which they recorded their listening. Those respondents who had not returned their diary by the Thursday of the week following their recording were followed up. The follow up was by either telephone or by mail. Respondents were randomly selected for either of the treatments. There was no significant difference in the response rate between those who were phoned and those who received a letter. However, the follow ups did increase the response. Any

diaries that were returned later than 10 days after the finish of the recording period were not included in subsequent analysis.

Table 14 shows a more macro view. Over the four weeks of the main study an attempt was made to contact a total of 4,980 respondents. Of those 13 respondents were ineligible (usually house sitters and unable to complete a full week’s radio recording) and a further 938 could not be contacted, in spite of up to three call backs being made. The final number of respondents contacted was 4,029, of which 1,399 agreed to complete a radio diary. Of the 1,399 respondents who agreed to complete the radio diary a total of 1,136 diaries were returned.

Table 14: Sample Sizes and Responses

Week One Week Two Week Three Week Four Total

Initial Sample 1315 1322 1345 998 4980 GNA/Ineligible 2 4 2 5 13 Adjusted Sample 1313 1318 1343 993 4967 Non-contact 233 295 264 146 938 Adjusted Sample 1080 1023 1079 847 4029 Sent surveys 360 371 382 286 1399 Returned Surveys 305 293 311 227 1136

Upon return of the diaries they were scrutinised for what could be assumed as “impossible” listening patterns - that is listening for more than 20 hours per day or multiple listening to stations within one quarter hour period. Only seven diaries were regarded as having “impossible” listening patterns and were discarded. The final number of useable diaries was 1,129.

Response Rates

The response rates for each of the weeks and an overall total are reported in Table 15. As with the Pilot Study, three different calculations for the response rates are considered, although each week is comparable. Response Rate 1 is based on attempted contact, and used the initial sample size less identified ineligible or non-

contactable respondents gone-no address (gna). Response Rate 2 is based on actual contacts--that is, initial sample less ineligibles, gna’s, and non-contacts. The Participation Rate is the proportion of contacted respondents who agreed to participate, and the Return Rate is simply the proportion of delivered diaries that were returned completed.

Table 15: Response Rates

Week One (%) Week Two (%) Week Three (%) Week Four (%) Total (%) N 1315 1322 1345 998 4980 Participation Rate 33 36 35 34 35 Return Rate 85 79 81 79 81 Response Rate 1 23 22 23 23 23 Response Rate 2 28 29 29 27 28

As can be seen from Table 15, each week is very similar with an overall participation rate of 35% and a response rate of 28%. However, as mentioned, of the diaries that were returned seven diaries were regarded as being unusable and were subtracted from the 1,136 received. This left a final sample of 1129 respondents giving an effective return rate of 80.7%.

A random sample of 1129 radio listeners has a maximum margin for error at the 95% confidence level of plus or minus 2.9%. That is a researcher can be confident that any result using the whole 1129 respondents (and not a sub sample) is within 2.9% of the true result excluding consideration of non-sampling error.

Sample

As already introduced, the only requirement for testing the applicability of the Dirichlet’s potential as an empirical generalisation is data on the listening patterns of a “reasonable” population. Thus, the research targeted a sample size of between 1,000 and 1,200 respondents that was thought reasonable for the population considered.

As such this sample is not required to be representative of the population as a whole. However how representative this sample might be was tested against the population of the entire Manawatu as shown in Table 16 below.

Table 16: Sample and Population Comparisons

Population 15 years and over

Sample Population Manawatu Population

(1129) (77,500) % % Gender Male 39 47 Female 61 53 Age 15 to 29 years 15 29 30 to 49 years 38 36 50 to 74 years 39 28

75 years and over 8 7

The sample population differs from the Manawatu population only in that there are proportionally fewer males in the former and in the sample population the age group 50 to 74 years is over represented at the expense of the 15 to 29 year age group. These discrepancies were not thought avoidable as they reflect the known fact that women and older people are more likely to respond to surveys compared to younger people and males. However, as noted earlier, how representative a sample might be compared to a survey’s population as a whole is not a major issue for Dirichlet research.

Non-response Bias

Some word should also be said about non-response bias. Most researchers are worried about the decline in survey response rates. Researchers such as Baim (1991), Meier (1991) and Brown (1994) have suggested that the declining rate is as a consequence of changing lifestyles. This includes increasing numbers of women in the work force, continued increases in urbanisation and even reduced leisure hours. Other reasons that have been mooted include unethical business practices such as

“sugging” and “frugging” (selling or fundraising under the guise of “research”) and the huge increase in direct marketing, especially telemarketing. Whatever the reasons, survey response rates are declining with a “concomitant potential for non- response bias to increase… the views of a considerable number in the original sample who refused to take part… are not incorporated in the results” (Hosie 1995, p1). Non response bias in this case is both a refusal to participate in the research by declining the request to complete a radio diary as well as not returning a survey despite previously having agreed to complete one.

Methods of addressing non response bias have been covered in the discussion on the Pilot Study. As mentioned, many of those techniques are not applicable in terms of this research. For instance it was not possible to send out repeat copies of the radio diary to be completed within a given week. However, phoning respondents during the week they were to complete their diary asking if there were any problems could have helped the return rate and even the accuracy of their recorded listening. Unfortunately, a lack of resources made this impossible.

However, on average 60% of all diaries were returned during the first half of the week following the recording of the respondent’s listening. Those respondents who had not returned their diary by mid way through the following week were contacted by either telephone or mail and asked, if the diary had been completed to send it back. If the diary had not been completed the respondent was asked to retain it and not send it back. This follow up resulted in an overall return rate of 81% - a rather pleasing result, which reduced the potential for non response bias.

Item Order Effects

The position of the radio station in the awareness questions, like all items in a list of attitudinal or behavioural statements, can be problematic in that it is acknowledged by researchers that respondents may pay more attention to items in various positions (for example near the top of the list). There is also the issue that respondents may become ‘fatigued’ by any long lists and make decisions on the items near the bottom of the list without the thought being given to the question that researchers assume has been given to earlier items. For details on research in this area see Payne (1951), Schuman

and Presser (1981), Belsen (1981), Sudman and Bradman (1982), Brennan (1995) and Wright and Lees (2003). To minimise item order effects the position of stations in the ‘awareness’ questions were reversed resulting in two versions of the radio diary. The ‘correct’ item order was re-established at the data entry stage by adjusting the data entry programme accordingly.

In terms of listing the stations on the diary pages the order across the top of the page, remained the same for all surveys. It was considered that any item order effect introduced by this minimal sequencing would be minimal at best.

Other Sources of Error

Potential sources of error are acknowledged, and those that pertain to non response have been discussed above. Clearly any research that achieves less than 100% response will produce estimates potentially affected by non response error. Its impact on this research’s results is mostly unknown but the effect of non response cannot be dismissed entirely.

Similarly, respondent error is unavoidable; researchers rely on respondents for truthful and accurate information. Research topics like radio listening may or may not be susceptible to respondent error. The possibility of social desirability bias (where respondents record a station as being listened to as they feel it is more appropriate than other stations) is also acknowledged, but its effects on the results can not be ascertained.

6.

MARKET REGULARITIES, THE DIRICHLET