CHAPTER 4: RESEARCH METHODOLOGY
4.6 Data Collection Technique
Researchers seek effective, low cost means of collecting high quality data. This quality of survey is achievable when researchers can procure techniques that can increase mail survey response rates, reduce item omission, and speed up responses. Fundamentally, three critical issues that affect the overall quality of survey have been investigated: response rate, response completeness, and response speed (Cavusgil and Elvery-Kirk, 1998; Zikmund et al., 2010).
Response rate of a survey is defined as the total number of completed questionnaires returned, divided by the net mailing sent, where the net mailing is calculated by deducting the undelivered questionnaires returned by the post office, from total mailing. Response completeness is measured as the proportion of the unanswered questions on a returned questionnaire, whereas response speed is determined by the number of days between the date of mailing the questionnaire to the respondent and the date of its return to the researchers (Cavusgil and Elvery-Kirk, 1998; Zikmund et al., 2010).
A prevailing assumption in survey research is that a high quality survey can provide a useful and accurate basis to help researchers with further data analysis (Bright and Smith, 2002). However, given the budget and time constraints for the doctoral research, it is essential to consider whether the benefits of marginal increases in the
quality of survey outweigh the costs incurred. Therefore, this study adopts a number of cost-effective techniques in order to obtain surveys of quality as high as possible, at a reasonable survey cost. The techniques designed to improve survey quality are demonstrated below.
(1) Pre-Notification Contact
The purpose of pre-notification contact is to gain commitment from target respondents prior to the delivery of the actual questionnaires. Some researchers believe that recipients are more willing to complete the questionnaire if they receive advance notification by either mail or telephone (Cavusgil and Elvery-Kirk, 1998).
However, some empirical evidence illustrated that a preliminary contact has little or no effect on return rates (Fox et al., 1998). Greer et al. (2000) noted that pre-contact may be an appropriate technique only for consumer or household survey. In industrial survey, business people seem to work under more rigid time constraints. More importantly, the cost and effort to mail a pre-notification letter or to call each firm are exceptionally high. Dennis (2003) noted that pre-notification contact is worth implementing only when the sample size of business organisations is small.
It can be seen that the effectiveness of pre-notification contact in improving response rates appears inconsistent in the literature. Malhotra and Birks (2007) pointed out that the follow-up technique may be a better investment than the pre-notification technique to accelerate the rate of response. Following this suggestion, this study omits the pre-notification technique.
(2) Colour of Questionnaire
It is typically believed that colour can attract recipient‟s attention to complete the questionnaire. However, previous studies have not found an increase in response rate with the use of colour in questionnaire design (Dennis, 2003; Newby et al., 2003), except the study of LaGarce and Kuhn (1995), who found colour to be a significant value for increasing mail survey response rates. Since the colour questionnaires are costly, this study uses a black and white questionnaire instead.
(3) Cover Letter
Cover letter proves to be one of the few direct opportunities for influencing respondents and motivating recipients to reply (Zikmund et al., 2010). Fox et al. (1988) noted that cover letter used for the original mailing is probably the most important single factor influencing a high rate of response. Thus, care was taken to compose the cover letter for this research survey. The cover letter encompasses a warm, recipient-friendly, and appreciative language. It begins with clear research objectives and potential benefits of collecting the data. It also explains the importance of completing the questionnaire by emphasising the limited sample size and selectivity of the sample. Moreover, the cover letter points out that all data will be held confidential. The respondent‟s identity will not be related to their answers and the data will be presented in aggregate form only. Highlighting the direct benefits of the response to the participants is also included. At the end of the cover letter, the researcher‟s name, his supervisors‟ name and title, and contact details are provided as well, as suggested by Dillman (2000).
(4) Monetary Incentive
Monetary incentive technique has long been implemented in mail surveys to enhance the rate of response. A considerable number of researchers, based on a meta-analysis research by Fox et al. (1988), asserted that a monetary incentive, even a small amount of money enclosed with the questionnaire, increases the response rate. The possible reason is that the enclosed money establishes a trust necessary for the social exchange between sender and recipient (Dillman, 1987). Cavusgil and Elvey-Kirk (1998) also agreed that once the respondents receive monetary incentive, they feel obligated to reciprocate by returning a completed questionnaire. More recently, Jobber et al. (2004) confirmed that the response rate increases as the value of the prepaid monetary incentive increases, in both industrial and consumer populations.
However, Yammarino et al. (1991) argued that the positive effect of monetary incentive on response rate is not always valid in all contexts. James and Bolstein (1990) supported this view by indicating that monetary incentive may be meaningless to serve as a motivating factor. The probable reason is that a single dollar incentive is not large enough to trade with their efforts for questionnaire completion, particularly industrial surveys. Based on this perspective, this study does not employ this technique.
(5) Non-Monetary Incentive
For industrial mail survey, the gatekeeper (usually secretary) may be under instructions to filter out items that may be considered as junk mail. Therefore, the inclusion of a non-monetary incentive may differentiate a mail questionnaire from others. According to Yu and Cooper (1983), non-monetary incentives, such as pens, pencils, books, and summary of survey results, are most commonly used in past studies. Interestingly, Jobber et al. (1991) found that a non-monetary incentive (e.g., bookmark) had a positive effect on response rate as well, whereas the promise of a summary of the study results did not. Kalafatis and Tsoga (1994) also concluded that a summary of results did not significantly enhance the effectiveness of mail responses. Thus, this study does not use non-monetary incentives.
(6) Prepaid Stamp
The inclusion of a stamped, pre-addressed return envelope seems to be a widely accepted practice in mail surveys, accompanying the cover letter and the questionnaire. Certain studies have investigated the relative effectiveness of stamped versus business return envelopes, on response rates. The majority of studies showed a higher rate of return achieved through the stamped reply envelope technique (e.g., Harris and Guffey, 1978). Two possible explanations may justify these results. First, the usage of a stamp represents money, while a business reply envelope does not (Erdos, 1970), which may raise the perceived importance of replying, in the recipient‟s perspective. Psychologically, the recipients may find it difficult to throw away an unused stamp because of its monetary value. Second, the form of return postage prepaid by the researchers can improve response rate as a result of the
convenience provided to the recipient in returning the completed questionnaire. In contrast, if return postage is not offered with the mailed survey, the potential recipients tend to disregard the questionnaire because they are then required to put in the effort and the money to procure a postage stamp (Cavusgil and Elvey-Kirk, 1998). Consequently, it was worthy providing a stamped reply envelope in each questionnaire pack for this study, although it ensued a higher postage cost.
(7) Day of the Week
The day of the week can be considered as a situational factor regarding when the target respondents receive the questionnaires. Greer et al. (2000) categorised two periods of the week, early week (Monday to Wednesday) and late week (Thursday and Friday). Dennis (2003) suggested that higher response rates could be easily achieved at no extra cost by only adjusting the mail schedule. If the questionnaires arrive on Friday, the potential recipients may be less willing to fill out the questionnaires since the weekend is approaching, and other work may hold higher priority. Alternatively, if the questionnaires reach on Monday with the heaviest mail volume received, the recipients may be less likely to notice and to respond. Another possibility is that if the questionnaires arrive between Tuesday and Thursday, when the mail volume is lighter, it is more likely for respondents to participate in the survey. Therefore, the mailing was undertaken on Monday in order to allow the questionnaires to arrive by Tuesday or Wednesday.
(8) Follow-up
Follow-up has been widely used with great success (Jobber and O‟Reilly, 1996; Dillman, 2000). Jobber and O‟Reilly (1996) noted that respondents may not respond to a survey initially, but prefer to do so when they receive a reminder. This may be because the recipients realise the importance of the survey from the researchers‟ effort in spending the time and the money to conduct a follow-up.
There are four types of follow-up techniques: (1) postcard follow-up, (2) second mailing follow-up, (3) telephone follow-up, and (4) multiple follow-up, known as Dillman‟s (1987) technique, which combines all three together. There is a trade-off between cost incurred and response rate achieved. Although it is more time- consuming and costly, Dillman‟s follow-up technique is expected to gain better returns than using single mailing strategy (Dillman, 1987; 2000). Thus, this study uses two follow-up techniques, a postcard reminder and a second mailing of the questionnaire.