• No results found

3.6 Research process 2 instrument design and testing pilot study

3.6.4 Survey questionnaire design pilot case study results and conclusion

A mail survey is very useful when seeking to gather information from a relatively larger number of respondents. The method allows researchers to obtain large amount of information from a large sample, gives respondents time to answer, allows respondents to remain anonymous and helps reduce interviewer bias (Mangione, 1995). Response rates have become one of the common yardsticks for judging successful survey research. Frohlich (2002), concluded that response rates are important for three main reasons: - when the percentage of non-response is high, there is risk that the data will be biased. Secondly, many statistical tests require a suitable number of cases and last but not least, high response rates indirectly reflect the relevance and rigor of the study. The respondents are more likely to return a questionnaire if they perceive that the study is important and warrants cooperation.

The survey questionnaire used in this research is relatively short with a total of 64 questions to be responded to. The pilot test provided information on the actual time needed to complete the questionnaire, which is between 20 and 25 minutes. As a result of the pilot study, corrective actions were undertaken, to resolve unclear, ambiguous questions and to reduce the time needed for completion of the questionnaire.

56 Non-response

Researchers are always concerned with the issue of non-response. According to Erdogan and Baker (2002), the most threatening issues for researchers conducting mail surveys is non- response. In general, the response rate depends on the motivation of the recipient to to answer the questionnaire and send it back. Reviewed literature supports use of prior notification, anonymity, and follow up mail in postal surveys as capable of raising response rates (Jobber and O‘ Reilly, 1996). Erdogan and Baker (2002) found in an experimental design of a mail survey, that the ‗original replacement condition‘ was the most effective follow-up technique. Greer et al (2000) studied the respondents‘ willingness to respond to a questionnaire in a business set up and made a conclusion from the results obtained that the content of the study is the most important factor in stimulating response participation. As such researchers are advised to pay attention to the format of the questionnaires and the length of the questionnaire. Obviously, shorter questionnaires are likely to produce better response rates than longer ones, especially because the questionnaires will be completed during company time. In general, the following measures were taken, aimed at reducing non-response:

 For the initial mailing, an introduction letter on University of Southern Queensland logo was sent out and this emphasized academic relevance of this research project.

 A summary of results was offered to the respondents, reporting on the main conclusions of the study.

 The questionnaire was pre-tested to improve readability, question order and to improve ambiguous questions.

 The format and layout of the questionnaire where aimed at reducing the number of pages

 Five weeks after the initial mail out, a replacement questionnaire was emailed to all non-respondents (follow-up mail).

 Two weeks after follow-up mailing, remaining response received an email, asking them for the third and last time to participate in the survey and a replacement questionnaire added as an attachment.

Even though the research knew very well that use of pre-notification was likely to affect the response rate, in this study the respondents were not pre-notified as there was no adequate time to do so.

Responses

After an extensive analysis of the response and non-response, presented below is the results of the survey study. Consistently, the researcher will focus on answering the research questions. In this research, the response rate is defined as the percentage of total usage questionnaires returned by the respondents (Wiseman and Billington, 1984). A total number of 120 questionnaires was sent out, 116 reactions were received of which 4 could not be used, because the questionnaire were not completed. If at least 93% of the questionnaires were completed, a response was seen as valid. This means that the total valid response of the survey is 112 to be used for analysis. The sample consisted of 28 companies resulting in a response rate of 28 / 112 = 25%. Chronologically, the following stages were undertaken as shown in table 3.5.

57

Table 3.5Chronological stages undertaken to issue and follow up on questionnaires

Week and year Activity Results

January 2008 First round of survey sent out

by email

69 valid responses received

February 2008 Second round of survey send

out by email to respondents

11 valid responses received

July 2009 First round of additional

survey sent out by mail

19 valid responses received

August 2009 Second round of survey send

out by email

13 valid responses received

September 2009 Mail box closed N = 112

Source:Developed from this study

Low response rates are typical of business mail surveys and 20 – 30% response rates are considered to be good in a business research (Erdogan and Baker, 2002). Some 20% of the respondents bear Director or Head as job title. The other procurement practitioners accounted for almost 80% of the respondents and were classified as procurement managers; sourcing managers; supply managers; procurement team leaders, senior buyers and small groups of others (Table 3.6)

Table 3.6 Profiles of respondents

Job title Frequency Valid percentage

Director / Head / CPO 28 25%

Purchasing Manager 20 21%

Supply / sourcing manager 8 4%

Procurement team leaders 28 25%

Senior buyers 28 25%

Total N = 112 100

Source: Developed for this study

All questions are answered by managers with procurement responsibilities in either buyer or supplier firms. Assessing respondents‘ perceptions is the most meaningful way to measure variables, since decisions will be made on managers‘ perceptions of relevant conditions (McCutcheon and Stuart, 2000). This is the main argument for measuring perceptions. Essentially all decisions in business are made on the basis of interpretations and perceptions of conditions and circumstances. According to Aldrich and Mindlin (1978), studies focusing on environmental uncertainty, as perceived by decision makers, have generally made use of subjective indicators of the environment.

The pilot study conducted on the survey questionnaires found that some of the questions were repetitive as there was very close meaning of other attributes of trust that were measures. Typical example was the closer meaning of ‗openness‘ and ‗honesty‘. Questions 2 and 3 asking about openness and honesty attributes of trust were changed several times as they were construed as easy to think that they were asking for similar responses. The decision taken was to specifically include the factor being measured in each and every question to avoid any confusions which may have led to wrong responses being provided hence affecting the research outcome. By rephrasing all the questions asked and the rearranging the sub questions in a systematic way made it much easier for the respondents to complete the questionnaire as well as for the researcher to input the responses. As such, both the

58

respondents and the researcher saved significant time as a result of the changes made to the survey questionnaire.

Initially, the question required the respondent to answer questions about perceived importance first and last about perceived performance. As such the instrument covered four A4 sheet of paper which was a significant cost particularly where these were to be printed and send out as printed copies. This layout made the questionnaire unnecessarily too long, hence attract high risk of low response rate as respondents showed less interest in rather too long questionnaires. This approach resulted in all the questionnaire being fitted on two A4 sheet of paper compared to initial 4 x A4 sheet size (refer. Appendix D).

Table 3. 7 Revised layout of the survey questionnaire

On a scale of 1-7, evaluate each of the four categories of staff in your firm extent to which they perceive performance and importance of: PERCEIVED PERFORMANCE PERCEIVED IMPORTANCE Ve ry low Moder ate High Ve ry high Ve ry low M ode ra te High Ve ry high 1 2 3 4 5 6 7 1 2 3 4 5 6 7

Commitment to each other by stakeholders during procurement arrangements as being critical to business success.

Source:Extract of the used survey questionnaire used in this study

Further, the researcher found out that the interviewees would potential half the time they would usually have taken to complete the new questionnaire as the instrument required the interviewee to fills one set of questionnaires to rate accordingly on a Likert scale of 1 -7 by marking ‗x‘ or for both perceptions as shown in table 3.7. The new method will encourage the respondent to quickly complete the questionnaire in less time. Furthermore, the result may be more accurate as the interviewees have a better perspective as they fill the perceived performance followed by performance. The researcher will also benefit from the new layout as he only needs to collect and record the research data faster.