69. The analysis in this report was conducted using information from questions placed in the October/November 2003 survey of small and medium sized businesses. That survey is a panel of 1800 small and medium businesses. It provides the most comprehensive coverage of small and medium sized businesses in Australia that is available from a regular survey.
70. The population of interest for this survey comprises private businesses with between one and two hundred full-time em- ployees in all industries other than the primary sector. Be- cause this is a panel the sample frame has two parts. The Þrst part comprises businesses that are members of the panel. The second part of the sample frame comprises the Desktop Mar- keting Systems database of telephone numbers of Australian businesses. Potential recruits to the panel are selected via telephone numbers chosen randomly from this database. Po- tential recruits are admitted into the panel if they a) agree to participate; and b) come from the same state and industry, and are the same size, as the Þrm that they are to replace in the panel. This feature of the selection of respondents ensures that, over time, the panel remains representative of small and medium sized Australian businesses.
Response rates
71. The response rate for this survey was between 20 and 22 per cent. It is important to be precise about what this response
rate does and does not mean. Most importantly a low re-
sponse rate does not necessarily mean that the estimates of population quantities obtained using design based estimators (DBEs) of population means or sums are biased.21 What it
does mean is that one needs to mount additional arguments in order to convince ones most trenchant critic that the design based estimators are unbiased for this survey.
72. The discussion below provides the additional arguments that we feel make a compelling case that the design based esti- mators used in this report are not ‘signiÞcantly biased’ when applied to the data obtained from this survey.22 We then pro-
21Design based estimators depend for their validity on the survey being
designed and administered so that the probability of selection into the sample is known.
22By ‘not signiÞcantly biased’ we mean that any bias is too small to
vide a discussion of the model based estimators that are also used in this report and which are unaffected by low response rates.23
73. Before turning to those issues it is also important to emphasise that the response rate of between 20 and 22 per cent does not mean that the data collected by the survey is of low quality. Indeed, the opposite is the case as in part the low response rate arises because major efforts were made to select businesses that match those which leave the panel. This requires that more businesses be contacted, and thus more refusals encountered, than would be the case with a one-off survey.
Reasons why the design based estimators are unlikely to be signiÞcantly biased
74. There are three main reasons to support the contention that the low response rates do not create a signiÞcant bias in design based estimators for this survey.
75. First, the decision not to respond was made before the business was told (at the preamble to question 11) that the survey contained some questions about Safety Net issues.24
76. Second, the non response was attributable to factors that are best described as random and would not, therefore, be ex- pected to over (or under) select respondents that,
• have minimum award wage rate employees;
• have passed on the 2003 Safety Net adjustment to over award employees;
• have reduced employment because of the 2003 Safety Net adjustment;
• would increase employment in response to a guarantee of no change to the Safety Net for a period of Þve years. 77. Third, the weights used in this survey are constructed via poststratiÞcation. They therefore are, by virtue of their con- struction, robust to missing at random non response; see Lohr (1999, p. 268).
78. Thus, we do not consider it likely that the non response causes any signiÞcant bias in the estimates we make in this report.
23Chapter 7 provides a more detailed discussion of the statistical issues
and deÞnes terms such as ‘design based estimator’ and ‘model based estimator’.
24All businesses that reached the preamble to question 11 completed the
79. The only additional evidence that could be provided here is from the analysis of a survey of non respondents. Such analysis would involve testing hypotheses that the non responses are, using the terminology in Lohr (1999, p. 265), either ‘missing completely at random’ or ‘missing at random’.25 If the Þrst
of these hypotheses cannot be rejected then no adjustments are required to produce unbiased design based estimators. If the Þrst hypothesis can be rejected but the second cannot be rejected, then all that is required to reduce the bias is the reweighting, via poststratiÞcation, that we have already done. Thus, the only way that such a survey of non respondents could modify the results obtained using design based estima- tors would be if it was found that, after controlling for the characteristics of businesses such as size, location, industry etc, the probability of non response was related to the busi- nesses answers to the Safety Net questions. Such a Þnding would not affect the results obtained using model based esti- mators which are discussed below.
Model based estimators
80. Model based estimators provide an alternative to design based estimators and have Þve main advantages. First, they are un- affected by non response. Second, they allow the investigator to provide some indication of the likely extent of any bias as- sociated with design based estimators. Third, they provide estimators that are more efficient than DBEs in the sense that they produce smaller standard deviations and tighter conÞ- dence intervals. Fourth, they bring to light important rela- tionships in the data. Fifth, they can provide a better basis for testing hypotheses than do design based estimators. 81. We have discussed the model based estimators that we believe
can be applied to this data set but have not yet applied those estimators to the dataset.
Summary
82. We have sought to be explicit about the statistical issues that arise in this work and we have sought to adhere to the high- est standards for the execution and presentation of empirical work. Our assessment is that we have dealt with the non
25A non response is said to be missing completely at random if the prob-
ability of non response is unrelated to both respondent characteristics and the responses to the questions of interest (ie the Safety Net ques- tions). A non response is said to be missing at random if, after con- troling for respondent characteristics, the probability of non response is unrelated to the responses to questions of interest. There are standard econometric procedures for testing these hypotheses.
response issue using the best statistical and econometric prac- tices available.
83. Of course, given the richness of the dataset obtained from this survey, and the importance of the issues under investigation, there is always more that could be done. But, our judgement is that the features that might be discovered in a more extensive model based estimation approach would be in the Þne detail rather than requiring major modiÞcation of the conclusions drawn in this report.
84. We have observed that further assurances about the validity of design based estimators could be obtained for this dataset by taking a random sample of non-respondents and analysing it to establish beyond doubt the hypotheses advanced above that the non response was random rather than systematic. We note that these assurances will not be required for those conclusions of the report that are subsequently conÞrmed us- ing model based estimators as the latter are robust to non response.