Appendix – Missing Responses & Applied Cleaning

The UK Community Innovation Survey is voluntary. One could thus expect firms with more limited resources and a lack of interest in innovation to be less likely to fill out the survey. To check if there is a bias in not responding to the survey, the ONS carried out the questionnaire over phone for a random sample of those that the CIS 2 survey was sent to but that did not reply and apparently found there to be no significant response differences (Tether, 2001) which suggests that no bias can arise due to firms not responding to the survey. Upon request the ONS has indicated that also for the more recent survey rounds telephone follow ups are conducted to obtain answers to missing questions and to affect enterprises to respond to the survey. It is not clear why unlike in many other countries the CIS is not simply made compulsory.

The extent to which enterprises have filed out the questionnaire varies considerably. If analysis is carried out without some firms due to missing observations for specific questions in the survey, weights would need readjustment so that observations are representative again. Since not providing an answer occurs across the different questions sets this would require re-weighting at each stage, for instance when generating a tabulation for a certain question and then latter doing a regression analysis based on several questions one would have to use different weights. As this is rather time consuming and as non-response to questions seems random the analysis simply sticks to the original95 weighting. The mean value for each variable with adjusted weights where responses were missing was visually inspected compared to the mean obtained without adjusted weight and they were very similar thus it is assumed that non-response to a certain questions is random96 and hence does not introduce a bias. Nevertheless the issue should not be put aside that lightly and is closely related to the discussion about measurement errors.

As noted the extent to which firms responded to the survey and the efforts in filling out the survey have differed. Consequently statistics may be biased. Some examples of efforts by firms in filling out the survey (or potentially problems with scanning of the answers by the ONS) are now provided. Firms were asked who their main customer is and were supposed to only choose one option. In the CIS4 about 300 firms chose at least 2 options. There are also a number of firms97 _{that indicated}

to have spent on the “acquisition of machinery, equipment and software” as well as on “marketing expenditures” in the last year of the survey which did not report to have carried out any such activities in the previous question set which asks whether firms have undertaken this sort of activities over the whole survey period. Also a few firms answering the question on which markets they sold to only indicated that they have not sold to a certain market while leaving the other two options blank. Lastly there is as noted previously considerable confusion about question q1110 in the CIS 5 which is about “whether firms did undertake any product or process innovation during the survey period”. Subsequently respondents are asked if they responded negatively to the above question to skip the questions about effects of innovations. Nevertheless around 900 that responded negatively and around 100 that did not respond at all to the qualifier question ignored the instructions and responded to this question set.

Those enterprises that have not filled out the relevant information needed for a certain analysis have simply been left out for these. A hierarchical approach in deletion of observations for the analysis has been adopted. This means at each stage the most information possible from the dataset is used. However if other information allows to conclude what the relevant answer is the variables have been recoded. Let’s now look at the specific cleaning procedure used for each CIS question in turn. For details of the questions discussed here please refer to the end of this section where the different survey forms are inserted.

The question regarding whether product innovations were new to the firm or the market (0710, 0720) if missing were re-coded in the affirmative if a positive innovative sales intensity (0810, 0820) had been reported. Also if respondents

97

answered yes to any of the questions relating to who developed their product innovations (0610, 0620, 0630) or the previous questions on whether product innovations were new to the market or new to the firm but had not indicated that they carried out any product innovation (both good or service) the created variable reflecting product innovation was recoded to reflect that they had actually carried out product innovation98_{. Similarly the question on process innovation (0900) if}

missing but the respondent had reported where their process innovation was developed (1010,1020,1030) or that new to the industry process innovations had been introduced (1100) was recoded accordingly.

Some cleaning seems to have been done by the ONS beforehand. Questions relating to innovative outputs (that is whether firms had goods or service innovations and consequently product innovations as well process innovations) had no missing values. If answers have simply been recoded to zero if missing this is quite a stark assumption, however this seems the only plausible explanation as to the approach followed alternatively the ONS may have discarded all surveys where no response to these three crucial questions was obtained – any of these approaches were denied by the ONS upon request. Firms not indicating whether they had carried out innovative activities does not necessarily imply that they had not carried out any, they may simply not want to disclose this information.

For those firms that reported any amount spent on innovative activities in the last year of the survey period (question 14) the respective answerers relating to whether such activities were undertaken during the survey period (question 13) were recoded to reflect that they had undertaken the activity if necessary. This had to be done for the newly constructed composite measure of ‘Acquisition of machinery, equipment and software’ and ‘Market introduction of innovations’99 for the CIS 5 and the CIS 6100. This suggests that firms could not really identify what exactly it was they were spending on in terms of the newly offered sub-options.

98

Though note that from this one could not infer whether it was good or service innovation.

Note though that while the information about innovative activity spending (question 14) refers to spending in the last year of the survey period only, their dummy counterpart in (question 13) refers to the whole of the survey period, thus this adjustment neglects if a firm would have reported spending figures in the two previous years but did not respond to whether they carried out innovative activities (question 13).

Beyond these adjustments for the question sets containing several questions with yes, no or other options in one group (questions 2, 12, 13, 16, 19, 21, 22 , 23)101 _all

missing observations were replaced by zero if the respondents had responded to at least one question in the question set. This may have introduced a bias, however since around 95% of the respondents at one point or another had missing answers in a question set while at the same time having at least one answer to the question set not recoding them would have meant an unacceptable loss of information. For each individual question set the adjusted proportion of the population has not been larger than 5%.

101 _{As question 7, about whether the firm introduced an innovation new to the market or just new to the firms, is}

very important this adjustment procedure has not been applied at the expense of losing observations. This was also done since for the CIS 4 the question’s wording may have lead respondents to believe that they should tick one of the 2 by 2 response matrix, starting from the CIS 5 the way to respond here was clarified.