Survey design and administration

The purpose of the design was to enable an evaluation of the effects of inter-firm relationships upon supply network performance. There exists a first-order and second-order aspect to all performance.

First order performance measures primary effects of adopted characteristics upon performance. This is relatively straightforward complicated only by the fact that characteristics are adopted for specific or multiple performance outcomes. Five performance outcomes (or success criteria) were identified during literature review (see Chapter 3, Section 5). These were validated during stage 2 of the research design. The 27 characteristics, created initially from literature review and further refined by interviews at Stage 2 of the research design (see Chapter 7, Section 2) were listed in a simple matrix with 5 columns for the 5 performance success criteria. Thus 135 cells (27x5) need to be completed for each Table I. Integer responses were required in the range 0-9. This matrix formed Table I of the survey instrument and an empty matrix can be found in Appendix H.

Risk with Survey Instrument

Action to mitigate, avoid or transfer

References

Recognise Common Methods Variance (bias in Key

Informants)

(Jick, 1979; Snow and Hambrick, 1980) The use of the key

informant methodology by researchers creates bias

Recognise issues of agreement and reliability when using multiple raters

(Kumar et al., 1993; Boyer and Verma, 2000)

Single-item measures are not sufficient to

operationalize inherently complex business concepts; Use of single items

preventing internal consistency reliability

Use of multi-item

measurement scales to reduce measurement error by

providing a more robust construct of complex variables through the averaging of several related items

(Stratman and Roth, 2002)

Corroborative evidence may be biased

Retrospective data and sources of data inaccuracy

(Huber and Power, 1985) Understanding the

measurement issues

(2003; 2000) Getting the measures right

– scale development Measurement quality – reliability and validity of

measurement instruments Measurement of item and scale reliability and validity; Scale development good practice

(Rosenzweig and Roth, 2004; Hinkin, 1995)

Falling through the gap between theory

development, construct production and survey items

Bridging the gap between theory and survey production

(Malhotra and Grover, 1998)

Use appropriate techniques to gather information

(Frohlich, 2002) Risk of not obtaining high

response rates

Understanding bias in non- responses

(Armstrong and Overton, 1977)

The design of Table I extends previous research in two ways. Table I of the survey employs methods used in Organizational Systematics (McKelvey, 1978) and

Cladistics (McCarthy, 1995; McCarthy and Ridgway, 2000) in which organizational forms can be detected based on the characteristics adopted. The extension to existing methods provided by this research is to recognize that characteristics are adopted not just for a single purpose but for a small number of key performance success criteria. There is a trade-off in the adoption of a characteristic in that its adoption has a cost and not always a high return in each key performance success criteria.

The second order effects of adopting these 27 characteristics are the inter-

characteristic effects. These were captured in Table II of the survey instrument. The Table II pair-wise matrix requires respondents to evaluate the effect of each supply network characteristic upon every other characteristic. Responses measure the inter- characteristic effects on an interval scale from -5 to +5; they are described as pair- wise effects in the following discussion.

Table II of the survey instrument thus consists of a pair-wise interaction matrix. This matrix formed Table II of the survey instrument and an empty matrix can be found in Appendix I. A matrix in this format, but with different characteristics, was used for automotive manufacturing as part of the ESRC Nexus Project to find bundles of synergetic characteristics (Baldwin et al., 2005). These bundles identified

organizational forms within the automotive manufacturing industries, using only pair- wise interaction matrix survey results.

The survey instrument for the purposes of this research therefore consists of two tables: Table I for first order performance scores, and Table II for pair-wise, inter- characteristic effects.

Whilst simple in design, administering the survey by interview was likely to be onerous because of the need for the respondent to ‘see’ the table at the same time. Using the experience of colleagues in Sheffield University’s Industrial Manufacturing Unit who had previously administered such surveys, it was decided to email the survey as an electronic spreadsheet, so that recipients could complete the survey via

their personal computers and without needing to write anything down. This was possible because the questionnaire design was simple. It achieves the highest quality of responses as the interviewer cannot influence answers and the respondent

completes the survey at a time (or over multiple sessions) to suit their own convenience. It is also the cheapest form of data collection as it avoids excessive researcher costs particular when the firms are located across the globe.

The questionnaire survey was addressed to the Director of Supply Chain. This person was asked to coordinate multiple responses for the firm. It was anticipated that this person or their direct reports would be able to interpret accurately the effects of supply network characteristics. The individual is probably the best informed of all potential candidates but could be judged to be remote from day-to-day interactions. This was mediated by responses from multiple respondents in the same firm.

Requests for completion of the questionnaire surveys were distributed to the largest (measured as either turnover or number of employees) aerospace manufacturing firms with SIC code 3530 (aerospace manufacturing found on FAME (electronic database of registered companies) in combination with known firms at Sheffield University. Survey responses were received from all tiers of firms: primes, 1st tier and 2nd tier, from individuals in the ratio 15:35:40 respectively reflecting the smaller number of primes. Implementation problems (de Vaus, 1999) arose, with difficulties in obtaining responses. All responses could be used since only valid responses were supplied. 3 null cells were the maximum incomplete per survey (from a total of 351 cells). The quality of the responses was judged by random sampling of survey responses, to check the spread of ratings (see Chapter 7, Section 6). The

questionnaire design was such that the respondent could add further characteristics and/or further performance criteria, although none did so. Most respondents expressed some exhaustion after completing Part II of the survey instrument which was the pair-wise matrix. 27 characteristics appear to be excessive for such as survey. The design of the matrix could be altered to break it into smaller components,

alternatively, the use of a real-time electronic means of collection which allows the selection of characteristics practised by the firm, would make implementation easier.