Validation

Verification of complex systems models is certainly difficult (Pathak et al., 2007a; Pathak et al., 2007b; Davis et al., 2007; Pathak et al., 2010). The main

problem being that any valid model is nondeterministic and emergent, inferring that small changes in some variables may produce disproportionate changes in outcomes. Verification is further complicated by the difficulty of gathering real- world empirical data against which the system can be compared. This is particularly true of systems such as supply networks that are of large-scale and geographical spread bound together by autonomous decisions made under conditions of bounded rationality.

This thesis uses two approaches to validating the computer model:  Sound theoretical anchors for the algorithms

 Verification of algorithm descriptions using an expert panel. 5.5.2.1 Theoretical Anchors

Table 5-16 summarises the theoretical anchors for each element of the model described previously in this chapter.

Table 5-16: Theoretical anchors for each element of the model

Sub-model / Core Component

Algorithm Theoretical anchor

Core component Supply risk TCE: (Kraljic, 1983;

Williamson, 1996)

Collaboration Relational risk SET: (Hunt and Morgan,

1994)

Information sharing Supply chain management: (Fisher, 1997)

Price Competition Price adaptation Economics: (Marshall , 1930; Williamson, 1996) Purchasing Strategy Single/dual sourcing TCE, purchasing (Kraljic,

1983)

Inventory Management Inventory management Operational research: (Waters, 2003)

Birth and Death New entrants CAS: (Bak, 1999)

Supplier Selection Supplier selection criteria Supplier selection : (Weber et al., 1991; Dempsey, 1978; Dickson, 1966) Relational inertia SET (Hunt and Morgan,

5.5.2.2 Panel Verification

In addition to the theoretical anchors provided above, the model was validated at a number of workshops comprising practitioners and academics from the field. The selection of the panel was designed to balance academics with practitioners. The composition of the panel is given in Table 5-17

Table 5-17: Composition of the validation workshop panel

Panel Member Background Title and affiliation

Colin Dulson Practitioner Supply Chain Director, AAH

Pharmaceutical

Mike Griffiths Academic Department of Informatics,

Shrivenham

Professor Peter Allen Academic Director Complex Systems

Group, Cranfield University

XX Practitioner World Wide Duty Free

Richard Street Practitioner Supply Chain Director,

Mothercare

In addition to the workshops the approach was presented at the Logistics Research Network conference and the European Operations Management Association conference.

The workshops consisted of the author presenting the algorithms described previously and inviting the audience to challenge the algorithms and the underpinning assumptions.

A number of challenges were presented, and these are summarised in Table 5-18 together with the conclusion reached by the author and the conclusion justification.

Table 5-18: Challenges to the model arising from the validation workshops

Challenge Response Justification

EOQ sensitive to holding costs and re-order costs

Agent checks that EOQ covers lead time demand

This amounts to a hybrid approach incorporating elements of both EOQ and order up to levels

Use of a single product does not reflect the complicated nature of real supply chains

The single product abstraction involves less assumptions regarding interdependencies between different products in different organisations which would only obscure the system response and response mechanisms

Single product adopted, but observation noted and reflected in recommendations for further work

Risk assessment is less formalised

This is probably true but it was agreed that it would include the principles of the proposed algorithm

Retain the calculations of risk proposed but note comments and include in

recommendations for further work

Supplier selection not as sophisticated as the model Births of new companies involves an assessment of the market

Undoubtedly true but the operationalisation of this within the model would introduce added complication

It was agreed that market assessments are typically based on bounded rationality and are often naïve

Responses to underutilisation are more varied than price adjustments

This is also true, but

suggestions include activities anticipated in the model such as bundling and strategic location

The single product abstraction does not permit bundling and re-location of resources would require extension of the model run time to beyond

reasonable limits within the broad range of experiments necessary

Product criticality ignored This is also true but the

assumption of using a single product requires the strategic purchasing options to be reduced to whether or not to collaborate, and whether or not to dual source

Whilst the implications of ignoring product criticality are clearly a simplification of Kraljic’s (1983) purchasing strategy they are supported by TCE, SET and indeed to a large extent by Kraljic

Why not allow multiple sourcing to be more than two suppliers

Extending the possibility of multiple sourcing including more than 2 suppliers would require agents to balance the benefit with the resources and costs required to manage the extra suppliers

Two suppliers was felt to be realistic if not all embracing

5.6 Summary

This chapter has described in detail how the underpinning conceptualisation of supply networks as CASs susceptible to normal accidents as described by NAT has been incorporated into the design of an ABM suitable for assimilation in an

experimental design to provide answers to the research questions identified by the literature review.

The model design has been verified at elemental levels through careful stepwise construction, staged testing of components, and integration of components. The algorithms, sub-models and consequently the model itself have been validated by anchoring the design in strong theoretical anchors drawn from SET, TCE, and supply chain management. Further validation was secured through the reflections of an expert panel on the components of the model.

The model was developed using Java programming language in the Anylogic 6 development environment. The model code is provided in Appendix A

The next chapter will describe the design of the experiments used to generate data from which the answers to the research questions can be developed through appropriate analysis congruent with the theoretical underpinnings of CAS and NAT.

6 Experiment Design

Chapter 4 showed how the collection of real-world empirical data using survey methods whilst notionally attractive has practical limitations when trying to understand a complex system’s behaviour in response to events that are unpredictable and often rare. Furthermore, the difficulty in establishing real- world systems boundaries is prohibitively resource expensive.

Chapter 4 also showed how computer models can be built using established supply chain practice as the basis for organisational behaviour. Although the model abstraction is founded on empirically validated organisational behaviours it extends the environment within which these behaviours play out, thereby providing a more realistic representation of the real world than that which would be deliverable from even the highest quality surveys.

Chapter 5 specified a computer model that is anchored in extant theory, but which accepts a new dynamic environment. Furthermore, the model specification, cognisant of the research questions identified in Chapters 2 and 3, provides the basis for submitting the model to a series of experiments designed to answer the research questions. As a consequence, this chapter is primarily concerned with describing a rigorous and robust approach to the definition of an experimental program.

This chapter starts by describing the design principles adopted and then uses these to specify an experiment design before considering the inference this has on the data collected with a view to the subsequent analysis. The chapter concludes by integrating design principles, design and analysis into a validation of the approach taken.