The project’s overall research approach can be classified as inductive, since it was initialized by explorations from related research, informing theories as well as practice of field observations and the qualitative study in ADR Cycle I, combined with the application of justificatory knowledge as suggested by Gregor and Jones (2007). This resulted in the development of design principles. The design principles evaluation in phase 4 of the ADR Cycle II follows a deductive approach as it tests if the design principles hold true for empirical evidence. The findings from the previous, inductive research thereby inform the definition of testable hypotheses.
To reach the research goals, the focus is set to the performance of professionals and mental effects when using the artifact. The unit of analysis is therefore the individual supply network professionals. The construct performance is thereby split into two independent variables, namely efficiency and effectiveness. This common form of performance operationalization was for instance employed by Benbasat and Schroeder (1977), Allen (2006), and Vessey and Galletta (1991).
As well as outcome-oriented measurements, it is important also to take account of psychological effects on the individuals who utilize supply network systems. Kernel theories can therefore be used to support the evaluation of an artifact as one important step in the DSR paradigm, as proposed by Venable (2006), Kuechler and Vaishnavi (2008), and Gregor (2006).
For the design principles conceptualization and evaluation towards positive effects on supply networks, the utility for people acting in supply networks plays an important role. A measurement is therefore needed to account for the individual effort imposed on business professionals when conducting supply network tasks with certain software tools.
Cognitive Load Theory (CLT) represents a kernel theory of this kind from cognitive psychology first introduced by Sweller (1988). It proposes that a human’s short-term memory (also known as working memory) is limited in its capacity and can therefore be hindered in problem solving when excessive cognitive load is imposed (Sweller 1993; Miller 1956). The aim should thus be to reduce cognitive load in order to assign more working memory to schema acquisition (learning) and problem solving.
Instructional design plays an important role in accomplishing this goal. Sweller (1993) argues that instructions are often ineffective because they ignore information processing limits. This is why some tasks are perceived as more difficult than others, “[…] not because of their intrinsic nature but because of the way they have been structured” (Sweller 1993, p. 3). Cognitive load, sometimes also referred to as mental workload (Rubio et al. 2004; Wiebe et al. 2010), is composed of intrinsic and extraneous load. While the former is defined by the intrinsic nature of the task or information, and is therefore not modifiable for a particular task, the latter can be influenced by the instructional design or information presentation (Pollock et al. 2002).
Extraneous cognitive load results from the way instructions are given or displayed. It becomes apparent when the subject has to “[…] mentally integrate [...] mutually referring sources of information” (Sweller 1993, p. 5), which is called the split-attention effect. Consequently, the instructional design is the key to lower cognitive load.
Redundancy in instructional design can also foster extraneous cognitive load, because incorporating the same information multiple times in different ways distracts attention from schema acquisition (Sweller 1989). In order to lower extraneous cognitive load, instructions should be organized in a way that minimizes the substantial cognitive resources which are required to mentally integrate disparate sources of information (Sweller 1989). This can be achieved by avoiding the split attention effect and by not displaying information redundantly. This leaves more time for schema acquisition and automation, which ultimately can yield substantial performance increases. Learning, here also called ‘schema acquisition’ reduces working memory load by chunking elements of information into a single element, thus reducing the number of elements that need to be processed (Sweller 1993).
4 Conceptualization 54
Cognitive overload occurs once the intrinsic and extraneous cognitive load combined exceed the working memory’s capacity during a specific time frame. This only applies however if the intrinsic structure of the task is complex enough to impose high (intrinsic) load, so that an unfavourable instructional design will have an effect which can lead to a significant performance decrease (Sweller 1993). Below a certain intrinsic cognitive load level, even inappropriate instructional design will not cause a cognitive overload, as the overall cognitive load imposed will not exceed the working memory’s capacity limitations.
Figure 14 summarizes these explanations. This makes it apparent that the instructional design is the key to lowering cognitive load, as task complexity is mostly inherent.
Figure 14. Cognitive Load Theory16
As a multidimensional construct, cognitive load consists of cause factors, such as the task’s environment, the subject’s characteristics, and the interactions between them. This can be conceptualized in the dimensions of mental load and mental effort (Paas and Merrienboer 1994; Paas et al. 2003). Mental load accounts for the amount of intrinsic cognitive load that is imposed by the task. It is thus fixed for a given task. Mental effort represents the human-centered aspect of cognitive load, and accounts for the aspect of cognitive load that refers to the cognitive capacity that is actually allocated (Paas et al. 2003; Rey and Buchwald 2011). Paas et al. (1994) observed that people can compensate for an increase in mental load by increasing their mental effort, thereby maintaining the outcome. With regard to measuring, this means that it is impossible to differentiate between the different types of cognitive load. Mental effort is therefore used to assess cognitive load as a whole. This is true in particular when mental load is kept at a constant level, for example by applying the same task to an individual while using different tools (Paas and Merrienboer 1994, Paas et al. 2003; Rey and Buchwald 2011).
16
mental effort is therefore considered the best estimator (Paas and Merrienboer 1994).
4.4.2 Testable Hypotheses and Research Model
To test the artifact design, individual performance in supply networks as the dependent variable can be divided into two variables, namely task efficiency and task effectiveness (Sharda et al. 1988; Vessey and Galletta 1991; Fuller and Dennis 2009).
With the artifact providing solutions for the integration of unstructured and structured data and processes, and for the prevention of information disperse, it is expected that supply network individuals using the artifact will perform a task faster than with a comparison tool, as they do not need to integrate different sources of information manually and do not have to keep track of the latest version of a document. The following hypothesis is therefore defined:
H1: Using the artifact results in higher task efficiency than using a comparison tool.
Task effectiveness can be measured by decision-making quality. Making the right decision that meets most pre-defined requirements from a set of choices results in high task-effectiveness. Prevention of document exchange also results in a reduced risk of making mistakes because of obsolete document versions. If integration of unstructured and structured activities results in more clearly presented information, this is expected to result in better decisions due to the reduced likelihood of important pieces of information being missing. This leads to the second hypothesis:
H2: Using the artifact results in higher task effectiveness than using a comparison tool.
A defined supply network task that has to be performed during the experiment determines the intrinsic cognitive load. In both cases of applying the artifact or a comparison tool, the intrinsic load maintains a stable level. What differs for both tools is the extraneous load exerted upon the individual buyer by the instructional design of the tool. The B-Zone artifact with the two design principles is assumed to reduce inappropriate information presentation, split-attention effects, information disperse and redundancy. It is therefore proposed that the artifact lowers the amount of mental effort compared to a comparison tool frequently used today in supply management. This leads to the following hypothesis:
H3: Using an artifact results in lower mental effort required to perform a task as compared to using a comparison tool.
With this operationalization, the resulting research model consists of constructs that can be measured for evaluation. By deriving testable hypotheses, it is possible to evaluate
4 Conceptualization 56
the artifact as a design product (Walls et al. 1992; Pries-Heje et al. 2008). The research model with the relations between the variables is shown in Figure 15.
Figure 15. Research Model
4.5 Summary
In this chapter, the iterative research process following the overall research design based on ADR for aggregation of the meta-requirements and conceptualization of the design principles has been elaborated. This covered the first iterations of the problem awareness in phase 1 of the ADR Cycle I, providing preliminary meta-requirements. Based on these, and informed by further prior research, the key challenges of supply networks to be addressed by respective system designs have been explored. In the following iterations in the DSR team, the final meta-requirements have thus been aggregated, and the two design principles for supply network systems with a high potential to fulfill the meta-requirements have been induced, namely ‘networked Business Objects’ (DP1) and ‘social augmentation’ (DP2).
To put the design principles in the context of common interaction patterns in supply networks, interlinking data and process integration with people integration dimensions, the overall design approach has been explained.
Finally, the theoretical foundations, introducing Cognitive Load Theory (CLT) and the development of testable hypotheses have been presented. This led to the layout of the research model with task efficiency, task effectiveness and mental effort as dependent variables, and the applied software tool as the independent variable. The variation of the independent variable is implied by either applying the B-Zone artifact incorporating the design principles, or a comparison tool, which does not include the design principles.