Interesting patterns

The purpose of this section is to provide further details about the frequency of the patterns that were found interesting in the previous section. We show in the following paragraphs how the ratio between the ‘reflective’ development and the explicit (‘simple’) refinement of design understanding changes for the different types of design problems. This distribution reflects the occurrences of the transitions 5 Æ

7 Æ * (i.e. satisfaction followed by an acceptability assessment) compared with transitions 5 Æ 3/6 (i.e. satisfy partially and refine without any acceptability checking).

Another interesting distribution that was identified on the levels 2 and 3 in the previous section would be also detailed for the different types of design problems. Thus, we present how frequently each of the following transitions occurred in a particular class of problems:

• 4 Æ 3/6 (admissibility check followed by an extension/refinement),

• 7 Æ 3/6 (acceptability check followed by an extension/refinement), and finally

• 7 Æ 8 Æ 3/6 (problem extension/refinement following in the amended frame)

In order to give a certain structure to these comparative analyses, we categorised the experimental design sessions into several complementary categories. These categories and association of a particular session with a particular category were not chosen randomly. We used the auxiliary assessment provided by a panel that was elicited before the RFD analysis, thus being sufficiently independent of any conclusions that were summarised in section 9.1. These findings also support the proposals for extending the RFD model that were hinted at in the previous section. The categories selected for the comparative analysis include the following:

1. Overall distributions (all experimental sessions);

2. Distributions for ‘innovative’ and ‘routine’ sessions;

3. Distributions for superficial solutions and more elaborated, complex ones;

4. Distributions for conceptually detailed and abstract solutions

Note that the classification of the individual sessions into particular categories was conducted for each particular design session based on the explicit records and drawings from that session. It is not the intention of this classification to argue that for instance, all designs of shock absorbers must always be innovative. Simply, this particular design session was found innovative or routine, simpler or more complex by the panel, and we accept this subjective decision for our detailed analyses.

Let us present first the ratios between the reasoning sequences for all the design problems that were annotated and analysed. The distribution showing the ratio between the occurrences of ‘reflective’

development and simpler, ‘explicit’ problem refinement/extension is shown in Figure 9–10. The numbers in data labels correspond to the absolute occurrences of respective reasoning sequences. In our experiment, the ratio of these occurrences was reflective : explicit = 61.7% : 38.3%.

Ratio between 'reflective' and 'explicit' developments

95 59

0% 20% 40% 60% 80% 100%

1

(percent)

Figure 9–10. Ratio ‘reflective development’ vs. ‘explicit’ refinement/extension of design

Reflective development

Explicit refinement (extension)

‘all design sessions’

Ratios of distribution

Figure 9–11. Ratios between the consequences following different assessments The bar chart in Figure 9–11 shows a similar distribution for the operations that followed the assessment of an explicit admissibility and tacit acceptability. The ratios and percentages presented below are later compared with those for the particular classes of problems thus revealing interesting distribution patterns for the specific classes. Nonetheless, the respective overall ratios are as follows:

• Tacit assessment (acceptability) vs. explicit assessment (admissibility) = 65.4% : 34.6%;

• Design refinement/extension vs. re-framing following acceptability check = 55.5% : 44.5%;

• Tacit refinement of design vs. tacit extension of design = 83.7% : 16.3%;

• Re-frame & refine design vs. re-frame & extend design = 59.4% : 40.6%

9.2.1 Distributions between ‘innovative’ vs. ‘routine’ tasks

The classification of the individual design sessions into ‘more routine’ and ‘more innovative’ ones is based upon the overall impression provided by the panel, which was presented in section 7.4 (specifically in Table 7–1). Most sessions exhibited features that were considered by the panel as

‘innovative’ (16 tasks) compared to 11 tasks showing ‘more routine’ features. Figure 9–12 shows the distributions between the ‘explicit’ and ‘reflective’ problem development for the two classes. The classes are compared using percentage ratios rather than absolute frequencies, and for a comparison, the respective ratios are:

• More ‘innovative’ problems: reflective : explicit = 69.5% : 30.5% (data series at the bottom);

• More ‘routine’ problems: reflective : explicit = 46.7% : 53.3% (data series at the top)

Ratio between 'reflective' and 'explicit' developments

Figure 9–12. Ratio ‘reflective development’ vs. ‘explicit’ refinement/extension of design

Admissibility

Ratios of distribution

Figure 9–13. Ratios between consequences of tacit vs. explicit assessments (innovative class)

Figure 9–13 and Figure 9–14 show the distributions of the individual reasoning sequences for the

‘innovative’ and ‘routine’ problems, respectively. In order to make a comparison with the ‘overall’

scores easier; the percentage ratios are presented below:

Innovative problems:

• Tacit assessment (acceptability) vs. explicit assessment (admissibility) = 69.5% : 30.5%;

• Design refinement/extension vs. re-framing following acceptability check = 48.8% : 51.2%;

• Tacit refinement of design vs. tacit extension of design = 84.7% : 15.3%;

• Re-frame & refine design vs. re-frame & extend design = 40.3% : 59.7%

Routine problems:

• Tacit assessment (acceptability) vs. explicit assessment (admissibility) = 46.8% : 53.2%;

• Design refinement/extension vs. re-framing following acceptability check = 70.2% : 29.8%;

• Tacit refinement of design vs. tacit extension of design = 81.8% : 18.2%;

• Re-frame & refine design vs. re-frame & extend design = 50.0% : 50.0%

Ratios of distribution

Figure 9–14. Ratios between consequences of tacit vs. explicit assessments (routine class) The first significant difference between the two classes is in the ratio between the ‘explicit’ and

‘reflective’ problem developments. The ‘innovative’ problems show a ratio significantly in favour of reflection, whereas with ‘routine’ problems, the ratio is in favour of simpler, explicit extensions and refinements (see Figure 9–12). The ‘innovative’ problems also show higher percentage of reflective sequences (69.5%) than the ‘routine’ problems (46.7%). This is precisely reversed for the ‘explicit’

extensions and refinements – 30.5% for the ‘innovative’ class and 53.3% for the ‘routine’ class.

Admissibility

A similar pattern is observed in the distribution between the methods of assessing partial solutions (Figure 9–13 and Figure 9–14 compared). For the innovative problems, the reflective and inarticulate assessment of acceptability (69.5%) is more typical than the assessment of explicit admissibility (30.5%). On the contrary, the ‘routine’ tasks feature a more balanced assessment, and they slightly tend to prefer the explicit checks of admissibility (53.2%) to the acceptability (46.8%).

Following from these ratios, the next feature is also not surprising. More ‘innovative’ problems show almost even distribution between problem refining/extending and re-framing (48.8% and 51.2%, respectively). The same distribution for the ‘routine’ class is shifted more significantly towards refinements (70.2%), giving less space to the problem re-framing (29.8%).

The prevalence of reflective reasoning for the innovative tasks and the lack of problem re-framing for the routine ones both seem to uphold the attributes typically associated with the respective classes.

‘Innovative’ problems tend to be vaguer and under-specified, thus requiring more reflection; also the amendments to problem understanding are more significant (including re-framing). On the contrary,

‘routine’ tasks are better defined; they may need only minor clarifications and refinements rather than significant perspective shifts. ‘Routine’ tasks are usually not ill-structured problems, therefore explicit criteria for the assessment of their admissibility are typically defined in advance. With a large number of explicit criteria available, there seems to be less need for tacit assessments. More vaguely defined or missing criteria of admissibility in the ‘innovative’ class seem to force the designer to rely more on the tacit assessments of acceptability.

In addition to these mutual ratios, the ‘innovative’ problems show scores that are well above the average scores calculated for all the experimental sessions. Subsequently, ‘routine’ problems seem to be below the average scores presented in Figure 9–10 and Figure 9–11. This observation seems to comply with the perception that ‘innovative’ problems are usually outperforming the familiar and

‘average’ solutions.

9.2.2 Distributions for ‘superficial’ and more elaborated, complex solutions

The classification of the individual design sessions in respect to producing ‘superficial’ solutions (often only control algorithms) and ‘more elaborated’ ones is based upon Table 7–2 in section 7.4.2. The number of sessions exhibiting features of these two classes is almost evenly distributed – 14

‘superficial’ and 13 ‘elaborated’ solutions/alternatives. The classes are compared using percentage ratios rather than absolute frequencies. Figure 9–15 shows distributions between ‘explicit’ and

‘reflective’ problem development in the two classes. For a comparison, the respective ratios are:

• More ‘elaborated’ design solutions: reflective : explicit = 68.5% : 31.5%;

• More ‘superficial’ design solutions: reflective : explicit = 55.0% : 45.0%

As we can see from Figure 9–15 and the respective percentages, the sessions producing more

‘elaborated’ (complete) solutions tended to rely more on reflective reasoning (68.5%) rather than direct refinements and extensions (31.5%). On the other hand, the gap between reflective development and explicit refinements/extensions was much less significant for the simpler, more superficial solutions – 55% reflective and 45% explicit. In comparison with the previous section, we can see remarkably similar distributions. Only in this case, more elaborated and complete solutions tended to force the

designer to reflect more frequently (68.5%) than the straightforward, rather superficial solutions (55%).

However, the gap is narrower than it was between ‘routine’ and ‘innovative’ tasks, which suggests that innovation does not necessarily come with more elaborated (complex) solutions.

Ratio between 'reflective' and 'explicit' developments

Figure 9–15. Ratio ‘reflective development’ vs. ‘explicit’ refinement/extension of design

Ratio of assessments of 'admissibility' and 'acceptability'

38

Figure 9–16. Ratio ‘admissibility assessments’ vs. ‘acceptability assessments’

of partial solutions

From Figure 9–16 it is possible to conclude that more elaborated, complete solutions may have required more tacit assessments of the acceptability than simpler, superficial solutions. Moreover, solutions that were more complex featured more inarticulate assessments of acceptability (73.4%) than the explicit checks of admissibility (26.6%). The ratio is somewhat similar for the superficial solutions – showing a similar trend. The acceptability checks were more frequent (62.2%) than checks of the explicit admissibility (37.8%). The justification would be similar to the one made earlier – complexity does not necessarily mean the same as ‘poor structure’ and the need for tacit knowledge.

Ratio between 'refine/extend' and 're-frame'

Figure 9–17. Ratio between ‘solution refinements/extensions’ and ‘re-framing’

‘superficial solutions’

The observation from the previous figures is confirmed also in Figure 9–17. According to the last chart, the ‘elaborated’ solutions seemed to require more effort from the designer in understanding the particular details he wanted to elaborate. A re-framing sequence in reasoning occurred in 50.5% cases, compared to only 40.5% for the ‘superficial’ design solutions. The superficial solutions tended to exhibit more refinements and extensions (59.5%) than re-framing (40.5%). This suggests that the designer needed to introduce less conceptual terms and/or requirements for superficially handled problems than for those that yielded more elaborated, complete solutions.

9.2.3 Distributions for conceptually detailed and abstract solutions

A similar pattern in the ratio between reflective development of a design problem and the direct, explicit refinements and extensions is also observable when comparing conceptually detailed and conceptually more abstract solutions. Conceptually detailed solutions often included some details of the implementation, whereas conceptually ‘abstract’ ones usually corresponded to block diagrams and strategies rather than implementations. Figure 9–18 shows distributions between ‘explicit’ and

‘reflective’ problem development in the two classes. For a comparison, the respective ratios are:

• Conceptually ‘abstract’ design solutions: reflective : explicit = 63.1% : 36.9%;

• Conceptually ‘detailed’ design solutions: reflective : explicit = 57.0% : 43.0%

Ratio between 'reflective' and 'explicit' developments

Figure 9–18. Ratio ‘reflective development’ vs. ‘explicit’ refinement/extension of design As we can see from Figure 9–18, there is no significant correlation between the level of conceptual detail and the frequency of reflection. Conceptually ‘abstract’ solutions tend to provoke more reflection (63.1%) than direct refinement or extension (36.9%). The same holds for the conceptually detailed solutions – reflection observed in 57% cases, direct refinement/extension only in 43%. There is a marginal difference in the number of direct refinements/extensions, which goes slightly in favour of conceptually detailed solutions. This seems to be understandable – if the solutions are conceptually high-level or abstract, it may be difficult to refine or extend them without any exploration or probing.

Nonetheless, the conclusion from this particular visual representation is that there is little correlation between the conceptual abstraction and tacit, reflective reasoning strategies. In certain cases, it was possible to produce an abstract solution from a re-used template – directly refining or extending it. In other sessions, even abstract block diagrams may have required more effort.

‘detailed solutions’