Principles for the annotation

Before presenting the RFD analysis of the annotated experimental scenarios, let us define ‘measures’

and principles for the annotation. These ‘measures’ represent the knowledge-level actions that were introduced as a part of the RFD model in chapter 6. Their chief purpose is to provide a succinct account of each analysed session in the form of a simple table that will relate the individual design episodes, the frequency of occurrences, and the order of the investigated actions. The same ‘measures’ are used also in chapter 9 to show and discuss the aggregated results from all the experiments, as well as interesting patterns that re-occurred in the experimental sessions.

First, let us clarify the vocabulary that is used during the annotation and analysis. The term design task or design session both refer to one particular design problem that was tackled by the designer over a certain period of time. Each design session started with the presentation of a design brief to the designer. The brief was written in a customer’s language, and its purpose was to express the essential

Selection of design problems

Observation of designers at work

Assessment by a panel

RFD analysis of

collected data RFD analysis of

the annotations Amendments to RFD model Correlation

analysis, patterns Legend:

 … tasks discussed in this chapter

 

requirements and expectations from the designed product. These briefs were prepared in advance by the investigator, in consultation with a panel of experts.

Each design session lasted for several hours, and typically contained several explicit ‘milestones’.

The designer articulated such a milestone in order to record and emphasise his explicit commitment to a particular problem specification and/or partial solution. One significant result of each explicit commitment was a definition or re-definition of a design context. A design context always consisted of a list of selected (attended) requirements, constraints, assumptions, and other statements that clarified the designer’s position in respect to the problem interpretation. In addition to the explicit specification, each context may contain a (partial) candidate solution. The evolution of the explicit commitments and respective design contexts is transcribed in Appendix C.

However, the records of the evolving and changing design contexts are too coarse-grained, and only show the design problem at the moment when a designer made an explicit commitment. The design contexts do not record any reasons, justifications or alternative choices that were available to the designer. Therefore, in order to complement design contexts, we use the term ‘design episodes’. These episodes consist of one or usually more reasoning steps, and their purpose is to provide the transitions between the explicit commitments to particular design contexts. Typically, there were two design episodes between two consecutive design contexts at the early stages of a design session. Toward the end of the session, there was usually one episode bridging the two explicit commitments.

Each design episode described a meaningful standalone portion of the designer’s reasoning.

Usually, we associated the episode with a particular thread or sub-thread from the justification records.

Nevertheless, we took into account not only the syntax of the recorded threads but also their content and focus. Thus, a single design episode was typically concerned with the elaboration of a particular viewpoint, analysis and comparison of the alternative viewpoints, product decomposition and development of the sub-components. In the identification of ‘boundaries’ between the design episodes we also took into account the order in which the justifications were recorded. For a typical transcript of the ‘raw’ justifications, see Appendix D.

Finally, the reasoning steps of a design episode corresponded to the ordered selection of the knowledge-level actions that are presented in Table 8–1. These actions followed from the review conclusions in chapter 5 and the RFD theory introduced in chapter 6.

Table 8–1. Definition of reasoning steps forming a design episode Reference Verbal description of referenced action

1a Articulate design frame (set 666 and S6 ⊆ ⁶⁶⁶6: specifies(S, $3$3$3$3)) 1b Articulate design frame (set 777 and/or 77 777^*)

2 Retrieve similar problem/frame ( φSIM = 〈τ, θ〉 ) 3a Knowledge re-use – articulate S, 6666 similar to θ 3b Knowledge re-use – articulate T, 777 similar to τ7 4 Is re-used/adapted T/777 and/or S/67 666 admissible?

5 Find candidate solution (T⊆ ^7777*: satisfies(T, S))

6a Develop S, 666 into details (incl. alternatives, refinements)6 6b Develop T, 7777 into details (incl. alternatives, refinements) 7 Is candidate solution T acceptable?

8 Re-interpret conceptual frame (i.e. ΦÆΦ^{’ )}

Let us briefly recapitulate how we assigned the individual reasoning steps to the annotated design episodes and contexts. Reasoning steps 1a and 1b were assigned to the episode that featured a formulation or a selection of design objectives/objects. Typically, these two steps represented an explicit commitment to the particular requirements, constraints, and conceptual primitives. Usually, these steps were associated with an articulation and recording of a design context.

Reasoning step 2, and consequently 3a and 3b, represented any reference the designer made to the previous design cases, applicable domain theories or design models/prototypes. These reasoning steps typically occur together as knowledge retrieval and re-use/adaptation. ‘Keywords’ that can be found in the justifications include for instance, ‘from the theory we know’, ‘in order to create a typical control loop’ and similar. Reasoning step 4 was concerned with the explicit assessment of admissibility. An admissibility check was assigned to the episodes, in which clear criteria existed against which the designer could evaluate his partial solutions, compare alternative approaches, or check the compatibility of the re-used/adapted design knowledge.

Reasoning step 5 corresponded to an attempt to articulate an explicit solution, solution model or a partial solution. Typically, this step was accompanied by a drawing or sketch depicting the essential components, behaviours or functionality of the (partial) solution. In similarity with steps 1a/1b this step was typical in the records of explicit commitments and design contexts.

The next two reasoning steps 6a and 6b correspond to further development of a particular design context and commitment. They may include the elaboration of existing (partial) solutions, further clarification of design objectives, and various deductive refinements in the problem specification or solution description. They differ from steps 3a/3b in the fact that no specific reference is made to past knowledge, nor do any new conceptual terms appear in these ‘refinements’. Unlike steps 1a/1b that merely state the facts explicitly, reasoning steps 6a/6b tend to show the refinement of such facts in the form of causal or deductive chains.

Reasoning step 7 was assigned to those episodes that exhibited certain signs of evaluation, although there were no specific and clear criteria defined at the beginning of the episode. Typically, the new criteria/objectives emerged from the episode featuring this acceptability check. Finally, step 8 is our reference to a potential perspective shift. In other words, when new concepts or criteria emerged in the design episode as a result of reflecting on the previous designs or experience, we assigned the episode this re-framing step. Often, frame shift or amendment was accompanied or followed by an introduction of new assumptions or preferences and a tentative commitment to them. Thus, we may see step 8 as the creation and application of an exploratory probe.

Let us begin with the annotation of the design session T11 in section 8.2, where we split the design session, and show several design contexts with various design episodes connecting them. Then, in section 8.3, we shall interpret the annotated design episodes and explicit commitments in terms of RFD theory and the reasoning steps we defined earlier. The same strategy is repeated for design session T21 in sections 8.4 and 8.5.