Theme 2: Iteration

Research theme 2 explored the concept of iteration in design thinking. As illustrated in Section 2.2.6, authors have proposed various non-linear design thinking process models which consist of several connected activities. For the purpose of this study, iteration was defined as the recursive movement through the chosen design thinking process. An accepted limitation to this measurement strategy was the fact that sideways iteration (e.g. iteration between different prototypes in the same process phase) could not be captured.

For every week of data collection, study participants were asked to indicate how they had spent their time working on their project during the previous week. Data was collected via the paper-based weekly survey instrument, which was administered at every face-to-face workshop (see Appendix A). To allow for a visual comparison of the iteration behaviour of the different

sample groups, the collected data was illustrated as a stacked diagram in Figure 4.4. The colours in each diagram correspond to the individual steps of the design thinking process model (see Section 2.2.6).

A new metric was created to express how much each individual participant iterated from week to week (see Section 3.2). For the purpose of this study, iteration was defined as either moving forward or backwards in the design thinking process. For each week, the data was coded to indicate how many hours a participant has either remained in the same process phase, moved forward, or moved backwards. Remaining in the same phase was coded as

“no iteration”. The resulting scores for moving forward and for moving backwards were added together to provide an iteration score for each participant during each week. Considering that the main focus of this

research project is the study of teams, average iteration scores for each team were aggregated. These scores ranged from “0”, indicating no iteration, to

“10”, indicating maximum iteration. The average team iteration scores for the different samples and weeks are shown in Figure 4.5. The thicker black lines indicate the average iteration scores for each sample group. The dotted line represents a linear regression model which was fit to the overall average

iteration scores. The coefficient of determination (R²) in each diagram

indicates the goodness of fit of the trend line and therefore how linearly each group approached the design thinking process. The coefficient of the slope was significant at the .05 level for the APEn group and not significant for the BA and APE^e groups.

To spot more global patterns in the data, the time periods were also sliced into quarters as shown in Figure 4.6. This mirrors the insight drawn from the previous Figure 4.5 that the APEn group seem to have iterated significantly more in the third quarter. Applying one-way analysis of variance indicated that there are significant differences between the four quarters, p < .01 (2-tailed). On the other hand, for the APEe groups the average iteration scores seem to have increased steadily from quarter to quarter. However, these quarterly increases were not significant.

Figure 4.4: Stacked Diagram of Time Distribution in Projects

The colours in each stacked diagram correspond to the colours in the design thinking process model used for both programmes (see Section 2.2.6). The more vertically separated the colour blocks are, the more linearly the teams structured their projects.

Figure 4.5: Average Iteration in Design Thinking Projects per Team

This figure shows the average amount of iteration per sample (min. = 0, max. = 10). Error bars indicate the 95 % confidence intervals. R² indicates the fit of the trend line for average iteration. For the BA group, insufficient data was available to provide a break-down per team.

Figure 4.6: Box Plot of Aggregated Iteration per Quarter

This figure shows the iteration scores in aggregated form per project quarter. For the APEn

group each quarter represents three weeks. For the APEe group the first quarter represents five weeks, while the other quarters represent four weeks each. Error bars indicate the 95 % confidence intervals.

Hypothesis 2a

Multidisciplinary design thinking teams iterate more than single-discipline teams.

To test Hypothesis 2a, the APEn (multidisciplinary) and BA (single-discipline) teams were compared. Figure 4.4 provides a visual comparison of how the different sample groups allocated their project time within the six phases of the design thinking process model. While examining this figure it became apparent that the BA teams spent less time in the “understand problem”

phase of the model than the APEn teams. It seems that the BA teams also had one larger iteration loop, when they moved back from generating ideas (21 November) to working on their “point of view” (28 November). The

corresponding Figure 4.5 shows the average amount of iteration per week for each sample group. Both APEn and BA groups overall seem to have

increased the amount they iterated over time, as indicated by the trend line.

An independent-samples t-test revealed that the total amount of iteration of the two compared sample groups is not significantly different from each other at the .05 level (1-tailed). On average, the APEn teams

(M = 3.600, n = 5 teams) seem to have iterated slightly more than the BA teams (M = 2.406, n = 3 teams). The APEⁿ teams (SD = .449) also seem to have been more consistent than the BA teams (SD = 1.340) in how much they iterated.

Discussion

Overall, the APEn teams seem to have iterated slightly more than the BA teams. However, this difference is not significant. Therefore, Hypothesis 2a, that multidisciplinary design thinking team iterate more than single-discipline teams, was rejected in favour of the null-hypothesis. The power of the statistical test was limited by the amount of cases which could be included in the analysis (n = 8 teams).

Hypothesis 2b

More experienced design thinking teams iterate more than novice teams.

This hypothesis was tested by comparing the novice APEn teams and the experienced APE^e teams. Figure 4.4 provides a visual comparison of how both sample groups had allocated their time during the design thinking project. It appears that the APEe teams approached the different steps in the design thinking process model more sequentially. They also seem to have assigned less time for the two initial research phases of “understand problem”

and “observe environment” in favour of spending more time making sense of the collected data in the “point of view” phase. An examination of Figure 4.5 revealed that the APEe teams tended to iterate in small iteration loops rather than evenly spread throughout the project. This was confirmed by comparing the R² coefficients of determination for the regression models which indicated that a linear model only provides a poor fit for the behaviour of the APEe

sample group (R² = 6.9 %) when trying to explain their iteration behaviour

throughout their project. Also, the coefficient of the slope in the linear regression model is not significant for the APEe sample group, whereas it is significant for the APEn group, p < .05.

An independent-samples t-test revealed that the total amount of iteration of the two contrasted sample groups is not significantly different from each other at the .05 level (1-tailed). When comparing the means for average iteration per group, there seems to be a slight indication that the opposite of the stated hypothesis is actually true. The experienced APEe teams

(M = 2.875, n = 4 teams) overall seem to have iterated less than the novice APEn teams (M = 3.600, n = 5 teams). The APEe (SD = .780) group’s iteration behaviour was slightly less consistent than that of the APEn group (SD = .449).

Discussion

The previous analysis showed that the observed experienced design thinking teams did not iterate more than the novice design thinking teams. In fact, the data provided some evidence that the opposite might be true. Research Hypothesis 2b, that more experienced design thinking teams iterate more than novice design thinking teams, was therefore rejected in favour of the null-hypothesis. A possible explanation for this behaviour might be found by linking this phenomenon with the research theme on perceived effectiveness and ease (see Section 4.6). Higher levels of experience, which coincides with higher levels of perceived effectiveness and ease, might make experienced teams feel better able to foresee how a project could progress. This, in turn, might lead them to structure design thinking projects more linearly than novice teams.

Hypothesis 2c

More iteration during a design thinking project leads to a better final performance.

To test Hypothesis 2c, the APEn and APEe groups were jointly analysed. A scatter plot, with the standardised mean performance plotted against the

mean total average iteration, did not reveal a direct correlation between these two factors (see Figure 4.7).

Figure 4.7: Scatter Plot of Standardised Mean Performance and Total Average Iteration per Team

A Pearson product-moment correlation analysis for the nine APE teams confirmed that there is no significant correlation between these two variables.

Repeating this analysis separately for the APEn and APEe groups to account for the different levels of experience, resulted in similar findings.

The analysis was extended to investigate the correlation of the standardised mean performance and the average amount of iteration for each week. The Pearson product-moment correlation analysis was conducted separately for the APEn and APEe groups due to the different length of their respective projects. For the APEn group, only the week starting from 7 January showed a significant effect. For this week the amount of iteration showed a strong significant negative correlation of r = .944, p < .05 (2-tailed). During this week

Standardised Mean Performance

teams were mostly prototyping and business modelling while moving out of the idea generation and slowly advancing towards testing their prototypes (see Figure 4.4). For the APEe group no specific weeks could be flagged as significant in the correlation analysis of standardised mean performance and average iteration per week.

Discussion

Overall, no significant correlation between the standardised mean

performance and the average amount of iteration per team, as measured by the amount of recursive movement in the design thinking process, was found.

Therefore, Hypothesis 2c, that more iteration during a design thinking project leads to a better final performance, was rejected in favour of the

null-hypothesis. Once the analysis was broken down week-by-week, only one week showed a significant correlation between iteration per team and final team performance for the novice APEn group. This week signalled the point at which the teams had locked into a specific idea and move on into

prototyping, business modelling, and the initial testing of the idea. At this point, higher levels of iteration seem to have a negative effect on final team performance. This might suggest that, once projects are in their final stages before being presented to clients or investors, teams should fully commit to their current idea and direction. They should focus their efforts on finalising that idea rather than iterating within the design thinking process model.