Theme 4: Perceived Effectiveness and Ease

Theme 4 investigated perceived effectiveness and ease in design thinking project teams. Both variables relate to the concepts of creative confidence and informed intuition, as discussed in Section 2.2.7. Initially, the variables of perceived effectiveness and perceived ease were treated as separate

Standardised Mean Performance

variables. However, a closer examination of the collected data revealed that feeling effective and feeling at ease in the application of design thinking are highly correlated. The visual analysis of the data displayed in Figure 4.12 suggested a positive correlation between how effective and how at ease individuals in design thinking teams feel during projects. A Pearson product-moment correlation for these variables supported this conclusion.

Both variables are significantly correlated, r = .673, p < .05.

To further investigate the correlation between these two variables, the average values of “feeling effective” and “feeling at ease” for each data collection interval were visualised in Figure 4.13. In this longitudinal view, both factors again showed a clear correlation pattern. Due to this strong correlation, both variables were merged into the single variable “perceived effectiveness and ease” for the subsequent statistical tests.

Figure 4.12: Scatter Plot of Average Perceived Effectiveness and Ease per Team

Figure 4.13: Correlation of Perceived Effectiveness and Ease over Time

These figures show the levels of feeling effective and feeling at ease (scale min. = 0, max. = 5) for each sample group. Both variables are highly correlated.

Hypothesis 4a

Perceived effectiveness and ease follows a U-shape throughout a project.

As previously described in the literature review, Brown (2009) posits that a team’s level of creative confidence is high at the beginning of the project, significantly decreases towards the middle, and then increases again towards the end. In other words, he expects creative confidence to follow a U-shape throughout a project. As both perceived effectiveness as well perceived ease in the application of design thinking were thought to be facets of creative confidence, they were expected to show a similar pattern of development throughout a project.

The first step in testing this hypothesis was to conduct a visual analysis of how the joint measure of perceived effectiveness and ease developed over time. Figure 4.14 presents the corresponding data for each of the three sample groups. For the APEn and APEe groups, enough data was available to analyse each team separately. In Figure 4.14, the thick black line

represents each sample group’s average level of perceived effectiveness and ease. At first glance, perceived effectiveness and ease seem to have

increased fairly linearly during the design thinking projects. No distinguishable U-shape could be detected.

As a second step, a linear regression model was fitted to the variable of perceived effectiveness and ease in Figure 4.14. The R² values > .5 for all three samples indicated that a linear model offers a good representation of the underlying data. For the APEn and APEe groups, the coefficient of the slope was significant at the .001 level. For the BA group, it was significant at the .01 level.

Figure 4.14: Perceived Effectiveness and Ease per Sample Group

This figure shows the level of perceived effectiveness and ease per sample (scale min. = 0, max. = 5). Each sample group average is shown as a thick black line. Error bars indicate the 95 % confidence intervals. R² indicates the fit of the trend line for sample group average of perceived effectiveness and ease.

Discussion

Based on the collected data, perceived effectiveness and ease seems to have slowly and linearly increased over the course of the design thinking projects. Hypothesis 4a, that perceived effectiveness and ease follows a U-shape throughout a project, was therefore rejected in favour of the null-hypothesis. Perceived effectiveness and ease actually appears to be mainly influenced by the amount of time a team spends working on a design thinking project.

Hypothesis 4b

An individual’s perceived effectiveness and ease in the application of design thinking carries over to new projects and teams.

To analyse Hypothesis 4b, individuals in the APEn and APEe sample groups were compared. It was assumed that during the first design thinking project, (APEn) novice participants had developed their perceived effectiveness and ease to some extent. It could subsequently be assumed that the participants exhibited higher levels of creative confidence during their final design thinking project (APEe).

In the previous Figure 4.14, the average levels of perceived effectiveness and ease for both the APEn and APEe groups were represented by the thick black line. As has been described in the previous section relating to

Hypothesis 4a, perceived effectiveness and ease in the application of design thinking seems to have increased linearly throughout a project. In Figure 4.14, the experienced sample group (APEe) seems to have started off with a

slightly higher base level of perceived effectiveness and ease compared to the novice sample group (APEn). The slope of the fitted trend line for the experienced group therefore appears less steep than that of the novice group.

Figure 4.15 shows a visual comparison of the individual differences of means for the joint perceived effectiveness and ease measure. Individual-level data for both the novice and the experienced project were needed for this

statistical comparison, which left a total of 22 cases to be analysed. An

examination of Figure 4.15 uncovered that for 16 out of the 22 participants, the level of perceived effectiveness and ease was higher for the APEe project than the APEn project. An independent samples t-test revealed that in five of these instances, this difference was significant at the .05 level (2-tailed). In one case it was significant at the .01 level (2-tailed). A closer examination of these significant cases revealed that the disciplinary background of all these participants is in management-related subjects. For the remaining six cases, the level of the average perceived effectiveness and ease seemed to have declined for the APEe project. In one case (participant #20) this difference was significant at the .01 level (2-tailed). The disciplinary background of Kathryn (participant #20) is in business and music.

Figure 4.15: Differences in Average Perceived Effectiveness and Ease of Experienced and Novice Design Thinkers

The bars represent the differences in perceived effectiveness and ease between

experienced and novice design thinkers (APEe minus APEn). Continuous data was available for 22 cases. Bars highlighted in light blue represent significant differences at the .05 level (2-tailed). Bars highlighted in dark blue represent significant differences at the .01 level (2-tailed).

Discussion

Overall, there appears to be some evidence that at least parts of individual perceived effectiveness and ease in the application of design thinking carries over to new projects and teams. However, in the current dataset, only six out of 22 cases showed a significantly higher level of perceived effectiveness

and ease in their final design thinking project. Therefore, Hypothesis 4b, that an individual’s perceived effectiveness and ease in the application of design thinking carries over to new projects and teams, was rejected in favour of the null-hypothesis. In six out of 22 cases, perceived effectiveness and ease had decreased from the first to the last project. One potential reason for this might be a form of over-confidence in some novices, which resulted in a drop of perceived effectiveness and ease between the first and the final project, once they had adjusted their level of confidence in relation to the other participants.

Hypothesis 4c

Teams comprised of individuals with high levels of perceived effectiveness and ease achieve a better final performance.

To test Hypothesis 4c, the joint variable of perceived effectiveness and ease was plotted against the standardised mean performance of the APEn and APEe teams in Figure 4.16. The scatter plot did not reveal any obvious linear correlation.

A Pearson product-moment correlation analysis confirmed that there is no significant linear correlation between the average perceived effectiveness and ease per team and its standardised mean performance. It should be noted that the joint variable of perceived effectiveness and ease did not show great variation across the different teams (M = 3.331, SD = .436).

Figure 4.16: Scatter Plot of Standardised Mean Performance and Perceived Effectiveness and Ease per Team

The analysis was extended to see if the levels of perceived effectiveness and ease during specific weeks correlate with the final team performance. This allowed the researcher to identify specific points in time during the observed design thinking project in which perceived effectiveness and ease might have had a stronger impact on the final performance than others. For the APEn

sample group, three weeks were identified as showing a significant correlation between these two variables. The levels of perceived

effectiveness and ease for week 7 (2 Dec, r = .924, p < .05), week 9 (16 Dec, r = .980, p < .01), and week 10 (7 Jan, r = .918, p < .05) all show a high significant correlation with standardised mean performance. For the APEe

sample group on the other hand, no specific weeks were identified to show a significant correlation with final team performance.

Standardised Mean Performance

Discussion

No significant linear correlation between the joint variable of perceived effectiveness and ease and final performance could be identified. Therefore, Hypothesis 4c, that teams comprised of individuals with high levels of

perceived effectiveness and ease achieve a better final performance, was rejected in favour of the null-hypothesis. For the APEn group, three individual weeks were shown to significantly correlate with final performance. These three weeks seem to coincide with the peak in the amount of iteration per week (refer to Figure 4.5 in Section 0). It should also be noted that the joint variable of perceived effectiveness and ease did not show great variation between the individual teams. This indicates that overall, teams exhibited fairly similar average levels of perceived effectiveness and ease,

independent of their final performance.