Nodes: A “node” in ENA space represents each epistemic frame

element. The location of these nodes remains constant in any network model generated with a given ENA set. The size of the nodes in each model are visual cues about the relative strength of the connections made to that

epistemic element because it is proportional to the sum of the strengths of its connections. Said another way, a larger node in a network model, compared to others, can reflect either a) more connections, or b) fewer, though very strong, connections to that node.

In order to compare the richness, density and “location” of connections made in discourse to epistemic frame elements in groups at different ranked uses of APT, I conducted ENA and interpreted results to determine what, if any, differences existed in the types of connections among epistemic elements were made by groups that used differing proportions of APT in their design team meeting discourse by conversation (i.e., room) and design cycle (i.e., over time). Additional analyses were conducted to test whether any group-level characteristics were associated with these differences (e.g., EQUALITY; GRP_SELFEFF). Network models were generated and tested using a

comparative structure based on the quartile ranking approach described above (see 3.3.1- 3). Table 3.15, below, describes the types of comparative variations tested using ENA. Table 3.15

Summary of structures used to compare groups of differing quartile rankings

Variable Characterization Included Rank(s) Variable Characterization Included Rank(s) High 3 vs. Low 0

High 3 vs. Non-high 0, 1, & 2

Higher 2 & 3 vs. Lower 0 & 1

Non-low 1, 2 & 3 vs. Low 0

After generating network models, I then used statistical supports embedded in the ENA tool to conduct and interpret t-tests to determine if the observed differences in the relationships between groups’ use of APT and epistemic frame elements in the epistemic network models were statistically significant (α-level, p<.05) on the x and/or y axes. Additionally, I interpreted Cohen’s d, which is a statistic used to determine the effect size (i.e., the strength of a phenomenon of interest) of differences, which is the standardized difference between two means25. Therefore, the use of this statistic in my analysis represented a measure of the strength of difference in the mean equiload projections comparing any two network models in ENA space.

Finally, because mean network models (the types of models primarily used in this study) tend to have high densities of connections, they can look very similar and

differences can be difficult to interpret. Therefore, for those models found to be

significantly different, I generated an additional network model – a “subtracted equiload” – to further highlight and reveal differences in the nature of each group’s discourse

25 _{Cohen’s d is generated, along with the t-tests, as part of embedded statistical calculations in the ENA}

regarding the development of their epistemic frame of professional practice in

engineering. To generate these comparative models, the ENA tool “subtracted” the edge weights of one network model from another, one edge at a time. This resulted in a network model with negative values associated with one color and positive values associated with another. This is a visual representation of how any two models were different in terms of the relative strengths of their respective connections.26

3.4.3 Research Question Three

Do students have a higher probability of reporting a positive change in their attitudes and perceptions toward engineering when they are in a group, or groups, that engage in higher proportions of academically productive talk in their discourse during design meetings (vs. those in a group, or groups, that engage in lower proportions)? What, if any, individual- and/or group-level characteristics are associated with a positive change in disposition toward engineering?

I answered my third research question using quantitative data from students’ survey responses, and qualitative data generated from my analysis of APT use in design meeting discourse. To answer this research question, focused on a positive change in a student’s attitudes and perceptions toward engineering after playing Nephrotex, I first calculated a continuous variable of the proportion of discourse that was APT for their group in design cycle one (APT_EXPERIENCE1) and design cycle two

(APT_EXPERIENCE2). I then analyzed student's pre- and post-survey responses to identify if there was a positive change in their post-survey scores to generate a code for

26 _{For instance, if two equiload projections (i.e., means of network models) are generated - one red and}

another blue - when they are “subtracted" in ENA, the blue lines that remain in the model would indicate that those connections were stronger, on average, than in the red equiload model, and visa versa.

each player (CHNGE_CONFID, CHNGE_SELFEFF, CHNGE_COMMIT). Finally, I fit a series of logistic regression models to test the significance (α-level, p<.05) of

participation in groups with higher APT use, controlling for particular individual characteristics (e.g., pre survey scores, level of experience, etc.), and group

characteristics (e.g., equality of contribution, gender balance, etc.), on the probability that a positive change in a student's self-efficacy, confidence, and commitment toward

engineering would occur. A sample model for this analysis, for each player, followed this format: log p(CHNG _ SELFEFF =1 p(CHNG _ SELFEFF = 0 ! " # $ %

& = β0+β1APT _ EXPERIENCE1+ β2APT _ EXPERIENCE2 + β3PRE _ SELFEFF + β4NOVICE + β5EQUALITY +ε

3.5 Summary

In this chapter I presented details about three research questions, their related hypotheses, and the research design that seeks to provide empirical answers to these questions. As indicated, prior to conducting the analyses described above, the first phase of research involved applying, and testing the inter-rater reliability in applying, the conversational coding scheme to the discourse data. Once inter-rater reliability was established, I proceeded with the analytic plan as outlined.

Addressing research questions one and three required the use of logistic

regression to determine: (1) what, if any, APT moves are associated with the presence of unique epistemic frame elements in group discourse during design team meeting; and (2) whether students have a higher probability of reporting a positive change in their attitudes and perceptions toward engineering when they are in a group, or groups, that engage in higher proportions of Academically Productive Talk in their discourse during design meetings (vs. those in a group, or groups, that engage in lower proportions). Addressing

research question two required the use of Epistemic Network Analysis, along with the qualitative review of the discourse data, to determine whether there is a difference in the connections among different epistemic frame elements that groups make in their

discourse with respect to Academically Productive Talk.

In accord with the research design presented in this chapter, I present the results of data analysis and a discussion for each research question in the subsequent three chapters. I address Research Question 1 (RQ1) in Chapter 4, Research Question 2 (RQ2) in Chapter 5, and Research Question 3 (RQ3) in Chapter 6.

Chapter 4