Documentation and Reflection

Documentation and reflection to produce design principles is the fourth and last phase of design-based research. After completing the three initial phases of a design-based research project, large amounts of data should normally be available. These data are usually suitable for both qualitative and quantitative analysis and the researchers will typically vary the methodological approach according to requirements and suitability. The main aim of quantitative research is to perform precise measurement of things, for example the behavior of students, achieved knowledge, opinions, or attitude.

Quantitative data analysis is used to answer questions related to how much, how often, how many, when, and who and the use of surveys is considered dominant for

quantitative researchers (Cooper & Schindler, 2008). Furthermore, quantitative research is associated with numbers as the unit of analysis and it is often associated with a specific focus such as theory testing. The researcher must maintain a distance from the research to avoid biasing the results since the whole point of quantitative research is to produce objective numerical data that exist independently of the researcher

(Denscombe, 2007).

However, in this doctoral project I had most focus on qualitative methods, although it is possible to quantify some of the data from the surveys that were conducted since a mix between free-response questions, dichotomous questions, multiple-choice questions,

checklists, and rating questions with the Likert scale were used. Qualitative research includes a set of interpretive techniques where the goal is to describe, decode, translate, and otherwise understand phenomena in the social world that occur more or less naturally (Cooper & Schindler, 2008). Qualitative research produces large amounts of data in non-standard format (Denscombe, 2007), and this was also the case in my doctoral project. Having been through the first three phases of design-based research as described in the previous sections, I had a lot of qualitative data in different formats. This included for example, video clips, audio recordings, completed questionnaires, interview notes, project documents, training materials, design models, and fully developed courses.

With all this data available, it was natural to begin to work with RQ3 concerning requirements for practical realization, i.e. what are the key requirements for a

concurrent design approach to the design of customized e-learning for corporate clients? In order to answer RQ3 I used qualitative data analysis and coding. Since, the data were in several different formats I decided to handle the data using NVivo 9, which is qualitative data analysis software from QSR International. My approach to the coding process was inspired by Bazeley (2007) and Saldaña (2009), in that I carried out iterative coding in three cycles.

In the first cycle a descriptive initial coding was used to name the ideas in the data and represented them as nodes in NVivo 9. A total of 74 nodes were identified and each node refers to ideas and works that help aggregate extensive and varied raw data into a brief summary format. In the second cycle I did axial coding to extend the analytic work from the first cycle. During this process all the nodes from the first cycle were

categorized in relation to what each node actually concerned. After completing this cycle I had five categories (i.e. adaption to the surroundings, stakeholders, activities, infrastructure, and results) containing their respective nodes. In the third cycle the nodes were reduced to a total of sixteen principles and I used data associated with the different nodes to describe each of the sixteen principles in detail.

Figure 6: Use of NVivo 9 for Qualitative Data Analysis and Coding.

Figure 6 shows a screen dump where NVivo 9 is used for qualitative data analysis and coding. In this case the node Intermediate results is selected in the List View, while the Norwegian text in the Detail View below, which is the result from a survey carried out during the project, states the importance of having fairly detailed requirements for intermediate results that should be completed at certain times while the project is in progress. The principle of requirements for intermediate results and the corresponding category (Results) was later identified based on this node. This example shows that the goal and fundamental reasons for performing any project is to pursuit results and it is important to know what results we want to achieve both while the project is ongoing and when it is completed.

This section is about documentation and reflection to produce design principles. Towards the end of the doctoral project I conducted qualitative analysis of data collected over several years in order to identify the principal characteristics of concurrent e-learning design. In this context were sixteen principles of concurrent e- learning design identified. These principles were grouped into five categories and they were described in detail in research paper P6. Furthermore, qualitative data analysis techniques were used in conjunction with other research approaches (i.e. literature review and design science), when the prescriptive approaches for distributed concurrent design was finally identified and described in research paper P5. In total this constitutes research contribution number 4 (C4), i.e. requirements and guidelines for concurrent design of customized corporate e-learning; which includes 16 principles of concurrent e-learning design and some additional prescriptive approaches that should be considered for distributed workspaces. This contribution resulted from RQ3, which was intended to figure out what are the key requirements for a concurrent design approach to the design of customized e-learning for corporate clients.