Information Needed and Alternative Methods and

This section reviews the information needed to address the research objectives and then summarizes the basis for selection of Dervin’s Sense-Making Approach over

alternatives.

3.2.2.1 Information Needed to Address the Research Objectives

Review of the research objectives and previous pre-testing and pilot testing established what information and evidence is needed to address the research objectives. This

section discusses the information required to address the research objectives and to support the overarching goal of the study.

General RO: To empirically describe faculty’s articulations of their cognitive question-

asking behavior during early conceptual instructional design. Refer to RO1 through RO4 for required information to address this General RO.

RO1: To explore questions faculty ask during early conceptual instructional design. Required information for RO1: Details of participants questions/concerns and question

context were needed to empirically describe faculty’s articulations of their cognitive question-asking behavior during early conceptual instructional design. This was necessary as a starting point for improvement of design support and to support future research.

RO2: To identify uses that faculty associate with the questions they ask during their

conceptual instructional design experience (Example: Did it help or hurt?).

Required information for RO2: Identification of uses faculty associate with their

questions required obtaining detailed information on question context, including why questions were asked and why participants had concerns or needs. ‘Why’ is important for understanding complex situations requiring critical decision making. Uses provide insight on problems.

RO3: To identify patterns of behavior in the descriptions faculty provide about their

Required information for RO3: Identification of patterns of behavior required having

adequate detail about the cognitive question-asking behavior of participants to be able to compare what people did, thought, asked about, struggled with, succeeded at, etc., across all participants. Designing is a complex behavior. More than one or two elements of behavior are required to describe designing behavior adequately to determine

specific means of design support.

Data from Sections One, Two, and Three of the interview protocol (Steps, Questions, Question Context) was necessary as a basis for deductive and semi-inductive content analysis to identify patterns of behavior. General patterns of behavior were also

identified through analysis of big picture data collected in Section Five of the interview protocol, such as learning. However, big picture data is based on very broad questions with little or no additional context, and may be less reliable.

RO4: To explore what faculty feel is important about question-asking during

instructional design. For example: How does it make a difference in the quality of instruction as compared to when they overlook or leave out questioning? What are the most important things faculty want to share about question-asking, such as critical issues, something they are confused about, or a question they wish they had asked?

Required information for RO4: Big picture data illustrating what faculty feel is

important about question-asking during instructional design is an additional contextual and personal component of participants’ instructional design experience. The open- ended big picture questions in Section Five of the interview protocol were required to obtain information that was used to address RO4.

Overarching Goal: To provide a basis for future research on interventions to aid

designers from all disciplines with question-asking during design.

Required information to support the Overarching Goal: Question and question

context data are required as a comparison platform for questions asked in other design disciplines. An ad-hoc design mapping analysis was used to explore transferability of questions asked by participants to other design domains. Refer to section 3.7 and Appendix O for details.

3.2.2.2 Dervin’s Approach versus Alternative Methods and Approaches

There were a lot of unknowns involved in the issues investigated in this study. That ruled out most traditional closed-ended such as surveys that target specific variables and criteria, and may be associated with certain elements of behavior. Experimental approaches wouldn’t fit for similar reasons – there are far too many variables to formulate realistic hypothesis.

Dervin’s Sense-Making approach fit most of the requirements to address the ROs. Dervin’s approach was designed to provide data on users’ cognitive activities. It is a proven framework for user-based research that focuses on identifying questions, concerns, uses, and patterns of user behavior.

However, there are some limitations. Dervin’s approach apparently has not been used to investigate a small part of a larger behavior (i.e. early conceptual instructional design), and has not investigated a complex iterative behavior such as designing. Live interviews about complex design situations can be very time-consuming, limiting the research scope. A case study could result in more detailed elicitation of data across a

longer period, resulting in more data on users’ actual questions. But a case study in a dissertation research timeframe would have to include very few participants, and might not provide an adequate variety of data. Critical Incident Technique is a good means of eliciting depth on cognitive behaviors in a short timeframe, but the increased focus on key events could result in loss of necessary details of designing behavior.

Use of timeline interviews for complex behavior could also have unanticipated

complications. Participants sometimes have difficulty expressing design experiences in terms of the interview questions. Document analysis could provide a variety of data without requiring face-to-face interviews, but would be extremely time consuming, data access could be problematic, and it’s likely that context would be too narrow.

Overall, Dervin’s approach appeared to be the best fit for investigating question-asking behavior during early conceptual design. It can effectively elicit the required data.