Task Developments

The task was developed using a spiral model that consists of four main processes: identification, structuring, organization and evaluation distributed into three phases. This is illustrated in Figure 4.8 and Table 4.6. These questions were answered iteratively throughout different periods of this research.

152

Figure ‎4.8: SQL Pattern’s Design Phases

Different methods are used to design SQL instructional materials. The analysis of data gathered during qualitative and quantitative studies of SQL acquisition in achieving objective 1 guides objective 2 designs and development. Different instruments were used to collect data related to the design of instructional material for SQL learning (see Table 4.6).

Process Method Participants Aims

Phase-1 Identification SQL Learning model

Text mining knowledge in SQL text books Collecting examples and Structuring Literature on problem

based approach and instructional design

following the process students use in solving the query Organization Literature on

Checklist Matching the given problem to a set of patterns. Evaluation Case study 3 PhD

students Evaluate the use of SQL patterns in the process of solving a complex query.

Interview 3 students Reflect learners’ point of view on the usability of the patterns. Questionnaire 5

academic Reflect educators’ point of view on the usability of the patterns. Phase-2

Identification Novice observation IM2 and DB3 students

To find out how students approach SQL. Content analysis IM2 and

DB3 students

Evaluate students assignment and analyse the errors Structuring PLOP Interview 2 pattern

153

Focus group 5 pattern Writers

Workshop during Euro PLOP Organization Component level

design learners performance through … To find out the best to increase Evaluation Interview 10 students Collect novice feedback

Phase-3 Identification Expert observation 2 expert

students

To find out how expert students approach SQL.

Structuring Previous phase 2&1 checklist, component level design

Organization Same as phase 2 Evaluation Experiments,

questionnaire students 90 efficacy of SQL patterns on novice performance and satisfaction

Table ‎4.6: Research Methods Used for Research Question 2

The following subsection explains the design of the process of identifying and defining the patterns using text mining, observation of novices, and observation of experts.

4.6.3.1 Problem Solving Strategy Identification via Mining

SQL knowledge was identified through text data mining or knowledge discovery process. Mining concepts is the method used to discover knowledge from existing data available, solutions, or designs. According to Tan [241] text mining is:

“The process of extracting interesting and non-trivial patterns or knowledge from text documents”. (p. 65)

This method involves a review of database texts used to teach SQL. Thus, it is possible to identify common knowledge that relate to the core of SQL concepts.

To do that, the first decision was on the list of books that might be used. It was decided to use database textbooks which are available to the researcher. The text mining process is mainly based on natural language processing techniques, including text analysis, text categorization, information extraction, and summarization. The following steps were followed:

154

 Collect a set of database textbooks that are used to teach SQL concepts and are available in the university library.

 Identify the SQL misconceptions from both literature review and empirical research (chapter 2 and 5) and limit the text mining to those concepts.  Analyze the text and search for SQL-relevant knowledge.

 Identify declarative knowledge from database texts and categorize the knowledge as follow:

- SQL concepts definition and syntax “what”

- SQL concepts application purpose “Why” and “When” - SQL concepts application method “How”

 Extract the information from the text and structure it into the following form

- Highlight the “Problem” or “what” SQL concept.

- Identify the related “Context” in which SQL concept Problem is likely to occur. In addition, determine the concern or the forces that make such a problem difficult to solve.

- Find the “Solution” to the identified “Problem”: how the concepts should be applied, relevant syntax, and rules. - Illustrate the solution with appropriate examples, which

shows step-by-step how such a solution could be applied. - Highlight the impact of applying such a “Solution” to the

“Problem” in the identified “Context.”

The process of text mining provides an initial stage, delivering only a static understanding of how SQL pattern knowledge is presented in textbooks. The actual process by which SQL concepts are applied cannot be predicted without empirical evidence. Therefore, it is important to identify such knowledge through another approach such as observing and analysing students’ work in applying SQL. To enhance the observation, research on problem solving strategies is conducted at phase 1. The next section discusses: observation of

155

real novices solving SQL queries during labs and Examining examples and samples of SQL queries from students’ submitted assignments.

4.6.3.2 Problem Solving Strategy Identification through Observation

Researchers in the field of pattern identification agree that patterns ought to be identified with reference to design solutions through observation, rather than being constructed from theory. Therefore, cognitive aspects need to be taken into consideration. Instruction methods that apply what educators know about how students learn, remember, and use related skills can make the learnt subject meaningful and help students to perform better [62]. To achieve that, cognitive science suggested giving learners a problem and observing everything they do and say while attempting the solution. The cognitive task aims to formulate the process of SQL problem solving strategy. Thus, it consequent SQL knowledge identification through this kind of cognitive task or observation.

Time Participants Number

2009/10 Students registered in Information Management (IM2) course 17 2010/11 Students registered in Information Management (IM2) course 21 2010/11 Students registered Database

(DB3) course 15

Table ‎4.7: Time Spent with Novice SQL Learners

Strategy identification by means of learner observation helps determine how learners apply such knowledge. Unstructured observations were conducted on a period of two semesters (see Table 4.7).

The process of SQL strategy observation and subsequent pattern refinement was important to understand how novices solved SQL problems; i.e. the steps followed to arrive at a solution to the given problem. These include:

 Remembering:

156  Searching (Not Remembering):

o How was the unremembered but required knowledge obtained? For example, did they refer to textbooks or teaching materials? or did they search the net to find similar problems and related solutions?  Problem Solving:

o Was the required knowledge identified correctly?

o Was the knowledge correctly matched to the given problem context?

o Did they search for visual examples on the Web?

o Did they try different solutions? If so, why was a particular solution selected?

o How did they react to their errors?

Different questions designed to direct the unstructured observation shown in Table 4.8 to find out participants strategy in solving the given tasks.

Table ‎4.8: The Questions Used to Direct the Unstructured Observation.

Question Aim

How do students start solving the given task? Are there any initial questions about the context of the question?

Explaining how queries might be solved. To illustrate the steps learner followed in solving the given task.

What are the methods students use to get the required knowledge for solving the question?

Determining resources used to gather the required information.

General behaviour during problem solving What kind of questions students ask during

problem solving? such as: SQL content “What” questions, application of SQL structure “How” questions or if there are any other high-level question about “when” and “why”.

Studying learner decision in the applied solution.

What are the frequencies of the questions students ask? Are there any common misunderstandings or confusions in the task?

Do the available knowledge need to support by data models to enhance learners’ understanding.

Does available knowledge need to support by visual examples to enhance learners’ understanding.

157

Content analysis has been used as a method for analysing messages and communication that participants have been asked to produce. The results of this method are reported in Section 6.3.2. The next sections describe SQL knowledge identification through expert observation.

4.6.3.3 Problem Solving Strategy identification Through Expert Observation

This section describes how experts use their knowledge to solve problems. Moreover, it discusses the related cognitive activity that they perform during problem solving through employing a “loud-talk” protocol. This made it possible to identify gaps in the novice knowledge since it supported comparison.

The experiment was run on personal computer to oversee each subject’s approach, using a tool called SQL Pattern Based (SQLPB) that was developed by the researcher using Netbean platform. All the information about SQLPB is discussed in section 4.7. Additionally, Camtasia studio4 was used to record all participants’ action in the screen and record all their explanation. All participants’ trials and errors were recorded as well. Two participants were given a task (see Appendix F) to perform. They were MSc students at University of Glasgow. The observed experts had a long working experience of SQL. The task involved two questions as shown in Figure 4.9.

Q1: Give the titles of books that have more than one author.

Q2: Display the names of borrowers who have never returned a book late

Figure ‎4.9: Expert Observation Task

All the related tables were available from the SQLPB tool. They were asked to write the SQL query that would help them to solve the given problem. The collected data were analysed using protocol analysis. The findings of observation are often difficult to interpret, because it is not clear why the participants’ are behaving as they are. The collected data were analysed using content analysis. The data and result of this method is reported in section 6.3.3.

158

4.6.3.4 SQL Knowledge Structuring Design Methods

SQL patterns are knowledge and skills that exist in the expert’s mind and are continuously applied in related scenarios. They need to be formulated in a structured way by either the experts themselves or by others in the same field. Once these knowledge and skills have been documented and approved by the experts (may be called pre-patterns at this stage) then they must be given to different users to try. If different users accept these pre-patterns, then they can be called patterns and can be published. SQL patterns are aimed to facilitate learner’s knowledge and hence improve their performance. SQL patterns’ identification and structure requires some specific knowledge in educational instructional design research. In addition, knowledge of how the patterns are structured in other fields would support this quest. Chapter 3 presented this literature review in patterns’ structure in Architecture, SE and HCI.

The results reported in the development of section 4.6.4 of stage 1 guided this research to draw the outline of how SQL knowledge and skills might be delivered to learners.

The analysis of observation activities made it clear that instructional materials, such as their notes, did not guide students towards productive activities or to support effective problem solving. To help novices to achieve this level of expertise, the research proposes that the SQL patterns should be designed to:

 Highlight both the basic knowledge required to solve the problem and the

advanced knowledge.

 Provide step-by-step SQL visual examples of the SQL being applied.  Help in understanding the context of the problem. This depends on

learners’ previous schemata. Here, we tried to find out how such knowledge can be delivered.

 Support matching a problem to a solution in a simple format such as a

159

 The impact of applied concepts in such a problem context; for example,

the reasons behind the chosen approach.

Figure ‎4.10: Instructional Materials Elements.

Figure 4.10 shows what kind of concepts or knowledge need to be available and how such elements interact with each other and with learner schemata. More details of the level of knowledge presented in this Figure 4.10 are given in section 6.3.2. The next section presents the methods of patterns structure and organization.

4.6.3.5 SQL Instructional Materials Organization Methods

The aim of this part of the study is to propose an approach for the management of designed materials viz SQL pattern collections. The goals are to support novices in two different tasks: a) the selection of the correct pattern from the collection; and b) the understanding of the relationship between patterns in the collection.

The pattern, within Alexander’s [125] pattern language, are hierarchically connected to one another, in the way that higher level patterns are made up of lower level patterns, and these relationships are made explicit within the patterns. Many researchers highlighted the importance of organizing patterns and suggested one or more organizing principles. According to Salingaros [170]

160

“A loose collection of patterns is not a system, because it lacks connections” (p.154)

Chapter 3 elaborated on different patterns’ collection in Architecture, Software and HCI and their techniques in organizing and structuring patterns in patterns language. During SQL patterns design, the finding from the literature in patterns’ organization was analysed and tested in terms of the applicability to SQL patterns. In stage 1, it was decided to use a checklist approach to relate SQL query to the related patterns which is a new approach in bridging the SQL problems and SQL patterns. Thus, novices could select the correct set of patterns.

Scriven [242] described checklist as a list of factors, properties, aspects, components, criteria, tasks, or dimensions, the presence, referent, or amount of which are to be considered separately, in order to perform a certain task. After the evaluation phase in stage 1, the researcher studied other possible techniques in linking the patterns. Scaffolding techniques were taken into consideration as well.

Here, solving a query problem might require the application of more than one pattern. The collection of SQL patterns was inspired by Alexander’s [125] approach. Alexander’s pattern language is hierarchically built. Each pattern is connected to one another: higher level patterns are made up of lower level patterns, and these relationships are explicit within the patterns.

It was believed that using an approach that students were more familiar with might lead to a better understanding. Therefore, Component-level design approach was employed to present the graphical representation of level of patterns to understand the relationship between the given problem and the checklist, the checklist and the patterns and the relation between patterns in the collection. Modelling component-level design were applied in software engineering to translate the design model into operational software [243]. More details are given in section 6.5.2.

161 4.6.3.6 Evaluation research Methods

SQL patterns were subjected to various evaluations throughout the identification, design, and usability stages. Instructional objectives were written up prior to the design of 5 patterns. According to Dick and Carey’s [104] recommendations on instructional design, SQL patterns underwent a number of evaluations while in the developmental stages. These evaluations were used to “obtain data that [could] be used to revise [the] SQL patterns to make them more efficient and effective” [104]. These developmental evaluations consisted of an aesthetics and usability evaluation, subject matter expert (SME) evaluations and one-to-one evaluations. After completion of the SQL patterns’ structuring and organization, they were also field tested to determine the effectiveness of the SQL patterns that were explored, as discussed later in chapter 7.

4.6.4 Results Analysis

Stage 2 data analysis, as subsequent to the previous step and as indicated in Figure 4.10 above, consists of classifying the collected data under instructional design phases of the following four processes: identification, structuring, organization, and evaluation of SQL knowledge. Chapter 6 and chapter 7 report all the data collection and results analysis of these four processes. Research question 2 findings agree on structuring SQL knowledge as SQL patterns and employing the later as instructional material to help novice master SQL skills. The next section describes the process of SQL patterns evaluation.

In document SQL pattern design, development & evaluation of its efficacy (Page 152-162)