Chapter 4 : A Framework for an Adaptable and Personalised E-Learning System
4.4 The Knowledge Extraction Model
4.4.2 The Ranking Phase
4.4.2.1 Categorising Learning Outcomes Statements
The suitability of the contents of the selected website should be evaluated to ensure that they
fit the learner’s needs. Matching the content to learning outcomes of curricula is very important when assessing the validity of the selected websites. Basically, learning outcomes are
statements of what a student is expected to know, understand and/or be able to demonstrate
after the completion of the learning process (Kennedy, 2006). Likewise, Mclean and Looker
(2006) described learning outcomes as explicit statements of what we want our students to
know, understand or be able to do as a result of completing our courses. Bloom’s Taxonomy
(Bloom, 1956), is one of the most important and popular frameworks for developing learning
outcomes in order to help students understand what is expected of them. The following sections
explain how to identify learning outcomes, which include Bloom’s Taxonomy and Nouns and
Verbs Extractor.
4.4.2.1.1 Bloom’s Taxonomy
The Bloom’s taxonomy can be used to identify a set of learning outcomes. Typically, a learning
outcome contains a verb and a noun. In one hand, the verb describes the intended cognitive
level of the Bloom’s taxonomy and includes Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation. In the other hand, the noun describes specific subject that
student wants to learn. For example: basic structure of the genetic material; nature of
chromosomes and the organisation.
Thus, a set of action verbs was used based on the Bloom’s taxonomy to analyse the learning
outcomes. Furthermore, the Bloom taxonomy identified a list of suitable action verbs into six
levels representing the following cognitive skills: Knowledge, Comprehension, Application,
Analysis, Synthesis, and Evaluation. For example, action verbs such as define, describe and
identify are used to measure basic levels of cognitive skills in comprehension, while action
76
measure basic levels of the application cognitive skills. Table 4-1illustrates a set of action verbs
associated with the intended cognitive level of the Bloom’s original taxonomy.
Table 4-1 Set of action verbs associated with the intended cognitive levels of the Bloom’s original taxonomy (Bloom, 1956)
In 2001, a former student of Bloom’s, Lorin Anderson, and a group of cognitive psychologists,
curriculum theorists and instructional researchers, and testing and assessment specialists
published a revision of the Bloom’s Taxonomy entitled A Taxonomy for Teaching, Learning,
and Assessment. The revision updates included significant changes in terminology and
structure. In the revised framework, “action words” or “verbs”, instead of nouns, are used to
label the six cognitive levels, and three of the cognitive levels are renamed. Figure 4-12
illustrates the differences between Bloom’s original taxonomy and the 2001 revised one. Cognitive Level Sample verbs to use in Writing Intended Student learning outcomes
knowledge Define, Identify, Name, Recognize, Retrieve, Duplicate, List, Recall, Reproduce, Tell.
Comprehension Calculate, Conclude, Predict, Discuss, Explain, Classify, Clarify, Translate, Reproduce, Exemplify.
Application Carry out, Demonstrate, Solve , Illustrate, Use, Classify, Execute, Implement, Practice, Utilize.
Analysis Discriminate, Compare, Differentiate, Examine, Infer, Attribute, Contrast, Distinguish, Select, Formulate.
Synthesis Check, Judge, Monitor, Critique, Reconstruct, Defend, Verify, Detect, Coordinate, Dispute.
Evaluation Construct, Design, Compose, Produce, Improve, Create, Invent, Generate, Plan, Combine.
77
Figure 4-12 Differences between Bloom’s original taxonomy and the 2001 revised one Table 4-2 A set of action verbs associated with the intended cognitive level of the revised Bloom’s taxonomy (Anderson et al., 2001)
4.4.2.1.2 Nouns and Verbs Extractor
In order to identify learning outcomes statements, first, two types of dictionaries are used. The
action verbs dictionary that contains the action verbs that have been manually defined based
on the Revised Bloom’s taxonomy (Anderson et al., 2001) and the topic name synonym
Noun Form Verb Form
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Remembering
Understanding
Applying
Analyzing
Evaluating
Creating
Original Taxonomy (1956)
Revised Taxonomy (2001)
Cognitive Level Sample verbs to use in Writing Intended Student learning outcomes
Remembering Define, Identify, Name, Recognize, Retrieve, Duplicate, List, Recall, Reproduce, Tell.
Understanding Calculate, Conclude, Predict, Discuss, Explain, Classify, Clarify, Translate, Reproduce, Exemplify.
Applying Carry out, Demonstrate, Solve , Illustrate, Use, Classify, Execute, Implement, Practice, Utilize.
Analysing Discriminate, Compare, Differentiate, Examine, Infer, Attribute, Contrast, Distinguish, Select, Formulate.
Evaluation Check, Judge, Monitor, Critique, Reconstruct, Defend, Verify, Detect, Coordinate, Dispute.
Creating Construct, Design, Compose, Produce, Improve, Create, Invent, Generate, Plan, Combine.
78
dictionary whose terms are retrieved from the ontology. Second, as previously mentioned in
chapter 2, the Stanford CoreNLP (Manning et al., 2014) is used in this research as a tool to
parse learning outcome statements and generate semantic representation in XML format,
including PoS tagging text and a typed dependency representation (see Figure 4-13).
Figure 4-13 Stanford CoreNLP
In addition, the XML format shows data in a tree structure format where tags are assigned for
each word. Theses tags include <root> (governor of sentences), <lemma> (word form), <
sentence id > (sentence number), tokens, word, PoS, parser, dependencies etc. as shown in
Figure 4-14. The Parser tag is very important as it represents the context-free phrase structure
grammar representation, which is used to describe the structure of sentences and words in
natural language that provide the structure derivations of the grammar.
Learning outcomes statement
Parsing Stanford
Parser
XML file
(Noun, Verb, Adverb, Adjective) (nsubj ,nmod advmod, dobj)
Input Conversion & Pre-processing PoS Dependency relation
79
Figure 4-14 XML format generated by Stanford CoreNLP
Consequently, from the product of the Stanford parser, all the tokens with the verb tag were
extracted and then check if it matches with a verb from the action verbs dictionary that contains
the action verbs that have been manually defined based on the Revised Bloom’s taxonomy
(Anderson et al., 2001).
A set of rules is used to identify learning outcomes statements by searching the pattern token
in the tagged verb in the action verbs dictionary that have been manually defined based on the
Revised Bloom’s taxonomy in order to identify learning outcomes statement. Table 4-3
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="CoreNLP-to-HTML.xsl" type="text/xsl"?> <root> <document> <sentences> <sentence id="1"> <tokens> <token id="1"> <word>define</word> <lemma>define</lemma> <CharacterOffsetBegin>0</CharacterOffsetBegin> <CharacterOffsetEnd>6</CharacterOffsetEnd> <POS>VB</POS> <NER>O</NER> <Speaker>PER0</Speaker> </token> </tokens>
<parse>(ROOT (S (VP (VB define)))) </parse> <dependencies type="basic-dependencies"> <dep type="root"> <governor idx="0">ROOT</governor> <dependent idx="1">define</dependent> </dep> </dependencies> <dependencies type="collapsed-dependencies"> <dep type="root"> <governor idx="0">ROOT</governor> <dependent idx="1">define</dependent> </dep> </dependencies> <dependencies type="collapsed-ccprocessed-dependencies"> <dep type="root"> <governor idx="0">ROOT</governor> <dependent idx="1">define</dependent> </dep> </dependencies> </sentence> </sentences> </document> </root>
80
illustrates the rules that are used to assign learning outcomes based on action verbs in the
Bloom’s taxonomy.
Table 4-3 Rules to assess learning outcomes statement using Bloom’s taxonomy