Bloom’s Taxonomy for Approximating Knowledge Utility

The main goal of our research is to develop automatic ways to generate exploration paths that can expand a user’s domain knowledge. Our work opens a new avenue which looks at the knowledge utility – expanding one’s domain knowledge while exploring the data graph. Knowledge utility measures to what extend the user expands his/her domain knowledge while exploring a path in a data graph of a particular domain. To approximate knowledge utility we need a reliable and applicable metric that approximates the changes in the user’s cognitive knowledge after exploring a path in a data graph. However, the assessment of learning in the field of information retrieval is considered to be rare [133]. According to [133], there are two main approaches have been used to assess learning after searching a topic:

The first approach uses a set of questions to the search topic, also referred as question answering approach [134]. This approach is considered to be the most direct and abstract way for assessing learning by asking the users questions about a particular topic after completing a search session about that topic. This approach is similar to the schema activation technique used for assessing how students expand their knowledge after reading text [135]. To assess a student’s knowledge of a target domain concept, the student is asked to name concepts in a schema activation test before and after reading a text. Comparing students answers before and after reading indicated the knowledge gain.

The second approach uses concept maps [136]. The users at first are asked to draw a concept map (in the form of set of nodes and links) about a topic prior their search session, search about that topic, and then re-draw the concept map of that topic. The differences between nodes and links of the concept maps drawn before and after the search session is used to assess the knowledge gain.

In our work, we are looking to an easy yet reliable way for approximating knowledge utility of an exploration path. For this, we adopt the well-known and established taxonomy for classifying and approximating knowledge introduced by Bloom (known as Bloom’s taxonomy) [27], as a question answering approach to approximate the knowledge utility of an exploration path. Recent research in the field of information exploration and search has suggested Bloom’s taxonomy as a reliable tool for assessing learning [4, 137]. The taxonomy identifies a set of progressively complex learning objectives that can be used to assess learning experiences over information search tasks, and offers a means of assessing the depth of learning that occurs through search [137]. The taxonomy has also been utilised to assess and support learning in the context of ‘search to learn’ in exploratory search tasks and applications [4].

The original Bloom taxonomy was introduced in 1956 as a tool for assessing six major cognitive categories in human cognition [138]. The six categories were Knowledge (recalling information) , Comprehension (understanding and interpretation of a situation), Application (using concepts in a new application), Analysis (separating concepts into structures), Synthesis (using concepts to build a structure or a pattern), and Evaluation (making judgments about solutions). The categories were ordered from simpler (Knowledge – knowledge of a terminology and some facts about the terminology) to more complex cognitive categories (i.e. Evaluation – evaluation in terms of evidence) [27]. Hence, a person who is functioning at the one cognitive category has also mastered the lower level cognitive categories.

A revised version of Bloom's Taxonomy was presented in 2002 [27]. It changed the original names of the six major cognitive categories from nouns into verbs since learning is described as an active process [27]. The new terms are remember (retrieving relevant knowledge from long-term memory), understand (determining the meaning of instructional messages), apply (carrying out or using a procedure in a given situation), analyse (breaking material into its constituent parts and detecting how the parts relate to one another and to an overall structure or purpose), evaluate (making judgments based on criteria and standards) and create (putting elements together to form a novel, coherent whole or make an original product). Among these cognitive categories, remember and understand are the two cognitive categories that are directly related to browsing and exploration activities. The remaining categories require deeper learning activities, which usually happen outside a tool, in our case a semantic data browser or a search engine, and hence will not be considered in approximating the knowledge utility of an exploration path. The remember cognitive category is about retrieving relevant knowledge from the long-term memory, and includes two steps: (i) recognition (locating the knowledge) and (ii) recall (retrieving it from the memory) [27]. For example, when we see a particular musical instrument such as Piano, we might remember a

famous musician who plays Piano (e.g. Sergei Rachmaninoff31_{), a musical performance} where Piano was played (e.g. Symphony No.232_{) or remember instruments related to} Piano (e.g. Grand Piano).

The understand cognitive category is about determining meaning, from which the most relevant cognitive processes from the understand category to a semantic browser are two cognitive processes: the cognitive process categorise (determining that an entity belongs to a particular category – e.g. the musical instrument Guitar belongs to the String Instrument category) and the cognitive process compare (detecting similarities between entity – e.g. the musical instrument Folk Guitar is similar to the musical instrument Classical Guitar) [27].

Approximating knowledge utility of an exploration path. We use a schema activation technique in a question answering format for assessing how users expand their knowledge. To assess the user’s knowledge of a target domain concept, the user is asked to name concepts that belongs to and are similar to the concept. The schema activation test is conducted before an exploration (i.e. pre-test) and after an exploration (i.e. post-test), using questions related to the cognitive processes of remember, categories, and compare of Bloom’s taxonomy described in Section 2.7.1:

 Q1 [remember] What comes in your mind when you hear the word X?;  Q2 [categorise] What categories does X belong to?;

 Q3 [compare] What entities are similar to X? .

The number of accurate concepts named (e.g. naming an entity with its exact name, or with a parent or with a member of the entity) by user before and after exploration is counted, and the difference indicates the knowledge utility of the exploration. For example, let us consider the following question for approximating the knowledge utility on the cognitive process compare about the musical instrument Biwa:

What musical instruments are similar to Biwa ?

If a user could name correctly two musical instruments similar to the musical instrument Biwa (Q3) before his/her exploration and then the user could name correctly six names of musical instruments similar to the instrument Biwa after his/her exploration, then the effect of the exploration on the cognitive process compare is indicated as 4 (i.e. as a result of the exploration the user learned 4 new similar musical instruments to the musical instrument

31_{https://en.wikipedia.org/wiki/Sergei_Rachmaninoff}

Biwa). If a user named one instrument after his/her exploration (i.e. knowledge utility is -1), in such cases the knowledge utility equal is to zero.