BCS HIGHER EDUCATION QUALIFICATIONS Level 6 Professional Graduate Diploma in IT. March 2013 EXAMINERS REPORT. Knowledge Based Systems

(1)

BCS HIGHER EDUCATION QUALIFICATIONS Level 6 Professional Graduate Diploma in IT

March 2013 EXAMINERS REPORT Knowledge Based Systems Overall Comments

Compared to last year, the pass rate is significantly better this year. Average marks are fairly consistent across questions 1,2, 3, 4, though, a little on the low side. Standard deviation results are fairly low across all questions.Thus, it may be concluded that the candidates appear to be fairly equated in their ability. However, the spread of results (range) is wide which would indicate that there were some high achieving candidates. Question 3 was a disappointment as it was worst answered, but has appeared in various forms on previous test papers. Question 5 was anticipated to attract the best performance since it has appeared on previous exam papers, and indeed, it was the best attempted. In general, candidates seem not to have prepared well for the paper, and have taken an instrumental approach in which they have revised previous exam questions and attempted to recreate the answers verbatim, as many answers across different test centres were quite formulaic and similar. Where variations in questions have been included, candidates have struggled.

Question A1

1. Data Mining has been capitalised on in the commercial world through the implementation of Business Intelligence Systems.

i. Discuss how Data Mining technologies have made the transition from research laboratory to business applications. Focus on the real business problems that data mining technologies address and the benefits perceived to have been realised. (20 marks)

ii. Discuss the implications of using intelligent data analytics to unfairly exploit data on user behaviour. (5 marks)

Answer Pointers and Marking Scheme:

Question 1.i: 20 marks. General distribution of marks according to salient features.

Candidates must demonstrate an appreciation of commercial usage of data mining technology, and how application tools have been developed for use by the non-technical business user.

Learning Outcomes/Assessment Criteria:

1. The learner will understand the main approaches used in AI problem solving. [1.2] 4. The learner will understand concepts and appreciate the significance of issues relevant to adaptive computing. [4.1, 4.3, 4.4]

6. The learner will know the major AI application areas and understand the AI techniques used within them. [6.1, 6.2, 6.3, 6.4, 6.6, 6.7]

7. The learner will know how AI technology has been used in real situations particularly on the Internet. [7.1, 7.2, 7.3,7.4]

(2)

Indicative solution

Data mining is often used in business, particularly on the web, to provide support for the “implicit web,” in which personal information about the user is indirectly

“discovered” from their normal interactions. Such valuable knowledge enables tailoring and personalization of services.

With the increased power of Web 2.0, in which users are generators of data and not just consumers, many new opportunities have emerged for technologies that can make value of the available data. Data mining as employed in business intelligence systems and data analytics are means by which Strategic management is

implemented. The knowledge derived from analysis of data using data mining helps a decision maker understand the nature of the problem better and supports better informed decision making. Data Mining can support the tactical detail about exactly how a strategy will translate to actions. In the business world, data mining is used extensively for personalized marketing, e.g. for profiling users and targeting advertisements to their interests and needs.

Question 1.ii: 5 marks. General distribution of marks according to salient features.

Candidates must demonstrate current awareness of an important application of artificial intelligence technology, data mining, and comment on its potential dangers in the context of privacy.

1. The learner will understand the main approaches used in AI problem solving. [1.1, 1.2]

5. The learner will know methods used in adaptive computing and understand how to apply them to problem solving. [5.1, 5.2, 5.3, 5.4, 5.5]

6. The learner will know the major AI application areas and understand the AI techniques used within them. [6.1, 6.2, 6.3, 6.4, 6.5]

7. The learner will know how AI technology has been used in real situations particularly on the Internet. [7.1, 7.2, 7.3, 7.4]

Data mining is now used in intelligent search engines on the web (Google), games, consumer robots with vision systems, handwriting reading in PDA, speech synthesis in word processors, intelligent tutoring, knowledge management, etc. As knowledge working grows, it is expected that such technologies would continue to become more prevalent and the potential for abuse of privacy thus grows.

A major factor impacting on the spread of data analytics is concerns with the manner in which data are collected, interpreted, distributed and manipulated to achieve a competitive advantage. The limits of what is acceptable to the consumer can often be pushed by companies seeking to capitalize on the value of data. As an example, employers these days can and do survey personal data on social network sites, such

(3)

as Facebook, to glean background data on prospective employees.

Answers could describe how monitoring of pages visited (times and location) can be used to develop a profile of users’ interests without their explicit consent to that information, which could be potentially sold on, or used to target marketing campaigns.

Answers must consider realistic and timely reasons, e.g. right now technological growth in terms of mobile systems enables data to be stored on location, which can be interpreted to the advantage of marketers.

EXAMINER’S COMMENTS

Question 1 has appeared on previous exam papers in various forms, so it was expected that performance would be better. However, students did not cope well even with the standard topic of data mining.

QA1(i): What was required

1. Data mining technology is now more reliable, cheaper and accessible.

2. Businesses now make use of large amounts of data and ways to exploit that data resource are in demand for better service and competitive advantage

3. Use of Internet and cloud means that data is being generated in large amounts and is relatively easy to gather in order to process.

4. Use of mobile technologies and the Internet of things contributes to the mass of Big data available on the Internet as an untapped resource.

5. Description of data mining and business intelligence required

6. Relationship between functions of data mining (patter generation, rule induction, clustering) and requirements of business (e.g. rules for business knowledge, market segmentation analysis) should be included.

QA1(i): What was included by students

1. Tend to focus on different hypothetical applications of data mining without specific cases being cited

2. Tend to confuse data mining with database access

3. Tend to confuse data collection of individual cases (e.g. from a customer) and processing of big data patterns and trends (without identification of individuals)

QA1(ii): What was required

1. Data analytics may be unreliable and too much dependence may be placed on them. 2. Checks must be in place to prevent unauthorised use/selling of data and avoid spam/fraud. 3. Consent should be gained before data are collected and used.

(4)

4. Process needs to be more transparent but business has vested interest in keeping data analytics processes opaque to third parties.

QA1(ii): What was included by students

1. Tend to look at the advantages and disadvantages of Data analytics 2. Tend to confuse data analytics with data mining

3. Tend not to consider privacy issues explicitly Question A2

2. AI technologies each offer different advantages and limitations; consider neural networks and case-based reasoning and

i. Explain why the chosen problem could not be solved by one technology alone and justify why a hybrid system must be used that would, in principle, overcome the limitations inherent in each technology. (10 marks)

ii. Describe how the two technologies could be used in combination to solve the chosen problem. (15 marks)

MODEL ANSWER POINTERS

Question 2.i: 10 marks. Five marks for each technology discussed.

Candidates must demonstrate an ability to critically evaluate the two AI technologies and be able to synthesise knowledge in to a cohesive vision of an application that integrates both technologies.

1. The learner will understand the main approaches used in AI problem solving. [1.1, 1.2, 1.3]

2. The learner will be able to critically compare and contrast various knowledge representation systems. [2.1, 2.2, 2.3, 2.4, 2.5]

Combining neural nets with case based systems must be addressed. Brief explanations of each technology should be included, with discussion of the

comparative merits of each. An ability to provide explanations for the outputs of the NN, to deal with applications in which knowledge is implicit, or an ability to learn and self-develop are possible benefits. Together they could be used for different aspects of a given task. The requirements of the task must be discussed, e.g. that it

embodies implicit knowledge, or that there are many existing examples of ‘cases’ available. Together, e.g. CBR can be used to train the NN, which will then be

employed for decision-making. CBR could be used to ‘explain’ or justify the solutions from the NN.

(5)

Candidates must be able to generate an hypothesised system architecture and produce appropriate representations of the knowledge embodied in the system, as well as be able to run a virtual simulation of a problem solving scenario.

3. The learner will understand methodological approaches to developing knowledge-based systems. [3.3, 3.4]

6. The learner will know the major AI application areas and understand the AI techniques used within them. [6.2, 6.3, 6.4, 6.5]

7. The learner will know how AI technology has been used in real situations particularly on the Internet. [7.1, 7.2, 7.3]

Candidates may choose any recognized knowledge representation formalism, e.g. rules, cases, frames, logic, etc. A clear explanation of the knowledge base must be provided from which it should be possible to understand how the case based-KBS would be utilised in combination with the NN to solve problems. It is important that the example shows collaboration between the two knowledge bases.

Question 2 is a variation of a question that has appeared on previous exam papers. Students were expected to perform well on this topic but they struggled to consider both AI technologies in combination.

1. Identify a specific problem and discuss its requirements in terms of AI technologies. 2. Explain the principles of hybrid systems and the potential to solve the chosen problem by combining benefits and compensating for limitations of each separate technology

QA2(i): What was included by students 1. Tend to not specify a particular problem

2. Tend to explain technologies separately but not relate to problem

(6)

4. Tend to confuse rule based systems with CBR and NN QA2(i): What was required

1. Focus on chosen problem and the process of problem solution

2. Demonstrate integration or co-operation between CBR and NN to solve problem 3. Highlight the dependency on hybridisation to solve the problem

QA2(ii): What was included by students 1. Tend to describe technologies separately

2. Do not focus on the steps of the problem solution process 3. Tend to be superficial and lack technical details

Question A3

3. AI technologies often emerge from insight to the way nature works. These ‘natural’ ideas are shaped in to forms that enable technological solutions to be developed. Consider genetic algorithms and

i. Describe the inspiration from nature that has most significantly contributed to its development. (10 marks)

ii. Specify a suitable problem of your choice (i.e. one that requires an intelligent approach) and explain how a genetic algorithm approach could be used to produce a solution. (15 marks)

iii.

Candidates must demonstrate an appreciation of the bases of much of AI in nature, and show direct contribution that nature has had on the development of AI

technologies – genetic algorithms.

1. The learner will understand the main approaches used in AI problem solving. [1.1,1.2]

2. The learner will be able to critically compare and contrast various knowledge representation systems. [2.1, 2.2, 2.4]

5. The learner will know methods used in adaptive computing and understand how to apply them to problem solving.[5.1, 5.2, 5.3]

Genetic algorithms – human gene evolution. Candidates should outline the principles of human gene evolution as they relate to the methods of genetic algorithms. The discussion should highlight similarities and differences, and evaluate the extent to which GA technology development can be traced to knowledge of the mechanisms of human evolution.

(7)

Candidates must demonstrate an understanding of the criteria that makes a problem or task suitable for a KBS approach, and show an ability to solve a problem using GA methods.

1. The learner will understand the main approaches used in AI problem solving.[1.1, 1.2]

4. The learner will understand concepts and appreciate the significance of issues relevant to adaptive computing. [4.1, 4.2, 4.3, 4.4]

5. The learner will know methods used in adaptive computing and understand how to apply them to problem solving.[5.1, 5.2, 5.3, 5.4, 5.5]

A clear explanation of the task must be provided from which it should be possible to understand that it requires an intelligent approach to solve it, and further, that the solution method is sound. It is important that the solution shows clear linkage to the natural phenomenon for its explanation. The task should be complex, require non-algorithmic methods, and be focused on optimisation or one that entails intelligent search. The answer should reference inheritance, mutation, selection,

and crossover. The concepts of population, generations, fitness, evolution, etc. should be mentioned appropriately.

Question 3 has appeared on previous exam papers as it covers some basic concepts of knowledge-based systems and AI. However, this question proved to be the most challenging for students.

1. Identify Theory of Evolution and DNA as relevant concepts from nature. 2. Describe Evolution and DNA processes as experienced in nature. 3. Relate GA processes to Evolution and DNA to show similarity

(8)

1. Failed to describe Evolution and DNA adequately or at all.

2. Tend to present GA processes in isolation without connexion to nature 3. Tend to confuse ANN with GA and focused on human neural system. QA3(ii): What was required

1. Identify suitable problem

2. Describe in detail step by step how GA could be applied to solve problem QA3(ii): What was included by students

1. Failed to specify suitable "intelligent" problem

2. Tend to provide superficial description of GA process

3. Failed to relate in detail the steps of GA with the problem requirements (i.e. no worked example)

Question B4

4. Knowledge elicitation involves modelling the knowledge used by an expert to solve problems. Consider an example application domain and construct a knowledge base for the domain by completing the following tasks:

i. Describe briefly general methods that could be applied to elicit the knowledge needed to solve a small complex problem. (10 marks)

ii. Present an example knowledge base using a knowledge representation formalism of your choice. Ensure that it is adequately annotated with a textual explanation of how it could be used to solve suitable problems. (15 marks)

Candidates must demonstrate an appreciation of how knowledge is elicited and acquired for a KBS.

1. The learner will understand the main approaches used in AI problem solving. [1.1, 1.2]

3. The learner will understand methodological approaches to developing knowledge-based systems. [3.1, 3.2, 3.3, 3.4]

5. The learner will know methods used in adaptive computing and understand how to apply them to problem solving.[5.1, 5.2, 5.3]

(9)

elicitation systems, machine learning, user maintenance should be discussed. Appropriate methodologies could be discussed too, such as KADS. Suitability of methods for task and situations should be explained.

Candidates must demonstrate an understanding of the criteria that makes a problem or task suitable for a KBS approach, and show an ability to solve a problem using KBS methods.

1. The learner will understand the main approaches used in AI problem solving.[1.1, 1.2]

5. The learner will know methods used in adaptive computing and understand how to apply them to problem solving.[5.1, 5.2, 5.3, 5.4, 5.5]

A clear explanation of the task must be provided from which it should be possible to understand that it requires an intelligent approach to solve it, and further, that the solution method is sound.

Candidates may choose any recognized knowledge representation formalism, e.g. rules, cases, frames, logic, etc. A clear explanation of the knowledge base must be provided from which it should be possible to understand how the KB would be utilised to solve problems.

For example, a rule based approach could be used and should include a set of self-consistent production rules: IF antecedent Then consequent

(10)

Question 4 has appeared many times in various forms on previous exam papers, and should be familiar to candidates. However, performance was disappointing.

1. Describe methods such as automated elicitation, knowledge discovery, observation, interviews, task analysis.

2. Identify example complex problem and relate methods to specific problem context. QA4(ii): What was included by students

1. Mentioned basic elicitation methods such as questionnaires, interviews, and observation but not automated methods (e.g. learning)

2. Did not specify specific problem or relate methods to context 3. Lacked details on presented methods.

4. Confused knowledge elicitation methods with inference methods (i.e. FW and BW chaining) QA4(ii): What was required

1. Identify specific problem context.

2. Generate small knowledge base with sample knowledge (e.g. using rules, frames, cases, logic)

3. Walk-through problem solution process with the knowledge base as it executes to solve the problem.

QA4(ii): What was included by students

1. Tended not to specify detailed problem context.

2. Tended to provide few knowledge items for the knowledge base (e.g. only 3 or 4 rules) 3. Tended to include multiple knowledge types briefly rather than one in detail.

(11)

Question B5

5. There are many alternative ways to solve a problem, each of which has its own merits in a particular situation.

i. Explain both brute-force and heuristic search methods and discuss their relative merits with the aid of suitable examples. (10 marks)

ii. Explain the difference between inductive and deductive reasoning and comment on the suitability of each for example problems of your choice. (15 marks) Question 5.i: 10 marks. General distribution of marks according to salient features.

Candidates must demonstrate an understanding of the methods employed in AI for search, and be able to evaluate each on particular criteria.

The solution should compare and contrast the two search techniques and make reference to the following points:

Uninformed search

An uninformed search algorithm is one that does not take into account the specific nature of the problem. As such, they can be implemented in general, and then the same implementation can be used in a wide range of problems thanks to abstraction. The drawback is that most search spaces are extremely large, and an uninformed search (especially of a tree) will take a reasonable amount of time only for small examples. As such, to speed up the process, sometimes only an informed search will do.

Informed search

In an informed search, a heuristic that is specific to the problem is used as a guide. A good heuristic will make an informed search dramatically out-perform any uninformed search.

There are few prominent informed list-search algorithms. A possible member of that category is a hash table with a hashing function that is a heuristic based on the problem at hand. Most informed search algorithms explore trees, such as the Best-first search, which is a search with a heuristic that attempts to predict how close the end of a path is to a solution, so that paths which are judged to be closer to a solution

(12)

are extended first. Efficient selection of the current best candidate for extension is typically implemented using a priority queue.

Candidates must demonstrate an understanding of the methods employed in AI for problem solving and reasoning, and be able to evaluate each on particular criteria. Learning Outcomes/Assessment Criteria:

1. The learner will understand the main approaches used in AI problem solving.[1.1, 1.2, 1.3]

Solutions should provide descriptions of the two general techniques and address the points of difference. It is essential that the answer contain example tasks that are each best solved using one of the techniques. The discussion should make it clear why the task requires a particular approach, i.e. the nature of the task (data driven or goal driven), and step–by-step explanations should be included to illustrate problem solution.

Question 5 has appeared on previous exam papers so it was expected that candidates would attempt it and perform relatively well.

QA5(i): What was required 1. Define search in general

2. Describe Brute-force as uninformed and comprehensive 3. Describe heuristic as informed and satisfying

4. Provide example methods of each type of reasoning (e.g. depth-first/A*) 5. Discuss merits of each and compare/contrast

(13)

1. Confused informed and uninformed when giving definitions. 2. Tend to give incomplete definitions.

3. Failed to provide adequate examples of each type of search. 4. Failed to compare and contrast search methods adequately. 5. Failed to include general definition of search.

QA5(ii): What was required 1. Define reasoning in general

2. Explain Inductive reasoning as ampliative (i.e. permits generalisation from specifics), uncertain (probabilistic), permits erroneous reasoning (e.g. false conclusions (generalisations) to be produced from available evidence (true premises))

3. Acknowledges notion of strong/weak conclusions from inductive reasoning.

4. Explain deductive reasoning as non-ampliative, confirmatory (asserts specifics from known generals), certain conclusions if supported by valid premises.

5. Acknowledges notion of valid/invalid arguments. 4. Compare and contrast using suitable examples. QA5(ii): What was included by students

1. Confused inductive and deductive and sometimes discussed the wrong way round. 2. Failed to provide general definition of reasoning.

3. Tended to provide brief definitions only.

4. Sometimes incomplete examples of reasoning provided.