Compiled by Dawit H.

_{Expert reasoning} _{Expert domains}

_{Production systems}

_{Rule-based expert systems} _{Case-Based Reasoning}

Inference Methods

 _{In this chapter, we will discuss various methods of reasoning or inference. This topic is} particularly important in expert systems because reasoning is the standard technique by which expert systems solve problems.

 _{Expert systems are commonly used when an inadequate algorithm or no algorithmic solution} exists and reasoning offers the only possibility of a solution.

 _{Whereas one could describe a conventional program (or at least, the part of it that produces} the results, as opposed to the user interface, etc) in these terms:

Program = algorithm + data

one would have to describe an expert system in these terms:

Expert system = inference engine + knowledge base + data.  The inference engine uses one of several available forms of inferencing.

 By inferencing I mean the method used in a knowledge-based system to process the supplied data, and the stored knowledge, so as to produce correct conclusions.

 _{Expert systems often present difficult knowledge representation and reasoning problems} which require artificial intelligence solutions.

Reasoning with Logic

 _{Modus Ponens}

 _{If A, then B}

 _{[A AND (A}  _B)] _B

 _{A and (A} _{B) are propositions in a knowledge base}

 _{Modus Tollens: when B is known to be false}

 _{Resolution: combines substitution, modus ponens, and other logical syllogisms}

Inferencing with Rules: Forward and Backward Chaining



Firing a rule:

When all of the rule's hypotheses (the “if parts”) are satisfied



_{Can check every rule in the knowledge base in a forward or backward direction}

_{Continues until no more rules can fire, or until a goal is achieved}

Forward and Backward Chaining



Chaining:

Linking a set of pertinent rules



_{Search process:}

_{directed by a rule interpreter approach:}



Forward chaining:

if the premise clauses match the situation, then the process

attempts to assert the conclusion



_{Backward chaining:}

_{if the current goal is to determine the correct conclusion,}

then the process attempts to determine whether the premise clauses (facts) match

the situation



Backward Chaining



_Goal-driven

_{- Start from a potential conclusion (hypothesis), then seek}

Forward Chaining



Data-driven - Start from available information as it becomes available, then try to

draw conclusions



_{What to Use?}

 _{If all facts available up front (as in auditing) - forward chaining}  _{Diagnostic problems - backward chaining}

Model-based Reasoning

 _{Based on knowledge of structure and behavior of the devices the system is designed to}

understand

 _{Especially useful in diagnosing difficult equipment problems}  _{Can overcome some of the difficulties of rule-based ES}

 _{Systems include a (deep-knowledge) model of the device to be diagnosed that is then used to}

identify the cause(s) of the equipment's failure

 _{Reasons from "first principles" (common sense)}

 _{Often combined with other representation and inferencing methods}  _{Model-based ES tend to be "transportable”}

 _{Simulates the structure and function of the machinery being diagnosed}  _{Models can be either mathematical or component}

Problem decomposition into an and-or graph

 _{A technique for reducing a problem to a production system.}  _{One particular form of intermediate representation.}

 _{A structured representation of the knowledge, which is not yet in the form of code that can be}

put into an expert system’s knowledgebase.

 _{A technique for reducing a problem to a production system, as follows:}

 _{The principle goal is identified; it is split into two or more sub-goals; these, too are split up.}  _{A goal is something you want to achieve. A sub-goal is a goal that must be achieved in order for}

the main goal to be achieved.

 _{A graph is drawn of the goal and sub-goals.}

 _{Each goal is written in a box, called a node, with its subgoals underneath it, joined by links.}  _{The leaf nodes at the bottom of the tree}

_{A tree is a hierarchical data structure consisting of nodes, which store information or} knowledge, and branches, which connect the nodes.

_{Branches are sometimes called links or edges and nodes are sometimes called vertices.} _{Figure,shows a general binary tree, which has zero, one, or two branches per node.}

_{In an oriented tree the root node is the highest node in the hierarchy and the leaves are the} lowest.

A tree can be considered a special type of semantic net in which every node except the root has exactly one parent and zero or more child nodes.

_{For the usual type of binary tree there is a maximum of two children per node, and the}

 _{If a node has more than one parent it is in a network In Figure, notice that there is only}

one sequence of edges or path from the root to any node because it is not possible to move against an arrow. In oriented trees the arrows all point downward.

_{Trees are a special case of a general mathematical structure called a graph.}

_{The terms network or simply net are often used synonymously with graph when describing a}

particular example of a graph such as a telephone network.

_{A graph can have zero or more links between nodes and no distinction between parents and}

children.

_{An artificial neural system is another example of a graph with cycles; because during training there}

is feedback of information from one layer of the net to another, which modifies the weights.

_{A simple graph has no links that come immediately back on the node itself.} _{Types of}

graph:-_{A circuit or cycle is a path through a graph that begins and ends on die same node.} _{An acyclic graph has no cycles.A connected graph has links to all its nodes.}

_{Graphs can be applied to many practical problems.}

_{A useful method of describing the behavior of an object is to define a graph called the state}

space.

_{A state is a collection of characteristics that can be used to define the status or state of an} object.

The state space is the set of states showing the transitions between states that the object can experience.A transition takes an object from one state to another.

_{As a simple example of state spaces, consider the} purchase of a soft drink from a machine. As you put coins into the machine, it makes a transition from one state to another.

_{Figure, illustrates the state space assuming that} only quarters and nickels are available and that 55$ is required for a drink.

_{The start and success states are drawn as double} circles to make them easier to identify. The states are shown as circles and the possible transitions to other states are drawn as arrows.

_{Notice that this diagram is a weighted digraph, in} which the weights are the possible coins that can be input to the machine in every state.

_{This diagram is also called a finite state machine diagram because it describes the finite} number of states of a machine.

_{The term machine is used in a very general sense. The machine can be a real object, an} algorithm, a concept, and so forth. Associated with every state are the actions that drive it to another state.

At any time, the machine can be m only one state. As the machine accepts input to a state, it progresses from that state to another.

_{If the correct inputs are given, the machine will progress from the start to the success or final} state. If a state is not designed to accept a certain input, the machine will become hung up in that state.

_{Finite state machines are often used in compilers and other programs to determine the} validity of an input.

 _{For example, Figure shows, part of a finite state machine to test input strings for validity.}

 _{Characters of the input are examined one at a time. Only the character strings WHILE,} WRITE, and BEGIN will be accepted. Arrows are shown from the BEGIN state for successful input and also for erroneous input going to the error state.

 _{For efficiency, some states, such as the one pointed to by "L" and "T," are used for testing} bothWHILE and WRITE.

 _{State diagrams are also useful in describing solutions to problems.}

 _{In these kinds of applications we can think of the state space as a problem space, in which some} states correspond to intermediate stages in problem solving and some states correspond to answers.

 _{In a problem space there may be multiple success states corresponding to possible solutions.} Finding the solution to a problem in a problem space involves finding a valid path from start (problem statement) to success (answer).

And-orTrees and Goals

_{Many types of expert systems use backward chaining to find solutions to problems PROLOG} is a good example of a backward chaining system that tries to solve a problem by breaking it up into smaller subproblems and solving them individually.

_{Solving a problem is considered by optimists as a goal to be achieved.}

In order to reach a goal, zero or more subgoals may need to be accomplished.

And-or-not

_{Another way of describing problem solutions is an AND-OR-NOT lattice, which uses logic} gate symbols instead of the AND-OR tree-type notation. The logic gate symbols for AND, OR, and NOT .

_{AND-OR trees and decision trees have die same basic advantages and disadvantages. The main} advantage of AND-OR-NOT lattices is their potential implementation in hardware for fast processing speeds.These lattices can be custom designed for fabrication as integrated circuits.

_{One type of logic gate such as the NOT-AND or NAND is used for reasons of manufacturing} economy rather than separate AND, OR, and NOT gates. From logic it can be proved that any logic function can be implemented by a NAND gate.

_{An integrated circuit with one type of device is cheaper to manufacture than one with} multiple types of logic gates.

A chip using forward chaining can compute the answer quickly as a function of its inputs because processing proceeds in parallel.

_{Chips like this can be used for real-time monitoring of sensor data and make an appropriate} response depending on die inputs.

_{The main disadvantage is that like other decision structures, a chip designed for logic} cannot handle situations it was not designed for.

_{In Chapter 3 we discussed the representation of knowledge by logic.}

_{Now you will see how inferences are made to derive new knowledge or information.} _{Deduction: Logical reasoning in which conclusions must follow from their premises.} _{Induction: Inference from the specific case to the general.}

_{Intuition: No proven theory. The answer just appears, possibly by unconsciously recognizing an}

underlying pattern.

_{Generate and test: Trial and error. Often used with planning for efficiency.}

_{Abduction: Reasoning back from a true conclusion to the premises that may have caused the}

conclusion.

_{Default: In the absence of specific knowledge, assume general or common knowledge by default.} _{Autoepistemic: Self-knowledge.}

 One of the most frequently used methods of drawing inferences is deductive logic, which has been used since ancient times to determine the validity of an argument.

 _{Although people commonly use die word argument to describe an angry "exchange of views,"} it has a very different meaning in logic.

 _Eg.

Premise:Anyone who can program is intelligent

Premise: John can program

Conclusion: Therefore, John is intelligent

 _{In an argument the premises are used as evidence to support the conclusions. The premises are} also called the antecedent and the conclusion is called the consequent.

 _{The essential characteristic of deductive logic is that the true conclusion must follow from} true premises. A line is customarily drawn to separate the premises from the conclusion, as shown above, so that it is not necessary to explicitly label the premises and conclusion.

 The argument could have been written more briefly as

Anyone who can program is intelligent John can program

 _{The main advantage of studying syllogisms is that it is a simple, well-understood branch of} logic dial can be completely proven. Also, syllogisms are often useful because they can be expressed in terms of IF..,THEN rules.

 _{For example, the previous syllogism can be rephrased as}  _{IF Anyone who can program is intelligent and}

John can program  THEN John is intelligent

_{Prepositional logic offers another means of describing arguments.} _{For example, consider the following propositional argument:}

If there is power, the computer will work There is power

.'. The computer will work

_{This argument can be expressed in a formal way by using letters to represent the propositions} as follows:

_{A -There is power}

_{B = The computer will work and so the argument can be written as} A  B

A

_{A rule is often referred to as a production rule.}

_{A production rule consists of two parts: condition (antecedent) part and conclusion (action,} consequent) part,

_{i.e: IF (conditions) THEN (actions)}

Example

IF Gauge is OK AND [TEMPERATURE] > 120

THEN Cooling system is in the state of overheating

_{This rule consists of 2 propositions given on separate lines (2 condition elements) and a} conclusion. The second condition element contains a variable. Condition elements in a rule can be connected by different connectives, the most used being AND, OR, NOT.

_{A production system consists}

of:-_{Working memory (facts memory)} _{Production rules memory}

_{Inference engine, it cycles through three steps:}

 _{match facts against rules} _{select a rule}

 _{Architecture of a typical production system:}

rule

memory

Inference

engine

New information rule memory working memory New information select Step 1 Step 2

_{The typical architecture of an KBS is often described as follows:}

user

interface

inference

engine

knowledge

base

_{Symbolic rules are one of the most popular knowledge representation and reasoning methods} (Ligeza 2006). Their popularity stems mainly from their naturalness, which facilitates comprehension of the represented knowledge.The basic form of a rule is the following:

if <conditions> then <conclusion>

_{where <conditions> represents the conditions of a rule, whereas <conclusion> represents its} conclusion.

_{The conditions of a rule are connected between each other with logical connectives such as,} AND, OR, NOT, etc, thus forming a logical function.

When sufficient conditions of a rule are satisfied, the conclusion is derived and the rule is said to fire (or trigger). Rules represent general knowledge regarding a domain.

_{Compact representation of general knowledge. Rules can easily represent general} knowledge about a problem domain in autonomous, relatively small chunks.

_{Naturalness of representation. Rules are a very natural knowledge representation method,} with a high level of comprehensibility, since they look like natural language expressions. Rules can emulate the expert’s way of thinking in many application domains.

Modularity. Each rule is a discrete knowledge unit that can be inserted into or removed from the knowledge base, without taking care of any other technical detail (as long as other rules are not affected). This characteristic grants flexibility during the development of rule-based systems, because it enables incremental development of the knowledge base as well as partial testing.

_{Provision of explanations. The ability to provide explanations for the derived conclusions in} a straightforward manner is a vital feature, given that explanations in certain application domains (e.g. medicine) are considered necessary. This feature of symbolic rules is a direct

_{Knowledge acquisition bottleneck. The standard way of acquiring rules through interviews} with experts is cumbersome and time-consuming. The chief reasons are the inability of an expert to express his/her knowledge and/or the unavailability of experts. Therefore, the acquired knowledge may be incomplete or even partially correct. Rule induction methods from machine learning can deal with some knowledge acquisition problems, but are still unable to recognize exceptions in small, low frequency sections of the domain.

Brittleness of rules. It is not possible to draw conclusions from rules when there are missing values in the input data. For a specific rule, a certain number of condition values must be known in order to evaluate the logical function connecting its conditions. In addition, rules do not perform well in cases of unexpected input values or combinations of them.

 _{Case-based reasoning is a paradigm for combining problem-solving and learning that has} became one of the most successful applied subfield of AI of recent years.

 _{CBR is based on the intuition that problems tend to recur. It means that new problems are} often similar to previously encountered problems and, therefore, that past solutions may be of use in the current situation

 _{What is Case?}

 _{A case can be said to be the record of a previous experience or problem. The information} recorded about this past experience will, by necessity, depend on the domain of the reasoner and the purpose to which the case will be put.

 _{The case base in the CBR system is the memory of all previous stored cases. There are three} general areas that have to be considered when creating a case base.

_{The structure and representation of the cases themselves} _{The memory model used for organizing the entire case base} _{The selection of indices which are used to identify each case}

_{Case-based reasoning is a methodology for solving problems by utilizing previous experiences.} It involves retaining a memory of previous problems and their solutions and, by referencing these, solves new problem.

 _{A case-based reasoning system can be viewed as a black box (see Fig. 2.4) that incorporates the} reasoning mechanism and the following external

facets:- _{The input specification or problem case}

 _{The output that defines a suggested solution to the problem}

 _{Most CBR systems, the case-based reasoning mechanism has an internal structure divided into} two major parts; the case retriever and the case reasoner.

 _{The case retriever’s task is to find the appropriate cases in the case base while the case reasoner} uses the retrieved cases to find a solution to the given problem description.

Reduction of the Knowledge Acquisition Task

_{By eliminating the extraction of a model or a set of rules as is necessary in model/rule based} systems, the knowledge acquisition tasks consists mainly of the collection of the relevant existing experiences/cases and their representation and storage.

Avoid repeating mistakes made in the past

In systems that record failures as well as successes, and perhaps the reason for those failures, the system can use the information about what caused failures in the past to predict any failures in the future.

Graceful degradation of performance

_{Some model based systems cannot even attempt to solve a problem on the boundaries of its} knowledge or scope, or when there is missing or incomplete data. In contrast case-based systems can often have a reasonably successful attempt at solving these types of problem.

Able to reason in domains that have not been fully understood, defined or modeled.

_{While insufficient knowledge may exist about a domain to build a causal model of it or derive} a set of heuristics for it, a case-based reasoner can function with only a set of cases from the domain.The underlying theory does not have to be quantified.

Reason with incomplete or imprecise data and concepts

Cases are retrieved, not just when identical to the current query case but also when they are within some measure of similarity, case based reasoning deal with since.

Avoid repeating all the steps that need to be taken to arrive at a solution

_{In problem domains that require significant processes to carry out the creation of a solution} from scratch, the modifying of a previous solution can significantly reduce this processing. By reusing a previous solution, the steps taken to reach the retrieved solution can be reused themselves.

 _{The problem solving cycle of CBR consists of four major processes : Retrieve, Reusing,}

Revise and Retain.

_{Retrieve: phase is an initial step which inquires about previous experiences that are similar to} the new case. In this phase most similar cases will be retrieved from the general knowledge. _{Reuse: phase is the second step which is responsible in suggesting a solution for the new case}

from the available solutions of the cases that were retrieved from the general knowledge.

Revise: the proposed solution from attempt to solve of new problem or adopt new proposed solution.

_{Retain: when the new solution was confirmed then new proposed solution was retained.}

_{The whole process of CBR cyclic in this order that new problem is solved by retrieving past} experienced cases, by reusing the pervious case revising the solution, and retaining the new solution of experience in to existing knowledge system.

 _{As shown in the figure, that from collection of pervious cases, the new case is used} RETRIEVES. The retrieved case is combined with the new case by reuse in to solve case and with the REVISE procedure the solution is also tested. In the RETAIN, useful experience is retained for future reuse, and existing case bases or updates with this case.

 _{There are step-by-step procedures involved in designing a CBR system.}

1. Case Representation

_{Case is a representing an experience. It contains the information which is content of cases and} situation where that information or experience can be used.

_{Cases in a case base can represent many different types of knowledge and store it in many} different representational formats. The objective of a system will greatly influence what is stored. A case based reasoning system may be aimed at the creation of a new design or plan, the diagnosis of a new problem, or the argument of a point of view with precedents. In general, a case consists of a:

_{Problem description, which depicts the state of the world when the case occurred;} _{Problem solution which states the derived solution to that problem; and/or}

_{In many practical CBR applications, cases are usually represented as two unstructured sets of} attribute value pairs, i.e. the problem and solution features. Cases can be represented as simple feature vectors, or they can be represented using any AI representational formalism such as frames, objects, predicates, semantic nets, or rules.

2. Indexing

Index is a computational data structure that can be held in memory and also can enable to search quickly.

Case indexing refers to assigning indices to cases for future retrieval and comparisons. This choice of indices is important to being able to retrieve the right case at the right time. This is because the indices of a case will determine in which context it will be retrieved in future. Index should have the following features:

_{Should show the purpose for which case will be used.} _{It should be easy to recognize it in feature.}

Case retrieval

_{Case retrieval is the process of finding} within the case base those cases that are the closest to the current case. To carry out case retrieval there must be criteria that determine how a case is judged to be appropriate for retrieval and a mechanism to control how the case base is searched.

_{The selection criteria is necessary to} decide which case is the best one to retrieve, that is, to determine how close the current and stored cases are, Figure

 _{There are four retrieval techniques to identify similar} cases:-_{K-Nearest-neighbor Retrieval} _{Inductive Retrieval} _{Knowledge-Guided Approaches} _{Validated Retrieval} K-Nearest-neighbor Retrieval

 _{K-Nearest-neighbor retrieval technique is to measure similarity between source case and} query case. If case is not matched with CBR library, then CBR system will return nearest match.

 _{In nearest neighbor retrieval, the case retrieved is chosen when the weighted sum of its} features that match that query is greater than the other cases in the case base. Nearest match can be computed using the following equation.

Where;

_{T= target case} _{S= source case}

_{n= number of attributes in each case} _{I= individual attribute from 1 to n}

_{f= similarity function for attributes I in cases T and S} _{w= importance weighting of attribute I}

_{Nearest-neighbor is not efficient technique. Because, when new case is introduced, indexing should be}

 _{The algorithm for nearest neighbor is as follows:}

 for each feature in the input case

 find the corresponding feature in the stored case base

 compare the two values to each other and compute the degree of match

 multiply by a coefficient representing the importance of the feature to the match

 add the results to drive an average match score

Inductive Retrieval

_{When inductive approaches are used to determine the case-base structure, which determines} the relative importance of features for discriminating among similar cases, the resulting hierarchical structure of the case base provides a reduced search space for the case retriever. This may, in turn, reduce the query search time.

In inductive retrieval, use past cases to extract rules or construct decision. This technique finds target case-based on index source case. Cases are divided into a decision tree structure. Inductive retrieval used to retrieve set of matched cases and then nearest-neighbor retrieval rank these cases according to their similarity with target case.

Knowledge-Guided Approaches

_{Knowledge-guided approaches to retrieval use domain knowledge to determine the features} of a case that are important for retrieving that case in the future. In some situations, different

Validated Retrieval

_{There have been numerous attempts at improving retrieval. One of these is validated retrieval,} which consists of two phases.

_{Phase one involves the retrieval of all cases that appear to be relevant to a problem, based on the}

main features of the present case.

_{Phase two involves deriving more discriminating features from the initial group of retrieved}

cases to determine whether these cases are valid in the current situation.

_{The advantage of validated retrieval is that inexpensive computational methods can be used to} make the initial retrieval from the case base, while more expensive computational methods can be used in the second phase, where they are applied to only a subset of the case base.

_{jCOLIBRI is an object-oriented framework in Java for building CBR systems that is an} evolution of previous work on knowledge intensive CBR.

_{COLIBRI is a platform for developing academic Case-Based Reasoning (CBR) software.} Its main goal is to provide the infrastructure required to develop new CBR systems and its associated software components. COLIBRI is designed to offer a collaborative environment where users could share their efforts in implementing CBR applications. It s an open platform where users can contribute with different designs or components that will be reused by other users.

Inferencing

_{Reasoning (Thinking).}

_{The computer is programmed so that it can make inferences and the inference is performed by the}

Inference Engine.

Inference Engine

_{The brain of the ES.}

_{The control structure or the rule interpreter and it provides a methodology for reasoning.}

Inference Engine Major Elements

_Interpreter _Scheduler

_{Considering that the Knowledge Base encodes domain knowledge and expertise in terms of} rules and facts there are three variations for the inference engine:

_{Forward Chaining or Data Driven (essentially Modus Ponens)} _{Backward Chaining or Hypothesis Driven}

_{Expert systems provide an advantage when dealing with uncertainty as compared to decision} trees.

_{With decision trees, all the facts must be known to arrive at an outcome.} _{Probability theory is devoted to dealing with theories of uncertainty.}

_{There are many theories of probability – each with advantages and disadvantages.}

What is Uncertainty?

_{Uncertainty is essentially lack of information to formulate a decision.} _{Uncertainty may result in making poor or bad decisions.}

Dealing with uncertainty requires reasoning under uncertainty along with possessing a lot of common sense.

_{Bayesian Probability} _{HartleyTheory} _{ShannonTheory} _{Dempster-ShaferTheory} _{Markov Models} Zadeh’s FuzzyTheory

_{Deductive reasoning – deals with exact facts and exact conclusions}

_{Inductive reasoning – not as strong as deductive – premises support the conclusion but do} not guarantee it.

_{There are a number of methods to pick the best solution in light of uncertainty.} _{When dealing with uncertainty, we may have to settle for just a good solution.}

Errors Related to Hypothesis

_{Many types of errors contribute to uncertainty.}

_{Type I Error – accepting a hypothesis when it is not true – False Positive.}