Classifying Related Work: Prior Knowledge Levels

When considering different approaches for application to a real world problem, the available prior knowledge on fraud is a crucial factor. While in one case, fraud is known to be a problem but otherwise a largely unknown concept, there may be extensive knowledge of revealed fraud in another case. Basically, three levels can be identified:

No or very low prior knowledge If a business is considered under the aspect of fraud pre- vention or detection only for a short time, knowledge based on experience is lacking. The fact that fraud detection is considered at all suggests however that a certain amount of fraud cases has been discovered and minimal knowledge is available. Only one paper mentions that the in-

troduced data-mining-assisted system constitutes the initial countermeasure and therefore prior knowledge is very limited [Major and Riedinger, 2002]. Such a system has completely different premises than a solution which can be based upon extensive expertise. A common approach to this problem is not to have a system identify “fraud” (as this is a widely unknown concept for the time being) but to search for “uncommon behavior” which, of course, has to be defined in an appropriate way. The identified behavior is subsequently delivered to human experts for closer investigation and evaluation. This approach assumes that there is an adequate concept of “un- common behavior” which is able to discriminate between fraudulent and normal behavior. Such a system requires extensive efforts by a human expert, which learns the concept “fraud”while working with the system. Ideally, the expert may feed the gained knowledge back to the system to gradually improve its discriminative power. Typically, the number of “false positives”(false alarms) is very high at the beginning and reduces as the human and the system refine their knowl- edge about the mechanisms of fraud.

Existing identification knowledge (”labels”) We define identification knowledge as follows: Identification knowledgeis knowing if an instance is fraudulent or not. In this case, the data is typically ”labelled”8 For example, phone fraud experts may know which of the calls logged in the past are fraudulent. They may, however, not have an extensive knowledge about the mecha- nisms and properties of the fraud cases, what we call model knowledge (see below). This situa- tion is ideal for an approach which Fawcett and colleagues describe as “discriminating method” [Fawcett et al., 1997]: based on known fraudulent examples a model is calculated which describes the discriminating factors between fraudulent and normal behavior. This approach can also be valuable in the presence of prior (human) knowledge about fraud mechanisms. Being able to consider more data than a human expert, calculated models can display previously unknown interrelations or can be used to support informal hypotheses. As the presence of a sufficient num- ber of identified fraud cases (used for model calculation) typically implies a certain prior model knowledge which lead to the discovery of those cases, this approach is assumed to offer model knowledge refinements rather than completely new knowledge. In other words, the models are, naturally, only calculated on the basis of previously identified examples and therefore will not have the ability to find completely new fraud tactics. Completely new, innovative fraud brings

the human experts back to the “no or very low prior knowledge” situation and the appropriate methods mentioned above.

Another crucial consideration when using calculated models is the interpretability. The active substantiation of a suspicion (for example by interrogating the suspect) is only possible if the investigator is able to reconstruct the reason for the initial suspicion. No suspect will be reported without a human expert knowing why (but only based on a score based on a black-box-model). If the calculated model is human-interpretable (e.g. a decision tree), suspicion validation can be accomplished directly. If the model is not human interpretable (e.g. a neural network), its output has to be validated by a human expert e.g. directly based on the data, where it is, to some extent, replaced by a human model. Some research work appears to completely disregard this aspect.

Existing model knowledge This type of prior knowledge is more elaborate than identification knowledge. Fraud experts may know exactly what they are looking for from experience, but they lack the appropriate tools for an efficient and effective search. In contrast to the approach mentioned above, the models are not calculated by an algorithm, but defined by human experts. These models can, e.g., be intentionally fuzzy to allow for an extension of knowledge about fraud when evaluating the results (“seeing the borders between fraud and normal behavior”), which again points back to the “low prior knowledge setting”. An advantage of this approach is that the model is not constrained by the model calculation method, but only by the richer expressivity of the human brain and the mechanisms built to retrieve the patterns defined by the model. A dis- advantage is the possible discrepancy in the identification knowledge and the model knowledge of the human expert (e.g. because the model is too complex to be captured by a human expert — or the amount of data is too big to consider exhaustively). Prior model knowledge motivates the use of pattern matchers to retrieve interesting data.

Another relevant categorization can be made concerning the available format of prior knowl- edge:

• Machine readable

Machine readable identification knowledge corresponds to “labels”or “targets” in Data Min- ing, which is a requirement for the model calculation with supervised Data Mining algo- rithms. Machine readable model knowledge is a typical result of Data Mining and other knowledge engineering techniques.

• Informal, not machine readable

Informal identification knowledge requires manual effort. The transfer to a machine read- able format suggests itself, but is not always possible (as in our case). Informal model knowledge is the typical example for human model construction, e.g. if model calculation is infeasible or not considered valuable.