A Web-based Intelligent Help Desk Support Environment
Foo, S., Hui, S.C., Leong, P.C. (2002). International Journal of Systems Science, 33(6), 389-402.
A Web-based Intelligent Help Desk Support Environment
Schubert Foo, Siu Cheung Hui, Peng Chor Leong
School of Computer Engineering, Nanyang Technological University Nanyang Avenue, Singapore 639798
Abstract
With the advent of Internet technology, it is now feasible to provide effective and efficient help desk service over the global Internet to meet customers' requirements and satisfaction. In this research, we have designed and developed a Web-based intelligent help desk support environment, WebHotLine, to support the customer service centre of a large multinational corporation in the electronics industry. This paper describes the basic architecture of the environment that supports the major functions of Web-based fault information retrieval, online multilingual translation capability, different operating modes of video-conferencing for enhanced support and direct intelligent fault diagnosis by customers or customer support engineers. As a result, WebHotLine helps to save cost in eliminating the expensive overseas telephone charges, reduction in machine down time and number of on-site visits by service engineers as in traditional help desk environment.
1. Introduction
A large multinational corporation in Singapore manufactures and supplies insertion and surface mount machines for use in the electronics industry. A hot- line service centre (or help desk) was set up by the Customer Support Department to service its worldwide customer-base and provide installation, inspection and maintenance support for its customers. Such customer support is often viewed as a key operation of any business. It is an important criterion to the customer when evaluating a product or a service. The level of customers’ satisfaction in this aspect will have a significant impact on whether new versions of the product or service will be supported by these same customers in future.
In the current situation, the service centre is responsible for receiving reports on faulty machines or inquiries from their customers via telephone calls. When a problem is reported, an on-duty service engineer will suggest a series of 'checkpoints' to the customers to implement or check as a means to rectify the reported problem. Such suggestions are based on past experience or extracted through a customer service database that contains previous service records that are identical or similar to the current one. Whe n an online session cannot resolve the problem, the service centre will dispatch the service engineers to the customer's site as soon as possible to carry out an on-site repair.
At the end of each service cycle, a customer service report is used to record the reported problem and proposed remedies or suggestions taken to rectify the problem. At present, this recording is subjected to the service engineers' vocabulary. No standard format or vocabulary is used to record these service records. Information from such records is eventually returned to the service centre for recording and storage in the customer service database for future support purposes. Over time, the information in these databases grows as more service records are produced and stored.
This research addresses this telephone-based scenario that is often typical of many existing customer service centres by proposing an integrated environment, WebHotLine, to harness existing computer, telecommunication and information processing technologies along with artificial intelligence techniques to provide a enhanced Web-based customer support system. The proposed environment provides four major functions including online retrieval of fault information, machine translation, video-conferencing support and intelligent fault diagnosis. Together, it provides an integrated and extensible platform to transform the customer service support into an effective and efficient activity that meets users’ requirements and satisfaction.
The rest of the paper is organised as follows. Section 2 describes the various functions in a Web-based help desk environment. In Section 3, the basic architecture of WebHotLine and its major components are given. Section 4 describes the customer service database. Sections 5 to 8 describe the four major functions of WebHotLine. The performance evaluation of the system is given in Section 9. Finally, Section 10 concludes the paper.
2. Web-based help desk environment
With the advent of Web technology that can provide dynamic, interactive, hypermedia, platform independent, distributed and client/server services, customers can access the online knowledge database via any Web browsers such as Netscape's Navigator or Microsoft's Internet Explorer. If the problem cannot be resolved through the use of the online help desk functions, the customers can fill out a standard help request form to document their problems. This form can then be automatically routed to the experienced service engineers for further response.
A number of commercial help desk products for customer support, such as WebLink and WebSupport (Muller, 1996), are available. In addition to providing general Web support for accessing its knowledge database of problem solutions, these systems also support automatic problem diagnosis through artificial intelligence technology. The current trend of Web-based customer service support is gaining acceptability and many multinational corporations such as Compaq and NEC have adopted such an approach. We have envisaged that a Web-based help desk environment will consist of the following four major functions: Web-based retrieval of fault information, machine translation, video-conferencing support and intelligent fault diagnosis.
2.1 Web-based retrieval of fault information
As the customer service (or knowledge) database contains past customer service records which serve as records of problems and solutions, a Web-based retrieval tool is needed to allow remote customers and service engineers access and retrieve these service records from the database. Different ways of retrieval can be supported to ensure flexibility. For instance, customers can retrieve all the checkpoints directly by selecting or defining the observed fault-conditions. These retrieved checkpoints can then be used to solve the problem. A hypertext-based system can be used alongside the system to provide additional help information to the users as necessary to outline the procedure for the various checkpoints. This can take the form of multimedia documents that primarily consists of text and images.
2.2 Machine translation
The Web-based help desk environment is used to provide customer service support for its world wide customers including Asian customers. However, as the service records are recorded and stored in English, they are retrieved and displayed in English only. This often poses problems for some of its Asian customers (such as Chinese and Japanese customers) who find difficulty in using the system. In an effort to support these customers and tap into the 'expertise' provided by the system, an online translation of service records from English to other languages has been carried out. Such a translation capability eliminates the need to create additional non-English service databases of identical content. At the same time, maintaining a central database of records offers significant advantages in terms of knowledge sharing, maintenance and extensibility.
Machine translation techniques (Goodman 1989, Goodman and Nirenburg 1991, Arnold 1994, Li et al. 1996, Liu and Zhang 1997, Fu 1997) have been successfully reported for English-to-Japanese, English-to-Chinese and to many other forms of bilingual translation. Recently, machine translation systems for English-to-Japanese (Isikawa et al. 1996, Harada 1996, Aizawa et al. 1996) have been developed for Web users. These systems translate the English information into the Japanese equivalent by maintaining the original layout of the Web page. By executing the translation program while receiving HTML documents, it gives users the impression of real-time on-the- fly translation. Such classical machine translation techniques are unsuitable for translating service records because of its technical nature and non-standard grammatical syntax of the service records. In this work, an approach using translation tables (containing only limited entries of controlled vocabulary) for English-to-Japanese and English-to-Chinese translation of technical service records is proposed.
2.3 Video-conferencing support
For certain difficult machine problems that cannot be resolved through the knowledge database or intelligent fault diagnosis, the customers need to seek help from the service centre directly. To cater for this functionality, a video-conferencing link over the Internet can be established between the customer and service support engineers. Video conferencing provides an invaluable 'sight and sound' response to allow service engineers carry out their advisory tasks.
Here, we have developed a stand-alone low bit-rate real-time Internet video-conferencing tool (Hui and Foo 1998) and integrated as part of the help desk environment. It supports different modes of operation according to users’ requirements during a communication session. The provision of various operating modes of sight and sound has greatly enhanced the whole customer support process. This tool was originally developed to study the feasibility of utilising various techniques to enhance the Quality of Service (QoS) of real-time audio and video communication between two users using Internet as the transmission medium to exchange real- time data. It uses the techniques of data compression, data buffering, dynamic rate control, data packet lost replacement, silence deletion, and virtual video play-out mechanism to provide a cost-effective and acceptable means of real-time communication.
2.4 Intelligent fault diagnosis
In traditional help desk service support, the identification of machine faults relies heavily on the service support engineers’ past experience or the information drawn from the service database. This method has a problem of training and maintaining a pool of expert service engineers. Thus, instead of relying on the knowledge of service engineers, an intelligent fault diagnosis tool that captures the expert knowledge of machine diagnosis to assist customers identify machine faults becomes extremely useful.
To support intelligent fault diagnosis for help desk applications, case-based reasoning (CBR) has been traditionally applied (House 1994, McCarthy 1994, Shimazu et al. 1994, Montazemi and Gupta 1996, Law et al. 1997, Liu and Yan 1997, Patterson and Hughes 1997). CBR systems rely on building a large repository of diagnostic cases (or past service reports) in order to circumvent the difficult task of extracting and encoding expert domain knowledge (Riesbeck and Schank 1989). It is one of the most appropriate techniques for customer service support as it learns with experience in solving problems and hence emulates human- like intelligence. However, the performance of CBR systems depends critically on the adequacy as well as the organisation of cases and the algorithms used for retrieval from a large case database. Most CBR systems (Watson 1997) use the nearest neighbour algorithm for retrieval from the flat- indexed case database, which are inefficient especially for large case database.
On the other hand, the artificial neural network (ANN) approach provides an efficient learning capability from detailed examples. Supervised neural networks such as Learning Vector Quantization (LVQ3) (Kohonen 1990) are used when the training data consists of examples with known classes. LVQ3 performs retrieval based on nearest neighbour matching, since it stores the weight vectors as the code-book or exemplar vector for the input patterns. The matching is based on a competitive process that determines the output unit that is the best match for the input vector, similar to the nearest neighbour rule. In LVQ3, the search space is greatly reduced because of the generalisation of knowledge through training. In contrast, the CBR systems need to store all the cases in the case database in order to perform accurate retrieval that greatly increases the search space. The CBR systems that store only relevant cases for an efficient retrieval lack the accuracy as well as the learning
feature of the neural networks. Hence, supervised neural networks become a very suitable complement for case retrieval.
As such, a hybrid CBR and ANN system (Lees and Corchado 1997, Papagni et
al. 1997, Richter 2000) that operates under the framework of CBR cycle (Aamodt and
Plaza 1994) is deemed to be one of the most effective and intelligent techniques for fault diagnosis. Such a system behaves most similar to the human beings in problem solving through recalling prior experience or cases and subsequently learning through experience in solving the problems.
3. WebHotLine basic architecture
Internet httpd Server Retrieval Engine Multilingual Translation
Customer Service Database Translation Tables Intelligent Fault Diagnosis Web Server Low-Bit Rate Video Conferencing System Audio/Video Capturing Device Web Browser Audio/Video Capturing Device Web Browser Figure 1. Web-based integrated help desk environment.
Figure 1 shows the system architectur e of the Web-based integrated help desk environment, WetHotline, which is designed to support the various functions shown in the figure. The WebHotLine system is developed on the Windows NT environment. The Netscape Enterprise Server 3.0 (Netscape Communications Inc. 2001) is used as the Hypertext Transfer Protocol (HTTP) (World Wide Web Consortium 2001a) server. The Retrieval Engine, Multilingual Translation and
Intelligent Fault Diagnosis modules are written in Visual C++ as Common Gateway
Interface (CGI) (World Wide Web Consortium 2001b) programs which are linked to the Web server. The Microsoft Access database management system is used for the
Customer Service Database. The CGI programs communicate with the database
system through Open Database Connectivity (ODBC) (Signore et al. 1995) to insert, delete or update information in the database. Hypertext Mark-up Language (HTML) (World Wide Web Consortium 2001c) is used to create the user interface as Web pages to accept user queries.
In addition, issues with user and data security are an important consideration in the design and delivery of the help desk environment since important and critical data is being transferred across the insecure Internet network that is prone to data tapping, data interception and user impersonation. In WebHotLine, security is achieved through password authentication, data encryption and verification (Liu 1999). Password authentication involves the setting up of user accounts and passwords in the Web server for user authentication prior to being able to access the facilities of the help desk environment. Such a mode of protection is still subjected to interception or tapping by hackers as the user and password information is being transmitted from the user to the Web server. Thus, data encryption has been utilised in this work to further protect the password authentication data.
Customers access the customer service database via any Web browsers such as Netscape's Navigator or Microsoft's Internet Explorer. The customer can interact with the system in many ways. One common method is to the define fault-conditions in an HTML form. Through the retrieval engine, the system searches, retrieves and displays the corresponding checkpoints from the customer service database. If the problem can be resolved by using these checkpoints, the service cycle is complete. Alternatively, the customer can make use of the video-conferencing tool to communicate with the service engineers to carry out a low-cost inquiry session to seek their advic e to solve the problem. Yet as another alternative, customers can use the intelligent fault diagnosis support to provide the system with information on system-generated error messages, fault description, component description, and state of these components. This will in turn allow the system to recall matching case(s) that matches with most (if not all) of the defined responses. The user then uses the proposed checkpoints to resolve the problem. When such actions fail to produce a solution, the customer can document the problem by filling in an on-line help form for onward transmission to the service engineers. If necessary, the service centre will then dispatch their service engineers for an on-site repair.
4. Customer service database
Service records (or reports) are currently defined and stored in the customer service database. Each service record consists of customer account information and service details, which contain two types of information: fault-condition and
checkpoint information. The forme r contains the service engineer’s description of the
machine fault, while the later indicates the suggested actions or services to be carried out to repair the machine, based on the actual fault-condition given by the customer. Checkpoint information contains checkpoint group name, and checkpoint description, with priority and an optional help file. The checkpoint group name is used to specify a list of checkpoints. Each checkpoint is associated with a priority that determines the sequence in which it can be exercised and a help file that gives visual details on how to carry out the checkpoint. An example of a fault-condition and checkpoint information for a service record is given in figure 2.
There are over 70,000 service records. Since each of the fault-conditions has several checkpoints, there are over 50,000 checkpoints. Apart from service records, the customer service database also stores information on over 4,000 employees, 500 customers, 300 different machine models and 10,000 sales transactions. Currently, two sets of service records are maintained. The first set contains all the service reports
the service engineers have reported during their on-site repair for the past few years. As the same fault conditions and its corresponding remedial actions can appear repeatedly in this set of service records, the second set is then created as a subset of the first set which contains service reports with unique fault conditions so that no repetition of service reports are stored. The latter set is used for Web-based retrieval of fault information.
Fault-condition 3008 PCB CARRY MISS ERROR. PCB WAS NOT TRANSFERRED BY THE CARRIER DURING LOADING BUT STAYED AT THE DETECTION POSITION OF PCB DETECTION SENSOR 2. Checkpoint group: AVF_CHK007
Priority Checkpoint description Help file
1 CONFIRM WHETHER THE CARRY GUIDE PINS ARE IN LINE WITH PCB. AVF_CHK007-1.GIF
2 CONFIRM WHETHER THE PCB IS IN CORRECT DIRECTION. AVF_CHK007-2.GIF
3 CONFIRM THE POSITION OF THE GUIDE LOWER LIMIT SENSOR. (I/O 0165) AVF_CHK007-3.GIF
4 CONFIRM THE TIMING FOR PCB 2 DETECT SENSOR. AVF_CHK007-4.GIF
5 CONFIRM THE TIMING FOR THE CARRIER START TIMING. AVF_CHK007-5.GIF
Figure 2. Fault-condition and checkpoint information of a service record.
5. Web-Based retrieval of fault information
Web-based retrieval of fault information provides the facility to allow customers have direct access to past service records of machines that are stored in the customer service database. The customer would obviously be only able to access those records pertaining to their type of machines and not all the machines in the database. In addition, these service records pertain to all existing records that are previously accumulated for the same machine type from various customers. This enhances the amount of supported fault conditions and provides better chances for a reported fault to be resolved. This is especially useful for repeat faults or those faults that are distinct and easily identifiable.
Figure 3. List of fault conditions for selected machine type.
Figure 4. Checkpoints for specified fault-condition (in English).
The workflow of the Web-based retrieval is simple and intuitive. Upon successful user authentication, the customer selects a particular machine type followed by a reported fault (figure 3). A set of corresponding checkpoints are retrieved from the database and displayed (figure 4) as the steps for the user to check or take in an attempt to resolve the problem. Whenever appropriate, hypertext help links (not shown in the figure) can be used in conjunction with the checkpoints to display additional help information or procedures that should taken to carry out the check or corrective action.
6. Machine translation
Customer Service Database (Fault-conditions
and Checkpoints) Fault-condition Analysis Checkpoint Analysis Fault-conditon Structure Checkpoint Structure Translation Table Generation Record Retrieval Fault-conditions or Checkpoints
Online Parsing Online Translation Multilingual Software Fault-condition Table Checkpoint Table Translation Tables Online Display
Technical Report Analysis
Online Translation
Client Server
Web Browser
Figure 5. Multilingual translation process.
Machine translation is carried out through the two processes of Technical Report
Analysis and Online Translation (Liu et al. 1998) as shown in figure 5. In Technical Report Analysis, a thorough preliminary and structured analysis of all the existing
records in the customer service database is first carried out to derive a set of new simplified common structures to adequately describe the contents of the service records. Figure 6 shows the structure for fault conditions and checkpoints. Based on these structures, translation tables for each unique word can be created.
Fault-condition Structure:
SUBJECT [+ VERB [PHRASE]] + STATE (ADVERB, ADJECTIVE or NOUN) Examples:
(1) THETA CANNOT RETURN TO ORIGIN -> (THETA + CANNOT RETURN + TO ORIGIN) (2) TRANSFER ARM DID NOT RETURN AFTER LOADING -> ((TRANSFER ARM + DID NOT
RETURN + AFTER LOADING)
(3) DISPENSER CANNOT DISPENSE ADHESIVE -> (DISPENSER + CANNOT DISPENSE + ADHESIVE)
Checkpoints Structure:
VERB [PHRASE] [+ NOUN [PHRASE]] [+ CONDITION] [+ REFERENCE] Examples:
(1) TURN POWER OFF ONCE -> (TURN + POWER + OFF ONCE)
(2) CHECK WIRES FOR CONTINUITY -> (CHECK + WIRES + FOR CONTINUITY)
During Online Translation, service records are first retrieved based on customer's input. The corresponding fault-conditions and checkpoints are parsed to the common structures identified earlier. Translation tables are then used to generate the specified language counterparts and displayed on the Web browser.
The common structures derived are subsequently used by all service engineers to document future service records. In order to make this method effective, the vocabulary used in the service records is limited to reduce the size of the translation tables and to improve efficiency. As these common structures are derived from existing service records, they are not too restrictive and pose no problems for service engineers to use and follow in future. In addition, using this approach circumvents the need to carry out deep- level analysis of syntax and semantics between the English and other languages. More details of the machine translation capability of WebHotLine can be found in (Liu 1999) and (Foo et. al. 2000). Figures 4, 7 and 8 show a typical set of fault-condition and checkpoints in English and its equivalent in Japanese and Chinese respectively. From the figures, it can be seen that the English text is maintained in the original form as stored in the database while stop words are eliminated. Short form technical words or abbreviations are retained in the translated text.
Figure 8. Online Chinese translation of fault-condition and checkpoints.
7. Video-conferencing support
A low bit-rate real-time Internet video-conferencing tool (Hui and Foo 1998) has been integrated into the WebHotLine environment. The tool can be selected to operate on one of three different operating modes:
• High video and audio priority. With the presence of adequate bandwidth, some of the mechanism designed for the occasional unreliable Internet network such as dynamic rate control, dynamic video frames reconstruction and virtual video playback becomes unnecessary and can therefore be de-activated. However, audio and video compression is still used to enable effective management of data packets by the underlying network layer. This is the 'normal' video conferencing mode for customers and service engineers to communicate.
• High video and low audio priority. When good quality video is desired, lower quality audio compression can be used or even terminated to allow for better video transmission. This is useful for sending a sequence of video frames to the service engineers for their evaluation. Such a mode of operation has been used to remotely monitor and observe the action of faulty machines at the customer’s site. In the case where a static but larger and higher-resolution image is desired, the video conferencing mode can be switched to the Picture Phone mode and allow the exchange of snapshots and images from the video-capturing device. Thus, only an enhanced single frame (of user-defined size) is captured and transmitted as shown in figure 9. Such a facility will allow images of critical components to be zoomed in and out during the diagnostic process.
• Low video and high audio priority. In this mode, emphasis is placed on the audio communication aspect so that a virtual video play-out is used to provide a better communication session. Since audio takes a higher priority, dynamic video frames reconstruction is employed to allow for low video transmission rate with minimal jerky play-out. This is done by reconstructing (through image interpolation) a series of frames within previously received video frames and repeating the play-out sequence. Virtual video play-play-out is used to simulate video presence when no video frames are transmitted. Such a facility is useful for service engineers to
concentrate and listen to a sequence of sounds that is generated by a faulty machine at the customer’s site.
Figure 9. Picture phone interface.
The provision of these various operating modes in contrast to traditional off-the-shelf video-conferencing stems from the realisation that during a service cycle, the important audio and visual aspect is to listen and view a faulty machine’s condition or an enhanced image of some machine component as opposed to the users’ faces.
8. Intelligent Fault diagnosis
In WebHotLine, a hybrid CBR-ANN approach has been adopted to implement the intelligent fault diagnosis (Jha et al. 1999). Instead of using traditional CBR techniques for indexing, retrieval and adaptation, LVQ3 is incorporated into the CBR cycle to extract knowledge from service reports of the customer service database and subsequently recall the appropriate service reports using this knowledge during the retrieval phase.
The process for the intelligent fault diagnosis is shown in figure 10. Apart from the traditional four phases of CBR cycle (i.e. Retrieve, Reuse, Revise and Retain), the fault-condition of each service record of the customer service database needs to be pre-processed and trained. The fault-condition is pre-processed into a format suitable for training, and the LVQ3 neural network is then trained to make it learn to solve future machine problems. This process is similar to the indexing process in traditional CBR systems. Through the process of training the LVQ3 with suitably pre-processed data from customer service database, the ANN becomes a knowledge base from which information can be retrieved.
LVQ3 Neural Network Input Pre-Processing Revise with user-feedback Neural Network Training Reuse of Service Records Retain the solution Customer Service Database Neural Network Retrieval previous solution confirmed solution User input (via Web browser) fault-conditions Preprocessing Fault-conditions
Intelligent Fault Diagnosis
Figure 10. Intelligent fault diagnosis process.
The Preprocessing Fault Conditions module preprocesses fault conditions of service records from the customer service database in order to extract keywords and key phrases. This involves the preprocessing of sentence(s) representing a fault condition by going through a process of elimination of all redundant non-alphanumeric and punctuation characters, extraction of all technical abbreviations from the machine domain using a list of abbreviations, word stemming, handling verb phrases with special meanings (e.g. 'flip flop', 'turn on' and 'shut up'), and elimination of stop words using a stop- list. The preprocessing process is implemented using verb-list, stop- list and algorithms from Wordnet (Beckwith et al. 1993). The result is a list of keywords and key phrases that are used for the neural network training process.
Keywords Extracted Index
CUTTER 3
ANVIL 1
NOT ENGAGE 8
PCB 9
CUTTER & ANVIL CANNOT ENGAGE IN AFTER 1ST PCB.
Preprocessing List of Keywords Fault Condition Index of keywords Weight Vector Index Weights 1 2 3 4 5 6 7 8 9 10 .75 0 .75 0 0 0 0 .75 .75 0 Index Keyword 1 ANVIL 2 BREAK 3 CUTTER 4 CAMERA 5 FAULTY 6 GUIDE 7 NC 8 NOT ENGAGE 9 PCB 10 SHAKY
The Neural Network Training process is needed to initialize and train the neural network before it can be used for retrieval. During the initialization phase, input nodes are connected to the output nodes through links. The weights of these links are assigned by considering one output node at a time and updating the weight vector corresponding to the links associated with it. During the Preprocessing Fault
Conditions phase, each fault-condition from the customer service database (the unique
set) is preprocessed and its keywords are extracted to generate a list of keywords. The indices of keywords from a fault-condition in the keyword list are used to form the weight vector as shown in figure 11.
The process is repeated for all the fault-conditions in order to initialize the weight matrix. The initial weight matrix is a sparse matrix of order equal to the number of keywords times the number of unique fault-conditions and consists of the values '0' and '0.75' only. The value '0.75' has been chosen instead of '1' to provide the learning capability of the network. After the neural network has been initialized, the ne twork will undergo a training process. The training set of the customer service database will be used for training. This is basically the first set of unprocessed data in the customer service database. The weights are updated using the LVQ3 supervised learning algorithm. The learning rate is set to 0.4. For each fault-condition, the less frequent keywords will have lower weights whereas more frequent keywords will have higher weights. The training algorithm is based on the LVQ3 network (Kohonen 1990).
Turning to the user side, the Input Preprocessing process handles the user input that takes the form of a fault description in natural language form. However, for the convenience of some more experienced users or service engineers, the system also accepts other forms of inputs such as the error code and error title, the name of the faulty components and their states. The user first selects the machine model from a list of models and enters the optional error code and error title. The user transcribes this error information that is shown on the monitor of the NC (numerically controlled) machine when a fault occurs. This error code uniquely represents the corresponding fault condition. A list of error codes and its corresponding fault conditions is maintained fo r efficient retrieval. If the error code is known, then no other information is required from the user for further processing, the corresponding fault condition can be identified and its checkpoints can be retrieved. Otherwise, the fault description can be entered in natural language or as a set of keywords. The user can also provide the names of machine components and their states as input. If the user input contains keywords that are not in the keyword list, synonyms of these keywords will be retrieved fo r user confirmation as input keyword. User fault input is then preprocessed to form the input vector for retrieval.
The Neural Network Retrieval process consists of case-recall with virtue of its learnt knowledge and ranking the cases recalled based on certain score that signifies the closeness of retrieved case to the input pattern. The retrieval algorithm is based on the matching process discussed in the training algorithm. The process of computing winners can be extended to compute successive winners fo r the input provided by the user. Each of the winners should have a matching score above certain threshold. The number of matches to be retrieved is based on this threshold. Then, the fault conditions corresponding to the winner nodes are presented to the user for feedback. Figure 12 shows the fault-conditions retrieved by the neural network when the user
enters the fault description 'UPPER LIMIT SENSOR IS NOT UP'. It can be observed that the fault-condition displayed at the top of the screen is the one closest to the fault description provided by the user.
Figure 12 - Retrieved fault-conditions and prioritized checkpoints for the specified fault-condition.
The Reuse of Service Records process reuses the checkpoint solutions of the fault conditions retrieved during the retrieval process. In the LVQ neural network retrieval process, the displayed fault conditions that match most closely to the input fault description provided by the user are ranked according to their matching scores. When the user selects a fault-condition displayed, the corresponding checkpoints are then retrieved and displayed in the order of their priority in terms of its capacity in solving the fault-condition. The user can view checkpoints of any displayed fault-conditions as shown in figure 12. In addition, help can be obtained for exercising these checkpoints by clicking the 'Help' option. This will cause the system to load the corresponding help file to help the user to carry out the remedial task.
In the Revise with User Feedback process, the user can try out each checkpoint corresponding to the winner fault conditions in the order of their priorities. The process stops once the problem is solved after exercising one of the displayed checkpoints. The checkpoint solution that solves the problem can then be marked. A higher priority will be given to this checkpoint, so that this checkpoint solution will be of higher priority for the user to use for the same fault condition in the future.
In the Retain the Solution process, the confirmed fault-condition and the checkpoints are retained within the system. The system learns from the current problem experienced by updating the weights of the neural network. The weight update is based on the LVQ3 training process. The input vector is then linked to the
class of the confirmed fault-condition. If the user has identified additional keywords that he has missed in the input during the feedback process, the input vector will be updated and the values corresponding to the keyword index location is altered from '0' to '1'. The input vector is accordingly modified before the weight update.
At this point, it should be evident that the intelligent fault diagnosis is an alternative to Web-based retrieval in cases when the faults are not immediately obvious. The work flow of these two processes are basically similar with users following all the checkpoints in attempting to resolve the problem, and making a report to the service centre for further action when this fails. Both functions access and share the common customer service database. However, the earlier function is for pure information retrieval without intelligence, while the latter learns and adjusts itself through user feedback.
9. Performance Evaluation
Performance evaluation has been carried out for WebHotLine. Here, we focus on measuring the performance of the intelligent fault diagnosis, which is the major function of the help desk environment. WebHotLine adopts the hybrid CBR-ANN approach that uses the LVQ3 neural network to perform the tasks of a CBR cycle. Traditional CBR system using k Nearest Neighbour (kNN) technique needs to store all the cases in the case database in order to perform accurate retrieval. The CBR-ANN approach greatly reduces the search space because of the generalisation of knowledge through training. While CBR system that stores only relevant cases for efficient retrieval lack the accuracy as well as the learning feature of the neural networks. In this section, two areas, namely, efficiency and precision of retrieval, are analysed and presented. The retrieval technique of the CBR-ANN approach is also compared with that of traditional CBR system using the kNN technique. All of these are standard algorithms found in the literature and implemented by the authors for comparison.
9.1 Retrieval Efficiency
An experiment had been conducted in measuring the retrieval efficiency of the LVQ3 neural network. The retrieval efficiency can be measured based on the time taken in pre-processing the fault-conditions, the total training time and the average online retrieval speed by the LVQ3 neural network. The experiment was carried out on a 333Mhz Pentium II system with 128MB RAM running under Windows NT operating system. In the experiment, the following data were used. At the time of the experiment, the company had logged a total of 70137 fault cases with regards to the products under review. Among all these cases are 9392 unique fault cases. The number of keywords in the keyword-list is 2,173. The number of words to be searched in the Wordnet’s dictionary is 121,962 and the maximum number of keywords in a fault-condition or in the user input is 20. Table 1 gives the resulting statistics of the experiment. As shown in table 1, although the total training time is quite high, it is still acceptable as the training is carried out only once in an off- line mode. In addition, the average online retrieval time is quite efficient and is in real-time.
Table 1. Statistics on retrieval efficiency of the LVQ3 neural network.
Description Time
Preprocessing of fault-conditions 638.7 sec
Training 96 min 44 sec
Average online retrieval speed 1.9 sec
9.2 Retrieval Accuracy
In supervised neural networks such as LVQ3, the retrieval accuracy can be determined by measuring the number of times the correct fault-conditions generated by the neural network and the number of iterations required. If the user input consists of many new keywords, that are not a part of the keyword list, the accuracy will be affected. However, the system learns to improve its accuracy as it goes on. The learning rate is another important factor in determining the number of iterations required for convergence. In particular, the convergence was found to be fastest for the learning rate of 0.4. For higher learning rate, the system tended to be unstable (i.e. no convergence achieved) whereas for lower learning rate, the number of iterations needed to converge was quite high.
A comparison of the neural retrieval technique of the hybrid CBR-NN approach with the k-nearest neighbour (kNN) retrieval technique in traditional CBR system is given. Two popular variations of kNN techniques were chosen for comparison. The first variation, denoted as kNN1, stores cases in a flat memory structure, extracts keywords from the textual descriptions and uses normalised Euclidean distance for matching. The second variation, known as kNN2, uses the fuzzy-trigram technique (Inference Corporation, 1998) for matching. The performance comparison of the ANN technique with the two variations of the kNN techniques is measured in terms of efficiency and precision. Two experiments were carried out. The first experiment was based on a testing set of service records, while the second experiment was based on user fault descriptions. The results are given in Table 2.
Table 2. Performance comparison between ANN and kNNs.
Retrieval Technique Average Retrieval Time Accuracy of Retrieval (Based on Testing Service Records) Accuracy of Retrieval (Based on User Description) kNN1 15.3 sec 81.4% 72% kNN2 16.7 sec 77.6% 76% ANN 1.9 sec 93.2% 88%
For the first experiment, a training set of 70,137 service records was used to train the neural network. The testing set of 15,850 testing service records was then used for testing. As shown in table 2, the ANN technique performs better than both variations
of the kNN methods for retrieval in both the speed and accuracy because of its ability to generalise information through training. In kNN1 technique using Euclidean distance for matching, it always assigns equal weights to the individual attributes (i.e. keywords). Therefore, the retrieval is less accurate. In kNN2 technique using fuzzy-trigram matching, it assigns positive score for every sequence of 3-letters matched. Although this technique may be useful to check spelling errors and grammatical variations, the retrieval is quite inaccurate when compared with the ANN technique. Moreover, the major drawback in both of these kNN techniques is that, new cases retained are indexed separately into the flat memory structure and thus the search space keeps on increasing, thereby decreasing the efficiency.
In the second experiment, a total of 50 fault descriptions were taken from non-expert users of the system. The purpose of this experiment is to test the performance when the input is less precise in describing the fault-condition. Unlike the service records tested in the first experiment, user input is less accurate when compared with that of service engineers. In this test, all the three retrieval techniques were found to have lower accuracy due to the impreciseness and grammatical variation in the user input. The accuracy of the neural network was 88%. This is in contrast to the 72% accuracy for the nearest neighbour version with Euclidean distance matching and 76% for that with fuzzy-trigram matching. Fuzzy-trigram matching has a better performance than the Euclidean distance matching because of its ability to handle spelling mistakes and grammatical variations in the user input. However, since it does not take into account of the synonymous forms of the input words as in the ANN system, its accuracy is lower than that of ANN.
10. Conclusion
This paper has presented a Web-based intelligent help desk support environment to replace or compliment the traditional phone-based help-desk environment. The two-year research project has yielded an integrated environment that comprises the four main functions of Web-based retrieval, online multilingual translation, video conferencing support and intelligent fault diagnosis.
The proposed system architecture forms a useful help desk framework that can be extended in future in view of emerging research in areas of human computer interaction, artificial intelligence, natural language processing and computer communications. The use of such a “plug and play” paradigm in the system design ensures that such emerging researches can be integrated into the existing framework to further enhance the quality and service level of such a system. The system has been deployed in MNCS for more than one year. As part of future work, a systematic usability study is the next logical progression to carefully evaluate the major functions to obtain user feedback in order to refine and improve the system further.
References
Aamodt, A., and Plaza, E., 1994, Case-based reasoning: foundational issues, methodological variations, and system approaches. Proceedings of AICom –
Aizawa, T., Katoh, N., and Kamata, M., 1996, English-to-Japanese machine translation for wire-service economic news. Transactions of the Information
Processing Society of Japan, 37, (6), 1041-1048.
Althoff, K.D., Auriol, E., Barletta, R., and Manago, M., 1995, A review of Industrial Case-Based reasoning Tools. AI Intelligence, Oxford.
Arnold, D., 1994, Machine Translation: An Introductory Guide (NCC Blackwell). Beckwith, R., Miller, G.A., and Tengi, R., 1993, Design and Implementation of the
WordNet Lexical Database and Searching Software. Cognitive Science Laboratory, Princeton University.
Foo, S., Hui, S.C., Leong, P.C., and Liu, S., 2000, An integrated help desk support for customer services over the World Wide Web – a case study. Computers in
Industry, 41, 129-145.
Fu, A., 1997, Dependency-based Semantic Analysis in Machine Translation.
Language Engineering (Tsinghua University Press, Beijing), pp. 292-297.
Goodman, K., 1989, Machine Translation - Special Issues on Knowledge Based MT. Parts I and II, Morgan Kaufmann Publishers, 4, (1,2).
Goodman, K., and Nirenburg, S., 1991, The KBMT Project: A Case Study in
Knowledge-Based Machine Translation (Morgan Kaufmann Publishers).
Harada, K., 1996, Machine translation software for the Internet. Sanyo Technical
Review, 28, (2), 66-74.
House, W.C., 1994, Automating Help Desk Operations Using Case-Based Reasoning: A Practical Application of Expert Systems Technology. Proceedings of The
Annual Conference of the International Association for Computer Information Systems (Washington, DC, USA), pp. 100-6.
Hui, S.C., and Foo, S., 1998, Towards a Standards-based Internet Telephony System.
Computer Standards and Interfaces, 19, (1), pp. 89-103.
Inference Corporation, 1998. CBR content navigator. http://www.inference.com/products/
Isikawa, T., Nakamura, Y., and Aso, S., 1996, Machine translation systems for Internet/World Wide Web. Oki Technical Review, 62, (157), 85-88.
Jha, G., Hui, S.C., and Foo, S., 1999, A Hybrid Case-Based Reasoning and Neural Network Approach to Online Fault Diagnosis. Proc. 3rd International ICSC Symposia on Intelligent Industrial Automation (IIA’99) and Soft Computing (SOCO’99), Genoa, Italy, pp. 376-381.
Kohonen, T., 1990, The Self-Organizing Map. Proceedings of the IEEE, 78, (9), part I, 464-480.
Law, Y.F.D., Foong, S.W., and Kwan, S.E.J., 1997, An Integrated Case-Based Reasoning Approach for Intelligent Help Desk Fault Management. Expert
Systems with Applications, 13, (4), 265-274.
Lees, B., and Corchado, J., 1997, Case Based Reasoning in a Hybrid Agent-Oriented System. 5th German Workshop on Case-Based Reasoning, pp. 139-144.
Li, T., Eric, H.N., and Jaime, G.C., 1996, Chinese Sentence Generation in a Knowledge-Based Machine translation System. Technical Report
CMU-CMT-96-148, Carnigie Mellon University,
http://www.lti.cs.cmu.edu/Research/Kant/Chinese-tr.ps
Liu, S., 1999, An Intelligent Web-based Helpdesk for Customer Service Support. M.Sc dissertation, Division of Information Studies, School of Applied Science, Nanyang Technological University, Singapore.
Liu, S., Hui, S.C., Foo, S., and Leong, P.C., 1998, Online multilingual translation of technical service records over the World Wide Web. International Journal of
Computer and Engineering Management, 6, (2), 8-22.
Liu, Z.Q., and Yan, F., 1997, Fuzzy Neural Network in Case-Based Diagnostic System. IEEE Transactions on Fuzzy Systems, 5, (2).
Liu, Q., and Zhang, X., 1997, A Software Component Approach to Machine Translation. Language Engineering (Tsinghua University Press, Beijing), pp. 280-285.
McCarthy, D., 1994, Automation of help desks using case-based reasoning. IEE
Colloquium on Case Based Reasoning: Prospects for Applications (London,
UK).
Montazemi, A.R., and Gupta, K.M., 1996, An Adaptive Agent for Case Description in Diagnostic CBR Systems. Computers in Industry, 29, 209-24.
Muller, N.J., 1996, Expanding the help desk through the World Wide Web.
Information Systems Management, 13, (3), 37-44.
Netscape Communications Inc., Netscape Enterprise Server 3.0 (Windows NT). http://www.netscape.com/comprod/server_central/product/enterprise/index.ht ml
Papagni, M., Cirillo, V. and Micarelli, A., 1997, A Hybrid Architecture for a User-Adapted Training System. 5th German Workshop on Case-Based Reasoning,
pp. 181-188.
Patterson, D.W.R., and Hughes, J.G., 1997, Case-Based Reasoning for Fault Diagnosis. The New Review of Applied Expert Systems, 3, 15-26.
Richter, A.G., 2000, PATDEX: A case-based reasoning tool for diagnosis. http://www.agr.infomatik.uni-kl.de
Riesbeck, C.K., and Schank, R.C., 1989, Inside Case-Based Reasoning (Lawrence Erlbaum Associates, Inc.).
Shimazu, H., Shibata, A., and Nihei, K., 1994, Case-Based Retrieval Interface Adapted to Customer-Initiated Dialogues in Help Desk Operations.
Proceedings of the Twelfth National Conference on Artificial Intelligence,
Seattle, WA, USA, pp. 513-18.
Signore, R., Creamer, J., and Stegman, M.O., 1995, The ODBC Solution: Open
Database Connectivity in Distributed Environments (McGraw-Hill, Inc).
Simoudis, E., 1992, Using Cased-Based Retrieval for Customer Technical Support.
IEEE Expert , 7, (5), 7-11.
Watson, I.D., 1997, Applying Case-Based Reasoning: Techniques for Enterprise
Systems (Morgan Kaufman Publishers).
World Wide Web Consortium, 2001a, HyperText Transfer Protocol. http://www.w3.org/hypertext/WWW/Protocols
World Wide Web Consortium, 2001b, Common Gateway Interface. http://www.w3.org/hypertext/WWW/Overview.html
World Wide Web Consortium, 2001c, HyperText Markup Language. http://www.w3.org/hypertext/WWW/Markup
