Web Log Mining

distance education environment (Ha, Bae and Park, 2000) [2]; e-articles for students based on key-word-driven text mining (Tang et al., 2000) [3], and to analyze learners’ learning behaviors (Zaiane and Luo, 2001) [4].Yan Li [5] presented a detail algorithm method for web usage mining implementation of the data preprocessing system. After identifying the user session, the referrer based method is used to find the user's access path which is attached with an effective solution to the problems with proxy servers and local caching. Hussain .T, Sahail Asghar [6] proposed in the preprocessing level framework for web session cluster of usage mining. It covers the steps to prepare the log data and it converted into numerical data. Doru Tanasa [7] the research describes two main contributions to WUM process (i.e.) for preprocessing the web logs and a divisive with three approaches for the discovery of sequential patterns with a support. The algorithm used for the processing the web log records and obtaining the set of frequent access patterns have been implemented by huiping Peng[8]. An improved preprocessing expertise has been used by ling Zheng [9] for the purpose of solving some existing issues in traditional information preprocessing in web log mining. JIANG Chang-bin and Chen Li [10] says that, even if the statistical data are not enough and absence of visiting user history records, the web log data preprocessing algorithm based on collaborative filtering identifies the session flexibly and quickly.

B.Uma Maheswari & Dr.P.Sumathi presented preprocessing method for web log mining is the important method & preprocessing plays a vital role in efficient mining process as log data is normally noisy and indistinct [2].Reconstruction of sessions and paths are completed by appending missing pages in preprocessing. Additionally ,the transactions which illustrate the behaviors of n users are constructed exactly in preprocessing by calculating the reference length of user access by means of byte rate. Using Web clustering several types of objects can be clustered into different groups for various purposes. Supinder Singh presented the review the existing web usage clustering techniques and proposed a swarm intelligence based PSO clustering algorithm for the

This paper aims at the research on the preprocessing procedure of web log mining, uses ID3 algorithm to improve Frame page filtration method, adds pages with largest information gain to Frame page, and adds the unsuitable ones to the SubFrame page, And the SubFrame has not been deleted, which reduced the step of upgrading the site structure in path supplement. Meanwhile, adopting dynamic correction method through time-out threshold value in session identification can strengthen the recognizing ability of comparatively long time session in preprocessing procedure. Applying the improved methods of this paper to the web log mining preprocessing procedure can increase the interest of mining results.

Web log mining technique is apply to improve web services and mining web navigation patterns efficiently. Sagar More [3] proposed a method for mining path traversal patterns. In that paper they first clear the concept of throughout surfing pattern (TSP) which predict the path of website visitor. In next part they apply modified graph traverse algorithm to make mining from TSP in efficient manner. The modified algorithm use the filtering technique which removes duplicate and unwanted data in web browsing session and show the effectiveness as compared to formal algorithms.

The objective of the paper is the verifi cation of the fulfi lment of the purposes of Basel II, Pillar 3 – market discipline during the recent fi nancial crisis. The objective of the paper is to describe the current state of the working out of the project that is focused on the analysis of the market participants’ interest in mandatory disclosure of fi nancial information by a commercial bank by means of advanced methods of web log mining. The output of the realized project will be the verifi cation of the assumptions related to the purposes of Basel III by means of the web mining methods, the recommendations for possible reduction of mandatory disclosure of information under Basel II and III, the proposal of the methodology for data preparation for web log mining in this application domain and the generalised procedure for users’ behaviour modelling dependent on time. The schedule of the project has been divided into three phases. The paper deals with its fi rst phase that is focusing on the data pre- processing, analysis and evaluation of the required information under Basel II, Pillar 3 since 2008 and its disclosure into the web site of a commercial bank. The authors introduce the methodologies for data preparation and known heuristic methods for path completion into web log fi les with respect to the particularity of investigated application domain. They propose scientifi c methods for modelling users’ behaviour of the webpages related to Pillar 3 with respect to time.

World Wide Web is a huge repository of web pages and links. It provides profusion of information for the Internet users. The growth of web is tremendous as approximately one million pages are added daily. User’s accesses are recorded in web logs. Because of the incredible usage of web, the web log files are growing at a faster rate and the size is becoming huge. Web data mining is the application of data mining techniques in web data. Web log Mining applies mining techniques in log data to extract the behaviour of users. Web log mining consists of three phases pre-processing, pattern discovery and pattern analysis. Web log data is usually noisy and ambiguous and pre-processing is an important process before web log mining. For discovering patterns sessions are to be constructed efficiently. This paper presents the existing work done to extracting patterns by using decision tree methodology in the technique of web log mining.

Abstract: The methodological approach to achieve the elicited research objectives is presented in this study. The study will employ an experimental methodology involving testing of algorithms to identify the suitable algorithm for an optimized process of information retrieval. The algorithm that presents a better precision in the personalized IR in web log mining represents the core value of the architecture that this study presents.

ABSTRACT: Today is the era of internet. People directly or indirectly associated with the World Wide Web. Most of the works from small electric bill payment to bank transactions, from shopping to big business deals all can be easily done via internet. It is treated as a new source of entertainment, communication, education, etc. Apart from using internet people also take interest in sharing their personal suggestions and feedback in various micro-blogging sites (such as twitter), blogs, forums, social networks, etc. These reviews are very useful for the buyers in making decision as well as for the manufacturers or retailers to improve their quality product or services. As a result, a wide collection of consumer reviews are available on the net. It is a tedious task to go through each and every review before any purchase, so here arises the need of such system which helps in information retrieval. Also one major problem is fake reviewer problem or opinion spamming in the system, which highly affect the marketing of the product. In this paper we propose a product ranking framework which rank the product according to their quality and also filter the spam reviews by adopting the web log mining techniques.

Web mining advantages makes this technology useful to corporations including the government agencies. In personalized marketing, E-commerce is enabled which eventually results in higher trade volumes. To classify threats and fight against terrorism, the government agencies are using this technology. By utilizing the acquired insight of customer requirements the corporations can find, attract and retain customers. In web mining client profiles are created which can be utilized by companies which can increase their profit. Companies can even retain the customer by providing promotional offers, thus reducing the risk of losing customers.

The temporal nature of the events themselves, however, can likely be misinterpreted by current algorithms. We present a new definition of the temporal aspects of events and extend related work for pattern finding not only by making use of intervals between events but also by utilizing temporal relations like meets, starts, or during. The result is a new algorithm for Temporal Data Mining that preserves and mines additional time-oriented information. [21] Also in “Effect of Temporal Relationships in Associative Rule Mining for Web Log Data” by Nazli Mohd Khairudin, Aida Mustapha, and Mohd Hanif Ahmad it has been discussed that with The advent of web-based applications and services has created such diverse and voluminous web log data stored in web servers, proxy servers, client machines, or organizational databases. This paper attempts to investigate the effect of temporal attribute in relational rule mining for web log data. We incorporated the characteristics of time in the rule mining process and analyzed the effect of various temporal parameters. The rules generated from temporal relational rule mining are then compared against the rules generated from the classical rule mining approach such as the Apriori and FP-Growth algorithms. The results showed that by incorporating the temporal attribute via time, the number of rules generated is subsequently smaller but is comparable in terms of quality [22].

Abstract: It is the strategy to separate the client sessions from the given log documents. At first, every client is recognized by his/her IP address determined in the log record and comparing client sessions are removed. Two kinds of logs ie., server-side logs and customer side logs are regularly utilized for web utilization and ease of use examination. Server-side logs can be consequently created by web servers, with every section relating to a client ask. Client side logs can catch precise, complete use information for ease of use investigation. Ease of use is characterized as the fulfillment, productivity and viability with which particular clients can finish particular assignments in a specific domain. This procedure incorporates 3 phases, to be specific Data cleaning, User distinguishing proof, Session recognizable proof. In this paper, we are actualizing these three stages. Contingent on the recurrence of clients going by each page mining is performed. By finding the session of the client we can investigate the client conduct when spend on a specific page.

The feature weights are used to represent the user sessions instead of the fuzzy weights. The occurrence of the URL session is taken as a reference for the feature weights. The time a particular user spends on the web page or the number of bytes the user downloaded from the page is taken as the frequency of the occurrence. In the access logs taken from the user is of larger size, hence the URL of the user access is taken. The log dataset is a high dimensional data so distance based clustering is less efficient in clustering the user logs. In order to improve the clustering results the logs are filtered by removing the references to low support URLs which are not supported by the specified number of user sessions. A fuzzy based theoretic approach is used to improve the dimensionality reduction. We propose this approach to improve the clustering accuracy. We assign weights to all the URLs using the fuzzy set theoretic function. All the URLs with the session count less than α1 are assigned the weight 0.The support count higher than the session count is assigned the weight 1. The remaining weight is assigned between 0 to 1 which the having the count between α1 and α2.

Log files are files that list the actions that have been occurred. These log files reside in the web server. Computers that deliver the web pages are called as web servers. The Web server stores all of the files necessary to display the Web pages on the user’s computer. All the individual web pages combines together to form the completeness of a Web site. Images/graphic files and any scripts that make dynamic elements of the site function. , The browser requests the data from the Web server, and using HTTP, the server delivers the data back to the browser that had requested the web page. The browser in turn converts, or formats, the files into a user viewable page. This gets displayed in the browser. In the same way the server can send the files to many client computers at the same time, allowing multiple clients to view the same page simultaneously.

Abstract - At present in our day to day life internet plays a very important role. It has become a very vital part of human life. As internet is growing day by day, so the users are also expanding at much greater rate. Users spend lot of time on internet depending on the behavior of different user. Internet provides huge amount of information and from this information knowledge is extracted for the users. This extraction of information demands for the new logics and method. The data mining techniques and applications can be used in web based applications for performing this job which is also known as web mining. Web based mining or web usage mining is one of the trending topics nowadays. When user uses internet or visits some web pages, the associated information are stored in the server log files. Using these log files of server the human nature or behavior can be predicted. This paper focus on the web based mining and how it can be can be used to predict the human behavior using the server log files. The paper contains some of the techniques and methods associated with web mining.

Figure1: Categorization of Web Data mining Web content mining (WCM) is to find useful information in the content of web pages[4] e.g. free Semi-structured data such as HTML code, pictures, and various unloaded files. Web structure mining (WSM) is use to generating a structural summary about the web site and web pages. Web structure mining tries to discover the link structure of the hyperlinks at the inter document level. Web content mining mainly focuses on the structure of inner document, Web usage mining (WUM) is applied to the data generated by visits to a web site, especially those contained in web log files. I only highlighted and discussed research issues involved in web usage data mining. Web usage mining (WUM) or web log mining, users’ behavior or interests is revealed by applying data mining techniques on web. Three main sources of web log file are

ABSTRACT: Web usage mining (WUM) is the technique of extracting useful and interesting patterns from web access log. Web access log is the log-file that contains registered user access by the server. WUM is otherwise called as web log mining that is used to understand visitor’s behavior and navigation or routing preferences of the visitor mainly for evaluating the effectiveness of the website and for enhancing the quality of the website respectively. The aim of discovering frequent patterns in web log data is to obtain information about the navigational behavior of the user. This can be used for advertising purpose, for creating dynamic user profiles, etc. The click-streams generated by various users often follow distinct patterns. The proposed distributed mining algorithm is Group Movement Pattern Mining that identifies a group of visitors with similar patterns. The proposed algorithm comprises of two phases that are local mining phase and cluster ensembling phase. In local mining phase, the system finds navigation pattern of each user and compute the similarity between the users to identify the local groups. In cluster ensembling phase, the global group can be identified by clustering local groups. Distributed mining algorithm achieves good grouping quality.

Now a day a Web is a distributed, very largest dynamic data repository, global information service [1] to provide the services such as advertisements, news, financial management, education, government, e-commerce etc. It contains web pages information, user accessing hyperlink information and usage information. Web Usage mining (WUM)[3] is a web mining technique which is used to discovery and analysis of web usage pattern from web logs. It is also called web log mining. Web Usage mining (WUM) is one of the method of identifying user’s browsing patterns, with the means of knowledge gained from web logs over web. The results of the WUM can be used in[5] improving the speed of the system, modification of the site, web personalization, usage characterization, business intelligence etc.

This paper has used consumer electronics oriented dataset which consists of 1000 sessions, 150 users, 120 products and 30 classes. The evaluation method of this work is based on the techniques introduced in [15] which define 3 parameters accuracy, precision and F1 Metric. The effectiveness of the recommendation is measured in terms of coverage and precision. 10-fold cross validation is performed for each of the data sets. Each transaction t in the test set is divided into two parts. The first part is the first n items in t for recommendation generation. n is called the window count. The other part, which is denoted as eval, is the remaining portion of t to evaluate the recommendation. Once the recommendation phase produces a set of products, which is denoted as Rec, the set is compared with eval products. This paper splits the session log to 90 % and 10 % at the last. For the users in the 10 % list, this paper finds the recommendation and matching to the recommendation already bought now.

information from WWW into more structured form and indexing the information to retrieve it quickly. Web usage mining is the process of identifying the browsing patterns by analyzing the user’s navigational behavior. Web structure mining is to discover the model underlying the link structures of the Web pages, catalog them and generate information such as the similarity and relationship between them, taking advantage of their hyperlink topology. Web classification is shown in Fig 1.

C.I. Ezeife and Y. Lu [6] in 2005 proposed a more efficient approach for using the WAP-tree to mine frequent sequences, which totally eliminates the need to engage in numerous re-constructions of intermediate WAP-trees during mining. This method builds the frequent header node links of the original WAP-tree in a pre-order fashion and uses the position code of each node to identify the ancestor/descendant relationships between nodes of the tree. It then finds each frequent sequential pattern, through progressive prefix sequence search, starting with its first prefix subsequence event. Sequential pattern technique is useful for finding frequent web access patterns. But the problem of mining sequential patterns from web logs is based on Web Access Sequence Database (WASD). WASD is used to indicate the number of access sequences in the database. This uses PLWAP algorithm for efficiently mining sequential patterns from web log. In order to avoid recursively re-constructing intermediate WAP-trees, pre-order frequent header node linkages and position codes are proposed. The PLWAP algorithm adapts the WAP-tree structure for storing frequent sequential patterns to be mined.