• No results found

Enhancement and Classification of Less Data Oriented Data set Attributes

N/A
N/A
Protected

Academic year: 2022

Share "Enhancement and Classification of Less Data Oriented Data set Attributes "

Copied!
7
0
0

Loading.... (view fulltext now)

Full text

(1)

P.Sirish Kumar, IJRIT 313

IJRIT International Journal of Research in Information Technology, Volume 1, Issue 12, December, 2013, Pg. 313-319

International Journal of Research in Information Technology (IJRIT)

www.ijrit.com

ISSN 2001-5569

Enhancement and Classification of Less Data Oriented Data set Attributes

1P.Sirish Kumar, 2M.Satish Kumar

1M.Tech – II Year, Akula Sree Ramulu College of Engineering (ASRCE), Tanuku, WGDt, AP

2HOD & Associate Professor, Akula Sree Ramulu College of Engineering (ASRCE), Tanuku, WGDt, AP

1sirishkumar@live.com, 2sk.muthina@gmail.com

Abstract

The Enhancement of Data set attributes is being performed by using data mining procedures that tests real data sets can provide all the information needed for a thorough assessment of their performance characteristics. The Problem in data mining has been studied extensively by several research area. The datasets for data mining applications are usually large and may involve several millions of records with high dimensionality, which make the task of classification computationally very expensive. In this paper we proposed a framework forward classification of such ever-growing large datasets. The framework is scalable with minimal data access easily. Mathematical background for incremental classification based on weighted samples and sampling techniques that extract weighted samples. Forward classification for large ever-growing datasets where fast development of decision tree used in this classification.

I. Introduction

The classification problem is one of the most common operations in data mining. Given a data set of N records with at least m+1 fields: A1,…,Am, and C, the problem is to build a model that predicts the value of the “distinguished”

field C (the class) given the values of m fields. C is categorical (otherwise the problem is a regression problem). The class is a variable of interest (e.g. will this customer be interested in a particular product?). The model can be used for prediction of to gain understanding of data; e.g. a bank manager may want to find out what distinguishes customers who leave the bank from ones that stay. A classifier is an effective means to summarize the contents of a database and browse it. Applications are many including marketing, manufacturing, telecommunications and science analysis [FU96]. Given attributes Ai, i=1,…,m, the problem is to estimate the probability of possible values C=c:

Pr(C cj | A1,..., Am) . There is a huge body of literature on this problem, especially in pattern recognition [DH73]

and statistics [PE96]. We consider classification algorithms in general, and give specific examples for clients that build decision trees [B*84,Q93,FI92b,FI94], classification rules [GS89, Q93], and simple Bayesian networks [K96].

Computational tools from statistics and machine learning communities have not taken into consideration issues of scalability and integration with the DBMS. Most implementations load data into RAM and hence can only be applied on data sets that fit in main memory. The straightforward solution in these settings is to move data (or a sample of it) from the server to the client, then work on this data locally. Moving data has many disadvantages including the cost of crossing the process boundaries and the fact that memory is consumed at the client.

Furthermore, such an approach fails to leverage the query processing and other data management functionality of a DBMS. We assume that data is stored in a SQL database, and focus on the problem of supporting a data mining

(2)

P.Sirish Kumar, IJRIT 314

“classification client”. We first note that most familiar selection measures used in classification do not require the entire data set, but only sufficient statistics of the data. We then show that a straightforward implementation for deriving the sufficient statistics on a SQL database results in unacceptably poor performance. Our proposed solution is based on a new SQL operator that performs a transformation on the data on-thefly and can help a standard SQL backend cope effectively with the problem of extracting sufficient. We demonstrate analytically that our method has better cost performance than other alternatives involving changing the physical layout of data in the database. We also discuss generalizations of the proposed operator.

II. Related Work

Classification algorithms can be de-coupled from data using the notion of sufficient statistics. This suggests a simple architecture (illustrated in Figure 1) in which the database server can support a classification client simply by supplying it with the sufficient statistics, eliminating the need to move data (or copies of it) around. Greedy Classification Algorithms and Data Access

Widely used prediction algorithms for classification in the data mining literature are decision tree classifiers and the Naïve Bayes classifier [K96]. We limit our attention to discrete data, hence we assume that numeric-valued

Q93, FI92a, FI93, DKS95, MAR96]. Decision trees are good at dealing with data sets having many dimensions (common in data mining applications), unlike other approaches for classification such as the nearest neighbor [DH73], neural networks, regression-based statistical techniques [PE96], and density estimation-based methods.

Decision trees can also be examined and understood by humans, particularly the leaves viewed as rules [Q93].

Algorithms reported in the literature generate the tree top-down in a greedy fashion. A partition is selected which splits the data into two or more subsets. There are several partition selection measures. Most are based on preferring a partition which results in locally minimizing the class impurity of each subset in the partition [B*84]. Several measures of impurity exist (e.g. Entropy [Q93], the Gina Index [B*84]). Other measures outside the impurity family have also been used: the Towing Rule [B*84], Gain ratio [Q86,Q93], the J-measure [GS89] and class orthogonally [FI92b]. The scheme proposed in this paper can support all of the above measures.

III. Forward Classification

The data sets for data mining applications are usually large and may involve several millions of records. Each record typically consists of tens to hundreds attributes, some of them with large number of distinct values. In general using large datasets results in the improvement in the accuracy of the classifier but the enormity and complexity of the data involved in these applications make the task of classification computationally very expensive. Since datasets are large, they cannot reside completely in memory, which makes I/O a significant bottleneck. Performing classification for such large datasets require development of new techniques that limit accesses to the secondary

(3)

P.Sirish Kumar, IJRIT 315 storage i8n order to mini8mize the overall execution time. Another problem that makes classification more difficult is that most datasets are growing over time, continuously making the previously constructed classifier obsolete.

Addition of new records constructing a a classifier for large growing dataset from scratch would be extremely waste3ful and computationally prohibitive. Many machine learning techniques and statistical methods have been developed for small datasets. These techniques are generally iterative in nature and require several passes over the dataset, which make them inappropriate for data mi8ning applications. As an alternative,

techniques such as discretization and random sampling can be used to scale up

the decision tree classifiers to large datasets. However, generating good samples is still challenging for these techniques. In some cases the techniques

are still expensive, and the samples may sti8ll be large for some large applications. In addition, these techniques may deteriorate the accuracy of the classifier for datasets

with large number of special cases. We present a framework that incrementally classifies a data base of records which grows over time, and that still results in a decision tree with comparable quality. The framework, named, incrementally builds decision trees for large gr4owing datasets abased on tree-based sampling techniques. The framework is described in greater details. The rest of the paper is organized as follows. A brief survey of related work on decision tree classifiers is given. The mathematical background of incremental classification is presented in together with a few sampling techniques for incremental classification. The framework for incremental classification is proposed in the experimental results and a comparison of the framework against random sampling are provided.

Measures for Classification

A key insight is that the data is not needed if sufficient statistics are available. We only need a set of counts for the number of co-occurrences of each attribute value with each class variable. Since in classification the number of attribute values is not large (in the hundreds) the size of the counts table is fairly small. Continuous-valued attributes are discretized into a set of intervals. All splitting criteria can be estimated in a straightforward fashion if one had a table, which we call the correlation counts table giving the set of counts of co-occurrences of each attribute with each class: : Sufficient Statistics for Classification (CC) Once obtained, there is no further need to refer to the data again. In a decision tree algorithm, the single operation that needs to reference the data are is the construction of CC table. We restrict our attention to the common case where the number of values makes the CC table size negligible compared to the size of the data. This results is a significant reduction in data movement from server to client.

Consider measures used by algorithms for constructing classifiers. Impurity measures, e.g. class entropy measure rely on the probability of any class value Ck in set S:

Pr(C=Ck|S), and measure class impurity of a set S as:

Let

Sj be the subset of S consisting of all data items satisfying Ai=Vij. The class information entropy of a partition induced by attribute Ai is defined

The probabilities are estimated by the frequencies of occurrence of each event in the data subset S.

let CC[i, j, k] denote the count of class Ck when Ai=Vj in S, for j=1,…, ri.

Let

We can equally well score a partition on a single value versus all other values (binary split) or other splits involving a subset of the attributes [F94]. Likewise, many other impurity and statistical measures can be computed strictly from the values in the correlation counts table CC. The same holds true for other measures such as Twoing [B*84]

(4)

P.Sirish Kumar, IJRIT 316 and Orthogonality [FI92]. Variants such as Gain Ratio [Q93] are obtained by dividing the above entropy by the entropy of the values of an attribute (without regard to class); i.e. using the terms Sum(i, j) above. It should be obvious to the reader how Gini Index measure in CART [B*84] is derived from CC. Many other measures of interest in statistics are a function of only the entries of the CC table. For example, a popular and simple model is known the Naïve Bayes classifier. If one were to assume that all the attributes in the data are conditionally independent given the class, then one can reliably estimate the probability of the class variable using a simple application of Bayes rule. The UNION is needed to obtain counts for each of the attributes A1,…,Am with the class. We introduce the condition some_condition to obtain the counts for subsets of the data (e.g. individual nodes in a decision tree, etc.) Most database systems will implement such UNION queries by doing a separate scan for each clause in the union since the optimizers will be unable to harness the commonality among the UNION queries.

Observe that the form of the SQL statement using UNION is different from the CUBE operation proposed in [GC*97]. Unlike CUBE, the grouping columns only share the class attribute and no grouping is required for combinations of other attributes. Since we intend to perform such queries over large tables with many columns (hundreds), m scans become quite expensive, we consider methods for avoiding the multiple scans.

IV. Test Points for Classification

The main idea of ANNCAD is to use a multi-resolution data representation for classification. Notice that the neighborhood relation strongly depends on the quantization process. This will be addressed in next subsection by building several classifier ensembles using different grids obtained by subgrid displacements. Observe that in general, the finer level the block can be classified, the shorter distance between this block and the training set.

Therefore, to build a classifier and classify a test point (see Algorithms 1 and 2), we start with the finest resolution for searching nearest neighbors and progressively consider the coarser resolutions, in order to find nearest neighbors adaptively. We first construct a single classifier as a starting point (see Algorithm ). We start with setting every block to have a default tag U (Non-visited). In the finest level, we classify any nonempty block with its majority class label. We then classify any nonempty block of every lower level as follows: We label the block by its majority class label if the majority class label has more points than the second majority class by a threshold percentage. If not, we use a specific tag M (Mixed) to label it.

Algorithm . BuildClassifier({x, y}|x is a vector of attributes, y is a class label.) Quantize the feature space containing {x}

Label majority class for each nonempty block in the finest level For each level i = log(g) downto 1

For each nonempty block B

If |majority ca| − |2nd majority cb| > threshold %, label class ca else label tag M

Return Classifier

Further Updates of Classifiers

The main requirement of a data stream classification algorithm is that it is able to update classifiers incrementally and effectively when a new tuple arrives. Moreover, updated classifier should be adapt to concept drift behaviors. In

(5)

P.Sirish Kumar, IJRIT 317 this subsection, we present incremental update process of ANNCAD for a stationary data, without re-scanning the data and discuss an exponential forgetting technique to adapt to concept drifts. Because of the compact support property, arrival of a new tuple only affects the blocks of the classifier in each level containing this tuple. Therefore, we only need to update the data array of these blocks and their classes if necessary. During the update process, the system may run out of memory as the number of nonempty blocks may increase. To deal with this, we may simply remove the finest data array, multiple the entries of the remaining coarser data arrays by 2d, and update the quantity g. A detailed description of updating classifiers can be found in Algorithm1. This solution can effectively meet the memory constraint.

V. Performance Evaluation

In this section, we first study the effects on parameters for ANNCAD by using two synthetic data sets. We then compare ANNCAD with VFDT and CVFDT we include the results of Exact ANN, which computes ANN exactly, as controls. Exact ANN: For each test point t, we search the area within 0.5 block side length distance. If the area is nonempty, we classify t as the majority label of all these points in this area. Otherwise, we expand the searching area by doubling the radius until we get a class for t. Note that the time and space complexities of Exact ANN are very expensive making it impractical to use.

Synthetic Data Sets

The aim of this experiment is to study the effect on the initial resolution for ANNCAD. In this synthetic data set, we consider a 3-D unit cube. We randomly pick 3k training points and assign those points which are inside a sphere with center (0.5, 0.5, 0.5) and radius 0.5 to be class 0, and class 1 otherwise. This data set is effective to test the performance of a classifier as it has a curve-like decision boundary. We then randomly draw 1k test points and run ANNCAD starting with different initial resolution and 100% threshold value. In Fig. 6(2), the result shows that a finer initial resolution gets a better result. This can be explained by the fact that we can capture a curve-like decision boundary if we start with a finer resolution. On the other hand, as discussed in last section, the time spent for building a classifier increases linearly for different resolutions. In general, we should choose a resolution according to system resource constraints.

Fig2. Effect on initial resolutions and number of classifiers

The aim of this set of experiments is to compare the performance of ANNCAD with that of VFDT and CFVDT on stationary and time-changing reallife data sets respectively. We first used a letter recognition data set from the UCI machine learning repository web site [2]. The objective is to identify a black-and-white pixel displays as one of the 26 English alphabet. In this data set, each entity is a pixel display for an English alphabet and has 16 numerical attributes to describe its pixel displays. The detail description of this data set is provided in [2]. In this experiment, we use 15k tuples for training set with 5% noise added and 5k for test set. We obtain noisy data by randomly assigning a class label for 5% training examples. For ANNCAD, we set g for the initial grid to be 16 units and build two classifiers. Moreover, since VFDT needs a very large training set to get a fair result, we rescan the data sets up to 500 times for VFDT. So the data set becomes 7,500,000 tuples the performance of ANNCAD dominates that of VFDT. Moreover, ANNCAD only needs one scan to achieve this result, which shows that ANNCAD even works well for a small training set. The second real life data set we used is the Forest Cover Type data set which is another

(6)

P.Sirish Kumar, IJRIT 318 data set from [2]. The objective is to predict forest cover type (7 types). For each observation, there are 54 variables.

Neural network (backpropagation) was employed to classify this data set and got 70% accuracy, which is the highest one recorded in [2]. In our experiment, we used all the 10 quantitative variables. There are 12k examples for training set and 90k examples for testing set. For ANNCAD, we scaled each attribute to the range [0, 1). We set g for the initial grid to be 32 units and build two classifiers. As the above experiment, we rescan the training set up to 120 times for VFDT, until its performance becomes steady. The performance of ANNCAD dominates that of VFDT.

These two experiments show that ANNCAD works well in different kinds of data sets.

VI. Conclusion

The field of data mining, and has remained an extensive research topic within several research communities. The data collection technologies and large scale business enterprises, the datasets for data mining applications are usually large and may involve several millions of records. Each record typically consists of ten to hundreds of attributes some of them with large number of distinct values. The enormity and complexity ofr the data involved in these applications make the task of classification computationally very extensive. In this paper we propose a scalable framework named for incrementally classi8fying ever-growing large datasets. The framework has the following properties temporal scalability, algorithm independence, size scalability, minimal data access, inherent parallelism, well-suited for distributed, flexibility to addition and deletion of partitions. We provide incremental classification based on weighted samples and a few sampling techniques that extract weighted samples from a decision tree built on a small data partition. The weighted samples from the previous partitions are used to build a new decision tree classifier together with the current partition are used to build a new decision tree.

References

1. R. Agrawal, J. Gehrke, D. Gunopulos and P. Raghavan. Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications. SIGMOD 1998: 94–105.

2. C. L. Blake and C. J. Merz. UCI Repository of machine learning databases.

http://www.ics.uci.edu/∼mlearn/MLRepository.html.

3. B. Babcock, S. Babu, R. Motawani and J. Widom. Models and issues in data stream systems. PODS 2002: 1–16.

4. J. Bentley. Multidimensional binary search trees used for associative searching. Communication of the ACM 18(9): 509–517 (1975).

5. F. Chu and C. Zaniolo. Fast and light boosting for adaptive mining of data streams. PAKDD 2004: 282–292.

6. C. Domeniconi, J. Peng and D. Gunopulos, Locally adaptive metric nearestneighbo classification, IEEE Transactions on Pattern Analysis and Machine Intelligence 24(9): 1281–1285 (2002).

7. P. Domingos and G. Hulten. Mining high-speed data streams. KDD 2000: 71-80.

8. L. Golab and M. ¨Ozsu. Issues in data stream management. ACM SIGMOD 32(2): 5–14 (2003).

9. J. Han and M. Kamber. Data Mining – Concepts and Techniques (2000). Morgan Kaufmann Publishers.

10. G. Hulten, L. Spence and P. Domingos. Mining time-changing data streams. KDD 2001: 97–106.

11. Jimenez, L., Landgrebe, D. High Dimensional Feature Reduction Via Projection Pursuit. PhD Thesis and School of Electrical & Computer Engineering Technical Report TR-ECE 96-5, (1995)

12. Jolliffe, I.T. Principal Component Analysis. Springer, New York, NY, (1986)

13. Kiang, M. A comparative assessment of classification methods, Decision Support Systems, Vol. 35, (2003) 441- 454

14. Kohavi, R., Sommerfield, D., Dougherty, J. Data mining using MLC++: a machine learning library in C++.

Tools with Artificial Intelligence, IEEE CS Press, (1996) 234-245

15. Liu, H. Feature Extraction, Construction and Selection: A Data Mining Perspective, ISBN 0-7923-8196-3, Kluwer Academic Publishers (1998)

16. Michalski, R.S.. Seeking Knowledge in the Deluge of Facts, Fundamenta Informaticae, Vol. 30, (1997) 283-297 17. Oza, N.C., Tumer, K. Dimensionality Reduction Through Classifier Ensembles. Technical Report NASA-ARC- IC-1999-124, Computational Sciences Division, NASA Ames Research Center, Moffett Field, CA, (1999)

18. Quinlan, J.R. C4.5 Programs for Machine Learning. San Mateo CA: Morgan Kaufmann, (1993)

(7)

P.Sirish Kumar, IJRIT 319 19. Tsymbal A., Pechenizkiy M., Puuronen S., Patterson D.W. Dynamic integration of classifiers in the space of principal components, In: L.Kalinichenko, R.Manthey, B.Thalheim, U.Wloka (Eds.), Proc. Advances in Databases and Information Systems: 7th East-European Conf. ADBIS'03, Lecture Notes in Computer Science, Vol. 2798, Heidelberg: Springer-Verlag (2003) 278-292

20. Tsymbal A., Puuronen S., Pechenizkiy M., Baumgarten M., Patterson D. Eigenvectorbased feature extraction for classification. In Proc. 15th Int. FLAIRS Conference on Artificial Intelligence, Pensacola, FL, USA, AAAI Press (2002) 354-358

21. Ch.Veeranjaneyulu, Ravi Mathey Attribute Information for Incremental Classification of Large DataSets

Author’s Bibilography

1 P.Sirish Kumar , studying M.Tech –II Year in Computer Application in Akula Sree Ramula college of Engineering.

2Mr.M.Satish Kumar, HOD & Associate Professor in ASR College of Engineering, Tanuku, WGDt, AP.

References

Related documents

Field experiments were conducted at Ebonyi State University Research Farm during 2009 and 2010 farming seasons to evaluate the effect of intercropping maize with

Al-Hazemi (2000) suggested that vocabulary is more vulnerable to attrition than grammar in advanced L2 learners who had acquired the language in a natural setting and similar

Results suggest that the probability of under-educated employment is higher among low skilled recent migrants and that the over-education risk is higher among high skilled

MAP: Milk Allergy in Primary Care; CMA: cow’s milk allergy; UK: United Kingdom; US: United States; EAACI: European Academy of Allergy, Asthma and Clinical Immunology; NICE:

It was decided that with the presence of such significant red flag signs that she should undergo advanced imaging, in this case an MRI, that revealed an underlying malignancy, which

Also, both diabetic groups there were a positive immunoreactivity of the photoreceptor inner segment, and this was also seen among control ani- mals treated with a

19% serve a county. Fourteen per cent of the centers provide service for adjoining states in addition to the states in which they are located; usually these adjoining states have

Aptness of Candidates in the Pool to Serve as Role Models When presented with the candidate role model profiles, nine out of ten student participants found two or more in the pool