PRODUCT SENTIMENT CLASSIFICATION AND VISUALIZATION USING NEURAL NETWORK

(1)

________________________________________________________________________________________________

IRJCS: Impact Factor Value – SJIF: Innospace, Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80

PRODUCT SENTIMENT CLASSIFICATION AND VISUALIZATION USING NEURAL NETWORK

Tushar Upadhyay

Electronics and Communication Engineering

SRM Institute of Science and Technology, Kattankulathur, Chennai, India [email protected]

Sayantan Saha

Electronics and Communication Engineering SRM Institute of Science and Technology

Kattankulathur, Chennai, India [email protected]

Diya Venugolpal

Electronics and Communication Engineering SRM Institute of Science and Technology

Kattankulathur, Chennai, India [email protected] Manuscript History

Number: IRJCS/RS/Vol.05/Issue10/NVCS10080 Received: 07, April 2018

Final Correction: 13, April 2018 Final Accepted: 21, April 2018 Published Version I: April 2018 Updated Version II: November 2018

Citation: Upadhyay, Saha & Venugolpal (2018). PRODUCT SENTIMENT CLASSIFICATION AND VISUALIZATION USING NEURAL NETWORK. IRJCS:: International Research Journal of Computer Science, Volume V, 492-500.

doi://10.26562/IRJCS.2018.NVCS10080

Editor: Dr.A.Arul L.S, Chief Editor, IRJCS, AM Publications, India

Copyright: ©2018 This is an open access article distributed under the terms of the Creative Commons Attribution License, Which Permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Abstract—Behavioural Science is the study of human behaviour in different contexts, situation and time.

Investigating about past human behaviour can help us calculate human behaviour in the future. In this paper we are analysing the public opinion of a product, as available on social media sites specifically Twitter. Our end goal is to visually represent the vital business insights that cannot be gathered from a plain dataset that can assist in developing further intelligent solutions. Sentence sentiment classification is a predictive modelling task achieved through supervised learning. Here, the extracted sentence is segregated into two target variables i.e. positive and negative stances using Natural Language Processing (NLP), through the utilization of neural networks.

Categorization of the public sentiment will help in market testing, public anticipation of the product and public sentiment analysis. Market testing involves assessing the risks involved, gathering the bias people cradle and determining our prospective customers. Once the product is launched it is essential to understand how people judge the product. This will provide a platform for developers to improve in the future. As text is a sequence of information and not simply a discrete representation we need an iterative process to train the model, thus the application of RNN. For training the model, we mined our own dataset from twitter API so that the model got accustomed to the natural trend of writing, as in tweets. Using specific keywords, we gathered all posts and tweets related to our product. After data collection and cleaning, the polarity of sentences is found. A special kind of neural network called as the convolution LSTM-RNN is used to train the machine.

(2)

________________________________________________________________________________________________

© 2014- 18, IRJCS- All Rights Reserved Page -493 Live social media data is given as an input to the model. Data is processed and distributed in its respective class. This result is stored in MSSQL server to which the Tableau’s dashboard is connected. Using Tableau, we perform visual analytics on the collected data, where we can classify tweets geographically to understand location wise reaction.

Having gathered this data a company might reach out to dissatisfied customers with solution to their predicament, thereby improving customer relation. Not only can we gather the notion people hold about a product but also about competing products.

Keywords—Sentiment Analysis; NLP; Convolution LSTM; MSSQL; Tableau; Business Insight; Visual Analytics;

I. INTRODUCTION

An important part of any well performing business model is customer feedback. Using this feedback, we get to know whether the product satisfies the needs of the consumer. It tells the manufacturer how well his product are performing in the market. Product reviews as such can point out any longstanding defect in productions that's causing product failures. We can draw links as to what the major drawbacks or USP of the product is. However, getting the feedbacks from a large customer base becomes a daunting task. The customers usually do not like filling huge feedback forms, even if they are forced, its filled blindly. Twitter reviews can be tapped to use as natural feedbacks within a word limit. User tweets may express pleasure or displeasure as to how the product appeals to them.

The rating system is widely popular. Nonetheless, the rigid format of rating does not permit the user to express himself freely and be vocal about his thoughts pertaining to the product. Different users have subjective levels as to how they rate. Some reviewers are very giving in nature while others aren't. Blogging/social media websites are often used by dissatisfied/pleased customers to rant about their experiences with a product. These tirades, when tapped into correctly, can provide useful insights into ameliorating the product and improving the business model.

For the purpose of this paper, the microblogging website twitter is used to obtain these responses.

A myriad of tweets are posted here on a daily basis, up to 6,000 tweets are made every second which corresponds to 3,50,000 tweets per minute and more than 500 million tweets per day[1]. This data can be used to analyse the sentiment of the users regarding a particular product, or rather, as an important parameter to gauge the popular public position onthe same. In paper [2]text data is understood with the help of deep learning and a sentiment analysis was performed on DBpedia ontology classification dataset and result of 99.96%Training accuracy and 98.7% Testing accuracy was achieved. In [3] TreeTagger was used to tag the training data with positive, negative and neutral tags which then was used to build a N-gram Naive Bayes Classifier for sentiment analysis.In [4] dictionary method was used to find the semantic pattern of adverbs and verbs and corpus-based methods were used to find the semantic pattern of adjectives, then a linear equation was used to obtain the sentiment of the sentence. Opinion words were used to get the sentiments from the data. In [5] numerous methods such as tree kernel model,100senti- features, unigram model and combination of the previously mentioned models were used to show how the models were more efficient than a simple unigram model. A dataset which was pre-classified by humans was used for training the models. In [6] emoticons were used as noisy data labels in the data set for training the distant supervised algorithms like Naive Bayes, Maximum Entropy Classification and Support Vector Machines. These models showed good performance in classifying the tweets into sentiments.

Fig 1. Convolution LSTM Neural Network

(3)

________________________________________________________________________________________________

© 2014- 18, IRJCS- All Rights Reserved Page -494 In [7] numerous different classifiers have been used alone such as rule based, statistics based etc. They have also been designed such that one classifier can be used piggy backing on the other while the results of one aids in the function of the next.

In [8] data from Facebook page were mined and a sentiment analysis was performed using the NLTK package in python, stored in Mongo database and displayed on Jango dashboard. In [9] a series of combination of recursive and convolution neural networks were used for sentiment analysis. Stanford tree level sentiments were used to show that phrase level features produced by convolution networks were combined by recursive neural networks.

Convolution LSTM Neural Network Our paper focuses on extracting the product related tweets which are then analysed using a neural network and the obtained results are displayed using Tableau. This Tableau dashboard is accessible from any device.In our paper we have used a combination of convolution and LSTM Neural Network for sentiment classification and its performance is observed. Tableau provides a very flexible platform for creating visually appealing dashboards which is used for visual analytics.

For our paper we use a combination of convolution and LSTM neural network for predicting the category of the sentiment of tweets. The following are the different layers present in the structure as shown in Fig 1.:

A. Word Embedding Layer

A word embedding is function with different parameters which maps words to greater dimensional vectors for the computer to examine the corpus (collection of a document) and pass the vector as input for the neural network.

W(words)⇾ n (1)

The different words are mapped to a function .

( ) = (2)

Wθ being the vector and θn being the function.

For the word “Bat”,

W(“BAT”)=(0.5,-0.7,0.14…..) (3)

Here word embedding provides a magnitude for each word in the corpus to showcase the significance of a word in a document. The RHS of the above equation shows the vector for the word “bat” in the corpus. Thus, the function gives an idea about the usage of a word in a sentence. In sentences, there are many words used and an analysis needs to be done in order to find the occurrence/significance of different possibilities of words coming one after another in a particular sequence.

Now, any sequence of words from i to j, are present as the following vector

Xi : j = xi ⊕ xi+1 ⊕ . . . ⊕ xj (4) where w(n) represents the vector function for each word.

X(i:j) is a matrix which stores the vectors for the individual words. In Eq(4).⊕ is concatenation operation, and this results in the matrix

Xi : j ∈ Rn×(j−i+1) (5)

B. Convolution layer

The convolution layer plays a vital role in focusing on learnable kernels. Each kernel has its individual characteristic activation maps which when stacked upon one another produce the convolution layer. Depth, stride and zero padding are the hyperparameters used for tuning the efficiency of the convolution layer. Depth is the depth of the output volume produced by the convolution layers, stride is used for setting the depth near the input in order to accommodate the receptive field to represent which point to begin the calculation from and zero padding is the procedure to pad the input to a specific dimension. Having provided these parameters beforehand the convolution layer by itself creates two dimensional activation filters on which the inputs are convoluted. It's called an activation map as it processes the input or is used to activate parts of the input to give us the output vector.

These kernels in the convolution layer have smaller spatial dimensionality (dimensionality of the kernels are smaller as compared to the input vector) but it tends to spread along the entire input which is lead to the convolution layer.

Once the input arrives at this layer, every filter across the dimensionality of the input matrix are convolved together.

Given F is the set of all filters present in the convolution layer with dimensionality window of size l.

The parameter θ(v) = {W (v), b (v) |W (v) ∈ R n×l ,b (v) ∈ R} (6)

(4)

________________________________________________________________________________________________

© 2014- 18, IRJCS- All Rights Reserved Page -495 Any filter “v” from “F” can be applied to any sequence of words from (i+l-1) using the following formula. V is one of the filters from the collection of filter set F

c (v) j = f (W (v) ⊗ Xi:i+l−1 + b (v) ) (7)

Where r ⊗ is the Hadamard product, b ∈ R is bias term and f is the activation function C. Max Pooling Layer: A pooling layer reduces the dimension of the input layer provided to LSTM but not the depth of it as the input is a three-dimensional vector. This increases the performance of the system, reduces the translation invariance and reduces the chances of overfitting the model. The input of this layer being- H1*W1*Depth*N (8)

The output of this layer being - H2*W2*Depth*N (9)

Where H2=(H1-K)/(S+1) (10)

And W2=(W1-K)/(S+1) (11)

K and S being the kernel and stride. D. LSTM layer A LSTM unit is composed of a cell, input gate, output gate and memory cell. The LSTM cell is made in such a way so as to remember certain information at some interval of time. The conventional RNN faces the issue of vanishing and exploding gradient problem while training which makes the training of such models for text processing unstable and unreliable. The LSTM on the other hand caters to this problem thus making the process of training the classifier system more efficient and reliable. The LSTM can be explained with the following formulae: wt = σ (W^(w)it + U^(w)ht−1 + b^(w)) (12)

ft = σ (W^{(f )}it + U^{(f )}ht−1 + b^{(f )}) (13)

ot = σ (W^(o)it + U^(o)ht−1 + b^(o)) (14)

ut = tanh (W^(u)it + U^(u)ht−1 + b^(u)) (15)

ct= rt ⊙ ut + ft ⊙ ct−1 (16)

ht = ot ⊙ tanh(ct ) (17)

The ⊙ operation denotes element-wise vector product. Usually, the notations wt,ftandotare for the input/write gate, forget/deallocate gate and output/read gateandctstands for the memory cell.

The functioning of the network is as follows:

● ht−1is observed as the short-term memory of the network.

● utis the information features from the input it and the short-term memory ht−1.

● Write gatewtdecides what information fromutwill be written in the memory cell ct.

● Informationpreserved in memory cellctis decided by the forget gate ft.

● otdecides what information will be read from the memory cell ct, that produces the short-term memory ht.

II. SYSTEMMODEL

Fig 2. System Model

(5)

________________________________________________________________________________________________

A. Data Collection

A specific keyword search is done for tweets relevant to the product and this data is mined using the twitter API.

Now the raw data is cleaned and stored as a table in the database server (MSSQL).

B. Analysis

The clean data is now retrieved from the database server and passed as input to a pre-trained neural network for analysing the sentiment of the mined tweets and categorizing into positive and negative reactions. The predicted sentiments of the tweets are concatenated to the mined data and stored as processed table in the database.

C. Display

The processed table is retrieved from the database server to tableau where the information (tweets, location of creation, time of creation and tweet user, along with the predicted sentiment) is visually represented. It gives an overall insight about how the product is received in different parts of the world, thus identifying potential markets of the product.

III. METHODOLOGY

Fig 3. Methodology

Fig 3. Display the general block diagram or the working methodology of the proposed system. For the ease of explanation we have divide the methodology in the following 3 steps, namely:

A. Training Dataset Preparation

Our primary objective is to make a training dataset with sentiments as its target variable. Instead of using the pre- existing twitter dataset the neural network model is trained on this dataset we created, to get the NN (neural networks) acquainted to the reactions of the people towards the product as well as including few product related functions/features in it’s dictionary of positive or negative search related words.

Twitter provides developers the access to tweet text and other related information in JSON format which can be accessed through scripting languages like python. This offers a chance to enhance ad-campaigns and innovate new twitter applications. For requesting data,a pair consisting of consumer key and consumer secret are passed to connect to the twitter API. Now when the connection is established, tweets are mined related to the product name.

For example by keeping the search word as” AI”, all the tweets containing the keyword “#AI” are mined, along with fields like user location, time zone, ID etc. The data gathered from twitter is in a mixed form. The field of twitter text is cleaned of hyperlinks, alphanumeric values (e.g. @,#, $, % etc.) and punctuation marks.

Then a sentiment polarity is obtained by using the TextBlob package, which sets a polarity in the range of (-1,1). The function of TextBlobis to tag the data as positives and negatives. The tweets with polarity in between -1 and less than 0 are labelled as negative and tweets with polarity between 1 and greater than 0 are labelled as positive. The tweets with 0 as polarity is removed and not taken into consideration. Now a training dataset consisting of 20 thousand records was compiled which consisted of 10 thousand positive tweets and 10 thousand negative tweets. For the purpose of avoiding biasing of the decision making capability of the neural network, the number of positive and negative tweets were kept balanced. This dataset was given as an input to the Convolution LSTM neural network for training purpose.

(6)

________________________________________________________________________________________________

A Convolutional LSTM NN layer was modelled using keras. Use of keras API to implement the neural network with TensorFlow running in the backend. This data acquired from twitter API was used as the input data. Input to the neural network cannot be given in the form of text, it needs to be converted to numeric data. For this purpose, keras provides a function “text_to_token ()” which first breaks the text into words (called tokens) and then creates a list of these words. Now by default the text gets split on the basis of spaces (“ “), punctuation and then converts all the words to lowercase. After which the text data is ready to be passed as an input. The sequences were padded using the “pad_sequence()” function from keras so as to pass equal length of input into to the neural network.

The training and testing dataset were prepared by making a split in the ratio of 75:25. So the model was trained on 15000 samples and was validated on 5000 samples. A Convolutional LSTM NN was designed with the sequential function to add four layers. The first layer of the model was set to be the Embedding Layer so as to convert the positive indices (of the tokenized and padded text input) into dense vectors of same fixed size. Next a convolution layer was added which convolved this dense vector input layer into single spatial dimension in order to produce a tensor of outputs. The input layer’s dimensionality was reduced by using the concept of pooling so as to allow assumptions to be made about the features contained in the sub region’s bins. This was done by using the

“MaxPooling1D()” function which carries out the process of pooling in strides.

Next LSTM layer was added by keeping the dropout value of 0.2 in order to avoid overfitting. At last a Dense layer was added by keeping the activation function of “sigmoid”. Using which the whole neural network was built as it is the only network layer present in the model.The model is compiled by keeping ‘binary cross entropy, as the loss function,’adam’ as the optimizer and ‘accuracy’ as the metrics. A Training Accuracy of 99.5% was achieved and a Testing Accuracy of 99.36% was achieved. Training and Testing accuracy were plotted with respect to the number of epochs and it was observed that these two accuracies were close, which indicated that the model had not undergone the problem of overfitting. This trained neural network is saved in .h5 format thus giving the ability of calling it in any script for making sentiment prediction thus avoiding the re-training of the model each time a set tweets were mined from twitter.

C. Data Scraping from Twitter for Analysis

For this purpose, we scrape tweets with the help of twitter API with respect to the product name. Followed by writing this data to a SQL server. A connection needs to be established between the server and the script, once it gets established the collected data is stored as a table in the server.

D. Data Analysis using RNN Model

There is a separate python script for performing the sentiment analysis on the tweets. This script needs to provide authentication through the engine to read the data (table in SQL form) from it. After retrieving the data from the server, the saved neural network model (in .h5 format) is called to make predictions on the text data, these outputs are labelled either positive or negative according to the polarity provided by the neural network. Now this data is stored back to the sql server as processed data by establishing a connection between the script and the server.

E. Retrieval of Labelled Data from Tableau

Tableau is a software for performing visual analytics on the data. The important factors present in the data usually gets missed when inspected in its raw format. Providing a visual representation helps in gaining hidden insights and presenting it in an appealing way for easy interpretation. Tableau has an option for connecting to various types of SQL servers. We connected Tableau to our MSSQL server to provide the data processed by the neural network.

F. Representation of Data on Tableau Dashboard Four sheets constituted the dashboard (Fig 4.), namely:

● The location of the generated tweet.

● Total number of positive and negative tweets.

● The actual tweeted text.

● The graph between the number of tweets created per minute.

The second sheet is selected as a filter, so when the positive representation of tweets was selected the other three charts were responsive to it showing the location, tweet text and time of creation of the tweets. Later the dashboard was published on the online platform of Tableau. Once logged into it would display the dashboard from any device, thus providing an online solution to monitor the live reactions of people towards the launch of a product.

(7)

________________________________________________________________________________________________

IV. RESULT

The online dashboard service provided by Tableau was accessible upon giving the login credentials through devices like mobile phone,laptop,tablet and any other device which can be connected with the internet .This facility provides the user flexibility to monitor the market scenario from anywhere, at any time.

Fig 5. Positive Tweets

Fig 6. Negative Dashboard

Fig 5. and Fig 6. Represents the final dashboard the proposed paper presents to aid data visualization. It comprises of 4 sheets representing different facts about the tweet such that product analysis become more efficient.

(8)

________________________________________________________________________________________________

Fig 8. Loss vs Epoch

The integrated system was able to gather tweets related to a specific product from twitter, analyse it using the neural network, store the results in SQL server and display the output on tableau dashboard. The designed neural networks attained a state of the art accuracy where the training accuracy was 99.5% and testing accuracy was 99.36% by running for just four epochs. The combination of convolution and LSTM trained neural network’s performance in classifying the sentiments was found to be pretty accurate and stable. On seeing the Fig 7. and Fig 8.

we see that the gap between the training and testing accuracies is pretty small indicating the case of no overfitting.

The developed dashboard was able to perfectly analyse the data and showed all the important information. On selecting the positive and negative category separately of tweets, all the information like location of tweet created, tweet text and time of creation of tweet was perfectly shown on the dashboard.

V. CONCLUSION

Our research on the proposed system architecture was able to show the business insights successfully on Tableau.

The use of convolution and LSTM layer was fruitful in getting the product related sentiments. The proposed system architecture was able to show the business insights successfully on Tableau. Using hashtag keywords proved to be an accurate way to mine tweets related to a specific product .In future the system needs to be scaled to get tweets on the go and keep updating the values on the dashboard as the users comment on twitter about the product. The use of online database servers could help in increasing the speed of the whole process of storing and retrieving data. The system with some minor changes with respect to different social media sites can be used to collaborate the reviews from all social media sites into one platform and show the results on a single dashboard.

VI. FUTUREENHANCEMENTS

The strength of this paper lies in the analysis of straightforward positive or negative responses. However, real life comments are often sarcastic in nature and further advanced research can make provision for the sentiment analysis of the same. The proposed system runs on an isolated system but can be scaled up to cloud services like Microsoft Azure and AWS for providing universal access to the system. This system is using TextBlob to find the polarity of the data while preparing the training dataset, a more efficient and accurate dictionary can be used for analysis. The involvement of neutral tweets can also positively impact the review process.

(9)

________________________________________________________________________________________________

© 2014- 18, IRJCS- All Rights Reserved Page -500 There is a fine line of difference between positive and neutral, deciding that line will provide greater efficiency to the system. For now, the model only processes English tweets, but twitter being a worldwide platform allows tweets to be created in many other languages which needs to be analysed too. Using of emoticons has not been incorporated in the paper, addition of which would definitely help in getting the essence of the tweets. The current proposed system does a sentiment analysis based on polarity of words present in the sentence, in future the system can be modified to handle semantic polarity. For example, in the sentence “Rocky defeats Monty”, the sentence is positive for Rocky and negative for Monty per se. The system can be designed to see who is affected in a situation and in which way to judge better. Right now, the proposed system only gathers tweets for sentiment analysis, the system can be extended to include the comments on the tweets as well.

The functionality of the system can be expanded to accommodate more social media platforms:

● Facebook: The comments made by users is usually private, so an analysis of individual posts would be difficult. Instead, the data from a business page can be mined for comments on which sentiment analysis can be performed.

● YouTube : YouTube is a video-sharing platform, the comments section has a lot of user opinion which can be scrapped to perform the analysis performed on twitter but prior to analysis a method to remove unwarranted comments or spam should be devised.

REFERENCES

1. Twitter Search Team (May 31, 2011). "The Engineering Behind Twitter's New Search Experience". Twitter Engineering Blog. Twitter. Archived from the original on March 25, 2014. Retrieved June 7, 2014.

2. ”Text Understanding from Scratch “,by Xiang Zhang [email protected] Yann LeCun [email protected] Computer Science Department, Courant Institute of Mathematical Sciences, New York University.

3. ”Twitter as a Corpus for Sentiment Analysis and Opinion Mining “Alexander Pak, Patrick Paroubek,Universit ́e de Paris-Sud, Laboratoire LIMSI-CNRS, Bˆatiment 508,F-91405 OrsayCedex, France.

4. “Sentiment Analysis on Twitter”Akshi Kumar and Teeja Mary Sebastian , Department of Computer Engineering, Delhi Technological University Delhi, India IJCSI International Journal of Computer Science Issues, Vol. 9, Issue 4, No 3, July 2012

5. Apoorv Agarwal, BoyiXie, Ilia Vovsha, Owen Rambow, Rebecca Passonneau.” Sentiment analysis of Twitter data”, Department of computer science, Columbia Univeristy

6. ”Twitter Sentiment Classification using Distant Supervision”Alec Go Stanford University,RichaBhayani Stanford University Lei Huang Stanford University

7. Rudy Prabowo,MikeThelwall.” Sentiment Analysis: A Combined Approach”, School of Computing and Information Technology,University of Wolverhampton

8. ”Product response analytics in Facebook “ RamyaMala.P,SeedhanaDevi.S,International Conference on Intelligent Computing and Control Systems ICICCS 2017.

9. ”Combining Convolution and Recursive Neural Networks for Sentiment Analysis”Vinh D. Van,ThienThai,Minh- ocNghiem,SoICT ’17, December 7–8, 2017, Nha Trang City, Viet Nam