Latent Semantic Analysis Implementation

Having the VSM totally defined in Darius, it was necessary to implement a more complex IR model, which in this case would be the LSA. The description of this model and its implementation can be seen on Subsection 3.1.3. In Darius, the implementation of LSA is defined as a class called LSA. As well as the class VSM, detailed on the previous section, the construction of an object of the LSA class, needs the passage of the document database or corpus in the form of a WordCommentMatrix object. As it is seen on Subsection 3.1.3, LSA relies on the use of the SVD to factorize the matrix and reduce its dimensions. As was seen before, this will permit a better approximation of words and documents (comments) that are semantically connected. However, taking in consideration the complexity of the development of a SVD algorithm, the decision was to incorporate the use of a third-party SVD algorithm, which in this case is a SVD algorithm included on the S-Space Package1_{. This algorithm takes a matrix, which in this case is the words × comments matrix,}

and factorizes it into three new matrices as defined on the Equation (3.4) on Subsection 3.1.3. However, this algorithm needs the attribution of the order number or was seen before, the number of concepts, that provides the best approximation to the original matrix, and establishes the best semantical relationships. According to the authors in [58] the optimal number of dimensions or order is between 200 and 300, so the decision was to choose a value in between, that is 250 dimensions for Darius.

Having the model completely defined, in the end Darius will have three matrices as defined on Subsection 3.1.3. The first one corresponds to the position of every word on the new semantic space and the third one will have the comments positions. The second one will contain the single values. With this matrices, it is now possible to execute queries to the model. To execute a given

query, this LSA class contains also a method that provides that function. It only needs a list of the words that take part of the query. And with it, this method ranks the query towards the comments, defining which ones are more relevant to the query. The first step in this case, is to represent this query into the new space that was build. To do so, Darius introduces an algorithm, fully detailed on [13], that represents the query on the new semantic space in order to perform appropriate ranking calculations. The mathematical model of this algorithm can be seen on Equation (7.1). q in this case corresponds to the array of the query, and U and S are the same matrices defined on Equation (3.4).

q = qTU S−1 (7.1)

With the query fully positioned in the new space, it is now possible to rank it in relation to the comments on the corpus. The same principle, as the one applied on the VSM, is used, as the cosine similarity is calculated to find out which of the comments are more related to the query defined by the user.

7.1.4 Graphical Interface

Having the implementation of the back-end for the second version of Darius, which includes the exploration of comment information for the enhancing of PC, through the search of Problem Domain concepts on the source code, it was necessary to create a graphical interface to support it, so that the users and programmers can increase the pace of its understandability tasks in a more effective manner.

Continuing the thought stated on Section 6.2.4, that there was not going to be put to many attention on extreme design details, but focusing only on the efficiency, the development of the graphical interface for the second version of Darius continued the basis of the first version. However in this second version, there was added an other tab onto the main tabbed pane, that corresponds to the new component, the concept locator, as it can be seen on Figure 7.2.

If a program or software project is correctly loaded, as described on Section 6.2.4, the user can now be able to start his concept location task. To do so, first he has to choose which IR model, that Darius provides, he wants to use. In this case, as it was seen before, Darius provides two

7.1. Darius: Version Two

Figure 7.2: Concept Location Graphical Interface

different models: the VSM and the LSA. To do so, the user can click on the Load VSM button or on the Load LSA button, corresponding to model he desires, as it can be seen on Figure 7.3.

Figure 7.3: Loading an IR model

In the example of the Figure 7.3, the user clicked on the Load VSM button, that corresponds to the loading of the VSM. If everything runs correctly, in the end the model will be loaded, and Darius warns that fact by writing the message VSModel loaded! on the down side of the centre stage. Now the user can create a query and execute it in this model (Figure 7.4). To do so, on the top right side of the centre stage, there is a text area where the user can collocate loose words corresponding to the query he desires to be executed. Doing that, the user is now able to execute that query, by clicking on the Query button, which is under the button correspondent to the model he previously loaded.

After the query is executed, the result will be something like what is seen on Figure 7.5. The results of the query will be presented in a form of a list on the table which is on the left down side of the centre stage. This table will have two columns: the comment and the value for the similarity of

Figure 7.4: Writing a query to execute

that comment to the query executed. The results on the table will also be presented in a decreasing order of similarity value. According to the desires of the user, he can navigate throughout this list. select one of its element and see what is the type an nature of the comment he chose. To do so, he must select one element and click on the => button. By doing so, the text area on the right down size of the centre stage will be updated. In this text area it will appear the content of the comment, the source code associated with it, the file corresponding to that comment and the position of that comment on that file. With all of this information, the user can now be able to search for concepts on code on a more efficient way.

Figure 7.5: Results from a query