Sentiment analysis at the sentence level involves determining whether the
sentence expresses a positive or negative opinion (e.g., Liu, 2012). NVivo software was used to carry out sentiment analysis at the sentence level for OSS developers’ comments.
Table 43 lists sentences from developers’ comments that were classified as containing
negative opinions and inferences from them. In the context of the first approach to issue
gathering in OSS issue repositories (which does not enforce classification at the time of
issue gathering), the negative opinions expressed by OSS developers include
disagreements that can arise among project developers over issue classification, limited
177
Table 43. Negative opinions of OSS developers
Classified as negative Inference
“depending on the number of developers, there may be disagreements over whether it is an enhancement or a bug”
When developers themselves have to classify issues, a challenge can be the disagreements that can arise over an issue type, especially if there are many developers in the project development team.
“Of course, the question is, is there any one with the time and temperament available to classify all the tickets being filed”
The limited time availability of OSS developers can be a challenge when it comes to issue classification.
“Leaving the project maintainers or issue triagers in charge of labeling reduces mislabeling but increases their workload”
Issue classification by project developers would mean reduced misclassification but increased workload.
“Interface two offers a lot more field to screw up”
The many label fields in interface two mean a higher likelihood of errors in the submitted issue data
“The second interface captures much more complexity, but it can possibly intimidate bug- reporters with its many options”
The many label fields in interface two can be overwhelming to issue reporters
“Users can be unreliable when it comes to this”
Non-technical users can be unreliable when it comes to issue classification
“An issue interface should not allow the reporter to indicate the type of issue, as they are more likely to give an incorrect classification than a developer who reads the issue”
There is a higher likelihood of incorrect classification when issue reporters submit classification information along with issues than when project developers read issues and then assign classification information to it.
“Two which looks like Jira etc. is far, far too complex and is likely to get people to mis- categorize or just give up”
The complexity of the second interface can facilitate misclassification and deter participation
“In addition to improper labeling, a complex UI can be confusing to users and inflexible to edge cases (e.g., what if the issue is both a bug and a feature)”
The complexity of the second interface can cause confusion to users. It lacks flexibility to accommodate issues that can be classified in multiple ways, for example, issues that are both bug and feature.
“The second (Jira?) is a total mess and asks for too much information.”
The second interface demands too much information from issue reporters which can be undesirable.
“In the first case there is no opportunity to gather any information and in the second it is likely to be wrong because it asks for too much and frustrates the user into picking false options”
The second interface demands too much information from issue reporters which can cause frustration and facilitate misclassification.
“Many times when I am submitting issues, I do not know that much information about what is happening”
Issue reporters may not have enough knowledge or information in mind to provide all the information that the complex second interface is asking for.
In the context of second approach to issue gathering in OSS issue repositories
(which enforces classification at the time of issue gathering), the negative opinions
178
errors in submitted data largely due to incorrect classification/labeling), cognitive burden
in issue reporters, reduced participation especially of non-technical users, the lack of
flexibility in accommodating many types of issue reporters (e.g., those who do not have
sufficient information in mind to provide all the classification information) and the lack
of flexibility in accommodating many types of issues (e.g., issues that are both a bug and
a feature). (e.g., those who do not have sufficient information in mind to provide all the
classification information) and the lack of flexibility in accommodating many types of
issues (e.g., issues that are both a bug and a feature).
Table 44 lists sentences from developers’ comments that were classified as
containing positive opinions and inferences from them. In the context of first approach to
issue gathering in OSS issue repositories (which does not enforce classification at the
time of issue gathering), the positive opinions expressed by OSS developers include
higher information quality (e.g., fewer errors due to project developers classifying issues
instead of issue reporters), higher usability and greater control for project developers.
There was also positive opinion about multiple individuals contributing to issue
classification (e.g., getting useful metadata from experienced reporters), a potential goal
179
Table 44. Positive opinions of OSS developers
Classified as positive Inference
“Leave the analysis to the developers who are familiar with the project”
OSS developers’ familiarity with OSS projects can be beneficial for issue classification.
“Assuming that project owners are more effective at classifying issues, it would appear self-evident that an interface that requires all classification be completed by project owners should deliver more accurate classification overall”
The first interface, by not asking any classification information from issue reporters and deferring issue classification to project developers, can improve the accuracy of issue classification.
“Therefore, (interface one) from GitHub is preferably simple because it allows the engineers to manage assignment”
Not asking for classification information makes issue interface one simple and (desirably) allows project developers to control issue classification.
“By encouraging the user to describe the problem rather than forcing them to tender non-applicable details, the developer who receives the issue is much more likely to get information that is more useful to addressing the issue at hand”
The simple interface that does not ask for any classification information can potentially encourage issue reporters to focus entirely on the issue at hand and subsequently, project developers getting more useful information.
“It is often useful to know particular version of application associated with a bug report and various other details”
Meta data associated with issues can be useful.
“One wins on human factors consideration” When it comes to human use, the simple interface is better.
“The more people who can contribute to the classification, the better the classification would be (generally speaking)”
More individuals contributing to issue classification can improve the quality of issue classification.
“Option one is much cleaner, but if it had optional things to add like assignee or category, it would be nice”
The simplicity of first interface is desirable. Some meta data accompanying issue description is desirable as well.
“Simple interface, some labeling, not all fields needed from second screen though”
The simplicity of first interface is desirable. Some meta data accompanying issue description is desirable as well.
“You would want android OS version, not just android, which a novice user might submit, but a more
experienced user would most likely have “android 3.4.4, HTC 1, gen.2” listed anyways”
There is a high likelihood of issue data from experienced users containing useful metadata such as version related information.
From recommendation two, it can be seen that when using a simple interface (that
does not require classification), experienced users and OSS developers could include
metadata (that can be useful for project developers during issue classification) as text data
accompanying the main issue description. There could be textual prompts or similar
180
wish to, but that it is not compulsory. In this way, an issue gathering approach (that does
not enforce classification at the time of issue gathering) can accommodate reporters who
do not have sufficient knowledge/information to provide useful meta-data as well as
reporters who are capable and willing to provide such information.
5.6 Discussion and Conclusion
As previously described, in open information environments different contributors
are likely to have different views/interpretations about some phenomena and, in order to
accommodate the diverse and evolving views of information contributors, a desired
characteristic in OIEs is that they should allow the capturing of information from
contributors independent of any form of classification (Parsons and Wand, 2014). When
an information system enforces a priori classification on contributors of user-generated
content, it can have negative impacts such as information loss and lower information
quality (Lukyanenko et al., 2014). A similar picture emerged from the research discussed
in this chapter about OSS issue gathering approaches that enforce classification at the
time of issue creation. An OSS issue gathering approach that forces issue reporters to
classify their issues at the creation time can potentially lead to misclassification. It can
also lead to the loss of issue information, especially when contributors are novice non-
developer users. An issue gathering approach that does not force issue reporters to
classify their issues at the time of creation of issues could potentially reduce
misclassification and the potential loss of participation from such enforcement. An issue
gathering interface based on such an approach could still accommodate metadata useful
181
experienced users) who can provide such information as additional text data
accompanying the main issue description. Simple textual prompts (that can be deleted if a
contributor wants to) could potentially serve as efficient reminders for such contributors.
182
CHAPTER SIX: THEORETICAL CONTRIBUTIONS, IMPLICATIONS FOR PRACTICE AND RESEARCH AND LIMITATIONS