Qualitative Analysis of General Error Reporting (Exploratory Analysis)Analysis)

Of the 126 participants, 43 said they would report errors in general whilst 82 said they would not (1 missing response). In order to explore this further, it was necessary to examine the qualitative comments participants made with respect to what motivates their error reporting practices. Comments were available from 121 of 126 participants. These comments were

6.4 Results 147

collected to explore why participants would and would not typically report errors. Several different themes emerged.

The reporting of errors may have been influenced by a variety of factors. The reporting appears to be subject to the extent to which participants sensed a potential threat (fear appraisal), or lack thereof. The first group of participants actually felt that the error message represented a threat (as an indicator of a Trojan or virus). Participants reported, for example, that they do not report errors because: “I always feel like the message is a virus rather than an actual warning;” “[it] could be a virus;” and “in case it’s a scam or a virus". The second group of participants did not perceive a threat and hence decided not to report an error citing reasons such as; “I have anti-virus software.” The third group of participants accepted the error message as they determined that it informed them about a legitimate threat, thus complying with the request to report the errors.

Another factor was the perceived efficacy of responding to the threat (including the benefit of response). Those who did not report the error message suggested that this would have negligible or no significant benefits to one’s self or others. Several commented that it would be “time consuming". A few additional quotes from participants who do not report errors reflect their logic for not doing so: “don’t really think that it is important, but of a time effort”. Not reporting errors may also be linked to uncertainty. For example, two participants reported that they were “not sure how it works” and “because I don’t know what it means or what it is". In addition, some participants believed action would not lead to an improvement (“when reporting incidents in the past nothing has happened”). Another group of participants did not comply because they expressed a lack of information. That is, they may not have been knowledgeable enough to know what is required of them or where the information would end up. This is shown in citations such as “because I don’t know what it means or where it goes". The third group of participants recognized the importance of error reporting not just for themselves, but others, leading them to comply with such error request generally:

“To hopefully draw attention to the problem and ensure it is more likely to be fixed"; “to bring the problem to the attention of the website administrator so they can sort it out faster";

“to try and stop it from happening from again"; “because it may improve future services";

and to “improve site.”

A third factor concerns the potential costs associated with error reporting (e.g., in terms of productivity costs incurred due to error reporting). Individuals who did not report errors were particularly attentive to the potential costs associated with reporting errors (including the time and effort involved). For example, one participant stated “makes it go away quicker if I say no". In addition, participants reported that “I just want to continue doing what I was previously and did not want to report an error because of the potential for disruption that may

result in terms of “time and redirection". Participants who decided to comply with the error requests did not comment on the immediate cost to themselves, only recognizing that the error “needed telling” and that reporting it will ensure “it can be fixed” and “to ensure it is solved.”

6.5 Discussion

Perceived redundancy and the lack of personal relevance of individual contributions are important work characteristics known to increase the social loafing of individuals in larger groups [138], [139]. Applying this knowledge to the user on a far grander scale, the current study aimed to use some of these social psychological findings to promote error reporting by avoiding perceptions of redundancy that many users experience and subsequently report when encountering such error requests. The next session summarizes our results and relates these findings to the existing literature.

Our first hypothesis examined whether or not the cognitive framing of an error message influenced error reporting in the current study. In contrast to our predictions, error reporting was significantly higher when participants were presented with a technical, but not security, framed error message. The difference may be attributed to several different factors. The first explanation relates to concerns that participants may have had when the error is security-related. Participants may be more likely to opt against reporting such an error as the label

‘security’ may imply potentially punitive or other unknown serious consequences [137]. This may evoke protection motivation that discourages compliance with such error requests [113].

In addition, a security framing may raise concerns about perceived control over what is implied. If individuals feel they are not in control over the consequences, they may not engage in error reporting behaviour [15]. In essence, the issue may be perceived as less manageable by the users themselves. This may explain why these participants were more likely to decide for inaction (not reporting the error). A third and related explanation concerns the lack of knowledge and experience as a motivation for inaction. Whether or not users understand and know what to do can also influence their interactions with a system [160].

This may discourage error reporting.

The second hypothesis addressed the lack of meaningfulness. The hypothesis proposed that when the message implies some purpose (specifically, the benefit of reporting), par-ticipants would also feel inclined to report the error message presented in our study. This hypothesis was supported. Error reporting frequencies were higher when the message implied a benefit to self, followed by a benefit to others. Error reporting frequencies were lowest when no information was provided. Since this control condition also represents the default in

6.5 Discussion 149

the majority of messages users will encounter, the results provide evidence for the role of meaningfulness in what a user is expected to do. By reducing redundancy of effort, the error reporting frequency was significantly increased. When considering both factors (framing and benefit) in combination, the results further indicated that both labelling of messages and the type of implied benefit could increase error reporting. Error reporting was highest when the error was technical as well as implied as a benefit to self. These results suggest that the combination of factors led to highest reporting rate; in line with protection motivation [113]. The findings also speak to the importance of providing information to the user about why certain actions are required rather than relying on uninformative requests, similar to the messages presented to the control groups in our study [173].

The third hypotheses considered the possibility that error reporting was a function of specific personality traits. In this study, the main traits of interest were risk-taking and impulsiveness. The suggestion for testing these was based on the fact that the continued rejection of errors may indicate (a) poor self-efficacy to deal with such challenges, or (b) an attitude towards ignoring risks and not considering the long-term consequences of ignoring such errors. There was no significant evidence that error reporting in any of the conditions (control or implied benefit) were impacted. Further analysis with other traits (such as sociability, privacy concern, security vulnerability, sociability or reciprocity) played no significant role in error reporting.

Exploratory analysis focused on the extent to which generic error reporting may indicate specific motives that encourage or discourage error reporting. This analysis involved the review of open response options and the coding of all reported rationales. The main themes that emerged included fear appraisals, response efficacy and productivity costs. These findings link to existing literature within the field.