Evaluating an Active Crowdsourcing Template

There was no evidence provided by the participants that government agencies in Canada are currently working towards an active crowdsourcing application for emergency information communication. In many instances, participants revealed that agencies are just beginning to experiment with the use of social media for both information dissemination and information gathering. The above barriers and constraints primarily relate to social media, with some participants describing potential barriers and constraints that a hypothetical crowdsourcing application could bring. With this in mind, participants were asked to comment on a list of criteria for a hypothetical crowdsourcing application and rank the importance of each criteria from 0 (“not important”) to 6 (“most important”). The average importance rankings are displayed in Table 8.

Table 8. The average importance of each feature proposed by the lead researcher for a hypothetical emergency crowdsourcing application. Importance was ranked from 0 (“not important”) to 6 (“most important”), see Legend.

FEATURES AVERAGE

IMPORTANCE

REPORTING FORM 4.5

WEB-MAP & SPATIAL ANALYSIS 4.3 ATTACHMENTS 4.2 LOGIN CREDENTIALS 3.3 PRIORITY CLASSIFICATION 3.2 STATUS UPDATES 2.7 ROUTING 2.5 POP-UP WINDOWS 2.5

Legend:

important Very important More important Important Somewhat important Less important Not important

6 5 4 3 2 1 0

The three highest ranked features were reporting form (4.5), spatial attributes (4.3), and attachments (4.2). This is not surprising, since credibility and liability of social media/crowdsourced information was a concern for 79% of participating agencies. A reporting form would ensure that the quality of the information gathered is held to a standard, thus increasing the credibility. In addition, the participants indicated that photo and video attachments and spatial data can also increase credibility (see Section 3.4.1).

Login Credentials an Indicator of Openness to New Technology

The fourth highest ranking feature for a crowdsourcing application was login credentials. This is also indicative of the concerns amongst the participating Canadian agencies around credibility and liability. While this feature did not make it into the top three, a deeper analysis of the variation in responses produces interesting results. The distribution of the importance rankings is displayed in Figure 12.

Figure 12. Distribution of the importance rankings of login credentials, where 0 represents “Not Important” and 6 represents “Most Important.”

The distribution in Figure 12 indicates a large variation in the participants’ importance rankings of the login credentials. Two of the four participants who ranked login credentials as 5, or “very important”, were found to not engage in social media for gathering information (i.e. passive crowdsourcing), and only use social media for information dissemination (Participant B, 2015; Participant J, 2015). The other two participants do engage in social media monitoring, but consider it a lower priority in their emergency operations (Participant K, 2015; Participant P, 2015). Conversely, all of the participants who gave login credentials an importance ranking of 3 or lower (3 = “Important,” 2 = “Somewhat important,” 1 = “Less important,” or 0 = “Not important”) do engage in social media for both information dissemination and passive crowdsourcing. Interestingly, the participant that gave login credentials a ranking of 1, or “less important”, assigned this ranking based on previous experience. The participant’s agency already uses an application and online “self-serve” platform that allows citizens to communicate with the agency. Initially these tools required citizens to create login accounts, and the agency found that the tools were not being used. Once the agency removed the login requirements, the usage increased by 300% within the first month (Participant O, 2015). The participant who assigned login credentials with the lowest possible ranking (i.e. “not important”) indicated that a login system would be a limiting factor, and that any crowdsourcing system should be “as wide open as possible” in order to allow the general public to participate (Participant T, 2015). The agency that

0 1 2 3 4 5 0 1 2 3 4 5 6 No Ranking Fr eq ue nc y Importance Ranking

Distribution of Importance Ranking for Login

Credentials

this participant represents uses social media for passive crowdsourcing, and has assigned social media monitoring and mining as an official task in the EOC, and brings in communication professionals in times of emergency (Participant T, 2015).

The ranking of login credentials is an indicator of an agency’s level of experience with crowdsourcing technology (active and passive), as well as of the agency’s openness to new technology (i.e. change). Agencies who gave login credentials a high importance ranking are less experienced with social media/crowdsourcing, and potentially unwilling to try new technology without approval from “higher up” first (e.g. Participant H, 2015; Participant P, 2015). Agencies who gave login credentials a low importance ranking are already engaging in passive crowdsourcing through social media, and even creating specialized teams for it (e.g. Participant O, 2015; Participant T, 2015). Agencies who gave login credentials a more intermediate importance ranking appear to be experimenting with social media, and are in the early stages of using it for passive crowdsourcing (e.g. Participants A, N1, N2, R, 2015). The only Canadian agency to use an active crowdsourcing model, a federal agency, ranked login credentials as 3, or “important” (Participant L, 2015). This is likely because the active crowdsourcing model that the agency employs still requires participants to be trained and listed in the agency’s database, and thus, credentials of the participants are present to ensure credibility (Participant L, 2015).

The results suggest that, in a hypothetical crowdsourcing application, there is a link between an agency’s level of social media/crowdsourcing engagement and the importance of login credentials. Agencies with a low level of social media engagement (i.e. do not use it, or only use it for information dissemination) tend to rank login credentials as of higher importance than agencies who are highly engaged in social media/crowdsourcing (i.e. actively monitor social media, develop social media teams, practice active crowdsourcing). However, this generalization comes with a limitation: the requirement for login credentials is also dependent on the purpose of the crowdsourcing application and the type of data being gathered. For example, the participating federal agency that practices active crowdsourcing to collect severe weather reports requires that the information gathered be highly credible, thus, only information from trained and registered volunteers is accepted. This is because the information is used “on the

fly” to update current severe weather watches and warnings. In order to do this, the

information must be highly credible and accurate.

The Importance of Credibility

It is apparent that, based on the importance ranking results of crowdsourcing features, participating government agencies are deeply concerned with the credibility of crowdsourced information. Anything that can increase the credibility of the data, such as photographs, videos, locations, and standardized forms, are highly important to the participating government agencies. While login credentials can help increase the credibility of the information, participants were not in agreement on whether login credentials are important or not; there is a concern that the requirement for login credentials will deter users.

3.5 Crisis Crowdsourcing in the Canadian Emergency