Sampling and Sample Frame

As the number of subscribers is commercially sensitive information to online game producers, companies have thus far been unwilling to grant access to most researchers to their databases, with the recent exception of Williams et al.’s (2008) access to Sony Online Entertainment. During the preliminary stages of this research, requests were made to a number of leading MMORPG companies to be able to access their subscriber database and to

use their records to define a sample frame; all requests were denied or not replied to.

The lack of engagement of game companies with researchers is more the norm than the exception though. As Williams et al. (2006) discuss “MMO space represents a special challenge to academic researchers seeking original empirical data…” (p.6) as “…MMO operators do not release data on their players” (Williams et al., 2008, p.999). Due to these issues, and the general lack of willingness of the gatekeepers of the sample frames (the subscriber databases) to engage with academic researchers, empirical researchers have treated online gamers as a hard-to-access population (Goode, 2000). As such, extensive use in gaming research has been made of purposeful sampling, snowball sampling and self selected sampling (Goode, 2000; Yee, 2006), leaving researchers thus far to infer and correlate relationships from large scale samples derived from web based surveys posted on well travelled and regarded MMO gaming websites (Seay et al., 2004; Yee, 2006; Williams et al., 2008; Yee, 2009)

This produces issues of coverage bias in the data, however online gaming researchers accept this bias as a limitation of their work. As Yee (2006), a researcher who has regularly used large scale online survey techniques, discusses, despite the general concerns of online web based surveys of online gamers, large scale surveys can still generate insightful findings,

“Thus, even with its limitations, the survey methodology can still provide important insight to many questions that rely on inferential or correlational data” (p.36).

Given these acknowledged issues, and given the hard-to-access nature of the population this research has used a self selecting snowball sampling method. This method should be seen as both an advantage in trying to reach a hard-to-access population and, however, a drawback in that it creates a non-random sample (Goode, 2000). To mitigate this, demographic benchmarking was undertaken to demonstrate that the sample population is similar to established demographic norms for MMO subscribers. However,

this is an overall limitation of the research, and will remain an overarching issue in all online gaming research unless and until researchers are able to develop relationships with the developers’ of online games.

7.3 Coverage

In keeping with online research recommendations (Kozinets, 2002), permission was requested from the venues identified by the pilot participants via e-mail to place the questionnaire on their website. These included leading MMO-gaming websites and blogs, community podcasts and social spaces. In addition e-mail requests were sent to the Community Management addresses of Blizzard Entertainment (makers of World of Warcraft), CCP (makers of Eve Online) and Ncsoft (makers of City of Heroes/Villains) and a number of other MMO games requesting permission to promote the research on their forums. Of the venues that replied, the venues which the survey was finally deployed on are listed below.

Classification Venue

Official websites and forums World of Warcraft European Forums Eve Online Official Game Forums City of Heroes Official Game Forums

Social Networking Sites

Facebook World of Warcraft Players Group

Linkedin World of Warcraft Players Group

MMO informational websites MMO-Champion Forums MMORPG.com Official Forums

Table 16: Coverage of Research Instrument

The use of in particular open online forums led to consideration of two key issues. Firstly, the issue that membership (and thus analysis of membership

numbers of these forums) of a non-official open online forum is not a pre-requisite of viewing these forums or linking through to the questionnaire survey instrument itself. The non-official web forums posted on are open to all internet users, which, given the literature on the effect of web “lurkers” in online survey research (Andrews et al., 2003) means that defining a sample frame becomes especially problematic, or indeed, as Couper (2000) discusses, impossible. Due to this, this study cannot classify the coverage of the questionnaire survey as the sample frame and notes this as a limitation of the research.

The second issue of a “Pass-along effect” (Norman and Russell, 2006, p.10) occurring with the research was also considered (in marketing often referred to as a viral marketing effect). This is where, because of the nature of the internet as a research medium for questionnaire surveys, the participants may start to refer friends or their social networks towards a particular survey link, or repost the link on other websites for people they think may be interested in replying. It was noted that an internet search of Google after the data collection period had ended, using the exact phrasing of the questionnaire, found that both the questionnaire preface and the hyperlink had been re-posted in 38 locations by participants beyond the initial websites.

As Norman and Russell (2006) discuss, this can be a positive addition to participation when investigating consumption based groups or subcultures in which “that person is to interact with individuals who also share that affinity, whether through formal organizations (e.g., fan clubs, owner's clubs) or through informal conversation” (p.10). In the case of MMOs where the very nature of the context is consumption, affinity based social groupings of individuals, it was deemed highly likely that the link to the survey would go beyond the original venues chosen (Norman and Russell, 2006, p.10). The

“pass-along effect” thus has the benefit to this study of reaching further into a hard-to-reach population (Goode, 2000), at the cost of the researcher losing control of the venues the research is distributed to. While the “pass-along effect” (Norman and Russell, 2006, p.10) and web “lurkers” in online survey

research (Andrews et al., 2003) complicate the interpretation of the web survey’s overall coverage this may be desirable for the purposes of snowball sampling for hard-to reach populations.

Both of these issues are over-arching concerns of all broad-based general internet questionnaire surveys. However, these issues are poorly documented and discussed in previous and similar research studies (Norman and Russell, 2006), and therefore they are highlighted by this study.