Submitted for publication as:
Van Rooij, A. J., Schoenmakers, T. M., Van den Eijnden, R. J. J. M., Vermulst, A. A., & Van de Mheen, D. (n.d.). Videogame Addiction Test (VAT): validity and psychometric characteristics.
51 Online Video Game Addiction VAT: validity and psychometric characteristics
Abstract
Aims
To explore the reliability, validity and measurement invariance of a new Video game Addiction Test.
Design
Large-sample paper-and-pencil questionnaire study, conducted in 2009.
Setting
Secondary schools in the Netherlands.
Participants
Large sample of schoolchildren (aged 13-16 years).
Measurements
Compulsive Internet Use Scale, Video game Addiction Test, Game Addiction Scale, weekly hours spent on various game types, and several psychosocial variables.
Findings
The Video game Addiction Test demonstrated excellent reliability, excellent construct (convergent and discriminant) validity, a one-factor model fit, and a high degree of measurement invariance across gender, ethnicity and learning year, indicating that the scale outcomes can be compared across different subgroups with little bias.
Conclusions
The Video game Addiction Test can be helpful in the further study of video game addiction and contributes to the debate on possible inclusion of behavioral addictions in the upcoming DSM-V.
52
4.1 Introduction
In light of the upcoming edition of the DSM-IV (the DSM-V), the phenomenon of internet addiction is receiving renewed attention (APA, 2010; Tao et al., 2010; Block, 2008). Historically, internet addiction is often modeled along the framework proposed by Mark Griffiths which is derived from the DSM-IV criteria for pathological gambling. Griffiths defined the following components of behavioral addiction: Tolerance, Withdrawal symptoms, Relapse, Salience, Conflict, and Mood Modification (Griffiths, 2005a).
Our group operationalised these criteria in the 14-item Compulsive Internet Use Scale (CIUS) (Meerkerk et al., 2009). In order to fit the nature of internet specific problems, the CIUS operationalised Relapse as Loss of Control, while Tolerance was not incorporated in the scale due to lack of relevance (Meerkerk et al., 2009, p. 2). The Internet Addiction Test (IAT), another well-known internet addiction scale, also does not incorporate tolerance (Widyanto & McMurran, 2004; K. S. Young, 1998). The CIUS contains the following components: loss of control, conflict, preoccupation/salience, coping/mood modification, and withdrawal symptoms. The CIUS is one of the few validated internet addiction scales which has been subjected to a confirmatory factor analysis (Caplan, 2010), i.e. it supports a one-dimensional measurement structure (Meerkerk et al., 2009).
While proposals for an Internet Addiction Disorder diagnosis are being made (Tao et al., 2010), it remains highly questionable whether people are actually addicted to the ‘internet’ as such. In fact, actual problem- atic behavior tends to centre around applications: i.e. people develop problems controlling their time spent in online video games (Mehroof & Griffiths, 2010; Wan & Chiou, 2006), online erotica (Meerkerk et al., 2006), or instant messaging services (Van den Eijnden et al., 2008). Utilizing the CIUS, gaming and online erotica were shown to have the strongest relationships with internet addiction for adults (Meerkerk et al., 2006). In a study focusing on adolescents, playing online video games was shown to have the strongest relationship with internet addiction (Van Rooij et al., 2010); subsequently, a follow-up study identified a small group of gamers scoring high on both internet addiction and on hours per week online gaming. This group is hypothesized to contain highly involved or ‘addicted’ online gamers (Van Rooij et al., 2011). While several studies have demonstrated the existence of a small subgroup of highly involved gamers (Gentile, 2009; Grüsser et al., 2007; Peters & Malesky, 2008), no consensus has been achieved regarding measurement of game addiction. The present study aims to address this issue by constructing a new scale, the Video game Addiction Test (VAT), which is a modified version of the CIUS that aims to measure video game addiction. Using data derived from the 2009 sample of the ongoing Monitor Study ‘Internet and Youth’, the current study explores various psychometric properties of the proposed VAT scale.
More specifically, the present study tests the reliability, validity and measurement invariance of this new Video game Addiction Test. Convergent validity is tested by cross comparisons with related scales, the CIUS (Meerkerk et al., 2009) and the Game Addiction Scale (GAS) (Lemmens et al., 2009), and discrimi- nant validity is tested by comparison with several discriminant variables, i.e. relevant measures of psycho- social well-being (Van Rooij et al., 2011; Caplan, 2007; Ha et al., 2007) and time spent on various game
53 Online Video Game Addiction VAT: validity and psychometric characteristics
types. Finally, our earlier study (Van Rooij et al., 2011) already identified a group of internet addicted online gamers using the CIUS and a measure of time spent on gaming in the 2009 sample of same dataset. This allows cross comparisons to be made between this group and the VAT.
4.2 method
Sample
The present study utilizes the 2009 sample of the yearly Monitor Study ‘Internet and Youth’, being the first sample to include the Game Addiction Test items in the survey. The sample includes ten Dutch secondary schools and contains information from 171 classes. Total sample response rate was 83%, which results in 4074 completed questionnaires. Non-response is mainly attributable to entire classes dropping- out due to internal scheduling problems: 12% of all classes did not return any questionnaires. The average per class response rate for the remaining classes was 92%. For specific details on the procedure see (Van Rooij et al., 2010; Van Rooij et al., 2011).
The proposed items of the VAT were skipped by adolescents who did not play games at all (n=1024). For the purpose of scale validation, adolescents who completed less than four of the items on the Video game Addiction Test were removed from the dataset (n=156). This resulted in a final dataset consisting of 2894 cases. Note that this subsample consists exclusively of gamers, which results in a skewed gender division: the percentage of boys is 62%. Furthermore, 80% of the sample has a Dutch or Western European ethnicity, while the remaining 20% has a non-Western European ethnicity. The sample was distributed over four learning years: 31% of the adolescents attended the first class (average age 13.3 years), 42% of the adoles- cents attended the second class (average age 14.2 years) and the remainder attended the third and fourth class (average age 15.5 years). Total average age was 14.3 (SD=1.0) years.
measures
Compulsive Internet use (CIuS). The 14-item version of the CIUS (Meerkerk et al., 2009) was used to measure CIU (with its Dutch phrasing slightly adjusted for adolescents), and includes questions such as “Have you unsuccessfully tried to spend less time on the internet?” and “Do you neglect to do your home- work because you prefer to go on the internet?” The CIUS contains the following components: loss of control (4 items), conflict (4 items), preoccupation/salience (3 items), coping/mood modification (2 items), and withdrawal symptoms (1 item). The CIUS has good validity (Meerkerk et al., 2009) and internal reliability (Meerkerk et al., 2009; Meerkerk et al., 2006; Van Rooij et al., 2010). It also shows good reliability in the current sample (Cronbach’s α=0.89).
Video game Addiction test (VAt). The items that compose the CIUS were rephrased into the 14 proposed items for the VAT (Table 4.1 in Results section). Note that while we strongly suspect that online games play a particularly important role in the development of video game addiction (Council on Science and Public Health, 2007), playing multiple game types is not unusual (Van Rooij et al., 2010, p. 54) and excessive use of other game types cannot be ruled out. Thus, the phrasing was kept universal (i.e. ‘game’ instead of ‘online game’).
54
Game Addiction Scale (GAS). The seven-item (brief) version of the Game Addiction Scale developed by Lemmens et al. was included (2009). Six of the seven items each measure one of Griffiths’ components of behavioral addiction, while one question measures ‘problems’ related to gaming (Mehroof & Griffiths, 2010). The seven-item version of the GAS showed high correlation with time spent on games (r=0.58), and medium correlation with loneliness (r=0.31), life satisfaction (r=-0.29), social competence (r=-0.18) and aggression (0.27), which was interpreted as evidence of good concurrent validity (Lemmens et al., 2009). The GAS shows good reliability in our current sample (Cronbach’s α=0.84).
Weekly hours online gaming, casual browser gaming, and offline gaming. Three types of games are distinguished: (multiplayer) online games, which are video games played with other people over the internet (e.g., Call of Duty, Team Fortress 2, Runescape, or World of Warcraft), casual browser games, which are simple games that are played online via the web browser (e.g., freebrowsergames.com), and finally offline games which do not utilize the internet (e.g. Sims 2). Hours per week spent on these game types were obtained by multiplication of results from two questions measuring days per week of gaming and average hours of gaming per day, similar to the procedure in our previous study (Van Rooij et al., 2011). Psychosocial measures. Psychosocial measures were incorporated in the present study in order to study
discriminant validity. The measures were the following: Rosenberg’s Negative Self-Esteem Scale (Van Rooij et al., 2010; Rosenberg, 1989), the UCLA Loneliness Scale (Ghassemzadeh et al., 2008; Russell et al., 1980), the Depressive Mood List (Engels et al., 2001; Kandel & Davies, 1982, 1986), and the Revised Social Anxiety Scale for Children (la Greca et al., 1988; la Greca & Stone, 1993; Gross, 2004). These scales have been used in Dutch studies and have demonstrated good reliability in the past (Meerkerk, 2007; Van den Eijnden et al., 2008; Van Rooij et al., 2011) and in the current sample (Cronbach’s α>0.80). For all four scales, a higher score indicates more reported problems.