Main Study 2: Web Content Analysis

Purpose and Rationale

The purpose of the web content analysis was to address the third objective of the study, i.e.

the in-depth assessment of civic youth and NGO / issue websites. The way in which civic organisations such as the ones sampled in this study employ the internet both at the strategic level (purpose of the site, political agenda, target groups) and at the tactical one (tools and facilities used, aesthetic and interactive approach) constitutes an important indicator of current political practice and, as such, it allows for comparisons across time and space. Combined with other data collection methods it can help us understand what does or does not work, as well as what the motivations and strategies of the organisations are. Despite the decline of public engagement with the formal institutions of parliamentary politics, Taylor and Burt (2001) note that public trust in voluntary organisations is high, while in recent times we have also witnessed an intensification of volunteering and mobilisation through civil society

organisations.

Content analysis has traditionally been used as a strictly quantitative research method (Bryman 2001) – or to be more precise, as a tool for the coding and analysis of existing material, as opposed to the generation of original data. Berelson – one of the “fathers” of content analysis – defined content analysis as “… a research technique for the objective, systematic and quantitative description of the manifest content of communication” (1952: 18).

This narrow definition soon widened to include “any technique for making of inferences by objectively and systematically identifying specified characteristics of a message” (Holsti 1969:

14). The scope and application of content analysis has evolved and expanded along with the proliferation of new media and new messages. While there has been a rapid rise in the number of studies using web content analysis (Weare and Lin 2000), their scope is still largely

quantitative (Gerodimos and Ward 2007, Gibson and Ward 2000).

The emphasis of this particular web content analysis was on the systematic, rigorous and reliable aspects of the method rather than on its quantitative ones. The analysis focused on the existence, usability and function of a large number of web features ranging from navigation menus and corporate logos to interactive message boards, promotional materials and participation tools, political agendas and youth-oriented facilities. The data was analysed

horizontally (i.e. comparing practice across different websites of the same or of different types /

“genres”), vertically (i.e. comparing aspects of the same website) and across time, all of which brought out interesting patterns.

Preliminary Study and Sampling

Twenty websites of UK civic organisations were coded and analysed. Primary coding took place between June 2005 and July 2006, while all sites were revisited between December 2008 and April 2009 for a follow-up evaluation. It should be noted that two of the sites, www.whiteband.org (which is the umbrella organisation for the Make Poverty History

campaign) and www.themeatrix.com, are international, but their inclusion was deemed useful because they have UK-focused sections, as well as for comparison purposes. A variety of sites focusing on youth engagement and on issue campaigns were chosen: youth forums, youth parliaments and consultation spaces, youth portals and training sites, charities and Non-Governmental Organisations, lobbying and campaigning sites (see Table 2 for the full list of sites, URLs and dates of coding).

It is widely accepted that sampling is one of the most challenging issues in internet research (Gerodimos and Ward 2007, Weare and Lin 2000). The mere size, lack of geographical boundaries and unstructured nature of the web means that it is virtually impossible to define populations and audiences. Subsequently, producing a sample of websites that can be characterised as random or representative (in the statistical sense of the term) may not be even relevant. In addition to established sampling techniques, such as word of mouth amongst researchers, prominence in UK civil society, appearance in links pages, search engines results and civic portals, all of which were employed both by the CivicWeb project (Banaji 2008a) and by this study, additional techniques have recently become available for researchers so as to tackle the problem of case selection.

One such technique is Hyperlink Network Analysis (HNA), i.e. the mapping of networks of websites based on the analysis of their in/out-links and the ensuing centrality of sites within clusters (Park 2003; Park and Thelwall 2003; Farrall and Delli Carpini 2004). HNA is an

increasingly popular, emerging method based on the principles of Social Network Analysis – a well established procedure for the dissection of structures in social systems based on the relationship between the main actors or elements of those systems, also known as the nodes

(Jackson 1997; Garton, Haythornthwaite and Wellman 1997). Hyperlink network analysis can produce valuable findings as a research method per se (e.g. see Shumate and Lipp’s (2003) study of NGO networks), although in this case it was used as a stepping stone for the content analysis. The choice of campaign and lobbying sites for our study was based upon a mapping of issue networks through an HNA of the UK’s civil society using the Issue Crawler software by the Govcom.org foundation. Initially, several issue networks were mapped on major current affairs (global poverty, the fair-trade movement, organic food & farming, climate change etc, see Table B2).

The websites featuring consistently as key nodes within and across all of those issue networks were then identified: Make Poverty History, The Fairtrade Foundation, Soil Association,

Friends of the Earth (FoE), Greenpeace (see Appendix F). All of those sites were included in the content analysis sample, and three of these sites (Fairtrade, Soil Association and FoE) were also chosen for the user evaluations that followed up the content analysis. The Live8 and Make Poverty History websites were coded four days before the July 2005 Live8 concerts, thus capturing the sites at the peak of the event when all facilities and tools were already in place and at the time when the sites were at the centre of mass media interest. Also, the Fairtrade Foundation website was coded at the start of Fairtrade Fortnight in March 2006, a national event which again would have been expected to attract more visitors to the site.

The other type of sites sampled was web spaces focusing specifically on young people. These included youth parliaments and forums across the UK: Funky Dragon (Wales), the Northern Ireland Youth Forum, the Scottish Youth Parliament and the UK Youth Parliament. Youth portals such as Young Scot, The Site and Connexions-Direct were also sampled. Finally, sites aiming to engage young people with politics were also coded: DoPolitics, Where Is My Public Servant (WIMPS) and HeadsUp. Coleman and Rowe (2005: 7) found that “there is a lack of systematic commitment across political institutions to actively engage with young people

through sites such as [HeadsUp, WIMPS, Funky Dragon and Young Scot]”, therefore including all of these sites in the sample can further our understanding of their uses.

Data Collection and the Coding Sheet

A 153-variable coding sheet was completed for every website sampled (Appendix E). The coding sheet focused on the availability, usability and relevance of a range of features (Features Analysis) considered vital from a civic usability perspective (see 2.8 above). The coding sheet benchmarks covered the site’s visibility, accessibility, maintenance, transparency, interactivity, community-building, media relations, youth focus, geographical focus, political agenda, empowerment and participation tools, promotional material, civic content, contextual and background information about the organisation and its managers, as well as links to other political or civic organisations. The coding sheet incorporates several variables used in seminal web content analyses such as the Government on the Web project (Dunleavy et al. 2002), benchmarks used in the usability industry (Nielsen and Tahir 2001), as well as others

discussed in the scholarly literature (Weare and Lin 2000). Most parameters covering the civic content (v56-85), agenda / focus (v86-96) and participation tools (v97-108) were designed for this study.

TABLE 2: CONTENT ANALYSIS – DATA COLLECTION