The quantitative content analysis

As Karlsson (2010, p. 1) highlights, transparency, as well as the more traditional quality defining concepts such as objectivity and balance, is an abstract concept and therefore needs to be interpreted as everyday “rituals” in order to enable a meaningful evaluation. This study thus required a set of indicators that would effectively describe the use of transparency in online news production. Many researchers have linked quantitative content analysis to simple word count, which is usually not enough to produce reliable results (Stemler, 2001).

Therefore, a more diverse technique was chosen for this study. Based on the commonly accepted definitions and measures of journalistic and editorial transparency as discussed in the review of the existing literature, six different indicators, or “codes”, were created for the purpose of the quantitative content analysis of the sample. All of these indicators were considered to be

transparency measures:

Indicator 1: “use of user-generated material”

This indicator referred to user contributions being published in the news item. In practice, visual contribution could be seen as consisting of either photo or video material, as well as any other visual illustrations, such as a drawing. Although the pilot study had clearly showed that the most commonly used forms of user contribution in news production tend to be visual content, the indicator also covered textual or any other formats of user material. Thus, user material was defined as any piece of material, textual or visual, that included a reference to a user contribution, such as “reader’s picture” (in Finnish, “lukijan kuva”), as

illustrated in the example below. The reference to a reader contribution could be located on the piece of material or next to the piece of material, as shown in the example below.

Figure 2. Example of the use of user-generated material (10 June 2012, www.iltalehti.fi)

Indicator 2: “linking to original sources”

This indicator reflected the hyperlinking of information to its original source. This could be, for example, an interview transcript, a blog post, a press release, a government or organization report, a study or a research abstract, a poll or a survey, a radio podcast, a video clip or any other material from the original provider of certain information. It is, of course, difficult to define what kind of information should be linked to its source, and it is clear that even the most transparent websites do not link all of the sourced information to the text.

Therefore, all of the articles that included at least one link to an original source were considered to be fulfilling this indicator. The hyperlink could be situated

either in the news story or under the news story, as illustrated in the samples below (hyperlinks marked in blue).

Figure 3. Example 1 of linking to an original source (10 June 2012, uusisuomi.fi)

Figure 4. Example 2 of linking to an original source (10 June 2012, www.yle.fi)

Indicator 3: “linking to secondary sources”

This indicator reflected the hyperlinking of information to its a secondary source.

This could be, for example, an article in another website, a blog post, or any other material which communicates information whose primary source is somewhere else. Parallel to the previous indicator, all of the articles that

included at least one link to a secondary source were considered to be fulfilling the criteria of this indicator. The hyperlink could be situated either in the article

text or under the article text, as exemplified in the examples below.

Figure 5. Example 1 of linking to a secondary source (14 June 2012, www.yle.fi)

Figure 6. Example 2 of linking to a secondary source (13 June 2012, www.aamulehti.fi)

Indicator 4: “providing a commenting space”

This indicator described the readers’ ability to comment on an article in the same news item frame. Commenting could be limited to registered readers only or it can be open to anyone visiting the website. Most commonly, the

commenting platform is located below the news story, but it could alternatively be situated elsewhere on the page. Separate discussion rooms, however, were not considered to represent a commenting space in this study because they are not located on the same page as the news article and thus play a slightly

different type of role for the audience.

Figure 7. Example 1 of providing a commenting space (14 June 2012, www.mtv3.fi)

Indicator 5: “providing a poll”

This indicator reflected publishing a poll in the news item frame. A poll could consist of just one or multiple questions. The questions can deal with any

possible subject material that relates to the content of the page. The poll can be located above, below or next to the news article.

Figure 8. Example of providing a poll (10 June 2012, www.uusisuomi.fi)

Indicator 6: “publishing the name of the writer”

Finally, the sixth indicator reflected the presence of a byline that gives the name, and often the position, of the writer of the article. Although the journalist’s photo, professional status and contact details are often published next to his or her name, simply publishing the name of the writer was enough to fulfill this indicator. The byline could be located either above or below the news story, as demonstrated by the following two examples.

Figure 9. Example 1 of publishing the name of the writer (12 June 2012, www.yle.fi)

Figure 10. Example 2 of publishing the name of the writer (12 June 2012, www.kaleva.fi)

The content analysis was conducted using a specific coding sheet created around these six indicators (for further details, see appendix 1). Before applying the coding sheet to the actual analysis, its functionality was tested through a simple inter-coder reliability test in which two external persons experimented with coding a couple of articles using this coding sheet. The test showed that

the external coders agreed with the researcher on the coding of the content.

This indicated that the coding scheme was clear enough. In other words, it indicated that the coding sheet would not be limited to the understanding and use of only one person, but that the analysis could be replicated later by

another researcher. Consequently, each individual news article in the sample of 700 articles was manually analysed by the researcher with the help of this coding sheet.