The use of content analysis in examining the quality of environmental disclosures

mechanistic approach to content analysis, with some undertaking interpretative approaches. Other studies have used a mixed quantitative-interpretative approach where the interpretations are predominantly denotative. Table 2.1 provides a summarised sample of studies that have adopted such a mixed approach.

Table 2.1: Prior environmental accounting studies showing a range of approaches to content analysis

Article Interpretative Quantitative

Wiseman, 1982 Disclosure per ‘categories and item of information’, e.g., expenditures, litigation, pollution abatement and others

Disclosure per ‘degree of specificity’ in each category

Frequency

Frequency and scoring

Toms, 2002 Disclosure by a number of ‘quality signals’, such as general rhetoric, policy, implementation and monitoring with and without published results

Using a rating scale Hasseldine et al.,

2005

Similar to the method followed in Toms (2002) Counting number of sentences

Cormier et al., 2005 Similar to the method followed in Wiseman (1982) Disclosure by categories

Disclosure per ‘degree of specificity’ in each category

Frequency and scoring

Brammer and Pavelin, 2008

Disclosure per selected quality ‘indicators’, such as policy, initiatives, improvement, audit and target

Frequency Beck et al., 2010 Content per theme, e.g., pollution, energy,

environmental risk etc.

Disclosure content per character (type 1, type 2 etc.)

Number of words per theme Number of disclosures per character

The mechanistic approach involving a frequency count through a dichotomous index as a basis of data capture was regarded as the simplest form of content analysis by Abbott and Monsen (1979). Here the presence of a disclosure item is recorded with a score of ‘one’ and those not present with ‘zero’. Although such an approach is regarded as purely mechanistic, it is argued that it has at least some interpretative element in the ability to inform the level of disclosure at a more complex level through sub-categorisation of disclosure items (Beck et al., 2010).

Wiseman (1982) was one of the early researchers to consider the quality of environmental disclosure through the adoption of a mixed approach. Wiseman (1982) employed a disclosure index consisting of eighteen items grouped under four main categories. The presence or absence of disclosures was rated for quality according to the degree of specificity of each of the information items (e.g., quantitative/non-quantitative specific and general disclosure). The weighting used by Wiseman is shown in Table 2.2.

Table 2.2: Weighting by Wiseman (1982)

Disclosure type Score

Monetary or quantitative 3

Non-quantitative specific 2

General 1

No disclosure 0

Freedman and Jaggi (1988) used a similar approach while analysing the quality of pollution- related disclosure, adopting an indexing and rating scheme. However, the introduction of arbitrariness to devise a scoring scheme and the subjectivity involved in interpreting the text to assign a score reduced the reliability and the resilience of the process in these studies.

Toms (2002) adopted a mechanistic approach to content analysis that was different to those looking at the amount of disclosure based on word/sentence/page counts in order to capture the quality of disclosure in corporate reports. The scoring system in Toms (2002) was based on the assumption that specific, quantified disclosures bear greater credibility than the ‘cheap rhetoric’ disclosures no matter how large their volume is, as the latter can be made without equivalent commitment or practice. Such an assumption reflects the work of Deegan and Gordon (1996) and Deegan and Rankin (1996), which indicated that in the absence of any environmental reporting legislation, companies’ disclosures tend to be increased in amount and ‘self-laudatory’ even in the presence of negative environmental performance. Table 2.3 presents the scoring system for quality analysis used by Toms (2002).

Table 2.3: Weighting by Toms (2002)

Disclosure type Score

No disclosure 0

General rhetoric 1

Specific endeavour; policy only 2

Specific endeavour; policy specified 3

Implementation and monitoring; use of targets, results not published 4 Implementation and monitoring; use of targets, results published 5

While Toms (2002) adopted quantitative content analysis to analyse the quality of disclosure, he completely ignored the volume of disclosures. This appears to be contrary to the main assumption behind the use of the quantitative approach as an empirical research

tool, which associates volume of disclosure with the importance of a disclosure (Unerman, 2000). Two major limitations restrict the strength and applicability of the method followed by Toms (2002). First, disclosures were identified with reference to the content of the whole report instead of individual items or categories of environmental disclosure. Second, the arbitrary setting of boundaries around some of the categories increases the level of subjectivity in scoring the disclosure. For example, the delicate distinction between concepts of ‘policy only’ and ‘policy specified’ would lead different coders to elicit the underling meanings from a narrative differently, and to interpret and score accordingly.

However, a major conceptual drawback appears to be associated with the rating of the disclosure along a numerical scale (Jones and Alabster, 1999). On a numerical scale, each two successive points has the same distance. But the variables (e.g., environmental disclosure categories) under study that are to be scored are categorical and hence, their importance or weight can vary unevenly. Therefore, using a numerical scale to score such variables is conceptually inappropriate for arithmetical addition or parametric statistical analysis (Jones and Alabster, 1999). Again, such scaling tends to create an average or ranking that fails to demonstrate specifically which qualitative criteria are missing or addressed (Kurt and Munis, 1998).

In a recent study, Beck et al. (2010) applied a modified approach, known as the ‘consolidated narrative interrogation approach’, which the authors claimed to be superior to the pure mechanistic or pure interpretative approaches. It involved three steps. First, each disclosure was interrogated for a sub-category of an environmental theme to which it belongs. The numbers of ‘phrases’ or ‘clauses’ were used as the unit of analysis and unit of measurement, which refers to a ‘group of words containing a single piece of information that was meaningful in its own right’ (Beattie and Thomson, 2007, p. 142). Second, following categorisation the disclosures were evaluated for information content along an information content scale to indicate the depth or detail of disclosure. This is shown in Table 2.4.

Table 2.4: Coding on the information content scale by Beck et al. (2010)

Definition of each type Disclosure type

No disclosure 0

Related to category definition; pure narrative 1 Related to category but provide details; pure narrative 2 Related to category in numerical way; pure quantitative 3 Related to category in numerical way with qualitative explanation; narrative

and quantitative

4

Any numerical disclosure to the category including qualitative explanation demonstrating year comparison; narrative, quantitative and comparable

5

Unlike prior studies, where the disclosures were allocated scores based on the level of detail, this study coded the disclosures under specific types (e.g., type 1 or type 2). Thereby, Beck et al. (2010) overcame the conceptual drawback identified by Jones and Alabster (1999) as discussed above. In the final step of their study, volumetric counts were recorded in phrases per content sub-category, as well as aggregated word counts per coded sub- category.

While attempts have been made in the extant environmental accounting research to improve content analysis in order to examine the quality of disclosures, there is still considerable scope for improvement. The studies mentioned under this section analysed the quality of disclosures using arbitrary determinants for capturing quality, such as narrative versus numerical and general versus specific. Quality of environmental disclosures is not examined in these studies as per the quality attributes prescribed in the environmental and accounting regulatory frameworks. This has been identified as a significant research gap in the literature (Guenther et al., 2007). In this project, attempts have been made to address this gap through the development of a tool that incorporates common quality attributes as suggested in the established regulatory frameworks and guidelines.