«Abstract This paper revisits the debate on the units of Content Analysis (CA) for the purposes of Corporate Social Reporting (CSR) research and also ...»
Woodward and Woodward, 2001; Dillard et al., 2005), to review the development of the field (Mathews, 1997; Carroll, 1999; Elkington, 1999; Bebbington, 2001; Gray, 2001; 2002; Line et al., 2002; Norman and McDonald, 2004; O‟Dwyer, 2005), to provide general explanatory accounts (Gray et al., 1997; Bebbington and Gray, 2000; Ramirez, 2001), to review specific theories (Gray et al., 1995a; Deegan, 2002), or offer some international perspectives (Capron and Gray, 2000; Bebbington et al., 2000; Aaronson, 2003), and proposals for the development of the field (Adams and Harte, 2000;
Lehman, 2004). Most of the authors implicitly seem to consider their methodological choice selfexplanatory and some describe how the literature relevant to their purposes is identified and segmented (Gray et al., 1997; Mathews, 1997).
-6consolidate what is known and identify any gaps in current knowledge leading to a more reliable and accurate picture of the current state of the field literature (Price et al., 2004).
In the second stage of data collection, the reference lists of previously conducted methodological reviews of CA in CSR (Guthrie and Mathews, 1985; Gray et al.
1995b; Hackston and Milne, 1996; Milne and Adler, 1999 and Unerman, 2000) were scrutinised and all the publicly available CA studies referenced were collected and reviewed. Finally, to further increase validity, some additional well referenced papers were collected, until it was realised that saturation in contributions was reached, when the review of more sources was not adding to the understanding but only to the paper‟s reference list.
The data collection resulted in over a hundred papers to be reviewed and a summary of these is presented in Table 2 under their CA approach (some sources employ more than one approach). Considering that a general major weakness of theoretical investigations regards the credibility of the collected documental sources (Denscombe, 2003; Yin, 2003), it was expected that the review of such wellreferenced sources would add to the validity of the research; it is albeit acknowledged that particularly in the final stage in data collection, where the additional sources were not collected according to specifically determined criteria, some original, less frequently referenced contributions, published in other sources may have been omitted..
Take in Table 2
The data analysis was based on the approach suggested by Miles and Huberman (1994) which will be discussed in more detail in the findings but in brief involves firstly the review, coding and synthesis of the raw data, subsequently the identification of key dimensions and development of classifications and typologies, and finally, building explanations for the identified patterns and making inferences to the research questions. The first two stages took place simultaneously: the methods‟ sections of the papers were reviewed and detailed notes for each paper of the precise
-7approach to CA, the implicit or explicit benefits and limitations identified and the references employed for the justifications, were kept (the development of coding schemes was not deemed necessary). These notes assisted in directly developing the explanatory accounts of the research, addressing the main set objectives. The succeeding sections discuss these findings.
Index vs. amount/volume approaches
As discussed earlier, an alternative distinction of the quantitative, more restrictive CA studies regards the one between the „index‟ studies and „amount-volume‟ studies. The index studies generally check for the presence or absence of specific items of information (what Stone et al, 1966 and Holsti, 1969a describe as „contingency analysis‟) whereas the volumetric ones check for the overall volume of disclosure, most frequently by counting words, sentences or proportions of an A4 page6.
Index studies appear to have been the most popular approach to CA up to the early 1990s (Guthrie and Mathews, 1985; Zéghal and Ahmed, 1990), and in fact, in a number of studies, it is this approach that is most commonly labelled as CA, as opposed to the „line‟, „sentence‟ or „page counts‟ terms generally attributed to the volumetric approaches (see e.g. Abbott and Monsen, 1979; Wiseman, 1982; Guthrie and Mathews, 1985, and more recently, Moneva and Llena, 2000; Patten and Crampton, 2004). As Table 2 illustrates, most frequently a simple binary coding scheme is used, where a score of 1 or 0 in the presence of absence of the item is respectively attributed. At times, the aggregated frequency of the presence of these items is further estimated (Cowen et al., 1987; Campbell et al., 2006; Bebbington and Larrinaga, 2007; Guthrie et al. 2007). Index studies, though, may further incorporate ordinal measures, to allow for the quality of the specific disclosure to be assessed (Beattie et al., 2004), with Wiseman (1982) and Freedman and Wasley (1990) employing a four-level index (quantitative Corporate Social Disclosure (CSD)=3, Note that the distinction between standardised vs. customised CA may still be applicable if one e.g.
considers the oft employed in the literature categories developed by Gray et al. (1995b), drawing on Ernst and Ernst (1978), as standard with regards to the volumetric studies, and e.g. the GRI (2002) guidelines as a potential standard index for contingency studies.
-8non-quantitative=2, general=1, absent=0) (see Marston and Shrives, 1991; Beattie et al., 2004, for more thorough discussions on types of index studies).
Although Marston and Shrives (1991, p. 195) assert that index scores “can give a measure of the extent of the disclosure but not necessarily the quality of the disclosure”, it is arguable how useful for evaluating extent these measures are, since as Abbot and Monsen (1979) concede, “The only meaning that may be attributed to the scale is that it reflects the variety of social involvement activities. It does not measure the intensity of each activity” (p. 507, emphasis in original). Further, as Milne and Adler (1999) note, although “from a coding perspective, the Ernst and Ernst [index] approach is likely to be more reliable than Gray et al.‟s [1995a,b] and Hackston and Milne‟s [1996, volumetric ones] because each coder has far fewer possible choices for each coding decision, and consequently, far fewer possibilities for disagreeing”, this increased reliability “is gained at the expense of potential refinements to the understanding of social and environmental disclosure” (p. 242). In addition, although this approach appears to have the advantage over the volumetric ones that the scores are not affected by repetition of information or grammar, this might also be misleading, since it might treat a company making one environmental disclosure as equal to one that makes 50 of the same type (Hackston and Milne, 1996;
Milne and Adler, 1999).
In volumetric approaches, on the other hand, it is assumed that the extent of disclosure can be taken as some indication of the importance of an issue to the reporting entity (Krippendorf, 1980; Campbell et al., 2003). Although, clearly not all the measured information would be of the same type or quality (Guthrie and Mathews, 1985, Gray et al., 1995b), this can be assessed by applying a wide range of distinctions (Krippendorff, 2004, a more detailed account of these is provided later in the paper)7.
Considering the evidence that CSR has been increasing across time, both in number of A colourful novelistic commentary on the constant biases arising from these volumetric arguments is given by Stern (1960): “The obituaries were Poppa Hondorp‟s measure of human worth. „There‟s little they can add or subtract from you then‟, was his view. Poppa‟s eye had sharpened over the years so that he could weigh a two-and-a-half inch column of ex-alderman against three-and-a-quarter inches of inorganic chemist and know at a glance their comparative worth. When his son had one day suggested that the exigencies of the printer and makeup man might in part account for the amount of space accorded the deceased, Poppa Hondorp had shivered with… rage…the obituaries were sacrosanct; The Times issued from an immaculate source” (p. 24, cited in Webb and Roberts, 1969, p. 320).
-9disclosing companies and in the amount of information being reported (KPMG, 2005), particularly over the last 15 years (Erusalimsky et al., 2006), and given the limitations of index studies, it seems reasonable that, as also evident in Table 2, the majority of CSR researchers are now employing volumetric approaches to CA in order to get a proxy of the emphasis given to particular content categories (Milne and Adler, 1999) rather than simply checking for the presence or absence of specific CSD.
It should be noted though, that this is not to say that index approaches to CA are necessarily less useful from volumetric ones. It is arguable, for example, whether volumetric studies would have been more useful in identifying general compliance to the GRI guidelines (as in Turner et al., 2006, study) or comparing the variety of CSR information across Annual Reports and corporate internet sites (as in Patten and Crampton, 2004). The suitability of each approach, therefore, as in most research efforts, should depend on the posed research question (Tashakori and Teddlie, 1998;
Ritchie and Lewis, 2003; Weber, 2004). Nevertheless, when Patten (2002b) and Patten and Crampton (2004) employed both index and volumetric measures and examined the correlations with lines and sentences, in both the cases quite high coefficients were noted, indicating that even with regard to determining the extent of CSR both approaches may be equally valid.
Units of analysis
In this section the units employed in CA are reviewed. For Krippendorff (2004) “units are wholes that analysts distinguish and treat as independent elements” (p. 97). As he
further points out, “In content analysis, three kinds of units deserve distinction:
sampling units, recording/coding units, and context units”. Given that Unerman (2000) has thoroughly discussed the choice on the sampling units and that researchers seem to generally agree (not necessarily on the terminology but) on the actual use of context units, more emphasis is paid in the discussion of the recording/coding units, which seem to sparkle a more lively debate.
In natural order the decisions on the definitions/context precede the ones for the sample and the measurement, as in e.g. the Gray et al. (1995b) review. However, in
Sampling units According to Krippendorff (2004) sampling units “are units that are distinguished for selective inclusion in an analysis… Content analysts must define sampling units so that (a) connections across sampling units, if they exist, do not bias the analysis; and (b) all relevant information is contained in individual sampling units, or, if it is not, the omissions do not impoverish the analysis” (pp. 98-99). One of the first decisions to be made, therefore, when undertaking CA is on the sampling units.
It seems that “the majority of studies into CSR worldwide… have used corporate annual reports as the exclusive sampling unit” (O‟Dwyer, 1999, p. 227) and a number of reasons for this have been put forward in the CSR literature. The Annual Report is “the most widely distributed of all public documents produced by a company” and it “can be accepted as an appropriate source of a company‟s attitudes towards social reporting” (Campbell, 2000, pp. 84–85, see also similar arguments by Gray et al, 1995b, Paterson and Woodward, 2006). It has also been argued that Annual Reports are the single most important source of information on corporate activities (Adams et al., 1998). Further, from an accounting perspective, explanations are most frequently being sought for voluntary CSD in the Annual Reports (Savage et al., 2000)8.
It has been argued, however, that the focus on the employment of Annual Reports may result in an incomplete representation of the quantum of CSR (Zéghal and Ahmed, 1990) since these are not the only medium through which companies report their socially responsible behaviour, a view empirically supported by Unerman (2000), Frost et al. (2005) and Guthrie et al. (2008). Erusalimsky et al’s (2006) affirmation that “in the years since 1990… the rise in the number of voluntary But note that “the exclusive focus on the corporate annual report has been accused of being selfsustaining given that its exclusive choice is often justified in the literature on the grounds that this is what most other studies have done” (O‟Dwyer, 1999, p. 229, a justification supported by Unerman, 1998).
- 11 standalone reports has been astonishing” (p. 12) and Guthrie et al‟s (2008) finding that “companies may use the annual report and corporate website for reporting different types of information” (p. 40) gives further support to Unerman‟s (2000) conclusion that “future studies focusing exclusively on annual reports might not produce particularly relevant results” (p. 674).
As a consequence, a number of CA studies now increasingly employ CSR sources in addition to or other than the Annual Reports as sampling unit. Quite frequently standalone reports are employed (see e.g. Laine, 2005; Jones, 2006; Turner et al., 2006; Bebbington and Larrinaga, 2007), although concerns have been expressed for the use in their analysis of the same protocols employed in the Annual Reports (Erusalimsky et al., 2006); and a number of studies also employ information from the internet (e.g. Esrock and Leichty, 1998; Adams and Frost, 2004; Patten and Crampton, 2004; Unerman and Bennett, 2004; Turner et al., 2006; Guthrie et al.
2008), a data source easier to be captured by the index rather than the volumetric approaches to CA, given its volatile nature (but note that Patten and Crampton, 2004, printed out all the environmental disclosures identified and also applied a volumetric approach). Notably Zéghal and Ahmed (1990), Tilt (1994) and Unerman (2000) have employed a much wider array of CSR sources as sampling unit, including e.g.
brochures and advertisements and several internally circulated bulletins and other ad hoc documents published in each year (see also Lewis et al., 1984, for a study considering publications to employees).
Nevertheless, as Unerman (2000) points out, “a limit must be set to the range of documents included in any research study… [due to the risk of] a researcher being overwhelmed by the number of documents… [and of not being] possible to ensure completeness of data” (p. 671. See also Campbell, 2004, for an account of feasibility factors that might also be at play). Further, it is almost impossible to identify all corporate communications that could possibly contain CSR information (O‟Dwyer, 1999; Guthrie et al., 2004) and it thus seems similarly impossible to identify all the CSR activities of the examined organisations. It, therefore, seems justifiable for studies to employ Annual and standalone reports as the sampling unit as these should contain the bulk of the disclosed CSR information. Ultimately, the sampling unit for each study depends on the set research objectives (Unerman, 2000) and also on the
Context units Context units are defined by Berelson (1952) as “the largest body of content that may be examined in characterizing a recording unit” (p. 135). As Krippendorff (2004)
“the meaning of a word typically depends on its syntactical role within a sentence. To identify which meaning applies to a word from a list of dictionary entries, one must examine the sentence in which the word occurs.
How else would one know, for example, whether the word go is meant to denote a game, an action, or a command? Here, the sentence is the context unit and the word is the recording unit” (p. 101).