«Abstract This paper revisits the debate on the units of Content Analysis (CA) for the purposes of Corporate Social Reporting (CSR) research and also ...»
- 19 The proportions of a page approach, however, has been also criticised for a number of reasons, mainly due to that it takes as data only dictated attributes of the content (statements about the content made by the analyst or participants in the research) and not the content itself, which results in arbitrarily created visual recording units (Ekman et al., 1969; Paisley, 1969). Given further that “it is difficult to place an objective measure on pictures and diagrams” (Deegan et al., 2000, p. 118) researchers often find it difficult to decide on whether this information is CSR or not, and then on how to classify it. As Newson and Deegan (2002) note, “in a number of cases, the expert coder regarded photographic evidence referring to employees in a work environment as less relevant than photographic evidence highlighting employees celebrating acts of achievement” (p. 193). In addition, as Wilmshurst and Frost (2000) note, although arguably pictures may be worth a thousand words (but which words?), “to include them in a measure based upon an unweighted word count is highly subjective” (p. 17).
Further limitations of the proportions of a page approach include that this approach is affected by different font and page sizes (Tilt and Symes, 1999; Paterson and Woodward, 2006); that an additional area of subjectivity is introduced with regards to the treatment of margins and blank pages (Gray et al., 1995b; Unerman, 2000); that the approach is similarly affected as words and sentences by the differences in grammar and repetition12 (Patten, 2002a); that the additional thoroughness and effort required in the use of the grid for recording increases the possibility for measurement errors (Milne and Adler, 1999); and that it is impossible to directly record data in an electronic (e.g. internet or pdf) or microfiche form (Campbell, 2004). It should be noted that, even if it is attempted to print (as Patten and Crampton, 2004, did) or type (as Ingram and Frazier, 1980, did) data in one of these forms, this would still result in some distortion to the size and context (e.g. margins, font and page sizes are often affected). Thus, the inherent limitations of this approach lend support to researchers to reject it and, albeit acknowledging the losses from the exclusion of pictorial or graphical evidence, to adopt words or sentences as recording units.
Although Unerman (2000) points out the possible limitations arising from differences in the use of grammar when he discusses sentences, it should be noted that grammar and repetition are context issues and thus affect all recording methods.
In an attempt to employ an alternative and more valid method for CSR measurements, a number of studies adopt, implicitly or explicitly, a page size approach (where “the written and pictorial part of a page… [is] considered to be the page itself”, Gray et al., 1995b, fn16, p. 90). The basic tenet of this analysis is that when measuring the extent of disclosures the collected data should be considered in conjunction with the physical source from which they are extracted. Most frequently researchers attempt to estimate CSD as proportion to the whole discussion in the report, often on a line-by line (Bowman and Haire, 1975; 1976; Trotman and Bradley, 1981) or on a sentence-bysentence (Salama, 2003; Hasseldine et al., 2005) basis. However, even when authors such as Dierkes (1979) report that they measure the extent of CSD in number of pages of each report (or as quarters of a page of each report, as in Gibson and Guthrie, 1995), this implies that a page – size rather than a standardised proportions of an A4 page approach was adopted. Major limitations of this approach are that researchers do not seem to include pictures when employing it and, even more importantly, that it would have been meaningless to adopt it in a research examining standalone reports, where it has been suggested (Buhr, 1994) that coding should commence on the assumption that everything should be considered CSD apart from some pre-specified information.
Hackston and Milne‟s (1996) study is a potentially more credible adaptation of a page size approach. The authors originally sentence-coded and measured their data sets and then constructed an approximation to page measurement from these data: firstly, “the average number of sentences per page of the chairman‟s report for each annual report was calculated. The average for each report was then divided into the total number of social disclosure sentences for that report to produce a derived page measurement for each company” (p. 86). Despite their acknowledged crudeness of this measure, when some refinements are made to it (more sentences from pages containing non-pictorial information are included in the estimation of the average and a detailed page adjusted
- 21 grid is also employed to account for the space of non-narrative disclosure) 13 it may provide more valid results with than the proportion of page approach.
As illustrated in Table 3, this approach is not affected by the report or font sizes, neither by the margins and blank pages, and although it provides less detailed measurements, it is easier to measure and further is not affected by a possible pdf/microfiche form of the text (even if printing affects the documents‟ size, it still has no effect on page size measurements); it thus seems to capture in a more valid way than the proportion of a page approach the information identified as CSR, particularly given its triangulation benefits from the additional use of sentences, and to further generate more reliable results. However, given that the generated data are in a page size and not in a standardised A4 page form, this implies that the derived measure (similarly to the proportions of the report one) could provide meaningful average CSD approximations but crude aggregate figures unless the average sentences per report of all added derived measures is known and adjustments are made; thus for the purposes of a database such as the one of the Centre for Social and Environmental Accounting Research (CSEAR), the employment of standardised proportions of A4 pages is deemed more suitable to avoid tedious adjustments.
The BA study utilises a refinement to Hackston and Milne‟s (1996) page size approach. To further increase the validity of their instrument all pages containing solely narrative information per report were considered for the calculation of the average number of sentences per page. Further, all pages in microfiche form containing non-narrative information were firstly printed in a A4 format; then as illustrated in Figure 2, a clear plastic A4 acetate was employed to draw a page-size grid with six rows of equal height and eight columns of equal width and the proportions were estimated to the nearest 1% of a page (for the documents in electronic form a specialised pdf reader software, incorporating a detailed 25x35 cell „grid‟ view, was employed)14. Despite the fact that the page grid employed for the Tinker and Neimark (1987) in an attempt to develop an aggregate textual and pictorial character measure of CSD “by counting the number of textual characters that would fit into the photographs that address the subject” (p. 80) seem to employ a similar measurement approach.
Note that a number of practicalities may arise when it is attempted to estimate the „average‟ page size, both in terms of sentences and in terms of written and pictorial space of the report: e.g. in BA (2000) there seem to be 2 main types of pages, including on average either 24 or 44 sentences, in which
- 22 data in microfiche form was not as detailed as the ones employed by Gray et al.
(1995b) or Unerman (2000), (but was still significantly more detailed than quantifications on the basis of the nearest tenth or even quarter of a page [e.g. Guthrie and Parker, 1989] that seems to have been employed in a number of earlier studies [Hackston and Milne, 1996]) since only tables, images and narratives in large typeface were recorded in this manner, it was considered that the use of a more detailed grid would add more to the possibility of measurement errors rather than to the validity of the findings.
Take in Figure 2
It should be noted, however, that when Hackston and Milne (1996) later employed a proportion of a page approach, they found “extremely high correlations between the three measures of disclosure amount (measured pages, derived pages, and number of sentences)” (p. 93, note that non-narrative information was excluded). In essence, this finding suggests that the theoretical advantages that the page size approach possesses (with regards to report and font sizes, margins and blank pages) do not hold in practice and both approaches are equally valid in measuring CSD. In this case, the remaining benefits of the page size measure is the increased reliability due to the ease in measurements and validity from considering pdf/microfiche in a more accurate manner (Hackston and Milne, 1996, did not check for those) and from triangulating with sentences, whilst the proportion of a page one maintains the benefits in detail and in straightforward aggregate comparability of the findings (but in A4 and not in actual page size terms).
It seems therefore that, given that words and sentences do not account for nonnarrative information, page size and proportion of a page approaches are more suitable. If detailed empirical investigation, involving all measurement techniques and examining both narrative and non-narrative information and possible effects of pdf/microfiche printouts reveals high correlations, this would grant validity to both and the choice should be then clearly subject to the research objectives. If however case a weighted average number of sentences per page was estimated. A similar approach was employed to estimate the average page size grid of the report to measure the pictorial space.
- 23 these correlations are not identified, and if the investigation does not provide any indication on which technique is more suitable, then probably page size data should be considered the superior recording unit, given the theoretical advantages and the additional validity gained from the sentence counts. It should always be remembered, though, that this discussion on the recording units of volumetric studies is based on the assumption that volume indicates importance. However, volume is not always equivalent to the content of CSD (Patten and Crampton, 2004) and it seems that “asking the right question of the data is even more important than the system of enumeration used to present the findings” (Holsti, 1969a, p. 12). The following discussion negotiates these issues.
In this section ways of defining/classifying the context units in CA to further increase the method‟s contribution to researching CSR are discussed. According to Krippendorff (2004), units in CA can be identified according to one or more of five kinds of distinctions: physical, syntactical, categorical, propositional, and thematic.
As the following discussion reveals, categorical and thematic distinctions are of more relevance to CSR research and are now respectively reviewed15.
Categorical distinctions Categorical distinctions define units “by their membership in a class or category – by their having something in common” (Krippendorff, 2004, p. 105) and are by far the ones most frequently employed in CSR research. Common characteristics of these Physical distinctions relate to time, length, size or volume but not information and they have been thus largely considered in the preceding section. Syntactical distinctions relate to the grammar of the medium of the data. Arguably “no content analysis can ignore grammar” (Deese, 1969, p. 42); to the extent however that these distinctions involve the selection of a recording unit from e.g. words or sentences, they have been considered in the preceding section whilst to the extent that these relate to the consequences on CA from differences in grammar in the context of the reports, a detailed discussion on those, albeit interesting, is beyond the scope of this paper. Propositional distinctions relate to decomposing language to identify certain linguistic relations and as with syntactical distinctions, despite their potential contribution (see e.g. Clatworthy and Jones, 2006) are beyond the scope of this paper.
- 24 distinctions are that they can all be quantified; that to a certain extent, they mostly focus on the manifest rather than the latent content and the syntactic and semantic dimensions of communication, and that they are all deductively defined (and thus, as opposed to subject categories, they all relate to a theoretical framework [Stone et al.
1966]). Clearly, however, these approaches are not equally useful in contributing to CSR research. As Table 4 illustrates four types of categorical distinctions were identified, one of them regarding the type of CSD (positive, negative and neutral) and three regarding the quality (monetary vs. non monetary; general vs. specific, and substantive vs. symbolic).
Positive vs. negative A number of large-sample studies have revealed that managers “attribute negative organizational outcomes to uncontrollable environmental causes and positive outcomes to their own actions” (Abrahamson and Park, 1994, p. 1302, see also Bowman, 1976; Bettman and Weitz, 1983; Staw et al., 1983; Salancik and Meindl, 1984 for similar arguments). Pfeffer (1981) suggests organisations would be expected to adopt strategies involving “the selective release of information which is… defined along criteria more favourable to the organization… measured along criteria which are more readily controlled by the organization, and… acceptable to those interested in the organization” (p. 30).
In an attempt to investigate these propositions, a number of CSR studies have often employed the positive vs. negative distinction for CSD, one of the earliest CA distinctions dating back to the 1640s and some studies of censorship (Stone et al., 1966; Rosengren, 1981) and e.g. Patten and Crampton (2004) illustrate that that the annual report contains proportionately more negative data than does the website suggesting that “the focus of Internet disclosure may be more on corporate attempts at legitimation” (p. 31). There are, however, some theoretical and practical limitations encountered when using this approach. In theory, for example, organisations may
- 25 disclose a significant quantity of positive information to satisfy targeted stakeholders (Pfeffer, 1981) indicating a more opportunistic organisational stance towards CSD (Gibbins et al., 1990) but they may also do so because this simply reflects their reported actions. And when conducting CA it is often difficult to e.g. differentiate between a positive and neutral disclosure and achieve consistency and comparability across studies. In these cases, as Weber (1990) underlines, “perhaps the best practical strategy is to classify each word, sentence or phrase in the category where it most clearly belongs” (p. 36).
It seems, therefore that the positive or negative CSD when considered per se can not contribute significantly to CSR analysis. On the contrary, more sustainable arguments seem to be built when the positive or negative CSD is related to similarly classified evidence from e.g. news papers or periodicals, related to community concern (Guthrie and Parker, 1989; Deegan et al., 2002), or when, as in the BA study, the changes in this type of reporting are considered as potential reaction to major social accidents.
Under all circumstances, this distinction is even more useful when viewed in relation to an approximation to CSD quality, as discussed below.
Substantive vs. symbolic
Often researchers attempt to assess the quality/evidence of the disclosure. Hammond and Miles (2004) offer a comprehensive account on quality assessment of CSR and note that what is most frequently regarded as quality disclosure is quantitative disclosure, third party verification and the adoption of reporting guidelines and standards. Most frequently CA index studies on the basis of disclosure/ non disclosure have been employed to assist in both third party verification (Ball et al., 2000;
O‟Dwyer and Owen, 2005) and adoption of reporting guidelines (Jones, 2006;