4. Methodology
4.2. Knowledge network methodology
The knowledge network is analysed using a combination of qualitative and quantitative sources. The analysis of texts, and collaboration, is based on a corpus of key works of Australian economic history written between 1950 and 1991. ‘Economic history’ has been defined relatively narrowly, including texts that predominantly discuss economic change over time (20 years or more). Texts are selected from wide reading of the subject, with
61 P. Bonacich, 'Power and centrality: A family of measures', American Journal of Sociology, 92, 5, 1987; Hanneman and Riddle, Social network methods.
62 Hanneman and Riddle, Social network methods.
63
further guidance from secondary analyses that focus on the published work of the field.63 This literature is helpful for determining the key authors, the main debates, and the major works for the community. The corpus has been filled out by including the main joint projects, and the contents of the community’s journal – the Australian Economic History Review (AEHR). This list of key texts is not exhaustive, rather the aim is to include a fairly representative sample of the Australian economic history literature at this time. The texts included in the corpus are listed in Appendix D.
4.2.1. Qualitative analysis
All texts in the corpus have been analysed qualitatively, focussing on the approach and interpretation of each author. Textual analysis is the primary methodology for intellectual history, with published works used to understand the ideas of scholars over time. One of the most enduring iterations of intellectual history, the history of thought, focusses on the analysis of ‘unit-ideas’ within texts. As the study of intellectual history has progressed, there has been greater emphasis on biographical, institutional or sociological forces, and the field now employs a wider variety of techniques and historical sources.64 However, ideas have remained the centrepiece of analysis, with contextual forces used to sustain a narrative around the analysis of texts.
The qualitative analysis is guided by the qualitative framework discussed in chapter 3.
Approach combines Lloyd’s epistemological scale and Coleman’s methodological spectrum, with distinction made between the statistical, deductive, and instrumental method of the economist, and the qualitative and realist analysis of the historian. Interpretation
incorporates Coleman’s internalist/externalist spectrum, as well as discussing the use of theoretical frameworks. While ‘approach’ and ‘interpretation’ are used here, both Lloyd and Coleman adopt other methods for classifying texts. Lloyd’s ontology scale makes the distinction between individualism and structuralism. The vast majority of work analysed in this thesis is individualist, meaning this method of classification does not yield much explanatory power. Similarly, Coleman’s distinction between ‘epochal’ and ‘episodic’
economic history largely depends on the research question of the scholar. It thus emerges through the discussion of ‘interpretation’.
63 These are Coleman, 'Historiography'; Jetson, 'Economic history'; Lloyd, 'Economic history and policy'; Lloyd, 'Analytical frameworks'; Schedvin, 'Midas and the merino'.
64 See discussion of different traditions within intellectual history in chapter 2.
64
Other criteria are used to analyse the material in the AEHR over this period. Morgan and Shanahan have determined the main themes in the journal’s output by using the Journal of Economic Literature (JEL) research codes. While these codes successfully highlight the changing interests and geographic reach of the journal, they do not capture the use of a common interpretive framework, or the bifurcation between internalist and externalist texts. Changes in methodology have been determined by measuring the numbers of tables and figures published in each article.65 This is a highly simplified proxy for complex methodological phenomena.
Qualitative analysis has been chosen as the primary way to determine ideas in this community. This may perpetuate a subjective interpretation of the scholar’s work. To address this, strict criteria have been used. This provides a detailed and rigorous analysis of ideas that is comparable with other work in intellectual history.
4.2.2. Citations
The qualitative discussion has been augmented with a quantitative analysis of citations.
Citations are the most widely-used method for quantitatively analysing intellectual communities.66 They are most often used to assess the impact of a text, author, journal, department, or university.67 Citations are seen as an important measure of intellectual influence between authors texts, representing shared pieces of information that connect the citee and citer.68 The Social Sciences Citation Index (SSCI) – a digital, open access resource for citation data from journal articles – is often used to construct these networks, with data for contemporary intellectual networks generally stored reliably.69 However, the SSCI is not available for non-digitised works such as books and historical texts. The
importance of these texts for the Australian economic history field necessitates manual
65 Morgan and Shanahan, 'Supply of economic history'.
66 Ding, 'Scientific collaboration'; Euske, et al., 'Management control'; Gondal, 'Knowledge production'; C. Hsueh and C. Wang, 'The use of social network analysis in knowledge diffusion research from patent data', 2009 International Conference on Advances in Social Network Analysis and Mining, 2009; McWilliams, et al., 'Who is talking to whom?'; Pieters and Baumgartner, 'Who talks to whom'; Siler, 'Citation choice'; Sorenson, et al., 'Complexity'; Studer and Chubin, Cancer mission; Wang, et al., 'Knowledge networks'; White and McCain, 'Visualising the discipline'.
67 Euske, et al., 'Management control'; T. C. Judge, DM, A. Colbert and S. Rynes, 'What causes a management article to be cited: Article, author, or journal?', Academy of Management Journal, 50, 3, 2007; G. Salancik, 'An index of subgroup influence in dependency networks', Adminstrative Science Quarterly, 31, 1, 1986.
68 Kaplan, 'Citation behavior'; McWilliams, et al., 'Who is talking to whom?'; Sharplin and Mabry, 'An alternative ranking'; Siler, 'Citation choice'; Small, 'Concept symbols'.
69 Euske, et al., 'Management control'; McWilliams, et al., 'Who is talking to whom?'; Pieters and Baumgartner, 'Who talks to whom'.
65
coding of citations. Although this is a laborious method of data collection, it improves the representativeness of the analysis by including books, book chapters and un-digitised articles. The citation index includes over 20,000 individually recorded pieces of
information, and thus human error may have affected its accuracy. However, this margin is likely to be fairly low.
Citations are analysed in part two and part three of this thesis. The number of citations in each text has been recorded, showing variations in the extent to which each author draws on others. Each citation is counted, with a few caveats. For in-text citations, if the same author is cited twice within the same paragraph, this is treated as one citation. If two texts by the same author are cited in the same paragraph, this is also treated as one citation. For footnotes, the same rule applies – if there are two citations of the same author in one footnote, this is treated as one citation. If there are multiple works by the same author cited in a single footnote, this is also given a score of one. Citations of primary sources (for example correspondence or historical reports) are not recorded, as the analysis is
focussed on the diffusion of ideas among secondary sources. If a text cites a co-authored work, a score of one is given to each of the co-authors. Similarly, if a co-authored work cites a text, it is recorded as one citation from each of the co-authors.
While citation analysis is widely used to study intellectual networks, it has been criticised for failing to capture other important sociological forces that affect the diffusion of knowledge.70 Citations don’t offer any insight into the author’s perceptions of the papers they have cited, and scholars may have a high level of similarity through disagreement about approach or interpretation.71 The prevalence of cronyism and other reputation-making activities is also a relevant issue, with authors tending to disproportionately cite their friends and colleagues.72 Another source of bias may be the Matthew Effect, where prominent individuals are cited with higher relative frequency, simply because they are seen as important.73 Finally, the importance of quantitative data in economic history means that citations in this field may favour those who establish the primary data sources.
70 McWilliams, et al., 'Who is talking to whom?'.
71 Leydesdorff and Amsterdamska, 'Citation analysis'; M. MacRoberts and B. MacRoberts, 'Problems of citation analysis', Scientometrics, 36, 3, 1996; Phelan, 'Citation analysis'; Sula, 'Visualizing social connections'.
72 Cozzens, 'Measure of science'; Crane, Invisible colleges; Gilbert, 'Referencing as persuasion';
Gondal, 'Knowledge production'; Kaplan, 'Citation behavior'; Kostoff, 'Citation analysis'; Phelan, 'Citation analysis'.
73 Merton, 'Matthew effect'.
66
Prominence for scholars such as Coghlan and Butlin, who were renowned for establishing the ‘quantitative infrastructure’ of the field, may simply reflect a reliance on their data.74
4.2.3. Analysing citation networks
With approximately 4,000 nodes, visualising the citation networks is not very efficacious.
The maps are dense and complicated, and they have very little explanatory power. As such, quantitative analysis of the citation network, based on connections between authors, is the primary method used here. Cohesion metrics have been used to suggest some overall trends in citations, and to compare the networks over time. Average degree is a normalised measure of the number of citations per node. Density indicates the number of ties held as a proportion of the number of possible ties. An expanding literature for scholars to draw on would be expected to increase average degree, and decrease citation density over time.
Centrality metrics have been used to indicate individual prominence in the network.
Citations indicate a one-way transfer of ideas. The centrality metrics thus separate into in- and out- scores. This distinguishes between actors who were central because they were cited frequently, and actors who were central because they cited others frequently. In-degree measures the number of citations the node received, and out-In-degree measures the number of authors the node cited. A high in-degree score indicates prominence, as
influential scholars are generally cited by a wider group. A high out-degree score suggests that the node was the culmination of published work for the community. A later entrant into a group may have a high out-degree score, as they would tend to widely cite the established literature.
Bonacich power indicates prominence due to dependence of others on the actor for ideas or connections. This distinguishes between in- and out- scores. A high in-bonacich power indicates that the actor was cited by otherwise disconnected authors, and a high out-bonacich power score suggests the scholar cited otherwise disconnected authors. The balance between in- and out- measures may change over the course of a career, with the scholar citing established literature in early pieces, but then becoming highly cited as their contribution to the field grows. Betweenness makes no distinction between in- and out- connections, with a higher betweenness score simply indicating that a node formed the path between otherwise disconnected groups. The author’s work may have been a
unifying element for the community, or the author themselves may have brought together
74 See chapter 7 and 9 for a discussion of this.
67
a lot of otherwise disconnected literature in their work. These metrics have been used in chapters 7 and 9 to indicate the relative influence of authors in this community.
Citation similarity has also been used to determine intellectual trends in this community.
Citations are assumed to represent shared pieces of knowledge between authors and texts, and so citation similarity indicates the extent to which authors drew on similar literature.
This may be associated with a shared approach or interpretation. Citation similarity is the key method for determining intellectual trends for contemporary knowledge domains.75 There are generally three methods for determining ‘similarity’. Co-citation analysis measures similarity between two texts based on the degree to which they are cited together by others.76 Bibliographic coupling deems two texts similar if they reference common works. Direct citations measure a link between texts only if they cite one another.77 Bibliographic coupling is used here, with similarity between authors determined by their common citations (including to each other’s work). The primary research themes for this component of the thesis – determining similarity between authors based on incorporating common pieces of knowledge – makes bibliographic coupling a close conceptual fit for the analysis. In UCINET, Pearson correlations, calculated by rows, are used.78 Scores vary between -1 (meaning the two actors have exactly the opposite ties), to 0 (meaning there is no association), to +1 (meaning the two actors have exactly the same citations). Citation similarity is an imperfect measure of intellectual trends for Australia’s economic history community. While it does reveal some of the qualitative and social groupings, there are a number of important omissions.79 A shared methodology or perspective may not be the sort of thing that would lead to common citations. Citations may also indicate social or positioning functions, and disagreement
75 White and McCain, 'Visualising the discipline'; S. P. Nerur, A. A. Rasheed and V. Natarajan, 'The intellectual structure of the strategic management field: An author co-citation analysis', Strategic Management Journal, 29, 3, 2008; K. W. Boyack and R. Klavans, 'Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most
accurately?', Journal of the American Society for Information Science and Technology, 61, 12, 2010;
Euske, et al., 'Management control'; Ding, 'Scientific collaboration'; McWilliams, et al., 'Who is talking to whom?'; Pieters and Baumgartner, 'Who talks to whom'.
76 White and McCain, 'Visualising the discipline'; Nerur, et al., 'Strategic management field'.
77 For a description and critique of these methods, see Boyack and Klavans, 'Co-citation analysis'.
78 This is the most common method of determining similarity in UCINET networks, see Hanneman and Riddle, Social network methods. The measure is calculated by rows because each of the ties in the network is set as a row in the citation index.
79 See the discussion of the knowledge network in chapters 7 and 9.
68
between scholars.80 Citation analysis is thus used cautiously, and is verified and complemented by qualitative analysis, oral history, and social networks.
4.3. Conclusions
This chapter has outlined the methods used to examine Australia’s economic history field in the post-WWII period. A range of qualitative and quantitative techniques are deployed to analyse the ways in which social and intellectual elements affected the development of this scholarly community. Each source is intended to compliment the others, with a more complete picture of the community emerging from the combination of techniques. The use of social network analysis is a particularly innovative technique for the study of
intellectual communities. This quantitatively and visually analyses social interactions between scholars. These maps are complemented by oral history sources, which provide additional details about the nature of institutions and collaborations, and the effect these ties may have had on ideas in the group. The qualitative analysis of texts is used to examine the knowledge network. This classifies texts based on ‘approach’ and
‘interpretation’. Citation analysis has recorded the pieces of literature included in texts.
This offers a quantitative assessment of intellectual trends, determining individual prominence, overall citation trends, and similarity between authors.
Each methodology used in this thesis has limitations. Oral history and qualitative analysis of texts may be subjective, with the current author, and the interview participants, imposing their own judgements on the analysis. Although the social networks and the citation analysis are more objective and verifiable, they are reductionist by assuming a uniform ‘effect’ from ties, and disregarding the nuance of relationships and intellectual influence in a community. Combining qualitative, quantitative, and visual techniques allows for verification between different sources, and the minimisation of bias in any one particular technique. It is through a range of methodologies, and an integrated
understanding of the social and knowledge networks, that the development of Australia’s economic history community is best understood.
80 Cozzens, 'Measure of science'; Crane, Invisible colleges; Gilbert, 'Referencing as persuasion';
Gondal, 'Knowledge production'; Kaplan, 'Citation behavior'; Kostoff, 'Citation analysis'; Phelan, 'Citation analysis'.
69