Defining Scholarship

Currently there are two principal views of scholarship. The first privileges the research and production of knowledge. The second is a more complex view incorporating teaching, service, knowledge creation and other factors. The shift to research production as the principal indicator of scholarship dates back to the 1980’s (Dirks, 1998). This transition is not without controversy. In Ernest L. Boyer’s book “Scholarship Reconsidered” (1990) sponsored by the Carnegie Foundation for the Advancement of Teaching, Boyer challenged the notion of using publication counts as a criteria for measuring scholarship. This book became a launching point from which institutions began to examine how they evaluated scholars. In the 1990’s there were many calls to reconsider the journal count, redefine scholarship, and to set clear criteria methodologies for evaluating scholarship (Boyer, 1990; Dirks, 1998; Glassick, 2000; Slanger, 1997) The Carnegie Foundation report created intent in examining ‘scholarship’ and the ‘criteria’ to measure ‘scholarship’, however in recent years the discourse has stagnated. In the last decade in IS we have seen the return to the status quo of using journal publication counts and has again become the de-facto method in evaluating scholarship.

7.3.1

Identified Critiques with Current System

A persistent challenge is the lack of a standardized and generally accepted definition of scholarship. Without such a definition there can be no measure of scholarship nor criteria for measuring scholarship. “Once scholarship is more broadly defined and evaluated according to common criteria, reward structures must be created to motivate and institutionalize the change” (Dirks, 1998). But even with a standard definition of scholarship, it may be impossible to create a single criteria for measuring scholarship (Krahenbuhl, 1998). An alternative may be analogous to the way the technology industry creates separate criteria for ‘technical’ and ‘managerial’ tracks to evaluate different types of workers.

The return to the use of journal citation counts has certain benefits and disadvantages. The benefits are that the journal citation counts are simple, easy to cross compare, and it is a single measure. The disadvantages of using citation counts include the lack of fairness, lack of openness and transparency, and a citation count criteria is power laden. The general approach institutions seem to follow is to use a journal ranking system to evaluate the worth of a scholar but this model has many flaws. First, this one-size-fits-all measure, while useful for administrators, is by no

means a fair measure for all institutions or their faculty. By instituting a research journal publication evaluation method across the board, professors at schools that are more traditionally ‘teaching’ school suffer.

Second, some of the consequences of emphasizing too much on research are that other areas of scholarship become victims to the system. The emphasis on research output can put a strain on our focus on teaching. Dirks (1998) identifies studies that indicated that a majority of scholars feel that the emphasis on research is compromising the quality of their teaching, that they are not awarded for good teaching, and that there is little overlap between their research work and teaching work. As some institutions “the reward system was heavily weighted toward published research, not effective teaching” (Glassick, 2000).

Third, by emphasizing ‘research’ we are getting further from key activities of the scholar such as teaching and service, and this is perceived as such by the public. “By the end of the 1980s, incentives for faculty work were very heavily focused on research and writing for publication, which the public had come to see as an increasingly serious distortion of the goals it sought for higher education institutions” (Dirks, 1998).

Fourth, these journal publication counts are just that. They are counts of ‘hits’ and do not address the quality of the works themselves. More than a third of faculty “supported the proposition that at their institutions, publications were ‘just counted, not qualitatively measured’” (Glassick, 2000). Journal publication counts are closely tied with journal ranking lists. Each institution adopts some sort of journal ranking list. The ‘A’, ‘B’, and ‘C’ level journals are identified and researchers strive to reach the ‘A’ level journal. The adoption and ranking of journal lists are wrought with politics where researchers try to place journals congruent with their affiliation to higher position in the journal list, whether they publish in those journals or are on the editorial board of such journal. This leaves researchers like Clyde Holsapple who predominantly publish in management journals in a quandary, where IS journal list do not have their target journals. They must decide whether to continue there IS research or try to conform their research to the journal lists adopted by their institutions. IS researchers continue to publish in non-IS research venues like finance, marketing, and management, and journal lists with IS journals continue to minimize contributions in non-IS journals. The use of bibliometric measures does not discriminate the way journal lists discriminate againsts non traditional IS research.

Therefore the initial step is to identify what scholarship entails. Scholarship can be defined widely. One of the two major problems that Glassick (2000) identifies is the fact that it is hard to pin point “the meaning of ‘the

scholarship of teaching’”. The next two sub-sections will identify what the literature has said about defining scholarship. Once scholarship has been defined the next problem is trying to measure scholarship for the researcher. The second major problem identified is “the question of how the quality of scholarship shall be measured” (Glassick, 2000). Section 2.4 will explore why measuring scholarship is a challenge.

7.3.2

Boyer’s Four Areas of Scholarship

Boyer identified four areas of scholarship: the scholarship of discovery, the scholarship of integration, the scholarship of application, and the scholarship of teaching (Boyer, 1997). The scholarship of discovery is the traditional notion of research and the creation of knowledge. The scholarship of integration is research that is at the boundary of a discipline and makes a connection with other disciplines. The scholarship of application is where the research is applied to a real life problem. Scholarship of teaching are activities that pertain to pedagogical endeavors (Boyer, 1990).

Others have proposed models similar to the Boyer typology and have worked off of the Boyer four area model. Slanger (1997) offers these four criteria “research that keeps faith with our training, research that makes a contribution to the field, research that serves as a model for students, and, finally, research that is personally and professionally renewing” (Slanger, 1997). Krahenbuhl offers a results oriented model focusing on knowledge transmission, knowledge generation, and knowledge application (Krahenbuhl, 1998). Glassick (2000) identifies “six standards of excellence in scholarship...clear goals, be adequately prepared, use appropriate methods, achieve outstanding results, communicate effectively, and then reflectively critique their work” (Glassick, 2000). A summary of how these different views of scholarship and how they are similar to Boyer’s four area model is given in Table 1.

Criteria/Typology of Scholarship

Boyer (1990) Slanger (1997) Krahenbuhl (1998) Glassick (2000) Traditional Research Discovery: Traditional research, creation of knowledge Research that keeps faith with our training Boundary/Cross Discipline Research Integration: Boundary research, cross discipline research Research that is renewing Knowledge generation and knowledge transmission.

Research that use appropriate

methods, achieve outstanding results, and communicate effectively.

Research that has clear goals, be adequately

Research Application Application: Real world application Research that contributes to field reflectively critique their work Pedagogical Teaching: Pedagogical endeavors Research that serves as a model for our students

Knowledge application

Not addressed

Table 1. Scholarship defined compared to Boyer model

7.3.3

Integrating Definitions of Scholarship

Some universities have even adopted and offered definitions of scholarship. For example, at Fort Hays State University (FHSU) in Kansas, scholarly activities is “defined as original, innovative intellectual contributions in the form of research, practice, creative activity, or performance. FHSU recognizes and values the diversity of types of scholarship, including discovery, pedagogy, integration, engagement, and application” (Boyer, 1997).

7.3.4

Measurement Challenges

The major problem that Glassick (2000) identifies after the problem of scholarship definition is “the question of how the quality of scholarship shall be measured”. In the current dissertation the theme has been concerned with after defining ‘scholarship’, then how to measure this ‘scholarship’. The citation measures and social network measures are concerned with this ‘measurement’ issue. The measurement criteria of scholarship are an important part of the academic process. Tying criteria to a reward and value system is crucial. Due to the lack of a clear or a too narrow of an evaluation criteria, scholars are calling for a clearer set of goals that integrate the institution’s mission and tie these criteria to faculty rewards (Dirks, 1998; Glassick, Huber, & Maeroff, 1997). “The measure of one's scholarship is the fundamental criterion for all meaningful rewards in colleges and universities, including retention, advancement, perquisites and recognition” (Dirks, 1998). Dirks (1998) is calling for a more broad definition of scholarship, to include not just research but teaching, service, and the integration with other areas outside of the typical scholarly world. Once ‘scholarship’ has been defined more broadly, measures and incentives need to be put in place to reward all component elements of scholarship.

The criteria would need an update depending on the definition of ‘scholarship’. By using a multi-tiered definition like Boyer has suggested, there is a need for criteria measuring each of the component elements. Dirk says that “Redefinition of scholarship must be combined with changes in incentives, which requires that all the forms of

scholarship in the broader definition be evaluated with parity” (Dirks, 1998), and Boyer tells us that “at my institution, we need better ways, besides publication, to evaluate scholarly performance” (Boyer, 1990).

To recapitulate a more diverse set of citation criteria, defining scholarly influence is needed and wanted by academics. These criteria would help require change in faculty reward structures. “Among other changes, diversity of faculty jobs would create a diversity of rewards for faculty that conflicts with many fundamental collegial values” (Dirks, 1998). A view of ‘scholarship’ that includes Boyer’s (1990) four components Discovery, Integration, Application, and Teaching, can also provide a basis for measuring scholarly influence.