Information Technology Use, Selected Learning and
Personal Development Outcomes, and Other
College Experiences
George D. Kuh Shouping HuThis study examines the relationships between student characteristics, student use of computers and other information technologies (C&IT), the amount of effort they devote to other college activities, and self-reported gains in a range of desirable college outcomes. Based on an analysis of responses to the College Student Experiences Questionnaire from 18,344 under-graduates at 71 four-year colleges and univer-sities, students appeared to benefit more from C&IT when they used it frequently and in a variety of ways. Equally important, using C&IT was positively related to educational effort with the effects of C&IT on outcomes of college being largely mediated through the educational efforts students put forth.
Computing and information technology (C&IT) is now almost ubiquitous on most college campuses (Dolence & Norris, 1995; Gilbert, 1996; Green & Gilbert, 1995; West, 1996). The 1998 National Survey of Information Tech-nology and Higher Education indicated that in 1994 about 8% of postsecondary classes were using E-mail (Institute for Higher Education Policy, 1999). By 1998, this percentage jumped to about 44%. Between 1996 and 1998 the percentage of classes using Internet resources increased twofold from 15% to 30%. In the 1980s, only 32% of students reported substantial progress in becoming familiar with computers during college. By the mid-1990s this per-centage had jumped to about 60% (Kuh, Connolly, & Vesper, 1998). About half of all institutions have a mandatory student fee to support information technology that is becoming increasingly interactive and distributed,
allow-ing users anywhere to access at any time a rich array of information and resources via the Internet and other sources (Green, 1996). Indeed, “nothing before has captured the imagination and interests of educators simul-taneously around the globe more than the World Wide Web” (Owston, 1997, p. 27).
Classrooms linked to global networking technologies allow students to interact with peers and master teachers around the world (Riel, 1993). In 1995, about one third of all colleges and universities offered distance education courses and almost a quarter had degrees that students could complete entirely on line (Merisotis, 1999). Accredited on-line degree programs are now available with electronic file exchange, Internet video conferencing tools, E-mail office hours, electronic libraries, virtual cafes, whiteboards, digitized movies, voice and chat tools, debate forums, and student opinion polls (Fetterman, 1996; Harasim, Hiltz, Teles, & Turoff, 1995). Through on-line learning apprenticeships, experts and learners can share their ideas and pose questions about those of others, thereby clarifying and extending their thinking and knowledge. “Used appropriately in concert with powerful pedagogical ap-proaches, technology is supposed to enrich synchronous classroom activities and provide students with engaging self-paced and asyn-chronous learning opportunities that enable students to learn more than they would other-wise at costs ultimately equal to or below that of traditional classroom-based instruction” (Kuh & Vesper, 2001, p. 87).
Thus far, studies examining the effects of using computer and information technology are George D. Kuh is Chancellors’ Professor of Higher Education at Indiana University Bloomington. Shouping Hu is Assistant Professor of Educational Administration and Supervision at Seton Hall University.
in some ways encouraging. For example, compared with traditional classroom activities, E-mail discussion, delayed text collaboration and file sharing, real-time idea brainstorming, and real-time text and graphics collaboration appear to enhance productive collaboration among students and encourage higher levels of student participation (Alavi, 1994; Bonk, Medury, & Reynolds, 1994; Oblinger & Maru-yama, 1996). Both delayed and real-time conferencing support tend to promote more frank discussion and equal opportunity among participants than traditional classroom instruc-tion (Sproull & Kiesler, 1993). A study of computer-mediated discussions among under-graduate education students found that an instructor’s informal conversational discourse style together with comments directed to individual students or to specific statements made in the conference fostered higher partici-pation, more complex interactions, and greater peer-peer interaction than when instructor comments were posed as either questions or formal statements to the entire group (Ahern, Peck, & Laycock, 1992).
Most of what is known about the effects on achievement of using computing and informa-tion technology is based on student performance in individual courses. Kulik and Kulik (1991) found that college students had about a .31 standard deviation advantage in course learning knowledge over peers who received instruction without the aid of computers. From their synthesis of studies of hypermedia and hypertext use, Dillon and Gabbard (1998) found only small and nonsignificant differences between the learning gains of students using hypermedia and hypertext and those exposed to traditional forms of instruction.
Two recent studies examined the impact of computing on outcomes other than achievement and content-specific knowledge across multiple institutions. Kuh and Vesper (2001) reported that after controlling for such factors as college grades, age, gender, hours worked per week, parents’ education, and educational aspirations, students’ self-reported gains in becoming familiar with computers were highly correlated with self-assessed gains in a variety of other
areas such as independent learning, writing clearly, and problem solving. Flowers, Pasca-rella, and Pierson (2000) also controlled for many of the same potentially confounding influences but also included precollege cognitive development and motivation. They found that computer and E-mail use had only trivial and nonsignificant effects on four standardized cognitive measures: end of first year composite cognitive development, reading comprehension, mathematics, and critical thinking. At the same time the use of computers and E-mail signi-ficantly affected the cognitive growth of students attending 2-year colleges. It was not clear why information technology had a greater effect on two-year college students than their counterparts at 4-year institutions.
Although research findings to date are generally promising, a substantial gap remains in our understanding of the effects of computer and information technology on student learning and other educational outcomes (Morrison, 1999). For example, little evidence is available beyond student performance in individual classes to determine the effects of different forms of technology on various aspects of the college experience including the acquisition of a range of desired outcomes of college (Morrison; Hibbs, 1999) or the most efficacious design and use of these new technologies (Ehrmann, 1995). Nor is it clear which aspects or forms of computer and information technology have the greatest effects for what types of students for what outcome areas.
In addition, some evidence suggests that the effects of computing and information technology use may not be uniform for different types of institutions or students. Institutional affluence, student ability and socioeconomic status (SES), and accessibility and use of computing and information technology appear to be highly correlated (Gladieux & Swail, 1999). The greatest learning benefits from using technology appear to be realized by higher ability students (Dillon & Gabbard, 1998), which is consistent with the Flowers et al. (2000) finding that students with the highest level of precollege cognitive development benefited the most from computer and information technology use as
reflected by end-of-first-year cognitive gains. Others worry that technology may dehumanize the educational process and that the quality of social relations between students and faculty will deteriorate if, for example, E-mail substitutes for rather than augments face-to-face inter-actions. Less frequent contact with peers could mute the development of interpersonal commun-ication and other skills as students increasingly rely on information technologies to obtain information, prepare class assignments, and communicate with one another and their teachers (Upcraft, Terenzini, & Kruger, 1999). Also, computers and information tech-nology can be a distraction if used primarily for noneducational or entertainment purposes, such as downloading and compiling music, playing games, or communicating with family, friends, and coworkers (Reisberg, 2000). Using com-puters for these activities reduces the amount of time available for students to be engaged in educationally purposeful activities such as taking advantage of cultural and performing arts venues, attending lectures by visiting scholars, and discussing substantive topics with instruc-tors and peers—activities that the research shows contribute to student learning and personal development.
The purpose of this study was to examine the characteristics of student use of C&IT and to determine the relationships between student use of C&IT and the amount of effort they put forth in other college activities and the gains they make in a range of important college outcomes. Three research questions guided the study.
First, what is the nature and frequency of use of various types of C&IT by undergraduates with different background characteristics and at different types of colleges and universities? Do students with certain characteristics or attending particular types of institutions use different forms of computer and information technology more frequently? For example, do students at private colleges and large research universities use C&IT more frequently, and are they therefore advantaged, because they attend an institution that can afford enough state-of-the-art
hardware to make technology accessible to virtually all students?
Second, what is the relationship between using C&IT and the amount of effort students expend in academic pursuits and engagement in other educationally purposeful college activities? Are students who use certain forms of C&IT more or less likely to be engaged in other meaningful aspects of the college experience, such conversing with faculty members and peers face-to-face or participating in clubs and organizations?
Finally, what is the relationship between using C&IT and the range of desirable outcomes of attending college? Is using C&IT a positive, neutral, or negative influence on overall gains from college or on specific areas as general education, personal or social qualities, or job-specific abilities?
METHODS
Data Source and Instrument
The data for this study are from the College Student Experiences Questionnaire (CSEQ) research program. The fourth edition of the CSEQ (Pace & Kuh, 1998) is designed for students attending 4-year colleges and univer-sities and gathers information about students’ background (age, major field, etc.) and their experiences in three areas. The first area is the amount of studying, reading, and writing students do and the time and energy (effort) they devote to various activities measured by items contributing to 13 activities scales (Kuh, Vesper, Connolly, & Pace, 1997). One of these scales, C&IT, includes nine items describing various forms and uses of computers and information technology (Table 1). The response options for all Activities items are: 1 = never, 2 = occa-sionally, 3 = often, and 4 = very often. The second area is student perceptions of the extent to which their institution’s environment emphasizes important conditions for learning and personal development measured by 10 Environment items. The final area is an estimate of what students think they have gained from attending college represented by 25 Gains items that load on five factors: general education,
intellectual skills, personal and social develop-ment, science and technology, vocational preparation (Kuh et al., 1997). Response options for the gains items are: 1 = very little, 2 = some, 3 = quite a bit, and 4 = very much.
Examinations of the validity of self-reported information (Lowman & Williams, 1987; Pace, 1985; Pike, 1999, 1995) such as that obtained using the CSEQ indicate that they are generally valid under five conditions: (a) when the information requested is known to the respon-dents, (b) the questions are phrased clearly and unambiguously (Laing, Sawyer, & Noble, 1988), (c) the questions refer to recent activities (Converse & Presser, 1989); (d) the respondents think the questions merit a serious and thought-ful response, and (e) answering the questions does not threaten, embarrass, or violate the privacy of the respondent or encourage the respondent to respond in socially desirable ways (Bradburn & Sudman, 1988). CSEQ items satisfy all these conditions. The questionnaire requires that students reflect on what they are putting into and getting out of their college experience. The items are clearly worded, well-defined, and have high face validity. The nature of the questions refers to common experiences of students during the current school year, typically a reference period of about 6 months or less. The format of most response options is a simple rating scale that helps students to accurately recall and record the requested information, thereby minimizing this as a possible source of error. The estimate of gains items ask students how much they think their college or university experience contributed to their own growth and development. In this sense the progress that students say they make is a value-added judgment (Pace, 1990b). Responses to gains items have been shown to be generally consistent with other evidence, such as results from achievement tests (Brandt, 1958; DeNisi & Shaw, 1977; Hansford & Hattie, 1982; Lowman & Williams, 1987; Pace, 1985; Pike, 1995). Pike found that student reports to gains items from the CSEQ were highly correlated with relevant achievement test scores and concluded that self-reports of progress could be used as proxies for achievement test results if
there was a high correspondence between the content of the criterion variable and proxy indicator. Based on their review of the major college student research instruments, Ewell and Jones (1996) concluded that the CSEQ has excellent psychometric properties and high to moderate potential for assessing student behavior associated with college outcomes. Additional psychometric properties of the CSEQ are described in Kuh et al. (1997).
Sample
The sample is composed of 18,344 under-graduates from 71 four-year colleges and universities who completed the 4th edition of the CSEQ in 1998 and 1999. The schools include 21 research universities (RU), 9 doctoral universities (DU), 22 comprehensive colleges and universities (CCU), 8 selective liberal arts colleges (SLA), and 11 general liberal arts colleges (GLA) as classified by the Carnegie Foundation for the Advancement of Teaching (1994). Although the mix of schools reflects the diversity and complexity of 4-year colleges and universities, the CSEQ data base is essentially a convenience sample in the institutions that use the instrument administer it in different ways and for different reasons. Women (63%), traditional-age students (92%), first-year students (48%), and students from private colleges were overrepresented compared with the national profile of undergraduates attending 4-year colleges and universities. About 77% of the sample were White students, 8% Asian Americans, 6% African Americans, 6% Ameri-can Indians and students from other back-grounds and 4% Latinos. Also, more than half of the students were majoring in a preprofes-sional area, 17% in math and science, 10% in social science, and 8% in humanities. Almost one fifth (19%) had majors from two or more of the major field categories. Descriptive statistics on student use of nine C&IT items on the instrument, C&IT total scores, EFFORT-SUM, GAINEFFORT-SUM, and five gain factors are reported in Table 1.
Variables
TABLE 1.
Descriptive Statistics of C&IT Items, Overall C&IT Score, EFFORTSUM, GAINSUM, and Five Gain Factors
VARIABLES M SD
1. Used computer or word processor for paper 3.72 0.61 2. Used E-mail to communicate with class 3.41 0.92 3. Used computer tutorial to learn material 1.88 1.00 4. Joined in electronic class discussions 1.71 1.00 5. Searched Internet for course material 3.16 0.92 6. Retrieved off-campus library materials 1.80 1.01 7. Made visual displays with computer 2.36 1.06 8. Used a computer to analyze data 1.95 1.03 9. Developed Web page, multimedia presentation 1.63 0.94
C&IT Overall Score 21.61 5.19 EFFORTSUM (Sum of activities items
excluding C&IT scale) 235.68 39.92 GAINSUM (Sum of gain items excluding C&IT item 63.48 12.34 General education 14.16 3.83 Personal development 14.42 3.37 Science and technology 6.97 2.53 Vocational preparation 8.17 2.13 Intellectual development 22.80 4.71
Note. N = 18,344
ability are highly correlated and affect college outcomes (Pascarella & Terenzini, 1991), two control variables were created, student SES and academic preparation. SES was represented by level of parents’ education and the amount parents contributed to college costs. This estimate of SES is a far from robust measure of SES, but it is the best approximation possible from the variables included on the CSEQ. Academic preparation is the sum of student self-reported grades and educational aspirations. In addition, institutional selectivity and control (public, private) were also controlled in all analyses with the selectivity measures taken
from Barron’s Profiles of American Colleges (1996). Student gender, race and ethnicity, major field, institutional type, and year in college were coded as dummy variables. The variables were coded as follows:
• Sex (0 = women, 1 = men);
• Age (0 = traditional-age students under age 24, 1 = students 24 and older); • Race or ethnicity was coded as a set of
dummy variables: Asian Americans, African Americans, Latinos, Whites, and Other Ethnicity (American Indians and
others), with Whites as the omitted reference group;
• SES (the sum of parent education where 1 = neither parent a college graduate, 2 = one parent a college graduate, and 3 = both parents college graduates and amount parents contribute to college costs where 1 = none to 6 = all or nearly all); • Academic preparation (the sum of grades
where 5 = A and 1 = C, C– or lower; and educational aspirations where 2 = Expects to pursue an advanced degree after college and 1 = Does not expect to pursue an advanced degree);
• Major field (humanities, mathematics and sciences, social sciences, preprofessional, and students in two or more major fields, with preprofessional omitted as reference group);
• Institutional type (RU, DU, CCU, SLA, GLA with RU omitted as reference group);
• Institutional control (0 = public, 1 = private);
• Institutional selectivity (6 = most com-petitive, 5 = highly comcom-petitive, 4 = very competitive, 3 = competitive, 2 = less competitive, and 1 = not competitive); • Year in college (first year, sophomore,
junior, and senior, with first year omitted as reference group);
• Number of term credit hours (1 = 6 or fewer, 2 = 7 to 11, 3 = 12 to 14, 4 = 15 to 16, and 5 = 17 or more);
• Hours per week devoted to studying and preparing for class (1 = 5 or fewer, 2 = 6 to10, 3 = 11 to15, 4 = 16 to 20, 5 = 21 to 25, 6 = 26 to 30, and 7 = more than 30); • Hours per week working on campus or off
campus (1 = none, 2 = 1 to 10, 3 = 11 to 20, 4 = 21 to 30, 5 = 31 to 40, and 6 = more than 40);
• Overall C&IT score (the sum of individual C&IT item scores). These
items on the fourth edition of CSEQ are presented in Table 1.
• EFFORTSUM (sum of all Activities item scores excluding C&IT scale items); • Gain factor scores (sum of Gain item
scores excluding C&IT item contributing to the general education, intellectual skills, personal and social development, science and technology, and vocational preparation Gain factors) (Appendix B); • GAINSUM (sum of all Gain items
excluding C&IT item). Data Analysis
The data analysis followed a four-step process. First we calculated and examined the descriptive statistics (unadjusted means and standard deviations) for the sample and students’ responses to the CSEQ C&IT score and other scales and factors. Then we used multiple regression to determine how student charac-teristics, institutional characcharac-teristics, and other student experiences during college were related to students’ overall use of C&IT. Next, we conducted a series of multiple regressions to examine the influence of overall use of C&IT and the use of various forms of C&IT on the total amount of effort students devoted to other college activities excluding C&IT effort (EFFORTSUM). Finally, a series of two-step multiple regressions was used to examine the total (gross) and direct (net) effects of C&IT on student overall gains (GAINSUM) excluding the C&IT gain item score and the scores from the five Gain factors (general education, personal development, science and technology, vocational preparation, and intellectual development). Therefore, in this latter set of regressions EFFORTSUM was treated as a mediating variable between C&IT and GAINSUM and gain factors (Pascarella & Terenzini, 1991; Wolfle, 1980).
The psychometric properties for C&IT scale are acceptable, with a reliability alpha of .784, item intercorrelations ranging from .102 to .642, and item-total score correlations ranging from .406 to .735. The moderate magnitudes of the
item intercorrelations reduced the threat of multicollinearity and permitted the use of the individual C&IT items in the multiple regres-sion analyses.
RESULTS
The three most frequent C&IT activities were using a computer for word processing, using E-mail to communicate with an instructor or classmates, and searching the Internet for course material (Table 1). The three least frequent C&IT activities were developing a Web page or multimedia presentation, participating in class discussions via an electronic medium, and using a computer tutorial.
Table 2 presents the unadjusted means of overall and various forms of C&IT use. Men used C&IT overall slightly more frequently than women and also preferred more advanced forms of C&IT. Women opted more often for word processing and E-mail, with men more fre-quently using visual displays, data analysis, and multimedia presentation options. Older students used C&IT less frequently than younger (traditional-age) students (unadjusted means were 20.49 and 21.71 respectively), though the differences were due primarily to the more common forms of C&IT such as word processing and E-mail. Students majoring in mathematics and the sciences used most forms of C&IT more frequently than did their counterparts in the humanities, social sciences, and those who were majoring in two or more areas. Seniors were more likely to use the less common forms of C&IT, such as to visually display information, analyze data, and develop multimedia pre-sentations.
Table 3 shows the regression results of the analysis of student characteristics, institutional characteristics, and student overall use of C&IT. Generally consistent with the results from the analyses on unadjusted means in Table 2, men used C&IT overall slightly more frequently than women. Older students used C&IT less fre-quently than younger (traditional-age) students. Students from higher SES backgrounds were more likely to use C&IT more frequently. However, compared with Whites, students from
other race and ethnicity had no significant differences in the overall use of C&IT. Student academic preparation was not related to overall C&IT use. C&IT use also differed by major field and institutional type and in predictable ways. Compared with students in preprofessional fields, students majoring in mathematics and the sciences used C&IT more frequently, and students in the humanities, social sciences, and those who were majoring in two or more areas used C&IT less frequently. Indeed, humanities majors used C&IT the least. Students at research universities had higher overall C&IT scores compared with their counterparts attending other types of institutions. Sector also made a difference as students at private schools had higher overall C&IT scores than students at state-assisted institutions.
Compared to the first-year students, seniors used C&IT more frequently, although students in other years of college had no significant difference in the overall use of C&IT. Also, students who took more credit hours, studied more (class preparation), and worked on campus were more likely to use C&IT more frequently. At the same time working off campus was not negatively related to C&IT use.
To better understand the relationship between C&IT and the college student experi-ence we first examined the effects of C&IT on EFFORTSUM (the sum of all CSEQ activity items excluding C&IT items) and then on gains from college. To accurately answer whether C&IT influenced gains, we determined both the net effects and the gross effects of C&IT on gains. Table 4 shows the results of the effects of C&IT overall scores and individual item scores on EFFORTSUM, and the overall effects and direct effects of C&IT overall scores and individual item scores on GAINSUM (the sum of all gains items) and the five Gain factor scores. In this analysis, the C&IT overall score and individual item scores can have both net and indirect effects on GAINSUM and the Gain factors. EFFORTSUM is, therefore, a mediating variable for GAINSUM and the Gain factor scores where C&IT affects EFFORTSUM, which in turn affects the different Gain variables (Pascarella & Terenzini, 1991; Wolfle, 1980).
TABLE 2.
Unadjusted Means for C&IT Item Scores and Overall C&IT Score
Item C&IT VARIABLES 1 2 3 4 5 6 7 8 9 Score Sex Men 3.68 3.28 1.93 1.76 3.14 1.88 2.48 2.17 1.81 22.13 Women 3.75 3.48 1.84 1.69 3.17 1.75 2.28 1.82 1.52 21.32 Age Nontraditional 3.54 2.70 1.85 1.59 2.99 1.91 2.34 1.94 1.62 20.49 Traditional 3.74 3.47 1.88 1.72 3.17 1.79 2.36 1.95 1.63 21.71 RACE OR ETHNICITY American Indians and Other 3.70 3.32 1.94 1.80 3.09 1.94 2.36 2.02 1.66 21.84 Asians or Pacific Islanders 3.66 3.43 1.96 1.90 3.12 1.86 2.49 2.09 1.83 22.34 African Americans 3.73 3.23 1.86 1.77 3.12 1.90 2.30 1.90 1.60 21.41 Latinos 3.66 3.32 1.76 1.66 3.09 1.85 2.25 1.90 1.61 21.11 Whites 3.73 3.43 1.87 1.69 3.17 1.78 2.35 1.94 1.61 21.56 MAJOR FIELD Humanities 3.67 3.25 1.67 1.62 2.95 1.77 1.94 1.49 1.47 19.85 Math and Sciences 3.70 3.46 2.03 1.72 3.16 1.84 2.68 2.31 1.79 22.70 Social Sciences 3.76 3.36 1.76 1.66 3.11 1.81 2.12 1.87 1.44 20.90 Two or more majors 3.72 3.39 1.90 1.73 3.19 1.80 2.36 1.94 1.65 21.68 Preprofessional 3.76 3.48 1.84 1.73 3.17 1.78 2.35 1.90 1.60 21.61 INSTITUTIONAL TYPE DU 3.71 3.32 1.91 1.89 3.15 1.84 2.30 1.94 1.61 21.67 CCU 3.69 3.24 1.78 1.57 3.13 1.82 2.30 1.90 1.57 21.01 SLA 3.85 3.71 1.86 1.54 3.20 1.85 2.20 1.90 1.55 21.86 GLA 3.75 3.40 1.81 1.71 3.14 1.91 2.49 2.07 1.58 21.69 RU 3.74 3.60 2.01 1.87 3.19 1.72 2.43 2.00 1.73 22.29 INSTITUTIONAL CONTROL Private 3.79 3.56 1.85 1.71 3.20 1.91 2.42 2.02 1.63 22.03 Public 3.69 3.36 1.89 1.71 3.14 1.75 2.33 1.92 1.63 21.42 YEAR IN COLLEGE Sophomore 3.68 3.42 1.86 1.64 3.11 1.75 2.33 1.91 1.60 21.30 Junior 3.65 3.27 1.82 1.66 3.10 1.76 2.38 2.00 1.65 21.29 Senior 3.74 3.22 1.81 1.77 3.19 1.92 2.58 2.16 1.81 22.20 First 3.76 3.53 1.92 1.74 3.18 1.79 2.27 1.87 1.56 21.61
As is evident from Table 4, a large part of the influence on gains of using C&IT was mediated by EFFORTSUM.
The C&IT overall score positively influ-enced the amount of effort that students expended on other educationally purposeful activities (EFFORTSUM). This reinforced that student use of C&IT may indirectly influence student gains mediated by EFFORTSUM. As demonstrated in Table 4, the C&IT overall score had significant and positive gross effects on all gain outcome measures (GAINSUM) and each of the five Gain factors. However, although the C&IT overall score had net positive effects on gains in science and technology, vocational preparation, and intellectual development, it had a net negative effect on general education. Apparently, EFFORTSUM mediated a large portion of the gross effects of C&IT overall score on gain measures.
Table 4 also includes the results from the analyses of the individual C&IT item effects on EFFORTSUM, GAINSUM and five gain factors. All individual C&IT activities had statistically significant and positive effects on EFFORT-SUM, with one exception: developing a web page or multimedia presentation. Again, this suggests that individual C&IT activities may affect student gains directly and indirectly by influencing EFFORTSUM. However, the results indicate that different C&IT activities had different effects on outcomes represented by the overall gains measure (GAINSUM) and the five gains factors.
Using a computer or word processor to prepare papers had a positive net effect on intellectual development but a negative net effect on science and technology. However, this type of C&IT activity had positive gross effects on GAINSUM, general education, personal devel-opment, and intellectual development. Using E-mail to communicate with an instructor or other students had a positive net effect on personal development but a negative net effect on general education as well as gross positive effects on personal development and intellectual devel-opment. Using a computer tutorial to learn materials had positive net effects on science and technology and vocational preparation as well
TABLE 3.
Standardized Coefficients of Student Characteristics and Other Predictors on
C&IT Overall Score
VARIABLES Beta
Men .074*
(Women)
Nontraditional –.057*
(Traditional)
American Indians and Other .018 Asians or Pacific Islanders .012 African Americans .005 Latinos –.018 (Whites) SES .059* Academic preparation .000 Humanities –.110*
Math and sciences .032*
Social sciences –.053*
Two or more majors –.035*
(Preprofessional) DU –.034* CCU –.132* SLA –.060* GLA –.110* (RU) Private .105* (Public) Selectivity –.026 Sophomore .002 Junior .010 Senior .086* (First-year student)
Number of term credit hours .067*
Hours on academic work .123*
On-campus work .053*
Off-campus work .000 Adjusted R2 .076*
Note. The omitted reference group in parenthesis. * p < .001.
TABLE 4.
Standardized Coefficients of Overall C&IT Score and Individual Items on EFFORTSUM, GAINSUM, and Gain Factors
Gen Per Sci & Voc Intel
EFFORTSUM GAINSUM Ed a Dev Tech Prep Dev
Model 1: C&IT Overall Score
C&IT Overall Score .470* .013 –.062* .006 .049* .029* .095* (.470)* (.291)* (.186)* (.219)* (.233)* (.177)* (.314)* Full Model Adjusted R2 .316* .385* .301* .233* .276* .194* .303*
Model 2: Individual C&IT Items
1. Used computer or word
processor for paper .057* .012 .023 .011 –.039* –.002 .039* (.057)* (.046)* (.053)* (.037)* (–.017) (.017) (.066)* 2. Used E-mail to
communicate with class .042* –.007 –.042* .043* –.024 .004 .017 (.042)* (.018) (–.019) (.062)* (–.007) (.018) (.037)* 3. Used computer tutorial
to learn material .088* .011 –.005 .019 .036* .022* –.002 (.088)* (.064)* (.041)* (.059)* (.070)* (.051)* (.040)* 4. Joined in electronic
class discussions .089* .001 .014 .008 –.007 –.010 –.003 (.089)* (.054)* (.060)* (.048)* (.028)* (.019) (.038)* 5. Searched Internet for
course material .118* .020 –.004 .028* –.002 .027* .049* (.118)* (.090)* (.058)* (.082)* (.044)* (.065)* (.105)* 6. Retrieved off-campus
library materials .157* –.026* .014 –.041* .010 –.036* –.039* (.157)* (.067)* (.096)* (.030)* (.051)* (.014) (.034)* 7. Made visual displays
with computer .085* .007 –.051* –.003 .048* .021 .042*
(.085)* (.058)* (–.006) (.036)* (.081)* (.048)* (.081)* 8. Used a computer
to analyze data .108* .015 –.046* –.007 .069* .013 .046* (.108)* (.079)* (.011) (.042)* (.111)* (.048)* (.096)* 9. Developed Web page,
multimedia presentation .015 –.014 .012 –.037* –.025* –.002 .005 (.015) (–.005) (.020) (–.030)* (–.019) (.003) (.012) Full Model Adjusted R2 .325* .386* .305* .238* .284* .196* .309*
Notes.Full regression models included controls for all the independent variables in Table 3. The upper line indicates the net effects and the lower line (in parentheses) indicates the gross effects of the C&IT Overall Score and individual items on outcome variables.
a Gen Ed = General Education; Per Dev = Personal Development; Sci & Tech = Science and Technology; Voc
Prep = Vocational Preparation; Intel Dev = Intellectual Development * p < .001.
as a positive gross effect on GAINSUM and all five gain factors. Participating in electronic class discussions had no significant net effects on any gain measures, but it did have a positive gross effect on GAINSUM and four of the five gain factors, with the exception of vocational preparation. Searching the Internet for informa-tion had positive net effects on personal development, vocational preparation, and intellectual development, and a positive gross effect on GAINSUM and all five gain factors. Retrieving off-campus library materials had negative net effects on GAINSUM, personal development, vocational preparation, and intellectual development; however, it had a positive gross effect on GAINSUM and all gain factors except for vocational preparation. Making visual displays with a computer had positive net effects on science and technology and intellectual development but a negative net effect on general education as well as a positive gross effect on GAINSUM and all gain factors except general education. Using a computer to analyze data had positive net effects on science and technology and intellectual development but a negative net effect on general education and positive gross effects on GAINSUM and all gain factors but general education. Finally, develop-ing a Web page and multimedia presentation had negative effects on personal development and science and technology; the gross effect on general education was also negative.
Limitations
This study is limited in several ways. First, the measures we employed were limited to those available on the CSEQ. For example, the C&IT items from the CSEQ are not an exhaustive list of the computing and information technology activities that students can use that might affect their learning in positive or negative ways. For example instructor-designed use of hypermedia and hypertext are not specifically mentioned nor are activities that represent noneducational uses of C&IT such as surfing the Web or playing games. Thus, these data do not shed light on such potential debilitating behaviors associated with C&IT such as Internet addiction or cocooning (Kandell, 1998). Also, the SES
measure used in this study is not as precise as one would desire and may not accurately reflect the underlying construct. Such data might provide a different view of the nature of the relationships between C&IT, student effort, and self-reported gains.
Second, this study was based on a con-venience sample of institutions participating in the CSEQ research program from a recent 2-year period. If data from other institutions were available or a longer period was covered, perhaps the results would differ. Another limitation is that the findings related to institutional type may be skewed due to practices at certain institutions included in the study. For example, some of the schools may require all matriculating students to purchase laptops. At the same time many schools that are not represented in the study have state-of-the-art networks, hardware, and software that provide unusually rich opportunities for their students to become familiar with and use information technology; the experiences of these students is not captured in the results. Fourth, we employed measures representing two levels of analysis— students and institutions. An analytical tech-nique such as hierarchical linear modeling (Bryk & Raudenbush, 1992) is recommended in such situations. Finally, as with other studies that use self-report data the findings may be affected by response set (Pascarella & Terenzini, 1991) and the halo effect (Pike, 1999) that create dif-ficulties in determining whether the outcomes are really being influenced by the use of C&IT or whether other variables are intervening. DISCUSSION
Computer and information technology repre-sents a substantial investment of university resources that fortunately seem to be generally beneficial. The findings from this study show that using C&IT is related in complex, statis-tically significant ways to the amount of effort students devote to educationally purposeful college activities. Although both positive and negative net effects on student gains were found, the gross effects of using C&IT, either in aggregate or individual activities, generally were
positive, though most of the effects on student gains were mediated through the effort students expended on other activities. The exception to this conclusion is the item regarding developing Web pages and multimedia presentations, which had a negative gross effect on some gain factors (e.g., personal development and science and technology). That said, the results of the study both confirm the popular view that C&IT use is positively related to college student learning and personal development and also raises some questions about the efficacy of certain C&IT activities.
First, the use of computing and information technology for word processing and E-mail is practically universal. Understandably, more time-consuming activities that require a higher level advanced knowledge about how to use C&IT were far less common, such as developing a Web page or multimedia presentation. We were somewhat surprised and disappointed that participating in class discussions using an electronic medium was one of the less frequently used forms of C&IT. Students in the humanities and social sciences were least likely to parti-cipate in electronic class discussions, whereas seniors and students at private colleges were the most likely. Encouraging students to engage more actively in learning through C&IT is primarily the responsibility of the instructor. Thus, institutions and instructors can promote active learning via technology if they con-sistently use good educational practices such as clarifying expectations, preparing course assignments that require active student engage-ment, teaching students how to appropriately use the technology, and giving students prompt and accurate feedback about their contributions (Chickering & Gamson, 1987).
The differences between institutional types and C&IT use favored students at research universities and private colleges and univer-sities. Their more frequent use of C&IT may be a function of institutional affluence, where such institutions have had more funds available to invest in technology, making it more available and accessible to most students. Some of these schools may even require all matriculating students to have a personal computer. At the
same time, institutional selectivity was not related to overall C&IT use.
Consistent with other studies (Flowers et al., 2000; Gladieux & Swail, 1999), students from higher socioeconomic backgrounds ap-peared to use C&IT more frequently. However, C&IT appears to create a level playing field for learning for students from different racial and ethnic backgrounds who have access to it, given that the use of various forms of C&IT did not differ to any great extent by race and ethnicity. Seniors used C&IT the most, including the more advanced forms (visual displays, data analysis, multimedia presentations). This is heartening in the sense that even though most students (83%) used the Internet for research or home-work in their senior year of high school prior to matriculating (Higher Education Research Institute, 1999), most students do not simply continue doing the same things with technology that they did in high school but also expand their use of C&IT.
General education was the only gains cluster that potentially suffered as a result of overall C&IT use; E-mail, making visual displays, and analyzing data with a computer were the most deleterious influences on general education gains, even after controlling for major field. It is not immediately obvious why developing a Web page would have a net negative effect on personal development or obtaining materials from an off-campus library would have net negative effects on GAINSUM, personal development, vocational preparation, and intellectual development. Perhaps the nature of the Web page being developed is unrelated to academic work, such as a page used to present a photo album of a recent social event, as contrasted with a Web page on which one posts an electronic portfolio or research paper. Implications
The findings suggest five near-term implications for policy, practice, and research. First, as Kuh and Vesper (2001) suggested, all students in all fields should be encouraged to become proficient with computers and the other forms of informa-tion technology available on their campus. C&IT seems to work for all students, though some
students, such as women, appear to be less likely to use more advanced forms of technology. Perhaps the manner in which the technology is presented, described, or formatted has some-thing to do with this. Some experiments with different approaches to getting women to engage in using the technology for course-based assignments may be in order, where appropriate. A student’s socioeconomic background also seems to matter, consistent with the findings from other studies (Dillard & Gabbard, 1998; Flowers et al., 2000), with those from higher SES categories being more likely to use most forms of C&IT. But some other unalterable background characteristics such as race and ethnicity do not seem to affect C&IT use or the advantages that C&IT can provide in terms of learning and personal development outcomes. Thus, the so-called “digital divide” in 4-year colleges and universities does not seem to be a problem if institutions make C&IT accessible (as in the case of most of the private colleges and research universities in this study).
At the same time, students in some fields do not seem to do much with C&IT other than use E-mail. For example, C&IT is underused by students majoring in the humanities and social sciences compared with their counterparts in preprofessional fields and in mathematics and sciences. Perhaps C&IT is introduced more reluctantly or at slower rates in such fields as history and literature where the immediate utility of technology is not realized. Electronic discussions of the material introduced in many of humanities and social science fields would seem to be an appropriate pedagogical strategy. Second, because C&IT appears to be positively related to learning and personal development in a variety of areas for most students, public and institutional policies must ensure that such resources are accessible by all students at every college and university. Systematic efforts are needed to monitor the extent to which various groups of students are using various forms of C&IT and the effects of institutional policies and practices on student access to and use of C&IT. This is an area to which student development professionals can contribute by collaborating with institutional
researchers to collect the needed information. Third, the costs and benefits of C&IT must be estimated and interpreted in the context of student learning data and other institutional priorities (Morrison, 1999; Upcraft et al., 1999). Productivity has always been difficult to measure in colleges and universities (McKeachie, 1982). Information technology typically demands nontrivial amounts of new money or realloca-tions of campus resources to establish, update, and upgrade software, hardware, and networks. Reallocating institutional resources for this purpose means that other potentially productive and useful activities cannot be supported. In the long run, such investments (including the replacement of out-of-date technology) may actually cost more than other uses of the funds when productivity measures are examined (Massy & Wilger, 1998). The prevailing view, however, is that technology will generate economies of scale following what can often be a relatively large front-end investment. That is, following the initial investment, the per-student cost is expected to be relatively low (Massy & Wilger). At the same time, although “most successful technology applications have im-proved net productivity (benefits, adjusted for quality changes, divided by cost), gross produc-tivity (number of output units divided by cost) generally declines (Massy & Wilger, p.51). Thus, the impact of the institutional C&IT investment on learning must be evaluated periodically.
Fourth, studies are needed for the effica-cious ways to promote the effective use of C&IT by different groups, especially faculty members and student development staff with instructional assignments. These challenges include helping faculty learn how to integrate technology into their instruction and providing adequate instruction for how to use the technology by students and other users (Green, 1999). Pro-viding new forms of hardware will not neces-sarily change the way people teach or cause them to modify course materials to take maximum advantage of the technology. In part, this is attributed to the inertia of the system where the widespread integration of information tech-nology into core institutional processes ought
not be expected in the short term (Gilbert, 1996). Without significant structural change, the innovations that technology promises to bring to teaching and learning are not likely to occur (Green & Gilbert, 1995).
Finally, additional research is needed to determine the extent to which C&IT is being used for noneducational purposes that are largely distractions and incompatible with the educa-tional purposes of postsecondary institutions. This is very important as the positive effects of C&IT are largely mediated through the amount of effort students spend in other educational purposeful activities. Perhaps the amount of time most students spend in noneducational use of C&IT is minimal and does not detract sub-stantially from engaging in more important activities. However, without this information institutions cannot act intelligently because they are at a loss to know first whether such a problem is significant and for whom. Student affairs staff can play an important role by systematically observing how much time students spend using computers for various purposes (e.g., surfing the web, playing games, producing academic work).
Conclusion
The results of this study show that students who benefit most from using C&IT are those who use it more frequently and in more advanced ways. Although women and students from lower SES backgrounds use C&IT less frequently and benefit less from its use, the effect sizes associated with these differences are trivial. Moreover, effort devoted to using C&IT appears to have generally positive gross effects on the development of most important outcomes of college. Equally important, using C&IT is associated with greater levels of educational effort with the effects of C&IT on gains being largely mediated through the other educational efforts students put forth. Thus, C&IT use appears to have a general salutary influence on the overall learning environment.
Correspondence concerning this article should be addressed to George D. Kuh, Center for Post-secondary Research and Planning, College of Education, Indiana University, Bloomington, IN 47405; [email protected]
Ahern, T. C., Peck, K., & Laycock, M. (1992). The effects of teacher discourse in computer-mediated discussion. Journal
of Educational Computing Research, 8, 291-309.
Alavi, M. (1994, June). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, 159-174.
Barron’s Profiles of American Colleges (1996). Hauppage,
NY: Barron’s Educational Series.
Biglan, A. (1973a). The characteristics of subject matter in different academic areas. Journal of Applied Psychology,
57, 195-203.
Biglan, A. (1973b). Relationships between subject matter characteristics and the structure and output of university departments. Journal of Applied Psychology, 57, 203-213. Bonk, C. J., Medury, P. V., Reynolds, T. H. (1994). Cooperative hypermedia: The marriage of collaborative writing and mediated environments. Computers in the
Schools, 10 (1/2), 79-124.
Bradburn, N. M., & Sudman, S. (1988). Polls and surveys:
Understanding what they tell us. San Francisco:
Jossey-Bass.
Brandt, R. M. (1958). The accuracy of self estimates. Genetic
Psychology Monographs, 58, 55-99.
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear
models: Applications and data analysis methods.
Newbury Park, CA: Sage.
Carnegie Foundation for the Advancement of Teaching (1994). A classification of institutions of higher
educa-tion. Princeton, NJ: Author.
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE
Bulletin, 39(7), 3-7.
Converse, J. M., & Presser, S. (1989). Survey questions:
Handcrafting the standardized questionnaire. Newbury
Park, CA: Sage.
Deden, A., & Carter, V. K. (1996). Using technology to enhance students skills. In E. DeNisi, A. S., & Shaw, J. B. (1977). Investigation of the uses of self-reports of abilities.
Journal of Applied Psychology, 62, 641-644.
Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative research literature on learner comprehension, control, and style. Review of Educational Research, 68, 322-349. Dolence, M. G., & Norris, D. M. (1995). Transforming higher
education: A vision for learning in the 21st century. Ann
Arbor, MI: Society for College and University Planning. Ehrmann, S. C. (1995). Asking the right questions: What does
research tell us about technology and higher learning?
Change, 27(2), 20-27.
Ewell, P. T., & Jones, D. P. (1996). Indicators of “good
practice” in undergraduate education: A handbook for development and implementation. Boulder, CO: National
Center for Higher Education Management Systems. Fetterman, D. M. (1996). Videoconferencing on-line:
Enhancing communication over the Internet. Educational
Researcher, 25(4), 23-27.
Flowers, L., Pascarella, E. T., & Pierson, C. T. (2000). Information technology use and cognitive outcomes in the
first year of college. Journal of Higher Education, 71, 637-667.
Gilbert, S. W. (1996). Making the most of a slow revolution.
Change, 28(2), 10-23.
Gladieux, L. E., & Swail, W. S. (1999). The virtual university
and educational opportunity: Issues of equity and access for the next generation. Washington, DC: College Board.
Green, D. C., & Gilbert, S. W. (1995). Great expectations: Content, communications, productivity, and the role of information technology in higher education. Change, 27(2), 8-18.
Green, K. C. (1996). The coming ubiquity of information technology. Change, 28(2), 24-28.
Green, K. C. (1999). The 1999 National Survey of Informa-tion Technology in Higher EducaInforma-tion. Campus Computing [On-line]. Available: http://www.campuscomputing.net Hansford, B. C., & Hattie, J. A. (1982). The relationship
between self and achievement/performance measures.
Review of Educational Research, 52, 123-142.
Harasim, L., Hiltz, S. R., Teles, L., & Turoff, M. (1995).
Learning networks: A field guide into teaching and learning online. Cambridge, MA: MIT Press.
Hibbs, J. W. (1999). Will universities be relics? What happens when an irresistible force meets an immovable object? On
the Horizon, 7(1), 10-11.
Higher Education Research Institute. (1999). Freshmen
embrace the Internet as an educational tool. Los Angeles:
UCLA Graduate School of Education & Information Studies.
Institute for Higher Education Policy. (1999). Distance
learning in higher education. Washington, DC: Institute
for Higher Education Policy.
Kandell, J. J. (1998). Internet addiction on campus: The vulnerability of college students. CyberPsychology &
Behavior, 1(1) [On-line]. Available: http://www.inform.
umd.edu/CC/Personal/~kandell/iacpbart.htm
Kuh, G. D., Connolly, M. R., & Vesper, N. (1998, April). A
comparison of educational gains of college seniors between the mid-1980s and the mid-1990s. Paper
presented at the annual meeting of the American Educa-tional Research Association, San Diego, CA.
Kuh, G. D., & Vesper, N. (2001). Do computers enhance or detract from student learning? Research in Higher
Education, 42, 87-102.
Kuh, G. D., Vesper, N., Connolly, M. R., & Pace, C. R. (1997).
College Student Experiences Questionnaire: Revised norms for the 3rd edition. Bloomington: Indiana University
Center for Postsecondary Research and Planning. Kulik, C., & Kulik, J. (1991). Effectiveness of computer-based
instruction: An updated analysis. Computers and Human
Behavior, 7, 75-94.
Laing, J., Sawyer, R., & Noble, J. (1988). Accuracy of self-reported activities and accomplishments of college-bound seniors. Journal of College Student Development, 29, 362-368.
Lowman, R. L., & Williams, R. E. (1987). Validity of self-ratings of abilities and competencies. Journal of Vocational
Behavior, 31, 1-13.
Massy, W. R., & Wilger, A. K. (1998). Technology’s
contribution to higher education productivity. (New
Directions for Higher Education No. 103, pp. 49-59). McKeachie, W. J. (1982) Enhancing productivity in
post-secondary education. Journal of Higher Education, 53, 460-464.
Merisotis, J. (1999). The “What’s the difference?” debate.
Academe, 85(5), 47-51.
Morrison, J. L. (1999). The role of technology in education today and tomorrow: An interview with Kenneth Green, Part II. On the Horizon, 7(1), 2-5.
Oblinger, D., & Maruyama, M. (1996). Distributed learning. CAUSE Professional Paper Series, No. 14. Boulder, CO: CAUSE.
Owston, R. D. (1997). The World Wide Web: A technology to enhance teaching and learning? Educational Researcher,
26(2), 27-33.
Pace, C. R. (1985). The credibility of student self-reports. Los Angeles: University of California, The Center for the Study of Evaluation, Graduate School of Education. Pace, C. R. (1990). College Student Experiences
Ques-tionnaire (3rd ed.) Los Angeles: University of California,
The Center for the Study of Evaluation, Graduate School of Education. (The CSEQ is now distributed by the Center for Postsecondary Research and Planning, Indiana University).
Pace, C. R., & Kuh, G. D. (1998). College Student
Experiences Questionnaire (4th ed.). Bloomington:
Indiana University Center for Postsecondary Research and Planning.
Pascarella, E. T., & Terenzini, P. T. (1991). How college
affects students: Findings and insights from 20 years of research. San Francisco: Jossey-Bass.
Pike, G. R. (1995). The relationships between self reports of college experiences and achievement test scores. Research
in Higher Education, 36, 1-22.
Pike, G. R. (1999). The constant error of the halo in educational outcomes research. Research in Higher
Education, 40, 61-86.
Riel, M. (1993). Global education through learning circles. In L Harasim (Ed.), Global networks: Computers and
international communication (pp. 221-236). Cambridge,
MA: MIT Press.
Reisberg, L. (2000, June 5). 10% of students may spend too much time online, study suggests. The Chronicle of Higher Education [On-line]. Available: http://chronicle.com/free/ 2000/06/2000060501t.htm
Sproull, L., & Kiesler, S. (1993). Computers, networks, and work. In L Harasim (Ed.), Global networks: Computers
and international communication (pp. 105-119).
Cam-bridge, MA: MIT Press.
Upcraft, M. L., Terenzini, P. T., & Kruger, K. (1999). Looking beyond the horizon: Trends shaping student affairs— Technology. In C. Johnson and H. Cheatham (Eds.), Higher
education trends for the next century: A research agenda for student success (pp. 30-35). Washington, DC:
American College Personnel Association.
West, T. W. (1996). Make way for the information age: Reconstruct the pillars of higher education. On the Horizon,
4(4), 1, 4-5.
Wolfle, L. M. (1980). Strategies of path analysis. American