Achieving the Dream has clearly raised awareness of the importance of using data about students in institutional decision-making. According to one president: “Our people are beginning to ask, ‘What does the evidence show?’ rather than, ‘What do you think?’ [It is] [t]he culture of evidence . . . rather than the anecdotal stories we’ve used for 30 years.” The vice president for education and student development at another college, who refers to Achieving the Dream simply as “the Dream,” said that: “The Dream project provided for us that avenue to talk about data and what it means to make decisions based on data. It used to be: ‘Who is in charge of that? Is that a campus initiative?’ Now people are saying: ‘What data do you have to support it?’” Similar views were expressed at most if not all of the other colleges. The “culture of evi- dence” concept clearly resonated with most of those interviewed at the participating colleges. Besides the focus on helping underserved students succeed, this seems to be the hallmark of Achieving the Dream, even for those not directly involved in the initiative.
At least two colleges indicated that the initiative has shown them the value of longitudi- nal tracking of students. The IR director at one college said that community colleges in Texas used to do cohort tracking of students through the state’s LONESTAR (LONgitudinal Evalua- tion, Student Tracking And Reporting) system, which was discontinued several years ago. Achieving the Dream has refocused the college on the importance of cohort tracking, according to the IR director. Other colleges pointed out the value of disaggregating their data so they could compare the outcomes of different student groups. According to IR staff members at one col- lege: “We had looked at our data for years, but had not disaggregated it. We were treating stu- dents as if they were all the same. So now, we began focusing on those at risk.”
Central to the Achieving the Dream model is the notion of “cognitive dissonance” — the idea that when faculty and staff see from hard data that their students are not doing as well as expected, they will be motivated to work to change how they teach and support students. This phenomenon was observed at several of the colleges. For example, a math instructor at one college said: “The math data were definitely good to see. Our Math 60 courses — [the] basic course — students don’t tend to do as well as we thought. . . . That’s one of the reasons we talked about rearranging courses. This will help.” A developmental English instructor at the same college said: “The data makes the campus community more aware of our [developmental education] students.”
At about half the colleges, faculty and staff expressed concern that data might be used to blame them for students’ lack of success. A fuller discussion of this problem is presented in the next section, but there is some evidence that participating in Achieving the Dream has helped colleges move beyond those fears. More and more faculty and staff are buying into the notion that Achieving the Dream is fundamentally about using data to improve student out- comes, not to punish faculty and staff.
47
Indeed, administrators at several colleges indicated that Achieving the Dream has helped them persuade faculty and staff of the value of measuring student outcomes. The associ- ate director of institutional effectiveness at one college credits the initiative with helping faculty ask the right questions so that they can better evaluate student outcomes. At another college, faculty said that they were now more aware of how many students are in developmental educa- tion, which has strengthened their resolve both to reduce the number of students placed into de- velopmental courses and to help developmental students advance more rapidly into college- level courses. Both developmental and college-level faculty at the college reported seeing more enthusiasm around data collection and analysis.
In several cases, Achieving the Dream has helped to elevate the role of institutional re- search. At one college, for example, IR staff members said that they are integrally involved in strategic planning for the first time. In at least one case, Achieving the Dream has led to in- creased sharing of student data with local high schools and universities.
Not surprisingly, Achieving the Dream has increased the demands on institutional re- search at nearly all the participating colleges. Despite the increased workload, IR staff at some colleges said they were pleased by the fact that their analyses are being used productively. The director of institutional planning and research at one college said: “We’re always under pres- sure, but it has made what we’re doing much more meaningful that data are being looked at and being looked at with a purpose, not just for the sake of reporting.”
Achieving the Dream colleges were permitted to use some of their grant money to in- crease their IR capacity; at least half of the 27 colleges have hired additional IR staff. Three oth- ers have replaced their IR directors, in at least one case hiring an individual with a broader skill set. A small college hired a research analyst to do more focus groups with students and to en- sure that the college’s strategies are properly evaluated. Participation in Achieving the Dream apparently encouraged one large college to expand the focus of its IR department to include as- sessment of learning.
Several of the colleges, including the one just mentioned, have purchased software to facilitate access and analysis of data on students, including software to better enable them to analyze longitudinal data from “data warehouses” and create “dashboards” of performance in- dicators for administrators and others.4
Customizing these systems to a college’s particular needs and training staff to use them effectively take time. One college had been trying to install
4
A “data warehouse” is a database that incorporates data from different functional areas and time periods. Several of the Achieving the Dream colleges have established data warehouses to facilitate analysis of longitu- dinal cohort data on students. “Dashboards” are software that provides timely information on an organization’s performance according to a limited set of indicators. They are typically used by senior administrators to gauge an institution’s progress on key indicators, such as enrollment, retention rates, and faculty diversity.
48
a front-end software system for over a year and did not expect to have a functional system in place until June 2007. Even after that point, the college expected continually to add to the basic functionalities of the system.