Annual Report on Degree Program
Assessment of Student Learning
University Assessment Committee
Office of Academic Assessment
Purpose: The purpose of the Annual Report on Degree Program Assessment of Student
Learning is to provide information about progress in assessment efforts for each degree program
within your academic unit. Only one report is requested of each academic unit, as this report will accommodate multiple degree plans. (You can still submit separate reports if you prefer.) The report will be made available publicly at the Office of Academic Assessment website and will be available to appropriate accrediting agencies. It is recommended that your unit use your
assessment report and results to celebrate achievements of student learning as well as to identify
potential areas for future curriculum improvement. The University Assessment Committee will
review your report to provide constructive feedback, as well as to identify particular academic units for potential assessment awards and/or mini-grants to support continuing assessment efforts.
Please email this completed form as an attachment to [email protected]. CONTACT INFO:
Academic Unit: Geography, Planning & Recreation Date: November, 2008
Name: Pam Foti, Tom Paradis Title: Chair (Pam Foti)
Email: [email protected], [email protected]
Phone: 523-6196 NAU Box: 15016
Degree Program(s) reported here: BS Geography, BS Geographic Information Science
ASSESSMENT REPORT:
Instructions: Please answer the following five questions to the best of your ability for
each degree program offered within your unit. You may use the table provided on the
next page, or you may create your own report format.
1. Summarize your assessment activities during the past year for each degree program. (e.g. faculty discussions, new survey design, data collection, revised assessment plans or learning outcomes, etc.).
• Assigned, collected, and analyzed nine reflective essays from GGR 480C in May, 2008. A total of nine senior students completed the three-week capstone course.
• Assigned, collected, and scored with rubrics the following GGR 480C projects for both course-level and program-level assessment:
o Written analytical essay: Glen Canyon Dam and positions for and against its
removal.
o Formal oral presentation: Learning from Los Angeles: Successes and
Challenges with Redevelopment and Urban Design.
• Assembled a summer subcommittee with four faculty members and instructors: Jim Sell, Ray Huang, Kathy Thatcher, Tom Paradis. Subcommittee met two times to review
sample projects from GGR 480C portfolios. Each student portfolio included all course work for GGR 480C and two student-chosen projects from other courses. Subcommittee members attempted to complete a comprehensive rubric that included all seven
program-level outcomes (see attached to report below). Several conclusions about student success and challenges were made with the input from the entire subcommittee. • Assigned a student research poster project, requiring students in GGR 480C to produce,
print, and present their team-based poster to the class following the final project. A scoring rubric will need to be designed for this project next year. In this case, the summer subcommittee reviewed the poster submissions, and three of the four
subcommittee members watched the students present in front of the class. The poster became a part of their final course portfolios, reviewed by the subcommittee.
2. Describe specific assessment findings related to the learning outcomes assessed for each degree program, including any pertinent context surrounding the findings. Please include the learning outcomes themselves. (e.g. 77% of
seniors performed at the “proficient” level of competency in problem solving, which is where we aimed to be this year using a new scoring rubric…)
a. Please attach any tables, graphics, or charts to the end of this report. Please see table below, followed by an Executive Summary of findings.
3. Describe how assessment feedback has been provided to students, faculty, and staff. (e.g. report for faculty, executive summary for the dean, web page for students,
alumni newsletter, discussion with students in class or club event, etc.)
• Poster presentation of 06-07 assessment findings (based on report from Fall 2007) displayed at the spring, 2008 NAU Assessment Fair. Poster then displayed
prominently in the Geography office area for several months.
• Assessment findings and recommendations were discussed and used as input for a year-long curricular revision process, beginning with a presentation of assessment findings at the fall, 2007 annual departmental retreat.
• Geography summer subcommittee discussed May, 2008 assessment findings from the capstone geography course (see findings below) and concurred on dominant conclusions and future recommendations.
• Presented this latest report with the faculty at a regular faculty meeting in November, 2008. Conclusions and recommendations discussed and considered for next year. 4. In what ways have you used assessment findings to celebrate student
achievements and/or to improve the curriculum this past year? (e.g. prizes to
students, hosting student parties, changes to curriculum, student projects, learning goals, assessment strategies, etc.)
Using the fall, 2007 program assessment report as one of several inputs, the entire geography faculty met once or twice a month throughout the 2007-2008 academic year to produce a major overhaul of the geography curriculum. This curriculum “redesign” consisted generally of an analysis of the entire core curriculum and the sequence of courses; several curricular changes within key courses to address various weaknesses; strategic additions and deletions of core courses themselves in an effort to produce a more efficient, comprehensive, and flexible
geography program. The year-long process began with the August, 2007 Geography Retreat and continued at the next faculty meeting with the distribution of numerous internal and external reports and related information related to student learning. Two primary assessment documents were distributed and discussed during early fall, 2007: 1) the annual Geography assessment report that incoporated two years’ worth of assessment data, and 2) a national NSF-funded geography report sponsored by the Association of American Geographers that included national
data from geography alumni and geography employers across the country. This latter report included lists of the most important academic skills and geography knowledge deemed necessary for successful college curricula and ensuing employment. The geography faculty considered the report’s conclusions and matched them with our own learning outcomes and assessment system. The national report confirmed our current focus on the real-world skills of writing, oral
communication, teamwork, and research methods.
The following bulleted list includes the dominant changes to the geography curriculum undertaken during 2007-2008:
• Assuring a balanced curriculum: roughly equal emphasis on dominant geography sub-disciplines, including cartography/GIS, human geography, physical geography, and regional geography.
• Providing enhanced flexibility for the Extended Geography major. The resulting core curriculum of 47 credits became the standard for both the BS Geography (with Minor) and the Extended BS. Those enrolled in the Extended major can complete a variety of active-learning experiences including travel abroad, internships, and/or
independent research.
• Improving efficiency and updating of Cartography/GIS course content through a revision of GGR 230, 330, and 331, consisting of the primary technical geography sequence.
• Improved “scaffolding” of the curriculum from sophomore through senior-year courses, where freshmen/sophomore level of course content serves as a stepping stone for more advanced learning during junior and senior years.
• Adoption of a four-year sequence of courses reflecting the revised core curriculum, useful as an advising tool. Students are encouraged to take courses in a logical sequence to help prevent past practices of students enrolling in courses largely out of sequence.
• Revisions to the Capstone geography course, GGR 480C:
o Moved course from traditional May intersession slot to spring semester. Will be taught during spring, 2009 for the first time. A full spring course will allow students more time to develop key skill sets prior to graduation, including oral and written communications, cartographic (mapping) production, and
competencies with research methods, analyses, and synthesis. o Scheduling the dominant student presentations in conjunction with two
departmental or university functions: 1) the annual department student
conference in Dubois, and 2) the NAU Undergraduate symposium. Both forums will allow students to present their research to a wider audience in both
PowerPoint and poster formats, further enhancing the stakes for development of oral presentation and research skills.
o An enhanced focus on in-depth undergraduate research focused on a topic of student’s choice. Rather than a series of separate field-based projects during the course, students will more thoroughly develop a particular research project including literature review, development of research questions, methods, and approaches, and reporting of research analyses and findings. Students will participate in a class “practice” trip to a nearby locale to practice their field research methods.
5. Describe any changes to your assessment plans, or any challenges or educational experiences with the assessment process this past year that you would like to share.
In large part based on the recommendations of the UAC, the following improvements have been attempted during 07-08 to the geography assessment process:
• A faculty subcommittee was formed to review GGR 480C student portfolios during summer, 2008. This was a first attempt to provide faculty members outside of GGR 480C to inspect and evaluate some student work from the capstone course, in addition to the traditional input from the course instructors. The subcommittee met twice to review a sample of student portfolios that included projects from the capstone course and other student-chosen projects. Though deemed a successful first attempt, time went by quickly at each meeting. The subcommittee gleaned various conclusions, but not a thorough analysis of student work. This process will need be made more efficient, especially should the number of students increase. • Developed a comprehensive rubric for use by Geography Subcommittee members
(see attachment below) to provide general scores of student work for all seven program learning outcomes. This process was partially successful though provided only a generalized overview. Still, the input from multiple (4) faculty and instructors provided enhanced validity of assessment conclusions.
• Created a new rubric to assess critical thinking of a specific student field project, applied to a class trip to Page and the Glen Canyon Dam. Rubric includes criteria that apply to program outcomes, including geographical analysis, synthesis, and writing.
• Rescheduled the geography capstone course from May intersession to spring semester, beginning in spring, 2009. This will allow more thorough assessment of cartographic products that were not possible during the 3-week intersession. It will further allow for student-peer assessment of oral presentations during the annual geography department conference at Dubois. Students from other classes and disciplines fill out simple rubrics to assess every student presenter. This will likely be built into the revised geography assessment plan.
• Developed a rubric to score writing assignments in the capstone prerequisite, GGR 380W. Instructors of 380W and 480C collaborated on a more standardized rubric for use in both courses, to provide consistent assessments from one course to the next, and to allow for program-level assessment outside of the field course.
a. Please submit any revised/updated assessment plans to the Office of Academic Assessment along with this report.
ALTERNATIVE TABLE FORMAT:
Degree-Program Learning Outcomes
Assessment Method and Analysis
Assessment Findings: How well are your students achieving the learning outcomes assessed?
Geographic
Description, Analysis, and Synthesis:
Students will be able to differentiate between geographical description, analysis, and synthesis and demonstrate their own cognitive skills by
• Writing and Oral Rubric Scores (2 projects) • Reflective Essays • Faculty Observations of
sample portfolios
• Synthesis: 9 out of 9 students met standards for proficient or advanced. • Analysis: 4 of 9 students scored at Novice
level.
• Essays: Students overall satisfied, felt they made progress throughout course.
• Portfolio samples: Team ranked samples from Novice to Advanced, with wide disparity.
effectively describing, analyzing, and synthesizing various geographical data and information.
Written Geographic Presentation: Students
will be able to effectively present physical and human geography content knowledge, description, analyses and synthesis in written form while
demonstrating
satisfactory writing skills including grammar, paragraphing, organization, and referencing.
• Writing Rubric Scores • Reflective Essays • Faculty Observations of
sample portfolios
• Mechanics: 4 of 9 students scored at Novice level. 5 of 9 were Proficient or Advanced. • Essays: Few comments on writing, though
essays themselves demonstrated that 9 or 9 students were Proficient or Advanced with writing.
• Portfolio samples: Group rated students from Unsatisfactory to Proficient, a wide range.
Oral Geographic Presentation: Students
will be able to effectively present geographic analysis and synthesis through the use of oral presentations. Students should demonstrate their ability to clearly
communicate
geographical information in a teaching situation to lay audiences and demonstrate skills with regard to time
management, organization, and speaking clarity.
• Oral Rubric Scores • Reflective Essays
• Rubrics: Delivery and Language: 9 of 9 students were Proficient.
• Intro, Organization, Knowledge, Evidence: 7 of 9 students rated Advanced, 1 Proficient, 1 Novice in one category each.
• Audience interaction and awareness: 9 of 9 students rated Proficient or Advanced.
• Documentation of Evidence: 6 of 9 students rated as Novice.
• Essays: Students overall pleased with progress in oral communication skills. Frequently mentioned in essays, thankful to have the chances to improve presentation abilities. Several students commented about continued nervousness in front of a group.
Cartographic Design and Production:
Students will be able to design and produce quality reference and thematic maps for appropriate audiences and purposes within at least one computer-generated production
• Reflective Essays • Faculty Observations of
Sample Portfolios
• Essays: Students generally desiring more cartographic experience, though realize limitations with 3-week course.
• Portfolio samples: All samples rated
Unsatisfactory. In part due to lack of attention to this outcome within the course.
method available, including software for computerized graphics and/or GIS software systems.
Conceptual Knowledge of Physical and Human Geography: Students will be able to demonstrate their knowledge and understanding of how physical processes shape the earth’s surface and interact with plant and animal life to create, sustain, and modify ecosystems. Students will likewise be able to demonstrate their knowledge and
understanding of how political, cultural, social, and economic activities collectively shape human landscapes and physical environments.
• Oral Rubric Scores • Reflective Essays • Faculty Observations of
Sample Portfolios
• Oral Rubrics: 9 of 9 students rated Proficient or Advanced.
• Essays: Students generally pleased with ability to incorporate knowledge into products. No key comments one way or another.
• Portfolio samples: Students rated on a range from Novice to Advanced. No real consensus. Very subjective.
Research Methodology and Design: Students
will be able to effectively apply the scientific method to conduct and interpret geographical inquiries utilizing a variety of primary and secondary research methods both qualitative and quantitative.
Students should be able to conduct research in both individual and teamwork situations through the development of specific research questions and problems.
• Writing and Oral Rubric Scores
• Reflective Essays • Faculty Observations of
sample portfolios
• Rubrics: Not yet available in rubric form. • Essays: Students provided numerous
comments with respect to qualitative and quantitative research: Very pleased with opportunities to improve qualitative research methods and move through the scientific research process. Very displeased with lack of opportunities to conduct quantitative research within the course. Focused more on human rather than physical geography.
• Portfolio samples: Similar findings: Excellent student work in conducting, analyzing, and reporting qualitative methods and findings. Lack of experiences to practice and assess quantitative research. Final student posters demonstrated thorough knowledge of the research process. Teamwork and Collaboration: Students will be able to demonstrate effective, professional teamwork and collaborative skills when working in groups toward a common goal.
• Written and oral rubric scores
• Reflective Essays
• Rubrics: 9 of 9 students rated Proficient or Advanced.
• Essays: Students commented often about teamwork, generally pleased with their own abilities to work within a team. Occasional comments about challenging team members and how they approached the situations.
EXECUTIVE SUMMARY OF FINDINGS:
Student Successes:
• Assessment indicators consistently reveal student success in the areas of oral
communication, teamwork, and qualitative research design and
implementation. All of these outcomes are emphasized within the capstone
course.
Areas for Improvement:
• Assessments revealed mixed results and high disparity among student
outcomes with respect to effective writing, knowledge of physical and
human geography, and geographic analysis and synthesis. Poster design
was acceptable though could be better improved and guided by the instructors.
Areas to add experiences and assessments:
• Assessment findings (or lack of them) reveal a need to include greater
opportunities for quantitative research experiences, such as in the area of
physical geography. Cartographic techniques should also be further
emphasized as a class project (or embedded within other projects) and
assessed accordingly.
Degree Program Assessment
2008 Outcomes from Program Assessment Plan (for scoring of Portfolios by Geography Subcommittee)
CRITERIA (Check one box at right for each criterion based on your sense of overall student
accomplishment.)
Unsatisfactory Novice Proficient Advanced
1) OUTCOME 1: Geographic Description, Analysis, and Synthesis: Students will be
able to differentiate between geographical description, analysis, and synthesis and demonstrate their own cognitive skills by effectively describing, analyzing, and synthesizing various geographical data and information.
2) OUTCOME 2: Written Geographic Presentation:
Students will be able to effectively present physical and human geography content knowledge, description, analyses and synthesis in written form while demonstrating satisfactory writing skills including grammar,
paragraphing, organization, and referencing.
3) OUTCOME 3: Oral
Geographic Presentation:
Students will be able to effectively present geographic analysis and synthesis through the use of oral presentations. Students should demonstrate their ability to clearly
communicate geographical information in a teaching situation to lay audiences and demonstrate skills with regard to time management, organization, and speaking clarity.
4) OUTCOME 4: Cartographic Design and Production:
Students will be able to design and produce quality reference and thematic maps for appropriate audiences and purposes within at least one computer-generated production method available, including software for computerized graphics and/or GIS software systems.
5) OUTCOME 5: Conceptual
Knowledge of Physical and Human Geography:
Students will be able to demonstrate their knowledge and understanding of how physical processes shape the earth’s surface and interact with plant and animal life to create, sustain, and modify ecosystems. Students will likewise be able to demonstrate their knowledge and
understanding of how political, cultural, social, and economic activities
collectively shape human landscapes and physical environments.
6) OUTCOME 6: Research Methodology and Design:
Students will be able to effectively apply the scientific method to conduct and interpret geographical inquiries utilizing a variety of primary and secondary research methods both qualitative and quantitative. Students should be able to conduct research in both individual and teamwork situations through the development of specific research questions and
problems.
7) OUTCOME 7: Teamwork and Collaboration:
Students will be able to demonstrate effective, professional teamwork and collaborative skills when working in groups toward a common goal.
WRITING MECHANICS
CRITERIA Unsatisfactory (D/F)
Novice (B-/C) Proficient (A-/B) Advanced (A/A+)
Mechanics Many or numerous
repeated errors in grammar, diction, sentence structure, and/or spelling that detract from writing effectiveness. Some errors in grammar, diction, sentence structure, and/or spelling that may or may not detract from writing effectiveness.
Few errors which do not detract significantly from writing effectiveness; sentence structure and word choice show variety. Few or no errors which do not detract at all from writing effectiveness; sentences and word choices varied and creative.
HIGHLIGHTS OF OUTCOMES THAT APPEAR TO BE SUCCESSFULLY LEARNED
