The school or program defines appropriate evaluation methods and measures that allow the school or program to determine its effectiveness in advancing its mission and goals. The evaluation plan is ongoing, systematic and well-documented. The chosen evaluation methods and measures must track the school or program’s progress in 1) advancing the field of public health (addressing instruction, scholarship and service) and 2) promoting student success.
B5.1. Present an evaluation plan that, at a minimum, lists the school or program’s evaluation measures, methods and parties responsible for review. See Template B5-1. (self-study document)
Below are the program’s evaluation methods and measures that we use to assess our effectiveness with our Mission and our Goals.
Mission: The MPH program promotes health equity through innovative approaches in teaching, culturally responsive community engagement, and robust scholarship embracing the Jesuit values of social justice, service to others, and critical self-reflection, emphasizing populations at-risk, vulnerable or marginalized.
Table B5-1. Evaluation of Goal Statements
Evaluation Measures Data Collection Method for Measure Review Process Goal Statement: Teaching
Prepare public health professionals committed to health equity grounded in competencies (knowledge, skills, beliefs, attitudes) in the foundations of public health to serve at-risk, marginalized and vulnerable communities through practice in health care ethics or health care management.
Measure:
Didactic instruction, learning resources and assignments in core courses reflect the program’s focus on health equity in at-risk,
marginalized and vulnerable populations.
• Core course syllabi reviewed annually to assess course objectives as they relate to health equity and priority populations.
• Content, reading assignments, other learning materials and assignments (discussion boards, written assignments, quizzes) reviewed annually to also assure relevance to the program’s commitment to health equity in our priority populations. (see ERF B5.3h Course Review Example)
The MPH Curriculum and Evaluation Committee is the lead committee for these reviews. While there are no quotas, per se, for the number of course objectives and course
activities/resources, the review looks for consistent evidence of our focus on issues of health equity in our priority communities, above and beyond merely mapping the courses to CEPH learning objectives and foundational
competencies. Any recommendations for change are reported to the Graduate Program Committee. Final decisions are made by the Program Director
Measure:
Students’ perceived mastery of core and concentration competencies.
• Online survey that all students take midpoint through the curriculum and then during their final course before graduating. (ERF B5.3b MPH Midpoint Survey; ERF B5.3c MPH Exit Survey).
• Survey results compiled and submitted to the MPH Graduate Program Committee for review and action, if warranted.
The MPH Curriculum & Evaluation Committee is the lead committee for review of survey results. Measures to improve the curriculum are forwarded to the Graduate Program Committee.
• IPE Modules and Passport Activities (ERF B5.3g CIPER Report)
• Practicum Supervisor Site Evaluation (ERF B5.3a Practicum Supervisor Site Evaluation)
• Capstone Project Final Paper
IPE Modules and Passport Activities:
The annual outcomes report is reviewed by the Curriculum and Evaluation Committee.
Practicum Supervisor Site Evaluation: A report of the summary data is prepared by the Practicum Coordinator and presented annually to the MPH Curriculum and Evaluation Committee.
This report includes documentation of student mastery of at least 3 foundational competencies and 2 concentration competencies per CEPH criteria.
Evaluation Measures Data Collection Method for Measure Review Process
• Alumni Survey (ERF B5.3e. MPH Alumni Survey) at one-year post-graduation.
• Alumni activity and updates available through LinkedIn.
• Results of these surveys are compiled and reported to the Graduate Program Committee.
The MPH Curriculum & Evaluation Committee is the lead committee for review of survey results. Measures to improve the curriculum are forwarded to the Graduate Program Committee.
Goal Statement: Scholarship
Foster public health scholarship to promote health equity within national and global communities that are at-risk, marginalized and vulnerable, while advancing public health knowledge and practice.
Measure:
Number of peer reviewed articles with a focus on vulnerable populations to advance health equity.
• Annual report using data from the university’s Faculty Database on research and scholarship activity amongst program faculty. (ERF B5.3d MPH Program Review Documentation)
• Faculty CV data.
• Annual survey of part-time and adjunct faculty regarding scholarship.
The Program Director assures compliance with annual reporting. The Scholarship and Service Committee is the lead committee for assessing the extent of faculty scholarship by part time and adjunct faculty.
• Annual report using data from the university’s Faculty Database on research and scholarship activity amongst program faculty. (ERF B5.3d MPH Program Review Documentation)
• Faculty CV data.
• Annual survey of part time and adjunct faculty regarding scholarship.
The Program Director assures compliance with annual reporting. The Scholarship and Service Committee is the lead committee for assessing the extent of faculty scholarship by part time and adjunct faculty.
• Annual report using data from the university’s Faculty Database on research and scholarship activity amongst program faculty. (ERF B5.3d MPH Program Review Documentation)
• Faculty CV data.
• Annual survey of part time and adjunct faculty regarding scholarship.
The Program Director assures compliance with annual reporting. The Scholarship and Service Committee is the lead committee for assessing the extent of faculty scholarship by part time and adjunct faculty.
Measure:
Increase by 50%/year over 2020 baseline the number of faculty-student
collaborations in research and scholarship.
• Annual report using data from the university’s Faculty Database on research and scholarship activity amongst program faculty. (ERF B5.3d MPH Program Review Documentation)
• Faculty CV data.
• Annual survey of part time and adjunct faculty regarding scholarship.
The Program Director assures compliance with annual reporting. The Scholarship and Service Committee is the lead committee for assessing the extent of faculty scholarship by part time and adjunct faculty.
Goal Statement: Service
Promote culturally responsive community engagement through outreach, education, and partnerships.
Measure:
• Annual report using data from the university’s Faculty Database on primary instructional faculty extramural service. (ERF B5.3d MPH Program Review Documentation)
• Faculty Annual Evaluation
• Annual survey of part time and adjunct faculty regarding service activities
The Program Director assures compliance with annual reporting. The Scholarship and Service Committee is the lead committee for assessing the extent of faculty scholarship by part time and adjunct faculty.
Measure:
Number of faculty-student service collaborations with vulnerable populations.
• Data collected from faculty (ERF A1.5a MPH Graduate Program Committee Meeting Minutes)
• Annual survey of the EAB to identify new opportunities for community engagement.
The Program Director, MPH Practicum Coordinator and Faculty report and review the results of the annual data collection. The EAB participates by reporting new community engagement
• A survey is conducted every semester for all actively enrolled students to assess their current engagement in community service.
• Results are assembled by program staff and reported to the Program Director
• The EAB is surveyed annually to identify new opportunities for student engagement.
• The Student Advisory Board reviews survey data collected from the first bulleted method above.
All faculty participate in this survey of the students enrolled in their courses.
Summary results are reported to and acted on by the Program Director.
Efforts to foster student community engagement occurs through faculty-student interactions through individual courses and through outreach efforts by program faculty, the Student Advisory Board, and the EAB.
Evaluation Measures Data Collection Method for Measure Review Process Measure:
Number of priority community-based service projects
• Quarterly Community Advisory Board Meetings include ongoing efforts within this Board to create new and document existing partnerships between the program and our geographically diverse communities. Minutes summarize these results and are reported to the Program Director.
• A Practicum Site Database, maintained by the Practicum Coordinator updates and reflects the scope of community engagement between the program and its community, based on student placement sites. Results are reported to the Program Director.
• Faculty are surveyed annually to document their specific community engagement activities.
Results are reported to the Program Director.
• The EAB is surveyed each year to identify professional development needs in our priority communities and to identify to opportunities for community engagement. (ERF B5.3f External Advisory Board minutes)
The Program Director is the lead for evaluation efforts regarding this measure. The MPH Practicum Coordinator, faculty, the EAB, and students participate in the data gathering process to identify the nature and extent of program-community partnerships. The Program Director assesses the results and works with each of the stakeholders to continue a process of community outreach and engagement.
The evaluation plan outlined in Table B5.1 is just one component of the overall assessment and evaluation activities for the program. In addition to the evaluation activities included in Table B5.1, the program participates in required evaluative processes such as the Academic Program Reviews and annual assessment reports.
CU’s Academic Program Review policy requires new academic programs to be reviewed within three years and every seven years thereafter. Program review includes completion of a self-study document which provides evidence and narrative components related to admission and enrollment trends, student outcomes, faculty outcomes, instructional productivity, and program strengths, and weaknesses or opportunities. A virtual program review is conducted by two external reviewer and one internal. The reviewers report is submitted to the program, which then prepares an action plan, which is reviewed by the graduate dean and approved by the Provost. Documents from the 2017-18 MPH program review can be found as ERF B5.3d.
To submit annual learning assessment data, all academic programs, including the MPH program, use Watermark/Task Stream, an online platform for collecting and reporting assessment data. Creighton’s Teaching and Learning Center collects annual reports from all academic programs and coordinates a systematic peer review of the results. See annual assessment reports, as well as reviewer comments, from the last three years in ERF D20.2b-e Assessment Reports. CU uses the Provost Data Management System, an internally developed online database, for faculty to manage, report, and track teaching, scholarship, service, and professional activities. The Data Management System includes a reporting feature for academic programs and administrators to review for continuous improvement and reporting to accreditation bodies.
The MPH Accreditation Committee is the direct internal review body monitoring the program’s success in meeting CEPH criteria, learning objectives, and competencies. This is an ongoing process during each academic year and includes responsibility for assuring that required data gathering methods are operational and the data are being assessed and acted on by the appropriate MPH committees.
B5.2. Briefly describe how the chosen evaluation methods and measures track the school or program’s progress in advancing the field of public health (including instruction, scholarship and service) and promoting student success. (self-study document)
The Teaching, Scholarship, and Service goals listed in this section reflect the vision, mission, and values of the program as outlined above in B.1. The chosen evaluation measures, methods, and review
processes allow us to track progress in advancing public health and promoting student success in the following ways.
GOAL 1 Teaching: Prepare public health professionals committed to health equity grounded in competencies (knowledge, skills, beliefs, attitudes) in the foundations of public health to serve at-risk,
marginalized and vulnerable communities through practice in health care ethics or health care management.
This goal advances the program’s role in the field of public health by ensuring that as students gain relevant knowledge and skills from mastery of CEPH learning objectives and competencies, they do so with a focus on health equity within our priority populations. The program measures this in the following ways:
• Measure 1 course review indicates whether or not core course objectives, learning materials and assessments reflect the program’s commitment to health equity in priority communities. This measure provides actionable data in support of ongoing program enhancement and
improvement.
• Measure 2 promotes student success through the collection of quantitative and qualitative data multiple times throughout the program on student perceptions of their own learning. It indicates that students graduate having mastered competencies that are foundational to the field of public health as well as those specific to their concentration area. This measure also permits program faculty to monitor student success and adjust the learning experiences of the students, if necessary, to improve their success.
• Measure 3 consists of the collection of experiential learning artifacts (i.e., ILE, APE, and interprofessional activities) as evidence of student mastery in real world settings. This not only advances the public health field by preparing qualified new members of the workforce, but it permits the program to promote student success through ongoing evaluation of capstone experiences with an intention to continue to improve student experiences in real world settings.
• Measure 4 promotes student success through the collection of quantitative and qualitative data on alumni perceptions of their ability to apply competencies on the job. Since the majority of our students are already employed in the field, this measure permits the program to assess the specific enhancement to their professional development, thus providing the program with reliable data regarding the extent to which the program is contributing to advancing the field of public health. Similarly, it provides actionable data upon which to continue to improve the program.
GOAL 2 Scholarship: Foster public health scholarship to promote health equity within national and global priority communities that are at-risk, marginalized and vulnerable, while advancing public health knowledge and practice.
This goal supports the program’s role in advancing the field of public health by prioritizing scholarship to promote health equity in our priority communities, and with intent to inform teaching and practice as well.
The program measures this in the following ways:
• Measure 1 tracks benchmarks regarding scholarly work on health disparities in priority
populations submitted for publication. This advances the field of public health by contributing new knowledge and skills to be disseminated to the broader public health community through peer reviewed publications. It also promotes student success through our commitment to teacher-scholarship and the diffusion of new knowledge into the program’s curriculum by faculty.
• Measure 2 tracks benchmarks related to faculty scholarly productivity that addresses health disparities in priority populations. This contributes to the public health field by generating new research activities leading to additional peer reviewed publications and/or presentations. It also offers opportunities to foster interprofessional collaborations. The measure promotes student success through our commitment to teacher-scholarship and the diffusion of new knowledge into the program’s curriculum by faculty.
• Measure 3 tracks benchmarks related to presentations and speaking engagements on health disparities in priority populations. Dissemination of knowledge and skills derived from faculty presentations advances the field and promotes student success.
• Measure 4 creates a structure to assess whether students are active participants in faculty scholarship through direct collaboration. The scholarship on health disparities in priority
populations will contribute to the field while students will gain valuable experience in scholarly collaboration contributing to improved student success.
GOAL 3 Service: Promote culturally responsive community engagement through outreach, education, and partnerships.
This goal advances the field of public health by engaging the program’s students and faculty with our priority communities, focusing on promoting health equity, and reflecting the needs of the public health workforce. Student success is promoted through individual and student-faculty engagement with our priority communities. The program measures this in the following ways:
• Measure 1 tracks benchmarks related to faculty involvement in extramural activities within our priority communities. The methods and review processes assure that we can assess the cultural responsiveness of all faculty community engagement. This will inform annual faculty evaluation, foster the integration of the community engagement in the curriculum, and permit the program to determine how to foster even greater engagement.
• Measures 2 and 3 tracks benchmarks to evaluate the extent of collaborative service work
between faculty and students, and individual community engagement activities by students within priority communities. The measures and review processes permit the program to evaluate the extent of such collaborations, identify areas for improvement, and in the process, both advance the field and promote student success.
• Measure 4 promotes the development and evaluation of new and existing culturally responsive community partnerships between the program and our priority communities. The collaborative nature of the review process assures that opportunities for engagement by students and faculty are identified, that the program is addressing the professional needs of public health practitioners in communities, and that the Service goal both advances the field and enhances student success.
B5.3. Provide evidence of implementation of the plan described in Template B5-1. Evidence may include reports or data summaries prepared for review, minutes of meetings at which results were discussed, etc. Evidence must document examination of progress and impact on both public health as a field and student success. (electronic resource file)
Documentation is provided in ERF B5.3i on implementation of the evaluation plan. A good example of how we “close the loop” using evaluation data is how the program addressed cultural diversity, inclusion, and cultural competence in the program in general and the curriculum in particular. In the ERF are the following relating to this one example:
• Minutes from a faculty retreat that led to adoption of revised guiding statements relating to advancing health equity within marginalized, at-risk and vulnerable populations
• Results of faculty and student surveys to determine the extent of cultural competence within the curriculum
• Results of faculty and student surveys on the climate of the program regarding diversity and inclusion
• The Curriculum and Evaluation Committed reviewed syllabi for alignment with CEPH learning objectives/competencies and the program’s guiding statements (see ERF A1.5a Program Level Communication-Curriculum and Evaluation Committee)
• Minutes of meetings with the Program Director during which decisions were made to hold regular faculty meetings to advance cultural diversity, inclusion and improve the climate within the program
See ERF B5.3 for the following documents that relate to additional examples of implementation of the evaluation plan:
• ERF B5.3a Practicum Supervisor Site Evaluation
• ERF B5.3b Midpoint Survey
• ERF B5.3c Exit Survey
• ERF B5.3d Program Review Documentation
• ERF B5.3e Alumni Survey
• ERF B5.3f External Advisory Board minutes
• ERF B5.3g CIPER IPE 400/500 Report
• ERF B5.3h Course Review Example
• ERF B5.3i Evaluation Plan Implementation
B5.4. If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area. (self-study document)
Strengths
• The program has clear goal statements in alignment with our guiding statements.
• Each goal has clearly articulated evaluation measures, specific data gathering systems, and a clear review process for decision making.
• Committee responsibilities are clearly articulated regarding each goal and corresponding measures.
• Each goal reflects collaboration for the benefit of the program by engaging all faculty, external advisors, and students.
• The data collected and the review processes employed permit the program to effectively assess how the program advances the field of public health and how we promote student success.
Weakness
• Prior to the 2019-2020 academic year, faculty-student collaborations in scholarship and service occurred but were limited in number.
• Prior to the 2019-2020 academic year, faculty-student collaborations in scholarship and service occurred but were limited in number.