December 1, 2020
Dear Creighton MPH Program Stakeholders,
We appreciate your interest in reviewing and commenting on our self-study for initial accreditation by the Council of Education for Public Health (CEPH).
Your comments may take the form of e-mail, printed mail or any other written form and sent directly to the CEPH Accreditation Coordinator listed below:
Zeinab Bazzi, MPH
Accreditation Coordinator
Council on Education for Public Health 1010 Wayne Ave
Suite 220
Silver Spring, MD 20910 Email: [email protected]
Comments must be in writing and must be specific. Comments received by phone or in person will not
be accepted or considered. CEPH will accept stakeholder comments about the program’s practices and
procedures until February 8th, 2021.
Submitted to the Council on Education for Public Health
October 1, 2020
Contact Person:
Dhitinut (DT) Ratnapradipa, PhD
Director, Master of Public Health Program Creighton University Graduate School 2500 California Plaza
Omaha, NE 68178-0690
Master of Public Health Program
Preliminary Self-Study Report
Table of Contents
List of Commonly Used Acronyms ... 7
Introduction ... 9
Table Intro-1b. Creighton University Degree Offerings ... 9
Figure Intro-2a. Internal Organization of MPH Program ...13
Figure Intro-2b. Organization of Academic Unit ...14
Figure Intro-2c. Creighton University Organizational Chart ...15
Table Intro-4.1. Enrollment by Degree and Concentration ...16
A1. Organization and Administrative Processes ...17
Table A1.1. MPH Program Standing and Ad Hoc Committees ...17
Table A1.2b. Curriculum Approval Processes ...19
Table A1.4. MPH Primary Instructional Faculty Involvement on Committees ...22
A2. Multi-Partner Schools and Programs ...25
A3. Student Engagement ...26
A4. Multi-Partner Schools and Programs ...28
A5. Degree Offerings in Schools of Public Health ...28
B1. Guiding Statements ...29
Vision: ...29
Mission: ...29
Program Core Values: ...29
Program Goal Statements: ...30
Goal Statement 1: Teaching ...30
Goal Statement 2: Scholarship ...30
Goal Statement 3: Service ...30
B2. Graduation Rates...32
Table B2.1 MPH Student Persistence and Graduation Rates...32
B3. Post-Graduation Outcomes ...35
Table B3-1. Post-Graduate Outcomes ...35
B4. Alumni Perceptions of Curricular Effectiveness ...37
Table B4.1. Alumni Perceptions of Curricular Effectiveness ...37
B5. Defining Evaluation Practices ...40
Table B5-1. Evaluation of Goal Statements ...40
B6. Use of Evaluation Data ...46
C1. Fiscal Resources ...48
Table C1-1. Sources of Funds and Expenditures by Major Category, 2016 to 2020 ...50
C2. Faculty Resources ...53
Table C2-1. Faculty Resources by Concentration ...53
Table C2.2. Faculty/Advisee Ratios in Advising, Mentoring, and the Integrative Experience ...55
Table C2.5a. Course Evaluation Data on Student Perception of Class Size ...55
C3. Staff and Other Personnel Resources ...58
Table C3-1. Staff Support ...58
C4. Physical Resources ...61
C5. Information and Technology Resources ...63
D1. Foundational Public Health Knowledge ...66
Table D1.1 - Foundational Public Health Knowledge for the MPH ...66
D2. MPH Foundational Competencies ...68
Table D2.1 MPH Degree Requirements ...68
Table D2.2 Mapping of MPH Foundational Competencies ...70
D3. DrPH Foundational Competencies ...75
D4. MPH Concentration Competencies ...76
Table D4.1a. Assessment of Competencies for MPH in Health Care Ethics Concentration ...77
Table D4.1b. Assessment of Competencies for MPH in Healthcare Management Concentration ...78
D5. MPH Applied Practice Experiences ...80
Table D5-1. Practice-based products that demonstrate MPH competency achievement ....82
D6. DrPH Applied Practice Experience ...87
D7. MPH Integrative Learning Experience ...88
Table D7-1. MPH Integrative Learning Experience (MPH 612 Capstone) ...88
D8. DrPH Integrative Learning Experience ...91
D9. Public Health Bachelor’s Degree General Curriculum ...91
D10. Public Health Bachelor’s Degree Foundational Domains ...91
D11. Public Health Bachelor’s Degree Foundational Competencies ...91
D12. Public Health Bachelor’s Degree Cumulative and Experiential Activities ...91
D13. Public Health Bachelor’s Degree Cross-Cutting Concepts and Experiences ...91
D14. MPH Program Length ...92
D15. DrPH Program Length ...93
D16. Bachelor’s Degree Program Length ...93
D17. Academic Public Health Master’s Degrees ...93
D18. Public Health Academic Doctoral Degrees ...93
D19. All Remaining Degrees ...93
D20. Distance Education ...94
E1. Faculty Alignment with Degrees Offered ... 101
Table E1-1. Primary Instructional Faculty Alignment with Degree Offered... 101
Table E1-2. Non-Primary Faculty Regularly Involved in Instruction ... 103
E2. Integration of Faculty with Practice Experience ... 106
E3. Faculty Instructional Effectiveness ... 109
E4. Faculty Scholarship ... 114
Table E4-1. Outcome Measures for Faculty (PIF/Non-PIF) Research and Scholarly Activities ... 118
E5. Faculty Extramural Service ... 120
Table E5.1. Primary and Non-Primary Faculty Community Engagement Activities ... 123
Table E5. Service Indicators for Primary and Non-Primary Faculty, FY 2018-2021 ... 126
F1. Community Involvement in Program Evaluation & Assessment ... 128
Table F1.3d. (Part 1) Employer assessment of program graduates’ ability to perform competencies in an employment setting – Quantitative Responses ... 131
Table F1.3d. (Part 2) Employer assessment of program graduates’ ability to perform competencies in an employment setting – Qualitative Responses ... 131
F2. Student Involvement in Community and Professional Service ... 133
F3. Assessment of Community’s Professional Development Needs ... 136
F4. Delivery of Professional Development Opportunities for Workforce ... 139
G. Diversity and Cultural Competence ... 142
Table G1. Enrollment Trends Data – Student Diversity Profile ... 144
Table G1.6a Student Perceptions of Program Climate of Diversity and Cultural Competence ... 152
Table G1.6b Faculty/Staff Perceptions of Program Climate of Diversity and Cultural Competence ... 153
Table G1.6c Student Perceptions of Diversity and Cultural Competence within the Curriculum ... 153
H1. Academic Advising ... 156
Table H1.1. Academic Advising Services Roles ... 156
H2. Career Advising ... 162
Table H2.4b. Student Satisfaction with Career Advising ... 165
H3. Student Complaint Procedures ... 167
H4. Student Recruitment and Admissions ... 169
Table H4.1 Outcome Measures for Recruitment and Admissions ... 170
H5. Publication of Educational Offerings ... 172
Table H5.1. Publications of Educational Offerings ... 172
ERF – Document Inventory ... 173
List of Commonly Used Acronyms
APRC Academic Planning Review Committee, Creighton University C-RAC Council of Regional Accrediting Commissions
CBPR Community Based Participatory Research CHI Catholic Health Initiatives
CHNA Community Health Needs Assessment
CHPE Center for Health Policy and Ethics
CIPER Center for Interprofessional Practice, Education and Research, Creighton University
CRM Customer Relationship Management
CU Creighton University
DPAC Distance Program Administrator Committee, Creighton University
EAB External Advisory Board
EdD Doctor of Education in Interdisciplinary Leadership
FC Foundational Competency
GPC Graduate Program Committee
GSG Graduate Student Government
HCE Health Care Ethics Concentration
HCM Health Care Management Concentration
HLC Higher Learning Commission
IDS Department of Interdisciplinary Studies, Creighton Graduate School ILAC Institute for Latin American Concern, Creighton University
IPE Interprofessional Education
IWC Master of Science in Integrative Health and Wellness Program MSOL Master of Science in Organizational Leadership
NetID User Identification Number
NCR Negotiation and Conflict Resolution Program
PBC Purpose Built Community
RAML Reinert Alumni Library, Creighton University SRC Student Resource Center (Online)
TLC Teaching and Learning Center, Creighton University
Introduction
Intro1. Describe the institutional environment, which includes the following:
a. year institution was established and its type (e.g., private, public, land-grant, etc.) Creighton University (CU) is a Catholic and Jesuit private, not-for-profit institution. Creighton College officially opened on September 2, 1878. Creighton College was founded in 1878 by two prominent Omaha families, Edward and Mary Lucretia Creighton and Edward’s older brother John, and his wife Sarah Emily. In 1879, the bishop transferred the institution to the Jesuits and the College was incorporated under Nebraska law. It is not clear as to when the name was changed to The Creighton University, but documents provided to the team indicate that in 1968 the name was amended to change the name to simply Creighton University.
CU is a comprehensive university committed to excellence in its academic health programs,
undergraduate/graduate programs and in preparing students to serve, promote justice and meet the complex needs of their future employers. The master's in public health program (MPH) provides an excellent example of the outgrowth of this mission.
b. number of schools and colleges at the institution and the number of degrees offered by the institution at each level (bachelor’s, master’s, doctoral and professional preparation degrees)
The university has nine major academic divisions, including the Graduate School, College of Arts and Sciences, Heider College of Business, College of Professional Studies, the College of Nursing, and Schools of Dentistry, Medicine, Law, and Pharmacy and Health Professions.
Table Intro-1b. Creighton University Degree Offerings College of Arts and
Sciences
• Bachelor of Arts (B.A.) Majors: American Studies, Art History, Classical Languages, Classical and Near Eastern Civilizations, Communication Studies, Cultural Anthropology, Economics, English, French and Francophone Studies, German Studies, Graphic Design & Media, Healthy Lifestyle Management, History, International Relations, Journalism, Justice and Society, Medical Anthropology, Music, Philosophy, Political Science, Psychology, Spanish and Hispanic Studies, Studio Art, Sustainability, Sustainable Energy*, Theatre, Theology
• Bachelor of Fine Arts (B.F.A.) Majors: Studio Art, Musical Theatre, Theatre
• Bachelor of Science (B.S.) Majors: Applied Physical Analysis, Biology, Biomedical Physics, Chemistry, Computer Science and Informatics, Elementary Education, Exercise Science and Pre- Health Professions, Health Administration and Policy, Mathematics, Neuroscience, Physics, Sociology, Sustainable Energy Science
• Bachelor of Science in Chemistry (B.S.Chm.)
• Bachelor of Science in Environmental Science (B.S.Evs.)
• Bachelor of Science in Mathematics (B.S.Mth.)
• Bachelor of Science in Physics (B.S.Phy.)
• Bachelor of Social Work (B.S.W.) College of Nursing • Bachelor of Science in Nursing (B.S.N.)
• Doctor of Nursing Practice (D.N.P.) Majors: Clinical Systems Administrator, Nurse Practitioner College of
Professional Studies
• Associate in Arts (A.A.) Majors: Organizational Communication, Theology
• Associate in Science (A.S.) Majors: Computer Science, Mathematics
• Associate in Science in Emergency Medical Services (A.S.E.M.S.)
• Bachelor of Arts (B.A.) Majors: Communication Studies, English, Healthy Lifestyle Management
• Bachelor of Science (B.S.) Majors: Elementary Education, Leadership, Health Administration and Policy
Graduate School • Master of Accounting (M.A.C)
• Master of Arts (M.A.) Majors: Christian Spirituality, English, Ministry, Theology
• Master of Business Administration (M.B.A.)
• Master of Education (M.Ed.) Majors: Elementary School Teaching, Secondary School Teaching
• Master of Finance (M.FIN)
• Master of Healthcare Management (M.H.C.M)
• Master of Investment Management and Financial Analysis (M.I.M.F.A)
• Master of Public Health (M.P.H)
• Master Physician Assistance Studies
• Master of Science (M.S.) Majors: Analytics, Bioethics, Biomedical Sciences, Business Intelligence and Analytics, Educational Leadership, Emergency Medical Services, Government Organization &
Leadership, Integrative Health and Wellness, Medical Anthropology, Medical Microbiology and Immunology, Medical Physics, Medical Science, Negotiation and Conflict Resolution, Organizational Leadership, Neuroscience, Oral Biology, Pharmaceutical Sciences, Pharmacology and
Neuroscience, Physics, School Counseling and Preventive Mental Health
• Master of Science in Nursing (M.S.N.) Concentrations: Clinical Nurse Leader, Clinical Systems Administrator, Nursing Education, post graduate certificates in Nurse Practitioner and Clinical Nurse Leader, Master of Science in Occupational Therapy (M.S.O.T.)
• Master of Science in Rehabilitation (M.S.R.)
• Doctor of Business Administration (D.B.A)
• Doctor of Education (Ed.D.) Major: Interdisciplinary Leadership
• Doctor of Philosophy (Ph.D.) Majors: Biomedical Sciences, Clinical and Translational Science, Medical Microbiology and Immunology, Pharmacology
Heider College of Business
• Bachelor of Science in Business Administration (B.S.B.A.) Majors: Accounting, Business Intelligence and Analytics, Economics, Finance, Management, Marketing, International Business, Prelaw Business
School of Dentistry • Bachelor of Science in Dental Hygiene (B.S.D.H)
• Doctor of Dental Surgery (D.D.S.) School of Law • Juris Doctor (J.D.)
School of Medicine • Doctor of Medicine (M.D.) School of Pharmacy
and Health Professions
• Bachelor of Science in Emergency Medical Services (B.S.E.M.S.)
• Doctor of Occupational Therapy (O.T.D.)
• Doctor of Pharmacy (Pharm.D.)
• Doctor of Physical Therapy (D.P.T.)
c. number of university faculty, staff and students
The 2018-19 Creighton University Fact Book lists the total student enrollment at 8,910, which includes all students enrolled among all the university schools and colleges. (see ERF Intro1-c.Enrollment and Persistence) There were 912 faculty members, which includes both full-time and part-time positions. A total of 307 faculty members also hold primary or secondary appointments as graduate faculty. (see ERF Intro1-c. Faculty and Class Size) As of June 2018, CU employed a total of 1,268 full-time and part-time staff members. In AY 2019-20 total enrollment was 8,200, including 4,100 undergraduate and 4,100 graduate students, 56% of whom were female and 44% male.
d. brief statement of distinguishing university facts and characteristics
Creighton is the most comprehensive of the 27 American Jesuit colleges and universities in that it is composed of nine colleges and schools which offer credit-bearing certificates, associate, baccalaureate, master’s, and doctoral degrees in over 130 programs. In order to reach a broad spectrum of audiences, consistent with advancing the Jesuit presence both globally and to diverse populations, programs are offered in face-to-face, hybrid, online, and accelerated formats. Despite this diversity of curricular
programming and delivery method, Creighton University collectively exists for “students and learning,” as noted in the Mission Statement. (ERF Intro1-d)
Located in Omaha, Nebraska, a metropolitan area with a population of approximately 925,000. The 130- acre campus is situated within walking distance of downtown Omaha. CU offers a mission-driven education in the Jesuit tradition for individuals who want to make a meaningful contribution to the world.
Our more than 8,000 undergraduate, graduate, and professional students enroll at Creighton each year to find their place in the world through lives of leadership and service. Our low student-to-faculty ratio (11:1) provides personalized attention.
Creighton is a top-ranked university and was ranked as the #1 Regional University in the Midwest (2019) for the 16th year in a row by U.S. News & World Report. In September 2019 the University was elevated to the newly created Carnegie Classification of Doctoral/Professional Universities category, resulting in U.S. News moving Creighton to the National Universities category in recognition of having a full range of undergraduate majors, plus master’s and doctoral programs, as well as for producing groundbreaking research. Creighton made its inaugural appearance in the prestigious National Universities category of U.S. News & World Report’s 2020 list of “Best Colleges.”
Over the past 8 years, Creighton has experienced an overall increase in the number of students of color enrolled both in undergraduate and graduate programs. In 2009, 17.6% of the undergraduate and graduate enrollment were students of color and in 2019 it increased to 18%. A review of the
Undergraduate and Graduate Enrollment Profiles over the past 10 years substantiates that the University has a history of attracting a diverse student population who want to be challenged to make positive changes in a global community while being guided by principles of ethical learning, service to others, the search for truth, and a faith that does justice.
e. names of all accrediting bodies (other than CEPH) to which the institution responds. The list must include the regional accreditor for the university as well as all specialized accreditors to which any school, college or other organizational unit at the university responds (list may be placed in the electronic resource file)
Creighton University is accredited to award degrees at the associate, bachelor’s, master’s, and doctoral levels by The Higher Learning Commission (HLC). The university’s last comprehensive visit was held in April 2017. In August 2017, the Institutional Actions Council of the Higher Learning Commission
reaffirmed accreditation with the next Reaffirmation of Accreditation in 2026-27. A list of other accrediting bodies can be found on the university’s Academic Specialized Accreditation webpage or as ERF Intro1-e.
f. brief history and evolution of the school of public health (SPH) or public health program (PHP) and related organizational elements, if applicable (e.g., date founded, educational focus, other degrees offered, rationale for offering public health education in unit, etc.) 2007-2008: A task force on graduate education recommended the University develop and grow interdisciplinary programs and graduate distance education programming. The master’s in public health program was identified as a program to be developed because of its alignment with Creighton’s academic health programs and professional schools. With Creighton’s growth in distance programming, a decision was made to utilize the services of Deltak/Wiley Education Services (Deltak/Wiley) to provide student recruitment, assist with student application processes, provide enrollment and retention services, onboarding and orientation for new students, proactive student support, and 24/7 technical helpdesk services.
2010: Academic Vice President, Patrick Borchers JD, and Health Sciences Vice President, Donald Frey, M.D., established a Public Health Task Force to explore the establishment of an educational program in public health. Initial activities of the Task Force focused on developing a program of study, including core courses and potential areas of specialization. Public Health Task Force members were assigned to these areas of specialization and tasked with identifying potential courses, developing course descriptions, and identifying student competencies.
2011: The Public Health Task Force recommended the MPH program be located within the Center for Health Policy and Ethics (CHPE) due to the Center’s experience with the online Master of Science in Health Care Ethics. The program and planning process transferred to CHPE Director, Amy Haddad, PhD, RN. A steering committee was convened for planning meetings with hired external consultants. Dr.
Haddad led the effort to develop a new program proposal to be presented to the Graduate Board. In December the CU Graduate Board approved the proposed MPH program and a committee was formed to conduct a search and hire a program Director. Deltak/Wiley was contracted to provide services for the MPH program. See ERF Intro-1f MPH Steering Committee for a list of committee members, meeting agendas, minutes, and reports.
2012: Sherry Fontaine, PhD, was named the first program director, and the recruitment process began for three MPH faculty. Shortly after the launch of the program, the university determined the program
should be re-located both philosophically and physically in the Department of Preventive Medicine rather than the Center for Health Policy and Ethics.
2013: The MPH program admitted its first student cohort in the spring of 2013.
2015-2016: The university adopted a Provost model, which led to the retirement of the Health Sciences Vice President, Don Frey, MD and the hiring of Thomas Murray, PhD as Provost. In 2016, the Program Director resigned, and a search team was formed, resulting in the hiring of Tanya Benedict.
2016 – 2017: The MPH program and faculty were structurally moved to the Graduate School within the Department of Interdisciplinary Studies, led by Department Chair, Cindy Costanzo and Dean Gail Jensen.
Under the Graduate School and Department of Interdisciplinary Studies the MPH joined several interdisciplinary online graduate programs, including the Master of Science in Integrative Health and Wellness Program (IHW), Master of Science in Health Care Ethics (HCE), Master of Science in
Negotiation and Conflict Resolution (NCR), Master of Science in Organizational Leadership (MSOL), and Doctor of Education in Interdisciplinary Leadership (EdD). The program has benefited from
interdisciplinary collaborative partnerships and knowledge-sharing related to curriculum, faculty development, service, and the scholarship of teaching and learning. In 2017 the CEPH self-study application was submitted.
2018: CU developed an internal infrastructure for graduate student recruitment, enrollment management and retention services, onboarding and orientation and proactive student support. The contract was ended with Deltak/Wiley Education Services (Deltak/Wiley).
2019: Tanya Benedict resigned as program director and returned to a faculty role. Cindy Costanzo was named the Interim Program Director and a search for a new program director was begun. An initial draft of the self-study document was submitted to CEPH in May. Following consultations with CEPH, the program submitted a formal request for an extension to receive additional time to work on the self-study;
this request was granted by CEPH in September, with a new deadline of October 2020 established.
Extensive work on the self-study continued through the end of the year. In November, following a national search, Dhitinut Ratnapradipa was named Program Director.
2020: Dr. Dhitinut Ratnapradipa assumed the role of Program Director on June 1.
Intro2. Organizational charts that clearly depict the following related to the school or program:
a. the school or program’s internal organization, including the reporting lines to the dean/director
As part of the Department of Interdisciplinary Studies, faculty report to the Program Director who in turn reports directly to the Department Chair. Additional administrative and staff support to the program is provided by Department personnel who report to the Department Chair. (see Figure Intro-2a).
Figure Intro-2a. Internal Organization of MPH Program
Internal Organizational Structure – MPH Program
Dr. Gail Jensen Dean
Dr. Cindy Costanzo Chair, Department of Interdisciplinary Studies
Staff
Monica Chapeau, Administrative Assistant Kate Johansen, Instructional Designer Amy Kitt, Sr. Financial Analyst Terri Mahaffey, Program Coordinator Sarah Meisinger, Academic Coach
Dr. Dhitinut Ratnapradipa MPH Program Director
MPH Faculty Tanya Benedict
Helen Chapple LaShaune Johnson Sarah Lux Kate Nolt Mark Robinson Special (Adjunct) Faculty
Practicum/Field Experience Coordinator Kate Nolt
b. the relationship between the school or program and other academic units within the institution. For programs, ensure that the chart depicts all other academic offerings housed in the same organizational unit as the program. Organizational charts may include committee structure organization and reporting lines
The MPH program resides in the Graduate School as part of the Department of Interdisciplinary Studies (see Figure Intro-2.b).
Figure Intro-2b. Organization of Academic Unit
Vice Provost, Learning and Assessment
Dean, Graduate School and College of Professional Studies (GAPS)
Senior Associate Dean, GAPS Chair, Interdisciplinary Studies
EdD in Interdisciplinary
Leadership (Program Director)
Master of Organizational
Leadership (Program Director)
Master of Integrative Health & Wellness (Program Director)
Master of Bioethics (Program Director)
Master of Public Health (Program Director)
Master of Negotiation
& Conflict Resolution (Program Director) Department Committees:
-- Assessment Commitee -- Currciulum Committee
-- Rank & Tenure Committee Department Staff
Teaching & Learning Center
Instructional Designers
Assistant Dean, Graduate School
Academic Coaches
GAPS Offices and Departments
EMS Education
Executive Assistant Chaplain
c. the lines of authority from the school or program’s leader to the institution’s chief
executive officer (president, chancellor, etc.), including intermediate levels (e.g., reporting to the president through the provost)
The university is governed by a 32-member Board of Trustees. The deans for each of the nine schools and colleges listed earlier in this section (see Intro-1b) report directly to the provost, and the provost reports directly to the president.
Figure Intro-2c. Creighton University Organizational Chart
d. for multi-partner schools and programs (as defined in Criterion A2), organizational charts must depict all participating institutions
Not applicable.
Intro3. An instructional matrix presenting all the school or program’s degree programs and concentrations including bachelor’s, master’s and doctoral degrees, as appropriate.
CU offers a Master of Public Health (MPH) degree consisting of 44 credits hours. All students complete 29 hours of core courses and complete 15 hours in one of two areas of concentration (see Table Intro- 3.1). The program is offered in a completely online, asynchronous format with no required residency.
Courses are delivered in 8-week modules or terms with students typically taking one course per term.
Information about the program is publicly available through the university’s online Graduate Catalog.
Table Intro-3.1. Instructional Matrix – Degrees and Concentrations Instructional Matrix - Degrees and Concentrations
Categorized as public health
Campus based
Executive Distance based
Bachelor's Degrees
Master's Degrees Academic Professional
Master of Public Health – Health Care
Ethics Concentration MPH X
MPH Master of Public Health – Healthcare
Management Concentration MPH X MPH
Doctoral Degrees Academic Professional
Joint Degrees (Dual, Combined,
Concurrent, Accelerated Degrees) Academic Professional
2nd Degree Area
Public Health
Concentration
Intro4. Enrollment data for all the school or program’s degree programs, including bachelor’s,
master’s and doctoral degrees.
Program enrollment was 55 for the 2019-2020 academic year. Table Intro-4.1 includes the enrollment numbers by concentration. All students who are admitted into the MPH program must choose between the two concentrations.
Table Intro-4.1. Enrollment by Degree and Concentration Degrees Offered Current Enrollment (2019-20)
Master’s Degree 55
Healthcare Management 34
Health Care Ethics 21
A1. Organization and Administrative Processes
The program demonstrates effective administrative processes that are sufficient to affirm its ability to fulfill its mission and goals and to conform to the conditions for accreditation.
The program establishes appropriate decision-making structures for all significant functions and designates appropriate committees or individuals for decision making and implementation.
The program ensures that faculty (including full-time and part-time faculty) regularly interact with their colleagues and are engaged in ways that benefit the instructional program (e.g., participating in instructional workshops, engaging in program specific curriculum development and oversight).
A1.1. List the school or program’s standing and significant ad hoc committees. For each, indicate the formula for membership (e.g., two appointed faculty members from each concentration) and list the current members. (self-study document)
Programs should generally focus the response on the specific committees that govern the unit of accreditation, not on departmental or school committees that oversee larger
organizational units. (self-study document)
The program maintains the following standing and significant ad hoc faculty committees. A brief description of each committee as well as the membership is provided in Table A1.1. Please note, program staff (not identified below as formal standing members of the committees) support each committee as warranted.
Table A1.1. MPH Program Standing and Ad Hoc Committees
Committee Membership Formula Current Members
1. Accreditation Committee Oversees institutional and CEPH accreditation activities, including a) preparation of CEPH self-study document, b) collection of data and documents, c) collaboration with other MPH faculty and department staff in accreditation efforts, and d) generation of annual reports.
• Program Director - MPH (Chair)
• 1-2 Health Care Ethics Concentration Faculty Members
• 1-2 Health Care Management Concentration Faculty Members
• Department Chair
• Assistant Dean, Graduate School
• Program Coordinator (non-voting member)
Dhitinut Ratnapradipa Sarah Lux
Helen Chapple Kate Nolt Cindy Costanzo LuAnn Schwery Terri Mahaffey
2. Admissions Committee Reviews admissions applications for the MPH program three times annually and makes
recommendations for acceptance into the program to the Academic Progression Committee.
• Program Director (Chair)
• One faculty member from each concentration
Dhitinut Ratnapradipa Tanya Benedict Mark Robinson
3. Academic Progression Committee
Meets monthly to discuss enrollment status of newly admitted students, plan interaction with prospective students, conduct a teaching and learning check in with adjunct faculty, and discuss overarching student progression issues. Submits recommended changes to
admissions and student progression policies to the Graduate Program Committee for approval.
• Program Director (Chair)
• All Full-time Faculty
• Academic Coach
• Graduate Enrollment Specialist
• All adjunct faculty teaching in the MPH program in the current term
Dhitinut Ratnapradipa Tanya Benedict Kate Nolt
LaShaune Johnson Helen Chapple Mark Robinson Sarah Lux Sarah Meisinger Denise Work Nick Baldetti Leah Casanave Tim Guetterman
Committee Membership Formula Current Members 4. External Advisory Board
The MPH External Advisory Board provides ongoing advice, council, and assistance to the leadership of the MPH program on all matters related to operations and curriculum.
• Public health professionals reflecting the geographic diversity of the program
• Program Director (Chair)
• 1 MPH Program Alum
Nick Baldetti, DBA, McPherson, KS Michael Baysinger, MPH, West Chester, PH Jaime Bland, DNP, Omaha, NE
Melissa Bottrell, PhD, MPH, Berkeley, CA Amy Krueger, MPH, Ph.D.(c), Atlanta, GA Jeff Kuhr, PhD, Grand Junction, CO Angela Lopez, MPH, Indianapolis, IN Carolina Padilla, Omaha, NE
Anthony Robins, PhD, Moon Township, PA Sarah Schram, MS, RD, Papillion, NE John Stone, PhD, MD, Los Angeles, CA Dhitinut Ratnapradipa, PhD
Dina Becirovic, MPH, Omaha, NE (Alumna) 5. MPH Graduate Program
Committee
As the primary governing committee at the program level, this group oversees the continuous development, maintenance, and evaluation of the MPH program.
• MPH Program Director (Chair)
• All other full-time faculty teaching in the MPH program
Dhitinut Ratnapradipa Tanya Benedict Kate Nolt
LaShaune Johnson Sarah Lux Helen Chapple Mark Robinson 6. MPH Curriculum and Evaluation
Committee
This committee reviews, makes recommendations, and approves matters related to curriculum and assessment, program improvement, and related policies.
• MPH Program Director (Chair)
• 1 faculty member representing each concentration
• Program Coordinator (non-voting member)
• Ad hoc faculty member
Dhitinut Ratnapradipa LaShaune Johnson Sarah Lux Terri Mahaffey
Jacqueline N. Font-Guzmán
7. Public Health Events Committee Committee plans and coordinates events and programming related to public health and/or the MPH program.
• Director’s Designee (Chair)
• Program Coordinator
• 1 Student Representative
• 1 Full time faculty member
• 1 Part time faculty member
Monica Chapeau Terri Mahaffey
Yolanda Reynolds, MPH Student LaShaune Johnson
PT faculty member TBD 8. Student Advisory Board
The Student Advisory Board, a group of current students who meet regularly to share their needs and expectations as well as emerging trends, provides input and assistance to the leadership of the MPH program on matters related to curriculum, policies and procedures.
• Staff Advisor
• 3-5 current students
Terri Mahaffey MPH Students:
• Norma Jean Aubain
• Melyna Avalos
• Camille Cuonzo
• Lindsay de Borba
• Tricia Griffin
• Yolanda Reynolds
9. Scholarship and Service Committee
Committee fosters collaboration and interaction among all program faculty engaged in scholarship, research and community engagement.
• MPH Program Director (Chair)
• All full time and part time regular faculty
• All adjunct faculty
• 1 Student representative
Dhitinut Ratnapradipa Kate Nolt
Tanya Benedict LaShaune Johnson Sarah Lux Helen Chapple Mark Robinson Jos Welie Nick Baldetti Roger Mustalish Leah Casanave Tim Guetterman
Melyna Avalos, MPH Student
A1.2. Briefly describe which committee(s) or other responsible parties make decisions on each of the following areas and how the decisions are made:
The program employs a collaborative leadership model that empowers faculty at all levels to be decision makers. Policy issues are brough forward to program standing committees, ad hoc committees, or the
program director. Issues are considered by the appropriate committee and when needed, voted on.
Faculty have considerable decision-making authority within their courses, recommend and vote on curriculum changes, participate on search committees and make recommendations in hiring decisions on program open positions for staff, faculty, program directors as well as university wide open positions.
Coordinators for each concentration provide direction for the concentrations; the Practicum Coordinator approves student field and capstone experiences; and the Program Director has ultimate authority over the program. Below is a description of how this is achieved for the specific areas identified.
a. degree requirements
Currently the MPH Graduate Program Committee, as noted above, is the primary governing committee at the program level. Chaired by the Program Director and consisting of a coordinator from each
concentration, Practicum Coordinator, and all MPH faculty, this committee is the decision-making body for all degree requirements. Items for consideration may originate from the University’s upper administration, as a result of our commitment to on-going program review, from our External Advisory Board (EAB), Student Advisory Board, site supervisors or directly from the faculty. If revisions for degree requirements are proposed, the Program Director works with this committee to review and agree on a course of action.
At that point, recommendations are shared with the EAB and the Student Advisory Board to secure feedback and additional suggestions. The Committee may refer a curricular item to the MPH Curriculum and Evaluation Committee for review, if warranted. Once the proposal has been thoroughly vetted, the Graduate Program Committee will take a formal vote. If modifications to degree requirements are approved, the changes are then forwarded to the Graduate Dean for final review and approval. If approved, the decision is communicated to all stakeholders.
b. curriculum design
The MPH Program Director provides leadership regarding curriculum development, design, and assessment. Curriculum design is a collaborative effort with the MPH faculty in consultation with the Department Chair and the Department Associate Director. Since this is a distance education program, as warranted, the program will engage the services of instructional technology specialists to assure that what we are designing can be delivered in an effective manner. The need for additions or changes to the curriculum may be identified by a single faculty member or arise from program-level discussion during a meeting. Faculty bring proposals to the MPH Curriculum and Evaluation Committee as a starting point for curricular design and revision. Upon approval by the committee, the proposal is forwarded to the Program Director and if required, onto the department curriculum committee for approval. Finally, the Dean of the Graduate School reviews and approves, or sends back to the committee for revision. The approval process for new courses, curriculum revisions, and new program proposals is as follows:
Table A1.2b. Curriculum Approval Processes
Process Process Steps
New Course Approval Process
1. Faculty member proposes new course, creates a syllabus and submits to the MPH Curriculum and Evaluation Committee.
2. Review/approval by program director and department curriculum committee.
3. Review/approval of the dean.
4. Documentation submitted to the Registrar for creation of the course Curriculum Revision
Approval Process
1. A proposal is introduced by the director or faculty followed by program-level discussion regarding curriculum revision.
2. Review/approval of curricular revision by MPH Curriculum and Evaluation Committee.
3. Review/approval by the program director and department curriculum committee.
4. Review/Approval by the dean.
5. Documentation submitted to the Registrar for Catalog/course updating as needed New Program Approval
Process
1. Program champion completes a New Program Proposal document with Dean’ approval.
2. Review by the Academic Planning Committee.
3. Review/approval by the Graduate Board.
4. Review/approval by the Deans Council.
5. Review/approval by the Provost.
6. Program announcement from the Provost to campus constituents 7. Registrar updates Catalog and creates courses as required
c. student assessment policies and processes
Policies and procedures related to student assessment exist at the program, department, and university levels.
Program Level Assessment: Recommendations and findings from committees are reported to the core faculty and Program Director. Faculty recommendation or formal vote informs all matters of importance.
The final decision-making responsibility rests with the Program Director. The MPH Curriculum and Evaluation Committee is responsible for the development, maintenance, and reporting of formative and summative measures embedded within the curriculum to assess student learning.
Like all other academic programs at CU, the MPH program completes an annual assessment report that documents assessment measure findings as well as other key indicators. The Committee also gathers accreditation specific data that is forwarded to the Accreditation Committee so that vigorous self-
assessment of student success with respect to CEPH learning objectives and competencies is assured.
This is the lead committee with responsibility to assure that CEPH’s 12 Learning Objectives representing Foundational Knowledge are documented, mapped and assessed within the curriculum. It is also the Committee responsible for documenting, mapping and assessing all 22 of CEPH’s Foundational Competencies as well as the 5 required Concentration Competencies. This mapping process involves Committee members meeting directly with all program faculty and carefully reviewing how competencies and objectives are reflected in course objectives, assignments, grading rubrics and student outcome measures.
The use of grading rubrics affords the program consistency from course to course in articulating
assessment criteria, conveying to students what is required for mastery of a given objective/competency, and affords the program a systematic tool to summarize student success with each competency. In the event there is evidence that student assessment processes are inefficient or non-compliant with CEPH requirements, the Committee works with the faculty member to rectify the deficiencies before the course is offered again. (It might be noted that no such deficiencies were uncovered during this review period.) The program also maps the curriculum to assure compliance with Graduate School and University learning objectives and student assessment policies.
Department Level Assessment: The Department of Interdisciplinary Studies Assessment Committee provides feedback and support related to the assessment activities of academic programs within the department. The committee also oversees initiatives to gather and analyze data on student learning.
University Level Assessment: Each academic program is expected to submit an annual assessment report which includes the following components: (1) a custom requirement report that focuses on the discussion of intended learning outcomes and actual results among the faculty, staff, and students involved in a program; (2) the learning outcomes or objectives for the program; (3) a plan for assessing program learning outcomes; (4) assessment findings; (5) a continuous improvement plan related to current findings; and (6) a status report on the prior year’s continuous improvement plans.
d. admissions policies and/or decisions
Admissions policies and recommendations for admission to the program are made by the MPH
Admissions Committee. Committee members review the applicant dossiers and submit their decisions to the Program Director through the university’s admissions review platform, Slate. Formal offers of
admission are made by the CU Enrollment Management Office. The MPH Graduate Program Committee reviews and establishes admissions criteria for the program in accordance with requirements and
expectations of the Graduate School.
e. faculty recruitment and promotion
CU is an Affirmative Action, Equal Employment Opportunity employer and all candidates for faculty positions, whether tenure-track, non-tenure track, or adjunct, are assessed on similar criteria and hired for mission. To ensure a fair and equitable process, the program follows Creighton’s hiring procedures (specification of qualifications, review of academic credentials, and background check for each position).
This process includes multiple phases. The Program Director prepares hiring documents, receives approval by the Dean and submits necessary documents to HR.
The Dean of the Graduate School names a Chair for the search committee who then leads the formation of a faculty search committee. MPH program faculty may serve on this committee. An HR representative is assigned to the process to provide guidance and ensure hiring policies and procedures are followed.
Decisions on the advertisement, scope of the search (internal and external), screening of candidates against established criteria, interview questions, interviews and the selection of the top candidates are completed by the search committee.
As warranted, faculty recruitment for the program strives to cast a wide net for qualified individuals. The EAB, individual faculty recommendations, and strategic placement of job positions within the wider public health community allows the program to communicate our hiring needs within a broad public health network on the regional and national level. The program is guided in identifying its recruitment needs by our commitment to diversity, and with a keen eye on addressing program determined underrepresented populations in our faculty ranks. Our recent hire of a new Program Director, effective June 1, 2020, is evidence of this commitment.
Final approval for hiring is conducted in tandem by the Dean and Provost based on the recommendations by the Department Chair, Program Director and faculty search committee.
Faculty can receive promotion in rank and salary based on achievement of specified attributes in the areas of teaching, scholarship, and service (and clinical work as appropriate). Faculty in the program follow the Graduate School guidelines for performance for the ranks of Assistant Professor, Associate Professor, and Professor, which are consistent with the Faculty Handbook, Section III.G. (see ERF A1.3c) Probationary faculty applying for tenure and/or promotion are required to be evaluated annually by the dean, chair, or other designated senior faculty member. Evaluations are provided in the context of current University and Graduate School Rank and Tenure Guidelines. This process is intended to assist faculty members in professional and career development. After review by the above-named entities, recommendations and dossiers for applicants for tenure and/or promotion are sent to the President, whose decision is final.
f. research and service activities (self-study document)
The teaching, research and service activities of each faculty member are determined in consultation with the MPH Program Director and the Department Chair. Generally, tenure-track faculty responsibilities are allocated as follows: 60% teaching, 20% research, and 20% service. Specific activities may be assigned based on the program’s mission and vision, or in response to needs within the program’s communities of interest (e.g. the at-risk, marginalized and vulnerable populations of the local Highlander Community).
The Graduate School has tenure and promotion documents that provide guidance on the allocation of effort among teaching, scholarship, service and clinical work (where applicable).
A1.3. A copy of the bylaws or other policy documents that determine the rights and obligations of administrators, faculty and students in governance of the school or program.
(electronic resource file)
The following can be found in the electronic resource file:
• ERF A1.3a – Graduate School Bylaws
• ERF A1.3b – Program Director Handbook
• ERF A1.3c – Faculty Handbook
• ERF A1.3d – Student Handbook
A1.4. Briefly describe how faculty contribute to decision-making activities in the broader institutional setting, including a sample of faculty memberships and/or leadership positions on committees external to the unit of accreditation. (self-study document) The MPH faculty are active participants in the decision-making processes at the University, Graduate School, and Department levels. Table A1.4 lists the primary instructional faculty who serve on or lead committees external to the program.
Table A1.4. MPH Primary Instructional Faculty Involvement on Committees Faculty Committee and Role
Benedict, Tanya
• Center for Interprofessional Practice & Research (CIPER) Curriculum Committee, Member (2018- present)
• Graduate School Program Directors Committee, Member (2015-2019)
• Graduate School Distance Education Administrators Committee, Member (2015-2019)
• Department of Interdisciplinary Studies Assessment Committee, Member (2016-2018)
• Department of Interdisciplinary Studies Curriculum Committee, Member (2016-2018)
• Department of Interdisciplinary Studies Rank & Tenure Committee, Member (2016-2018)
• Interdisciplinary Studies, EdD in Interdisciplinary Leadership Faculty Search Committee, Member (2016-2017)
Chapple, Helen • MS in Bioethics Admissions Committee, Member (2018-present)
• Pain Conference Planning Committee, Member (2016-present)
• Plagiarism Committee for the College of Nursing, Member (2017-present)
• Planning Group for IPE 515 Interprofessional Palliative Care Course, Member (2017-present)
• Research and Scholarship Committee for the College of Nursing, Member (2016-18)
• Traditional Curriculum Planning Committee for the College of Nursing, Member (2019-present) Johnson,
LaShaune
• Committee on Computing and Academic Technology, Member (2014-present)
• Diversity and Inclusion Strategic Implementation Team for the Inclusive Excellence Dissertation Fellowships/Post Doc Program, Member
• Health Disparities Implementation Group, Nebraska Cancer Coalition (NC2), Co-Chair
• Highlander Purpose Built Community Internal Advisory Group, Member
• Mario Digital Strategy Planning Team / IT-Project Prioritization Review Team, Member (2015- Present)
• Public Health and Global/Community Health Planning Group Committee, Member (2015-present)
• School of Medicine Distinguished Lecture Series Planning Committee, Member (2014-2016) Lux, Sarah • University Assessment Committee (UAC) – Member (2018 – present)
• UAC Professional Development Sub-Committee – Co-Chair (2018 – present)
• University Policy Committee – Member (2018 – present)
• Assessment Committee, Department of Interdisciplinary Studies – Chair (2018 – present)
• Curriculum Committee, Department of Interdisciplinary Studies – Member (2017 – present)
• Admissions Committee, EdD in Interdisciplinary Leadership – Reviewer (2017 – present)
• Health Care Ethics (Bioethics) Graduate Program Committee – Chair (2017 – present)
• Teaching and Learning: Evaluation and Improvement Writing Team – Member (2015 – 2016) Nolt, Kate • Department of Interdisciplinary Studies Assessment Committee, Member (2018 - present)
• Graduate School and College of Professional Studies Wellness Committee, Member (2017 - present)
• Department of Interdisciplinary Studies Curriculum Committee, Member (2018 - present)
• Center for Promoting Health and Health Equity, Member (2018 - present) Ratnapradipa,
Dhitinut • Curriculum Committee, Department of Interdisciplinary Studies – Member (2020 – present)
• University Research Council, Member (2020-present)
• Public Health Collaboration Committee, Creighton School of Medicine - Ad-Hoc Member (2020 – present)
• Endowed Chair Executive Director of NEHII/CIPER Search Committee - Member (2020-present) Robinson, Mark • EdD in Interdisciplinary Leadership Admissions Committee, Reviewer (2017-present)
• Health Care Ethics Admissions Committee, Member (2017-present)
• Health Care Ethics (Bioethics) Graduate Program Committee, Member (2017 – present)
A1.5. Describe how full-time and part-time faculty regularly interact with their colleagues (self- study document) and provide documentation of recent interactions, which may include minutes, attendee lists, etc. (electronic resource file)
Program Level: The MPH Graduate Program Committee (GPC) meets twice monthly and the
Curriculum and Evaluation Committee meets as needed. The Academic Progression Committee meets once a month and includes all full-time faculty and special faculty (adjunct) currently teaching in the program. This meeting allows faculty to discuss timely instructional topics via the Teaching & Learning Check-In. The format and agenda of these meetings facilitates knowledge sharing and provides an opportunity for faculty to discuss their service and scholarship activities. All meetings include time for faculty to learn of new developments at the program, department, school, or university level. A sample of meeting minutes can be found as ERF A1.5a Program Level Communication.
100% of the faculty-full/part time; regular/adjunct-have been active participants in the self-study process.
Emphasis has been placed on assuring direct interaction between regular and adjunct faculty during this process to strengthen interactions among our community of teacher-scholars. A specific example has been the regular conversations among regular and part time faculty regarding mapping of CEPH learning objectives and foundational competencies in all courses. This has led to pedagogical discussion,
enhancements to course assignments, and identification of common interests in scholarship and service that will foster future interactions.
Full-time, part time, and special faculty are encouraged to collaborate on research projects, pedagogical approaches, mentoring, and scholarship. A specific outcome of past interactions has been the expressed desire of adjunct faculty to participate at the committee level in the program on a Scholarship and Service Committee along with regular faculty, to promote enhanced opportunities for collaboration. (see ERF A1.5a) This Committee has been formed and began activities in the spring 2020 term. Given the logistical realities of adjuncts having other employment obligations during the day, this committee functions as a
“virtual committee” via Zoom, email, and, when feasible, in-person meetings.
A few examples of collaborations among faculty include Drs. Kate Nolt’s and Tanya Benedict’s work on improving the content and assessment of the practicum course. Drs. Benedict and Nolt also collaborate in research and service related to facilitating a children’s kitchen in a marginalized community of Haitian migrants living in the Dominican Republic. Recently they collaborated on a poster presentation at the 2020 Global Health Conference Midwest to present the findings from a community health needs
assessment conducted to determine the impact of the feeding program on the children of the community served. (see ERF A1.5b) Drs. LaShaune Johnson and Kate Nolt have collaborated on planning public health activities related to National Public Health Week (NPHW) observance in 2019 and 2020.
In 2019, the NPHW celebration included a series of virtual live, pre-recorded, and in-person (recorded) events.
In addition to the Creighton-created posts and speakers, MPH faculty also used their blog and listserv to encourage student involvement in the national activities, such as the Student Twitter takeover and opening day speaker. The ethnically diverse group of speakers for NPHW 2019 were from a range of universities and organizations across the globe.
In 2020, the MPH Student Advisory Board joined the planning activities for National Public Health Week coordinated by Drs. LaShaune Johnson, Kate Nolt and Roger Mustalish. Due to the COVID-19 crisis, the planned in-person activities needed to be cancelled. In response, the Creighton NPHW planning team, including the MPH Student Advisory Board, created a website featuring a series of pre-recorded activities that were advertised on various social media sites, listservs, and Creighton news outlets. As in previous years, the team encouraged students to participate in the national observations in addition to taking part in the Creighton initiated events. The Student Advisory Board participated in NPHW planning meetings with faculty and are collaborating on MPH social media posts to be featured later in the year. The multicultural group of speakers for NPHW 2020 included health educator and Creighton MPH alum, April Dixon; Creighton Medical School professor, Caron Gray; local health department supervisor and MPH adjunct instructor, Leah Casanave; health disparities researcher, Tess Thompson, from a regional university; part-time environmental health faculty Roger Mustalish; and Cydney Franklin, COO of
Seventy-Five North Corp. at Highlander Accelerator, an Omaha Purpose Built Community, where faculty member LaShaune Johnson conducts community engagement work. (see ERF A1.5 MPH Event
Committee Minutes)
Drs. Kate Nolt and Lisa Boyd collaborated on data analysis of a community health needs assessment conducted in two rural vulnerable communities in the Dominican Republic in 2019.
Department Level: All full-time faculty are expected to attend quarterly Interdisciplinary Studies
Department meetings, quarterly faculty meetings, and weekly MPH program meetings. In addition, Health Care Ethics Concentration faculty have academic responsibilities outside the MPH and regularly interact with faculty in the MS in Bioethics program. A sample of meeting minutes can be found as ERF A1.5b Department and College Level Communication.
Each year the Department hosts an off-campus day-long retreat attended by regular and adjunct faculty from the MPH program, as well as faculty from other programs in the Department. The theme for the most recent, on October 4, 2019, was “Care for the Common Home”.
Graduate School and University Level: As a group, all graduate program directors meet two times each fall and spring semester with the Dean and Senior Associate Dean of the Graduate School to receive university updates, discuss issues related to policies or procedures, and receive reports from marketing and graduate enrollment. MPH faculty, whether they are primary, secondary, or special faculty, are invited to participate in University and Graduate School events, such as commencement, hooding, workshops, seminars, public health forums, presidential town halls, and symposiums on relevant topics.
A1.6. If applicable, assess strengths and weaknesses related to this criterion and plans for improvement in this area. (self-study document)
Strengths
• The distributive leadership model employed by the program assures that all faculty are empowered to actively participate in the program.
• Primary instructional faculty and non-primary instructional faculty are engaged in program
development and governance. Decisions about curriculum, coursework, assessment, and policies are made collaboratively.
• There is an active and effective array of strategic committees.
• The program is well-represented on campus and university committees with at least one primary instructional faculty and/or staff member participating on all major committees. MPH
representatives relay the committee’s activities and decisions to the MPH Graduate Program Committee when appropriate.
• The EAB has a diverse membership, including alumni, who provide essential feedback and guidance to the MPH program faculty.
• There is a formal process to assure student participation in the program to inform faculty decisions (See A3.)
Weaknesses
• As a distance learning program with a national, even international, student body, and employing both regular and adjunct faculty, the program is always challenged to assure that our structure, and operations embrace and empower the diverse stakeholders in our program. We recognize a need to be mindful of ongoing ways to enhance engagement.
Plans for Improvement
• The newly formed virtual Scholarship and Service Committee will begin activities during 2020 to foster enhanced collaboration among all MPH faculty.
A2. Multi-Partner Schools and Programs
Not applicable.
