H Data analysis is used to foster ongoing program improvement.

Elaboration: The program uses outcome data for improvement. Data regarding completion, licensure, certification, and employment rates; other program outcomes; and formal complaints are used as indicated to foster program improvement.

• Data regarding actual outcomes are compared to expected outcomes.

• Discrepancies between actual and expected outcomes inform areas for improvement.

• Changes to the program to foster improvement and achievement of program outcomes are deliberate, ongoing, and analyzed for effectiveness.

• Faculty are engaged in the program improvement process. Program Response:

The responsibility of monitoring outcome data and input from students, faculty members, and others is shared with faculty members and administration, and involves these groups: curriculum committees, entry-level course coordinators, MS specialty directors, academic deans, Deans and Chairs Committee, Faculty Council, and Administrative Council.

BSN Program

Entry-Level Course Directors from UMB and USG co-prepare and submit an annual report to the Entry-Level Curriculum Committee (ELCC), describing quality review and process improvement activities for their courses. Each course director presents course related data, e.g., course evaluations, grade distribution, and student outcomes, to identify strengths and weaknesses and present a plan to address areas for improvement. For example, faculty members teaching NURS 407 Maternal, Newborn, and Women’s Health Nursing: A Family Perspective noted in their annual course report for spring 2013 semester that the UMB section (N = 68) had a grade distribution of 43 A’s and 25 B’s, while the USG section (N = 32) had 10 A’s, 21 B’s and 1 C (see January 2014 course report and ELCC minutes, 1/27/2014). Faculty members teaching the courses met to further analyze the data for this inequality in grade distribution, and reviewed input from clinical faculty members, standardized test scores, and faculty observations of the semester’s learning activities in an effort to arrive at solutions for greater equity in grading across both sections.

Clinical site evaluation aggregate data are also presented by semester, section, and campus in the annual reports to ELCC. Clinical evaluations indicating problems with preceptors, sites, and/or students are discussed and corrective actions proposed. For example, the course director reported clinical site evaluation data for

90 NURS 402 Psychiatric Mental Health Nursing for fall 2012 and spring 2013 semesters (see January 2014 course report and ELCC minutes, 1/27/2014). Students (N=18) from one hospital appeared more dissatisfied with their clinical site as compared to other clinical sites used for this course. Only 35% of the students strongly agreed or agreed that the clinical site offered a range of learning experiences, 50% that it provided a supportive learning environment, 56% that it provided effective role models and was consistent in meeting course objectives, and 39% that is provided a variety of interpersonal and/or technical opportunities for clinical growth. Follow up with the clinical instructors at this particular site by the course director identified them as being novices in their roles. In response, the orientation of clinical instructors was reviewed and enhanced, one of the clinical instructors was replaced, and a new clinical coordinator was assigned to the course with a plan to make periodic onsite visits to ensure higher quality, more meaningful and effective student experiences.

Standardized testing data are also presented by semester to the ELCC, by campus and by course. NCLEX pass rates are compared to the UMSON benchmark (≥ 80%) and trends over time. Pass rates are reviewed by course

directors to address further processimprovement initiatives. No action has been required since the NCLEX pass

rates consistently exceed the UMSON benchmark. RN-BSN

Changes to the RN-BSN option were partially based on anecdotal feedback from current students, recent graduates, and potential applicants at recruiting events and Open Houses. They have included comments such as dissatisfaction with being in classes with traditional BSN students, a preference for coursework that would minimize duplication from their prior registered nurse preparation, and requests for courses specifically designed for RN students. In the past, RN-BSN students were in the same classes as traditional BSN students, so course evaluations have been co-mingled. Beginning in the fall 2014 semester, the RN-BSN option will be a totally separate plan of study from the BSN traditional students, and student course evaluations will guide the evaluation of this new option. Changes to the current RN-BSN plan of study were required due to revision of curriculum for the traditional BSN Program. This revised RN to BSN option is based on the Baccalaureate Essentials and broadens registered nurses’ knowledge base.

MS Program

The MS and DNP Specialty Directors produce a comprehensive evaluation of each specialty every three years, and present their findings for review to the Master’s/DNP Curriculum Committee. The continued quality, relevance, and congruence with professional standards and guidelines of the curricula in each MS specialty are ensured by application of an evaluation tool that examines three key categories: program quality, market trends, and fiscal data (see Program Assessment Tool in Appendix IVH-1). The evaluations include

programmatic data and comments related to continual course content updates, sequencing and quality of courses, and alignment with national standards. Student data, faculty performance, and engagement are tracked. Each specialty reports on interdisciplinary activities that incorporate the teaching, research, and practice missions of UMSON. Market trends are analyzed, including inquiries, enrollment, future employment opportunities for students, and faculty recruitment. Finally, each specialty conducts a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis to identify direction and priorities (see Reports under Master’s/DNP

Curriculum Committee on intranet at http://www.nursing.umaryland.edu/Intranet/Committees/Faculty-

Organization/Masters-DNP-Curriculum-Committee/. MS Specialties

Curriculum changes for individual MS specialties have been implemented in response to evaluations from students and faculty members, as well as from the evaluation of each specialty by the Master’s/DNP Curriculum Committee. Each specialty director reviews core course evaluations regularly so they can use student input to enhance courses and refine curricula. They also look carefully at student retention issues and the availability of resources such as scholarships and mentoring opportunities.

For example, the Health Systems Leadership and Management (HSLM) tri-annual specialty report revealed a 23% attrition rate, which alerted them to examine the contributing factors and develop a plan to increase needed financial support for students through a MHEC Nurse Support Program II grant. Low enrollment numbers led the specialty to de-activate its MS/JD option, while increasing the MS/MBA track with the University of Maryland

91 College Park and University of Baltimore in response to demand. The HSLM specialty evaluation also highlighted the importance of using many local, state, and regional organizations for practicum experiences. Many of these opportunities evolved into mentoring relationships, with the positive outcome of students being well-prepared for leadership roles and employers recognizing their abilities.

Another example exists in the Adult Gerontological Primary NP specialty, which increased the instructional review of their diagnostic reasoning simulation and added content on interprofessional collaboration. They also added an assignment to incorporate policy issues into the curriculum and produced a focused plan to help students with the development of their Comprehensive Exam Papers. Genetics and HIV content have also been added to the coursework.

The Nursing Informatics specialty faculty members examined data on market trends and found that numerous online and face-to-face health care informatics programs nationwide demonstrated the importance of

branding. The faculty then defined their unique strengths in preparing informatics nurse specialists who can support clinicians in practice settings, and patients and their caregivers in community centers. The success of this branding has been evidenced by the continuously increasing number of the applications despite full scholarships offered by other health care informatics programs in other university settings.

Nurse Anesthesia made a change in their pharmacology course a few years ago, and continues to see marked improvement in SEE and NCE exam scores; feedback from students and preceptors is positive. The 2012 cohort had a 100% first-time pass rate, possibly due to increased simulation time (hands-on experience and practice). Previously, simulation was integrated in only the last year of study, and now it is included in all three years. Nurse anesthesia students are required to take a self-evaluation exam, which mimics certification questions, at the beginning of their second and third years. The nurse anesthesia specialty implemented MediaSite capture of all core curriculum lectures, coupled with use of oral board exams in the first and third years of the program.

MS-CNL

The CNL option director meets regularly with cohorts of CNL students to identify program and course issues. While the majority of CNL students are satisfied with most aspects of the CNL option, the causes of CNL student and graduate dissatisfaction are related to a variety of factors. Some of these are external to the UMSON environment. Some graduates have expressed dissatisfaction with the lack of integration of the CNL role into the health care system. Faculty members continue to communicate with health care system

administrators about roles that CNLs can assume in health care reform. While most health care institutions do not have a specific CNL titled role, some local hospitals have actively recruited UMSON graduates and shared their positive experiences with hiring them in an AACN faculty webinar, Transitioning Second Degree CNL Graduates into Practice: Innovative Models and Successes, in June 2011.

The overall program evaluations have reflected student dissatisfaction with courses taught jointly with BSN and CNL students. For the last several semesters, the only courses in which the two groups have been combined were three first semester courses, i.e., NURS 501 Pathopharmacology, NURS 503 Health Assessment, and NURS 505 Introduction to Professional Nursing Practice, and a second semester course, NURS 514 Adult Health Nursing. All laboratory and clinical sections have been separate since the implementation of the option and CNL students have different objectives and a number of different assignments. Students have also stated that there is insufficient CNL role-specific content taught in the curriculum. With the revised CNL curriculum being implemented in the fall 2014 semester, all CNL classes will be distinct and separate from BSN Program classes. In the revised curriculum, role-specific content has been enhanced, and a CNL-related text will be required. The length of the program has been adjusted to eliminate the accelerated option, to further improve the quality of the educational process. Course content is continually updated based on evidence in nursing and related literature. Course revisions will assure that the CNL curriculum meets the baccalaureate essentials, master’s essentials, and the revised CNL competencies, and that the role of the CNL is emphasized as appropriate.

Most CNL students maintain satisfactory academic standing. However, a small proportion fall below the required 3.0 GPA. Student progress is regularly monitored. Individual academic support (remedial) plans are required by the Graduate School for students with academic difficulties. They are encouraged to work with their advisors and course faculty members and take advantage of the Student Success Center (SSC). The SSC

92 runs a CNL Mentoring Program, which is based on the RWJF Mentoring Plan for Scholars. In spring 2013, this program grew to 14 mentors and 28 mentees. The Mentoring Program is evaluated on an ongoing basis and revisions are made as needed, with the goal of high retention, academic achievement, and successful

transition into practice. To further bolster their success, standardized tests are administered to CNL students to assess their knowledge. Remediation is encouraged, if necessary, using available testing materials, faculty suggestions, and participation in the SSC. A structured review course is required at the completion of the plan of study to assure that students receive a required content review and are familiar with NCLEX-style testing. Post-Master’s DNP Program

Faculty and student input and analysis of student outcome data resulted in several modifications to the DNP Program. For example, data on student attrition was reviewed in conjunction with course evaluations to make course and progression rule modifications. Students were previously required to maintain continuous

registration of at least four credits during the fall and spring semesters, and one credit during the summer term for the entire length of the program. However, students reported that at times they were unable to manage this load, and therefore dropped courses. In fall 2012, the rule for a minimum number of credits was removed, with only continuous enrollment being required. In 2013, students reported in course evaluation questionnaires that they were overburdened with assignments in NDNP 802 Methods for Evidence-Based Practice and NDNP 804 Theoretical and Philosophical Foundations of Nursing Practice. The courses were redesigned with a reduction in the number of assignments, and the changes were reviewed at a Master’s/DNP Curriculum Committee meeting (8/12/2013).

Summary of Standard IV

UMSON has developed a comprehensive evaluation plan to monitor program outcomes. Each program is evaluated by several components, including the course evaluation, end-of-semester program assessment, student licensure pass rates or certification pass rates, completion rates, and employment rates. The data and results of the evaluative components were shown to be above UMSON and CCNE benchmarks. The only area of concern was the completion rate of the post-MS APRN certificate program, which has not consistently met the benchmark. Given that it is a small group, strategies to improve the completion rate have been undertaken. On the other hand, although graduates’ three-year employment rates for all programs were above the benchmark, the response rates for the surveys were low (between 22% and 42%). In an effort to receive more responses from new graduates, the Office of Evaluation initiated a change in 2013 to collect employment data twice in the first year after graduation. In addition, alumni surveys were conducted to evaluate the program outcomes. Despite the low response rates, the majority of our alumni were satisfied with their program of study. The exception was the MS-CNL program, however, the curriculum has been totally revised based on student evaluations, and revisions will be implemented in fall 2014. In the area of the faculty outcomes, the data provided in the report indicated that faculty members’ teaching achievements are above the UMSON benchmarks. Even though no benchmark has been set for faculty scholarly activities, faculty engagement in scholarly activities and national and international recognition for outstanding professional contributions have continued to increase.