Health
Resident Intensive Summer Experience (RISE)
III-F. The curriculum and teaching-learning practices consider the needs and expectations of the identified community of interest.
Elaboration: The curriculum and teaching-learning practices (e.g., use of distance technology, didactic activities, and simulation) are appropriate to the student population (e.g., adult learners, second language students, students in a post-graduate APRN certificate program) and consider the needs of the program-identified community of interest.
Program Response:
The curriculum and teaching-learning practices consider the needs and expectations of our
community of interest. Our programs unique combined online and on-site teaching-learning practices clearly meet the needs of students and faculty. Online learning experiences based on sound
distance-learning pedagogy provide unique academic enrichment to students that may be more difficult to achieve in a face-to-face setting. For example, when every student is expected to participate actively in a discussion conducted only on-line, the breadth and depth of the discussion benefits from the increased level of participation. Because most discussions are conducted
asynchronously, students have time to do the necessary reading and synthesis that leads to an informed contribution. Further, this modality benefits students of whom English is a second language, as they can more easily participate. Many faculty have commented that student
discussions are much richer than they were when the class was conducted on site. Each course has specific criteria for discussion (e.g., frequency of posting, grading of quality). Another benefit of the online learning environment is that the student community at the College of Nursing is geographically and culturally diverse, providing valuable socio-cultural enrichment to the programs. Students in the DNP programs hail from diverse geographical locations such as Alaska, Hawaii, California and New York and live and work in rural, border, and Native American communities.
In 2012 the college administered an alumni survey using Survey Monkey software. The survey was issued to 21 DNP alumni and had a 48% response rate (10 respondents). During 2013 we analyzed various programs that would improve data collection on a larger scale, specifically data pertaining to alumni and employers. The college made the decision to move forward with Educational
Benchmarking INC. (EBI) Assessments (www.webebi.com/assessments), an educational benchmarking assessment system which allows us to survey alumni as well as evaluate performance and develop standards for continuous improvement. We launched EBI in Spring 2014 with the administration of three DNP surveys: alumni, exiting graduates, and employers. The alumni survey response rate during the Spring 2014 was 15%. We later administered the survey again in August, this time using our own Qualtrics survey software and offered a small incentive to participate. The response rate grew from 15% to 20%. The college will continue to use EBI to survey alumni, exiting graduates, and employers and surveys will be administered each spring. We are testing the use of incentives in order to
encourage participation and elicit responses from our communities of interest. In addition to alumni, exiting students and employers, we elicit input from our College Advisory Board, and during
discussions with student preceptors. Input from preceptors provided a quality improvement change in the time when the CSI would be offered for example. Furthermore, faculty is encouraged to give input and feedback during meetings such as doctoral forum, Clinical Supervising Faculty meeting and the DNP-CISC subcommittee. Faculty who are not members of the DNP-CISC committee regularly attend the DNP-CISC meeting to provide input to the subcommittee. Another example is we used the faculty input, discussed in informal interactions and in doctoral forum, regarding the DNP Project and comprehensive exam to formulate a task force to redesign the DNP Project and comprehensive exam for a more structured, optimal student learning experience.
III-G. Individual student performance is evaluated by the faculty and reflects achievement of expected student outcomes. Evaluation policies and procedures for individual student performance are defined and consistently applied.
Elaboration: Evaluation of student performance is consistent with expected student outcomes. Grading criteria are clearly defined for each course, communicated to students, and applied consistently.
Processes exist by which the evaluation of individual student performance is communicated to students.
In instances where preceptors facilitate students’ clinical learning experiences, faculty may seek input from preceptors regarding student performance, but ultimately faculty are responsible for evaluation of individual student outcomes. The requirement for evaluation of student clinical performance by
qualified faculty applies to all students in all programs. Faculty evaluation of student clinical performance may be accomplished through a variety of mechanisms.
Program Response:
Defined evaluation policies and procedures are consistently applied in faculty evaluation of individual student performance, to reflect achievement of expected individual student learning outcomes. General grading criteria and student evaluation policies are defined and communicated to all students through online student handbook
https://www.nursing.arizona.edu/OSA/PDF/OSA/DNP_Student_Handbook_2014_2015.pdf and course syllabi. Individual student performance is evaluated by the faculty and reflects achievement of expected individual student learning outcomes. For all courses in the DNP Program, evaluation policies and procedures are described in detail in each course syllabus (see examples in the Resource Room). Students are evaluated in didactic content courses with multiple methods including
discussion contributions, journaling, concept papers, presentations, and examinations. For example, students in NURS 751 Evidence Based Practice Methods, a core DNP course, are evaluated using a) assessment of depth, breadth, and frequency of discussion; b) accuracy in quizzes; c) critical appraisal of literature; d) written application of evidence based practice theoretical models to a clinical problem, and e) planning an implementation and updating of a clinical practice guidelines in practice. All work is individually assessed until the final project which is assessed as a group and supported by a peer evaluation.
Additionally, students in NURS 629a Advanced Family Psychiatric-Mental Health Nursing conduct and document a complete psychiatric evaluation including history, examination, and health promotion plan in order to measure the students’ ability to apply advanced practice nursing principles of risk assessment, coping skills and support systems of individuals experiencing a psychiatric disorder or mental health condition. Students also conduct and document a complete psychiatric evaluation that includes cultural assessment and culturally sensitive plan in order to measure the students’ ability to apply advanced practice nursing techniques of psycho-education and therapeutic interventions within a holistic and culturally competent approach. A de-identified exemplar of an evaluation is in the resource room.
Expectations for clinical performance are carefully described for each student level of practice through detailed clinical performance objectives (See Appendix III-G-1 for a sample of clinical evaluation criteria). These criteria were recently modified to reflect the new standards and guidelines appropriate to the Consensus model for specific specialties. Faculty use course-based evaluation of expected individual student learning outcomes and University-based course and teaching evaluation in the DNP program. College of Nursing faculty ultimately have direct responsibility to assure that learning experiences are appropriate for students and for evaluating students’ progress and competencies. Grades are assigned by faculty and are based on the criteria that are found in each syllabi. Clinical SOAP notes and activities are pass/fail, but student's clinical expertise is assessed by responses to case studies and other activities. A minimum of one site visit per semester is conducted by the faculty member either to the clinical setting on-site or by telephone or videoconferencing using the same evaluation criteria, and for more novice students, additional visits are made if deemed appropriate by faculty and/or preceptors. The CON, led by Jason Shuffitt, is in the process of developing an application which will allow clinical supervising faculty to evaluate students' on-site using technology and it is anticipated this ‘app’ will be ready for testing in 2015.
Some students may be located in geographically remote areas in Arizona or practice in a different state. If a site visit is deemed necessary, college faculty will travel to the site after exhausting other ways to communicate with the student and preceptor to resolve any issues. We also have the ability to require students come to Tucson to work with faculty at the College of Nursing. In 2013, one DNP faculty member visited a student at a Native American clinic as there was a concern the student may not be meeting individual student learning outcomes. The site visit assisted with the student’s remediation and facilitated student/preceptor interaction. No further action was necessary.
Communication among faculty, student and preceptor is essential and is performed by frequent telephone and email contact, with formal written evaluations completed by the clinical supervising faculty (CON faculty) twice per semester. Students are formally evaluated by the clinical preceptor twice a semester and conduct informal evaluations with faculty as needed. Students submit weekly patient notes and course assignments to faculty for assessment and evaluation. Further, students maintain a clinical log through an electronic student tracking system, the TYPHON Student Tracking System. TYPHON was described in Standard II. It is a software program used to track patient encounters provided by students who are enrolled in a clinical course. DNP student progress is evaluated annually in the Fall semester by the DNP-CISC Committee with assistance from the appropriate Academic lead [Badger], Assistant Director of Student Affairs [Amanda Gluski], who monitors submission and review of the students’ annual self-reflection and completion of the portfolios. The DNP Student Annual review includes advisors reviewing for completion of program benchmarks such as date of course completion, names of committee members and comprehensive exam completion. When students do not progress according to the benchmarks for Satisfactory Progression published in the Student Handbook,
https://www.nursing.arizona.edu/OSA/PDF/OSA/DNP_Student_Handbook_2014_2015.pdf, the DNP
program committee, student advisor, and student address the situation to facilitate progress.
Examples of de-identified students’ portfolio can be found in the resource room.
III-H. Curriculum and teaching-learning practices are evaluated at regularly scheduled intervals to foster ongoing improvement.
Elaboration: Faculty use data from faculty and student evaluation of teaching-learning practices to inform decisions that facilitate the achievement of student outcomes. Such evaluation activities may be formal or informal, formative or summative. Curriculum is regularly evaluated by faculty and other communities of interest as appropriate. Data from the evaluation of curriculum and teaching-learning practices are used to foster program improvement.
Program Response:
Faculty evaluate curricula and teaching-learning practices of the DNP Program on a regular basis. Our College Evaluation Plan (Table III-H) details data collected, when collected, and party responsible to ensure data are disseminated for quality improvement. Curriculum oversight is held by the DNP-CISC subcommittee along with the Academic Administrative lead for the DNP and the DNP Program
Director. Individual graduate faculty conduct formative and summative evaluations of course content and teaching-learning practices for each course each semester taught for ongoing
improvement. These evaluation data are obtained from Teacher Course Evaluations (TCEs) and via surveys collected during the on-campus RISE and clinical skills intensive (CSI) sessions. The
University and College of Nursing use student ratings obtained with the TCEs
(http://aer.arizona.edu/teaching/Guide/TCEGuide.asp) to improve teaching and document teaching performance on the faculty annual review. Examples of TCEs from selected core courses and clinical management courses will be available in the resource room.
Table III-H Evaluation Table
Evaluation
Dimension Data Source Benchmark
Target/Follow-up Action Sample, measurement &
interval Person(s) or group to receive data Teacher & Course
Evaluation (TCE) UA TCE reports 95% of Faculty will be within 1 standard deviation of UA mean Noted on faculty annual review, improvement plan - Division Director
50% response rate of students in each course
Every course End of each semester RISE, CSI
Individual reports:
Faculty of record, Course Chair, AF, PD
Online Course
Evaluation Quality Matters Rubric
100% of courses will meet all components of the QM rubric.
Faculty & ID to refine
On a schedule = 6-8 courses/semester (rotated) Students will evaluate as part of TCEs
Faculty of record, Course Chair, AF, PD
Program Evaluation UA OIS-led focus group
Volunteers from each student cohort EBI reports
100% of evaluation surveys will be reviewed by the CISC-DNP for change/improvement
Anchor course students- in Health Assessment or CSI Each year
AF, PD, CISC-DNP (SA to file decision and follow up report)
Individual Student
Annual Report Student input to
Portfolio 100% completed by last
day of class/semester All students
Each semester mandatory for academic progress file decision and follow up report)
Evaluation Certification Pass
Rate Certifier Report 80% will pass exam on
1st attempt All certification seekers Report Schedule
AF, PD, OSIQA
Coordinator to enter into database
Alumni Evaluation Alumni Survey - employed,
satisfaction, prepare for role
80% of graduates are satisfied and employed in APRN role within the 1st year of graduation
All graduates Years 1-5
Administrative Council
Employer/Colleagu
e Evaluation Semi-structured telephone interview and EBI
80% of employers are satisfied with our
Achievement EBI Surveys 2 achievements/ per
cohort in 5 years Present/publish DNP-CISC
Faculty Evaluation Annual report 100% achieve 3 or better on teaching, scholarship, and service
All faculty/annually Peer Committees, Division Directors Key: AF = Administrative Facilitator; CISC = Curriculum & Instructional Support committee; CSI= Clinical Skill Intensive, EBI = Educational Benchmarking Institute; ID = instructional designer; OSIQA = Office of Strategic Initiatives and Quality Advancement, OIS = Office of Instructional Services; PD = Program Director; DP= DNP Project; QC = quality control; RISE = Resident Intensive Summer Experience, SA=Staff Associate, QM= Quality Matters
The Office of Instructional Assessment can also assist faculty to obtain evaluation data from students for quality improvement of each course through discussion with students and occasional written questions for student to express opinions on courses. Through the TCE system, students have the opportunity to evaluate individual faculty teaching, faculty teaching teams, and courses and provide feedback on teaching-learning practices, assignments, clinical placements, classroom experiences, and course timelines. Table III-H Summary of Evaluation of the Curricula for the DNP details
evaluation dimension, data source, benchmark, sample measurement and interval, quality oversight group, person or group to receive data, and database & report storage. The College, until recently, had all shared data on the ‘G’ drive, but is migrating to the use of SharePoint. Data will be found on both the ‘G’ drive and SharePoint and visitors will be assisted to find documents as needed while onsite.
TCE data are used to improve teaching-learning practices. For example, after reviewing Fall 2013 student feedback of NURS 681, Issues in Geriatric Health, faculty made changes to course content for the Fall 2014 course offering. Student input included 1) student expectations of course curriculum and delivery; 2) usefulness of content; and 3) expected coverage of course material. Content
additions for NURS 681 included 1) content on alcohol and prescription drug abuse in older adults; 2) content on elder abuse from the provider perspective; 3) more inclusive content on acute care, in addition to outpatient management; and 4) changes to the functionality of group discussions within D2L to allow for easier class discussion. NURS 652, Methods for Scholarly Inquiry, was also revised based on Course Evaluations. For example, in Spring 2013, students voiced that they would like more discussion during the course. Therefore, the number of required discussion board postings was
increased from 4 weeks to 11 weeks of discussion board for the Spring 2014 course offering.
Additional changes based on student feedback included replacing the Synthesis-Evidence Table exercise with Evaluation Tables. This curriculum change enabled students to more readily synthesize findings based on articles in their evaluation tables and subsequently use the findings as the basis for the DNP Project.
Several other examples of data driven change come from survey data following the 2013 CSI.
Students who attended CSI gave high ratings to the chest and extremity lecture but requested an additional lecture on radiology test selection. Based on that feedback, a “What Test Do I Order?”
seminar was developed. This new seminar is held after the basic chest radiology seminar. In addition, all radiology seminars are now more interactive. Secondly, after reviewing student feedback, PNP students were given more “hands-on” learning opportunities during CSI including practice and return demonstration for splinting and foreign body removal. Also, a scholar session was added so that PNP students could present their DNP Project and receive feedback from PNP faculty and peers. For the FNP specialty, joint injections with practice on shoulder models were added into the Spring CSI as well as Punch Biopsies. Furthermore, students requested less time on
musculoskeletal assessment because this was redundant to content covered in NURS 609a Health Assessment.
Another example of the way student data are used for continuous quality improvement was to change the sequencing of courses. It was apparent to faculty and students that students had difficulty applying theory and developing their research methodology for their DNP Project.
Discussed during regularly scheduled doctoral forum meetings (attended by on-site and distance faculty through distance technology), a decision was made to modify the curriculum and reverse the order of some courses to include theory and methods earlier in the program of study (beginning in 2010). The intent of this change was to provide students with conceptual tools needed to connect theory, research, scholarship and practice early in their program. It allowed students to apply foundational knowledge in subsequent courses and in clinical practice and integrate this learning in work toward their DNP Project proposal. Ongoing evaluation is occurring to monitor student ability to commence work on the DNP Project proposal earlier in the curriculum as a benefit of these changes. Curriculum examples can be found at
http://newsite.nursing.arizona.edu/academics/doctor-nursing-practice-dnp/bsn-to-dnp.
Plans are in place to initiate a DNP Project Proposal course beginning in 2015. The course will be offered in the spring of the 2nd year as a strategy to facilitate earlier proposal development. The course is offered in this semester to allow students to obtain foundational knowledge (e.g., theory, methods, core content) prior to beginning their DNP project proposal. We have a current task force of the DNP-CISC subcommittee that is evaluating the procedures and quality of the written portion (portfolio, final self-reflection) and oral portion of the comprehensive exam procedure, as well as the DNP Project. Based on the NONPF white paper published in June of 2013, we changed the name of our DNP Project (formerly called Practice Inquiry) to DNP Project to facilitate recognition of the DNP degree and DNP Project results. Implementation of this change in fall 0f 2014 resulted in changes to the Handbooks, D2L website and Graduate College directions. We are recommending streamlining the portfolio (examples will be found in the resource room) and are developing policies, procedures and grading rubrics for the DNP Project to ensure quality and better consistency. We anticipate implementation in 2015 after we have had adequate time for faculty and student development.
Strengths:
1. Ongoing revision of the DNP curricula has assured objectives are explicit and expected
outcomes relate to the roles for which students are being prepared. Processes are in place for
outcomes relate to the roles for which students are being prepared. Processes are in place for