• No results found

A PRELIMINARY EVALUATION OF A NURSING PROGRAM S ADMISSION INTERVIEWS. Dale M. Maughan. A thesis submitted to the. Faculty of Utah Valley University

N/A
N/A
Protected

Academic year: 2021

Share "A PRELIMINARY EVALUATION OF A NURSING PROGRAM S ADMISSION INTERVIEWS. Dale M. Maughan. A thesis submitted to the. Faculty of Utah Valley University"

Copied!
42
0
0

Loading.... (view fulltext now)

Full text

(1)

A PRELIMINARY EVALUATION OF A NURSING PROGRAM’S ADMISSION INTERVIEWS

by

Dale M. Maughan

A thesis submitted to the Faculty of Utah Valley University

In partial fulfillment of the requirements for the degree of Master of Science in Nursing

Department of Nursing Utah Valley University

(2)

Utah Valley University Department of Nursing

Committee Approval

of a thesis entitled

A PRELIMINARY EVALUATION OF A NURSING PROGRAM’S ADMISSION INTERVIEWS

submitted by Dale M. Maughan

This thesis has been read by each member of the student’s thesis committee and by majority vote has been found to be satisfactory.

_________________ _______________________________

Date Gary J. Measom, Chair

_________________ _______________________________

Date Susan K. Rasmussen

_________________ _______________________________

Date Joohyun Chung

(3)

Abstract

Significant challenges exist for nursing education programs in their responsibility to prepare students for licensure and to be clinically competent as graduate nurses. Concerns regarding a marked reduction in NCLEX-RN passing rates and poor student performance issues led to the addition of an interview component to the undergraduate nursing admission process at Utah Valley University, to assess essential non-academic attributes in addition to prerequisite academic performance. With active faculty involvement, essential attributes of nursing applicants were identified and rubrics measuring those attributes were developed to ensure valid, objective evaluation during interview activities.

The purpose of this study was to evaluate data from the first four applicant cohorts, answering the questions: (a) Was there a difference in applicant ranking based on prerequisite academic performance compared to ranking based on interview

component scores? (b) Was there a difference in prerequisite academic performance between applicants who were accepted into the program and those not accepted based on the interview component? (c) What were the applicants’ perceptions of the likelihood of being accepted into the program by participating in the interview component?

Results revealed a significant difference in applicant rankings between

prerequisite academic performance and interview component scores. Some difference was found in prerequisite academic performance between those accepted into the

program and those not. Average applicant response was significantly positive; indicating their perceptions of being accepted into the program was more likely by participating in

(4)

the interview component. The interview component was effective in identifying applicants with the desired essential non-academic attributes.

(5)

Acknowledgement

Earning a graduate degree, let alone more than one, was definitely not in my plans as I initially began pursuit of a degree in nursing 32 years ago. The completion of this thesis marks the end of my formal educational pursuits. Admittedly I had no desire for yet another master’s degree but because of my love for teaching and the wonderful experiences I have had teaching the students in our program, I was willing to do it. I have learned so much in the few years that I have had the privilege of being a faculty member in the Department of Nursing and in pursuit of the MSN. I have a love for learning and I will continue to learn as much as I can, that I might be the best I can be for my students and patients.

There are so many people to thank for the patience and support shown me as I worked to complete this thesis. Fellow nursing faculty members have been so supportive of the implementation of the interview component into our program admission process, which became the subject of my study. I am particularly grateful to a mentor and fellow faculty member, Susan Rasmussen, who has had a clear vision of the interview

component from its beginnings and what it needed to be. I have been fortunate to be along for the ride as she has willingly shared her vision and the opportunity to participate with her over the past three years. I am also very grateful for and acknowledge the time and efforts of my committee members who provided assistance with the statistical analysis of my data and thoughtful suggestions and guidance with the many writes and rewrites of the thesis document. Lastly I thank my wonderful wife and children who have been so patient and supportive while a husband and a father has spent so many hours away from them in school. I can’t say I am any smarter but I hope in some way I am a better person now than when I began my nursing education experience 32 years ago.

(6)

Table of Contents

Chapter 1: Introduction ... 1

Background ... 1

Purpose ... 2

Conceptual Framework ... 3

Chapter 2: Literature Review ... 9

Academic and Non-Academic Factors ... 9

Attrition ... 11 Interviews ... 12 Chapter 3: Methods ... 15 Design... 15 Participants ... 15 Instruments ... 15 Procedure ... 15 Interview Component ... 17 Data Analysis ... 18 Chapter 4: Results ... 19 Chapter 5: Discussion ... 21 Implications ... 21 Limitations ... 23 Conclusions ... 24 References………..26 Appendix ... 29

Appendix A Institutional Review Board Approval ... 30

Appendix B Faculty Survey ... 31

(7)

List of Tables

Table 1. Cohort Acceptance Rates………..19 Table 2. Academic Indicator Means………....20 Table 3. Survey Item Response Mean………...20

(8)

Chapter 1: Introduction Background

Significant challenges exist for nursing education programs as they are responsible to prepare nursing students for licensure as well as ensure their clinical competence as graduate nurses. Concerns regarding a marked reduction in National Council Licensure Examination for Registered Nurses (NCLEX-RN) passing rates as well as poor performance observed in several first-year nursing students during the 2009 academic year resulted in active involvement by faculty members in the Utah Valley University (UVU) Department of Nursing to find appropriate solutions to quickly resolve these issues. Faculty members recognized the importance of honoring their commitment to prepare nurses to meet the needs of the local community in an ever-changing and complex health care environment despite the ongoing nurse shortage and limited program and clinical resources. Several possible solutions, including changes to the curriculum and program standards, were introduced formally and informally in faculty discussions.

Nursing program admissions was suggested as one of the areas possibly needing revision. Faculty voiced support for strengthening the undergraduate admission process by identifying and admitting only those applicants most likely to succeed in meeting program objectives, including first time passing of the NCLEX-RN. Faculty discussions led to a proposal to include applicant interviews to the admissions process; something that had never been done previously in the Department of Nursing. Proponents suggested that interviewing applicants would provide an opportunity for faculty members to meet and interact with program applicants for the purpose of assessing non-academic essential attributes not evident in the present admission criteria, which was based solely on

(9)

pre-requisite course grade point average (GPA) and scores on the Test of Essential Academic Skills (TEAS).

During the summer of 2010, with faculty, college, and university support, the UVU Department of Nursing implemented an interview component into the program admission process. The first nursing applicants to experience the change in admission process made up the Fall 2010 cohort.

Purpose

The UVU Department of Nursing faculty members who were responsible for the implementation of interviews into the admission process received Institutional Review Board (IRB) approval to conduct a 3-year longitudinal research study of the interview component to determine its role as a predictor of student success during the program and in taking the NCLEX-RN upon program completion (see Appendix A). That study is presently in progress. Using the data collected from the first four cohorts of students who were admitted to the UVU Nursing Program after the interview component was

implemented, the purpose of this particular preliminary study was to answer the

following questions: (a) Was there a difference in applicant ranking based on TEAS and GPA scores when compared to ranking based on the interview component scores? (b) Was there a difference in TEAS and GPA scores between applicants who were accepted into the nursing program and those not accepted based on the interview component? (c) What were the applicants’ perceptions of the interview component regarding the

(10)

Conceptual Framework

The majority of faculty members were supportive of adding interviews, but questions regarding the resources and costs required to complete the interview process as well as concerns about the validity of the interview data collected were raised. With enough interest and support behind making the change to the admission process, an investigation began, attempting to answer the questions that had arisen regarding the addition of applicant interviews. A few faculty members assigned to the Admissions, Progression, and Graduation Committee were given the responsibility to research

information regarding the development and use of applicant interviews as a component of the admissions process for the program. The following is an overview of the conceptual framework describing the steps taken in the implementation of the interview component into the UVU nursing admissions process.

Recognizing the impact the proposed change might have on faculty members as well as student applicants, the interview committee responsible for the implementation followed a process similar to that recommended by Grol and Wensing (2005) which described the steps to be taken when implementing changes in healthcare practice. Their model for effective implementation of change recommended consideration of the

following:

 Complexity of the practice to be changed with all its factors  Attention to the specific innovation or change to be implemented  Implementation using a step by step process

 Beginning implementation with a diagnostic or problem analysis

(11)

 Linking measures and strategies of making the change to the problem analysis  Multifaceted strategies for implementation

 Distinguish between strategies for dissemination and implementation  Plan to monitor ongoing progress of implementation

 Incorporation of implementation plan into established structures for professional development and quality management

According to Grol and Wensing (2005), changes in practice are more likely to be successful if the complexity of the practice, with all its key factors is considered.

Knowing this, a considerable amount of attention was paid to every detail of the admission process. Though the influence of tradition is very strong in the nursing department, the environment is supportive of learning and provides opportunities for change. Appreciating the seriousness of not meeting program outcomes provided the incentive as well as a supportive environment in which faculty members were willing to try something different. After the completion of four cohorts since the implementation of the applicant interviews, we continue to learn more about the application process and how to better serve the students.

One of the first steps taken in the implementation of the interview component was considering those most to be affected by the change. The change was certain to affect the faculty, as the proposed changes in the admission process needed to address the concerns they had initially voiced regarding qualified applicants to the program and how we could clearly identify those students most likely to be successful. The second group to be affected by the change was the applicants to the program who had to be guaranteed

(12)

the opportunity to represent themselves in the best and fairest way possible during the interview aspect of the admission process.

According to Grol, Bosch, Hulsher, Eccles, & Wensing (2007), communication theories address the importance of the exchange of credible information. In the

implementation of applicant interviews consideration was given regarding the critical nature of the information to be provided to the applicants. It was determined that they needed to have credible, succinct information in order to prepare themselves for their interview experience. The interview planners recognized the importance of providing the same amount and quality of information to all applicants to avoid any unfair advantages for any who would participate. This included simple and easy to understand instructions prior to, during, and following the interview experience. Faculty members received updated progress reports regarding the process to ensure their ongoing support.

The change model created by Grol and Wensing (2005) emphasized the importance of involving the entire target group to be affected, in order to secure their commitment and support for the change. Faculty involvement was one of the critical factors that would influence successful implementation of the interview component into the admission process. Active involvement would translate into faculty support as the change was likely to increase the complexity and subsequent workload associated with the admission of new nursing students. Planners felt strongly that one of the best ways to secure faculty member support was to seek their input regarding the specific applicant attributes to be evaluated through the proposed interviews. This input was obtained through a substantial brainstorming session at one faculty meeting. Faculty-identified attributes were then clustered into related groups called “Valued Characteristics” and

(13)

placed in a newly developed, five-point Likert scale survey to allow each faculty member to prioritize the importance of individual attributes and to identify attributes considered as critical criteria (see Appendix B). Analysis of the survey data clearly revealed a list of essential attributes and among those attributes, which ones were critical for success in nursing. Faculty input was regarded seriously and guided the development of all interview questions and activities.

According to Rogers (1995), theories of social network and influence take into account the importance of social networks and opinion leaders when making changes in practice. In the beginning stages of the project it was recognized that social influences would be significant in contributing to a successful implementation of the interview component of the admission process. It was believed that if faculty members who were involved initially had favorable experiences with the process, that they would have a positive influence on other faculty members for future participation.

Concerns related to maintaining the confidentiality and integrity of the interview component led to a decision to not seek input from students. However, a decision was made to survey students following their interview experience to determine their level of satisfaction with the process.

Referencing theories of leadership, Grol et al. (2007) stressed the importance of gaining support from those at the administrative and management levels, as well as those who might be considered informal leaders in the department. This is particularly true when asking for resources such as funding and/or personnel. Planners sought for and received approval from the department, college, and university administrative levels. This was necessary as funding was required to assist with the purchase of supplies and

(14)

refreshments initially and most recently, to include involvement in the interview component as faculty workload.

The Grol and Wensing (2005) model also emphasized the importance of linking all aspects of the proposed change to be implemented, to the problem addressed by the change. This suggested the need for refinement, meaning that each component of the interview process had to serve a clear purpose. The planners wanted to create a

component that was well-defined and would stand the tests of validity and reliability, as the seriousness of admission decisions was recognized.

Citing observational studies (Burgers et al., 2003; Foy et al., 2002; Grol et al., 1998). Grol et al. (2007) suggested improved compliance with changes in practice were more likely when changes “were based on evidence and compatible with existing values and that precisely defined the desired performance, that did not require new knowledge or skills, and that had limited consequences for management” (p. 99). Literature reviews and ongoing faculty input provided the evidence and support needed to proceed with implementation plans. Although the Department Chair did not have an active role in the actual implementation of the interview component, she was informed and consulted regularly. She monitored procedures closely and provided valuable input and approval as needed.

Lastly, one of the most important recommendations from Grol and Wensing (2005) was to closely monitor and evaluate changes in practice after their

implementation. Their recommendation included identifying and addressing factors that were likely to interfere with the success of the changes once implemented. The interview component of the admission process has been and will continue to be subjected to

(15)

rigorous scrutiny. Although a commitment was made early on in the implementation to limit major changes to the interview component until quantitative outcomes could be obtained, minor refinements have been made, such as editing rubrics to improve clarity.

(16)

Chapter 2: Literature Review

It became evident that there was a paucity of studies regarding nursing program admissions. Consequently, studies regarding admission to medical and other health-related programs of study were also reviewed. The development and institution of the applicant interviews into the admission process in the nursing program at UVU was based on the experience and research findings of other programs as expressed in the literature.

Academic and Non-Academic Factors

Many significant challenges exist for nursing education today. According to landmark Institute of Medicine reports, Crossing the Quality Chasm: A New Health System for the 21st Century (2001) and the Future of Nursing: Leading Change,

Advancing Health (2010), nursing should be leading the charge in making improvements being called for in the quality of healthcare as it has been and is presently being provided. Nursing education programs are responsible to prepare students not only for successful completion of the NCLEX-RN and licensure, but also to be clinically competent as graduate nurses (Ehrenfeld & Tabak, 2000). With many more well-qualified applicants than classroom space and clinical sites will allow, it is imperative that nursing education programs make every effort to recruit, retain, and graduate well qualified students to meet the expectations of all major stakeholders (Rosenberg, Perraud, & Willis, 2007; Sayles, Shelton, & Powell, 2003; Timer & Clousen, 2010). However, some suggest that with the ongoing nursing shortage, large applicant pools, and the ease of filling available seats with those who appear to be academically prepared, programs are more likely to fail to

(17)

take into account those attributes and attitudes recognized as most critical for those who will ultimately work in the nursing profession (Sadler, 2004; Trice & Foster, 2008).

To identify nursing program factors most predictive of successful completion of the NCLEX-RN, Grossbach and Kuncel (2011) recently published a meta-analysis showing that nursing program GPAs and standardized admission test scores are accurate predictors of NCLEX-RN success. They also found that preadmission GPAs were predictive but to a lesser extent than nursing program GPAs. This seems to confirm conclusions of others who have demonstrated that GPA is the most stable predictor of academic success in nursing as well as other health-related professional programs of study (Blackman, Hall, & Darmawan, 2007; Salvatori, 2001; Timer & Clauson, 2010).

Academic success and attaining licensure, both critical goals for nursing education, may not be enough to adequately respond to the demands of the healthcare environment and ultimately the consumers of healthcare services. Academics alone may not be the best measure of a graduate’s ability to provide competent, compassionate nursing care. According to Trice and Foster (2008), limiting admission criteria to academic factors alone may exclude applicants that are well qualified and will make contributions to the nursing profession, including those from minority groups. Regarding accepting students strictly on academics, Newton, Smith, and Moore (2007) stated, “Schools of nursing are harmed when they use policies that do not take student attributes into consideration when admission decisions are made” (pg. 443).

Despite the clear connection between academic factors and successful completion of nursing program requirements and licensure attainment, the assessment of factors that may predict the presence or potential for desirable, non-academic attributes in graduate

(18)

nurses has not been well studied. According to Timer and Clousen (2010) the valid predictability of supplemental application components beyond academic measures is equivocal and need to be the focus of more study. Grossbach and Kuncel (2011) report that much of the past research regarding the predictability of various admission factors as they relate to student success has been contradictory.

Attrition

Attrition continues to be a concern of nursing education programs today, wherein applicants meet academic criteria and yet fail to complete their courses of study for various reasons. The ability to account for the attrition experienced by nursing programs and predict those applicants less likely to complete all education requirements would be extremely valuable. Trice and Foster (2008) suggest that academic indicators cannot predict attrition attributed to “personal, non-academic factors such as stress or unrealistic expectations of nursing in general” (pg. 524). Admission interviews, commonly used in many professional level programs of study, have been suggested as a way to evaluate non-academic attributes such as values, levels of motivation and self-confidence, personality, and character, which may be more likely to contribute to attrition and/or clinical performance (Ehrenfeld & Tabak, 2000; Soule, 1995; Trice & Foster, 2008; Utterback, 1984). According to Ehrenfeld and Tabak (2000), over half of the attrition in one specific nursing program over a five-year period was attributed to non-academic factors such as behavior issues, personal problems, among others. Interviews were instituted as one attempt to identify factors more likely to contribute to attrition in that program. Although they were able to show a decrease in the level of attrition

(19)

Interviews

Interviews have been shown to provide a way to predict clinical performance and non-academic success because of their ability to measure key applicant attributes such as motivation, problem-solving, and personal goals (Carpio & Brown, 1993; Goho & Blackman, 2006; Karstadt, 2007; Lemay, Lockyer, Collin, & Brownell, 2007).

Interviews are not, however, without weaknesses or challenges despite their usefulness (Gorman, Monigatti, & Poole, 2008). One such challenge to interviewing applicants is the number of resources required, including faculty time (Ehrenfeld & Tabak, 2000; Trice & Foster, 2001). Who should conduct the interviews and how many applicants should be interviewed are difficult decisions to make. Another challenge is ensuring the validity of interviews as interviews are thought to be subject to evaluator bias (Gorman, Monigatti, & Poole, 2008) and the inability of interviewers to distinguish accurately between candidates, particularly those who are average and unacceptable, has been demonstrated by Standsfield and Kreiter (2007).

Salvatori (2001) and Ehrenfeld and Tabak (2000) suggest that the validity and reliability of interviews can be improved through evaluator training and the use of specific evaluation guidelines. The use of group interviews has also been suggested as a way to increase the objectivity of interviewing as well as provide an opportunity to observe applicants’ abilities to interact with others. Observations by Trice and Foster (2008) indicate that interviewing applicants in groups allows faculty to evaluate such things as interpersonal communication, leadership, and demeanor. In earlier studies, Fong and Leton (1978) reported advantages of group interviews including the use of fewer faculty resources, and the ability to measure applicant attributes not easily

(20)

measured in a structured one-on-one interview, such as the use of interpersonal or group skills.

Another option provided by some medical and nursing programs is offering multiple mini-interviews, which according to Eva, Reiter, Rosenfeld, and Norman (2001) and Eva, Rosenfeld, Reiter, and Norman (2004) are more reliable than traditional

interviews and have the ability to limit subjectivity and bias. Lemay, Lockyer, Collin, and Brownell (2007) more recently reported their findings from implementing multiple mini-interview assessment of medical school applicants and demonstrated the validity and reliability of using mini-interviews to determine which applicants were most qualified for medical school. Although the logistics associated with offering multiple mini-interviews are challenging according to McBurney and Carty (2009) the effort is worth the challenge as satisfaction of both interviewers and applicants were measured at high levels.

One must also consider the applicant’s view of a nursing program’s admission process. If the admission criteria are solely based on academics, those applicants with the best academic records are the ones admitted. On the other hand, if applicants are

provided an opportunity to interview and/or demonstrate through other measures their non-academic abilities and attributes, particularly for those who may not rank highest on the list academically, they may consider such an admission process more fair to those who really do want to be a nurse. Minimal research has been published regarding student views of admission processes. Trice and Foster (2008) report anecdotal evidence

following the addition of interviews to their admission process that suggests students are positive about meeting and interacting with other applicants and that they have the

(21)

opportunity to meet and interact with nursing faculty. Results from other studies

similarly confirm that students as well as faculty interviewers view admission interviews as satisfying, positive experiences (Harris & Owen, 2007; McNelis et al., 2010).

Based upon a thorough review of the literature, in the summer of 2010 an

interview component was implemented as one of three factors used to select students for entry into the undergraduate nursing program at UVU. In addition to the academic measures of GPA and TEAS, a select group of applicants were invited to participate in interview activities. The activities have been standardized to ensure that each candidate interviewed is evaluated by all interviewers, using rubrics to rate different levels of an attribute, all with the goal of providing an equal opportunity to be objectively evaluated. In addition to the academic requirements that candidates must meet, the implementation of an interview component has been viewed as a means of assisting nursing faculty in the selection of those applicants most likely to succeed in the program and who, not only possess the required academic competencies, but demonstrate the critical attributes essential for students and entry level nurses.

(22)

Chapter 3: Methods Design

This was a quasi-experimental one group before and after study. The independent variable for the study was the interview component. The dependent variable was the ranking of the applicants. The ranking based on GPA and TEAS scores was compared to the ranking of applicants following their participation in the interview component. Participants

Convenience sampling was used for this study as all participants were qualified students who submitted applications to the UVU Nursing Program for Fall 2010, Spring 2011, Fall 2011, and Spring 2012 cohorts. Qualified students were those who met all application criteria, including successful completion of prerequisite courses.

Instruments

A 14-item survey, the UVU Nursing Program Admission Interview Process Feedback Survey (see Appendix C)was created to gather information from participants regarding their interview experience. The content validity of the instrument was ensured through review by interviewers and subsequent revisions made by the planners of the interview component. Each item was a Likert scale type with four options: Strongly Agree, Agree, Disagree, and Strongly Disagree. Responses to survey item 10, “The likelihood of my being accepted into the nursing program is higher due to the interview process” were used to answer research question 3.

Procedure

All potential nursing students who met application requirements were ranked based on a composite score of prerequisite course GPAs and the TEAS scores. It was

(23)

determined that the top 80-90 students would be invited to interview, however, because of a change in a limit for repeating courses and number of attempts on the TEAS, the applicants for the Fall 2010 cohort totaled 56 and the Spring 2011 cohort totaled 43. In both cases all qualified applicants were interviewed. Cohorts three and four had

sufficient applicants to limit interviews to the top 80-90. Each applicant was personally contacted by the Department of Nursing Academic Advisor to establish an interview appointment on days pre-designated as interview days. Information was sent to applicants with appointments, regarding location, date and time of interviews in the application instructions. Information included instructions to applicants to prepare for their interview appointments as they would for a professional job interview. Instructions also included a recommendation for applicants to review the UVU Nursing Program Mission and Values statements posted on the Department of Nursing website.

Prior to interview days, face to face and written training were provided for faculty members acting as interviewers/evaluators in the use of standardized measures in rubric form to ensure evaluations were made as objective as possible for of every applicant in each of the interview activities.

Scores for each applicant were entered into an Excel spreadsheet where they had been assigned a specific weighting, determined by the attributes’ level of importance as rated by faculty surveys. All scores were verified and checked for accuracy with data entered by multiple reviewers. Each applicant’s total score resulted in her/his ranking in the pool of applicants who participated in the interview component. Applicants were offered positions into the program as determined by their ranking on the list and the total number of allowable positions available.

(24)

Interview Component

Applicants reported to the interview site in groups of six. When applicants arrived for their interview appointments, each was greeted by a faculty or staff member and provided with an overview of the interview activities in which they would be participating. Each applicant participated in four separate activities that made up the interview component of admissions, lasting approximately two hours. The first was a group activity in which four to six applicants were given a problem to solve together. Two nursing faculty members were assigned to the activity, one to facilitate and the other to evaluate the applicants’ participation. The second activity was performed individually wherein each applicant was provided with a brief scenario and questions to which she/he was required to respond. The third activity was rotating through three, 8-minute mini-interviews, each one with a different nursing faculty member. The fourth and final activity had each applicant provide a hand-written response to a spontaneous essay question on a topic based on nursing values.

The multiple interactions/activities, all evaluated by faculty members, provided applicants with an opportunity to demonstrate their ability to problem solve and

communicate with other applicants and faculty members. Upon completion of the four activities, applicants were provided an opportunity to complete a survey regarding their experience with the interview activities. The survey face sheet acted as a consent form, requesting consent from the applicants to use the data collected from the interview component as well as their feedback regarding their interview experience in order to make improvements for future admission processes. Upon completion of the survey they

(25)

were thanked for participating and handed an information sheet which explained when and how notification would be made regarding acceptance into the program.

Data Analysis

Data analysis was performed using IBM Statistical Package for the Social

Sciences (SPSS), (Version 19). The answer to research question 1 was based on rankings of applicants before and after participating in the interviews, using data obtained from student applications and interview component scores. The before ranking listed applicants from highest to lowest according to their academic scores obtained by combining GPAs and TEAS scores, with GPA weighted at 60% and the TEAS at 40%. Similarly, the after ranking listed applicants from highest to lowest according to their interview scores obtained by adding points earned in each of the separate interview component activities. The ranking data was analyzed using the Wilcoxon signed-rank test.

Research question 2 was answered by comparing average GPA and TEAS scores of students accepted into the nursing program, to those students interviewed but not accepted, using the t-test for independent means.

Research question 3 was answered using descriptive data obtained from surveys completed by applicants following their interview experience. Using a scale of one to four, with 1 = Strongly disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly agree, the mean for all responses to survey item 10 was obtained.

(26)

Chapter 4: Results

There were 340 participants total in the four cohorts. The majority were

Caucasian, with 247 (73%) female and 93 (27%) male. Table 1 shows the breakdown of each cohort including the number of applicants interviewed and the corresponding acceptance rates.

Table 1. Cohort Acceptance Rates

Cohort Fall 2010 Spring 2011 Fall 2011 Spring 2012

Applied 56 43 136 105

Interviewed 56 43 79 64

Accepted 51 (91%) 38 (88%) 38 (48%) 40 (63%)

There was a significant difference between student rank based on prerequisite academic performance and the rank based on the interview component scores, p = .000, alpha = .05, CI = 95%. This result demonstrates the ability of the interview component to identify applicants with the essential, non-academic attributes as previously identified by UVU Department of Nursing faculty members.

There was a significant difference found in average GPA between interviewed applicants who were accepted into the nursing program and those who were not, t = 3.188, df = 240, p = .002. The t test examining scores on the TEAS between the two groups showed no significant difference, t = -.487, df = 240, p = .627. Although no difference in average TEAS scores between groups seems to support the answer to research question 1, further showing that there is no difference in academic qualifications between those accepted and those not, the difference seen in average GPAs between groups suggests otherwise. Table 2 shows the mean GPA and TEAS scores with respective ranges for applicants accepted and those not accepted.

(27)

Table 2. Academic Indicator Means

Academic Indicator All Applicants Accepted Not Accepted GPA 3.75 (3.16 - 4.00) 3.79 (3.26 – 4.00) 3.85 (3.47 – 4.00) TEAS 82.33 (57 - 96) 84.59 (63.30 – 96) 84.22 (68.20 - 93)

In response to survey item 10, the applicants’ perceptions of the interview component regarding the likelihood of their being accepted into the program were positive as their average response was 3.2 out of four, meaning that applicants did perceive that the likelihood of being accepted into the nursing program was increased because of their participation in the interview component. Table 3 shows the mean response score for all survey items.

Table 3. Survey Item Response Means

UVU Nursing Program Admission Interview Process Feedback Survey Items Mean

Scores 1. The email communication I received regarding the interview process clearly stated

where and when my interviews would occur.

3.90

2. The location for the interview process was appropriate. 3.92

3. The interview process appeared to be well organized. 3.80

4. Nursing Department staff members who facilitated during the interview process were helpful and welcoming.

3.81

5. Instructions I received during the interview process were clearly written. 3.62

6. The interviewers and staff were receptive to answering my questions during the process.

3.79

7. I was treated fairly by the interviewers. 3.79

8. The interview questions were clear. 3.26

9. The interview process provided an adequate opportunity to represent myself to the nursing department.

3.28 10. The likelihood of my being accepted into the nursing program is higher due to

the interview process.

3.22 11. I believe that interviews should be one of the criteria used to determine admittance

into the nursing program.

3.50

12. I prepared myself adequately for the interview process. 3.46

13. My preparation for the interview process included seeking information about what to expect at a professional job interview.

3.38

(28)

Chapter 5: Discussion Implications

Individuals in the first four cohorts of applicants, who were not accepted on their first attempt, were allowed one additional opportunity to apply. There were some repeat applicants who were accepted after their second attempt. This could imply that the initial exposure to the interview component prepared them for the second attempt, possibly providing them with an advantage over first time applicants. However not all second time applicants were accepted. This may imply that the interview component had the ability to identify repeat applicants who lacked the essential attributes for which it was designed. To control for this problem, different faculty members were selected to participate as interviewers when known repeat applicants would be interviewed. Further study to evaluate reliability is recommended.

With the limited data collected over the first four cohorts of this study, it appears that the interview component’s ability to identify applicants with essential attributes in applicants may have had an influence on the attrition rates of the nursing program at UVU similar to other programs who have instituted interview components into their admission processes. Ehrenfeld and Tabak (2000) as well as Trice and Foster (2008) have reported reductions in nursing program attrition attributed to non-academic factors. Data collected for the UVU Nursing Program from Spring 2006 through Spring 2010 show attrition rates ranging from 2% to 12% (Farmer, 2011). The attrition rate since the implementation of the interview component beginning in Fall 2010 has been less than 1%, with only one student leaving the program through Spring 2012. Monitoring

(29)

attrition rates, including the reasons for a student leaving the program, is recommended as another area needing further study.

Despite having the lowest response mean of all the survey items, the overall positive response to survey item 10, “The likelihood of my being accepted into the nursing program is higher due to the interview process”, implies that applicants favored participating in the interview component of the admission process as it provided them an opportunity to present themselves in person and interact with faculty, which they

perceived to increase their likelihood of being accepted. Similar positive response means as expressed in Table 3 for survey item 9, “The interview process provided an adequate opportunity to represent myself to the nursing department” and survey item 11, “I believe that interviews should be one of the criteria used to determine admittance into the nursing program”, support applicant perceptions that the interview process was a favorable factor increasing the likelihood of acceptance into the nursing program. These results are similar to findings of Ehrenfeld and Tabak (2000) who found that most

applicants to their nursing program expressed satisfaction with opportunities to interview as part of their admission process. Positive response means for the remainder of the survey questions imply that the organization and administration details of the interview component were also perceived favorably by the applicants, which is reassuring to those faculty members involved who have tried to create the best interview environment possible for participating applicants. This is congruent with the observations of

McBurney and Carty (2009) who found that applicants who participated in multiple mini-interviews reported positive perceptions of their interview experiences, including “97 per cent [who] reported that interviewers put them at ease” during their experience (pg. 9).

(30)

Further research is recommended to test the reliability of the interview component in identifying the attributes thought to be essential and predictive of nursing student success. Another area needing further research is inter-rater reliability. Standardization of the rubrics, pre-interview training, and having every applicant interviewed by every interviewer will likely contribute to a higher level of consistency among interviewers. Eva, Rosenfeld, Reiter, & Norman (2004) suggest that these types of controls may contribute to consistency, but cannot guarantee it.

Limitations

As this was a preliminary study, the primary limitation of this study was the number of cohorts and applicants from which data were obtained. The number of applicants to the UVU Nursing Program for the first and second cohorts, 56 and 43 respectively, were fewer than the number of applicants for the third and fourth cohorts due to changes in application criteria specific to prerequisite courses and TEAS. As the number of applicants was small enough in each of the first two cohorts, all qualified applicants were invited to participate in the interview component and with few exceptions all were accepted. A decision was made by the researcher to not include the data

obtained for the first two cohorts in answering research questions 1 and 2 to prevent skewing the data, leading to erroneous results.

Another limitation of this study was the lack of data related to student outcomes. The specific research questions addressed by this preliminary study did not address student outcomes. A longitudinal three-year study of the interview component that is in progress at the time of this study will be evaluating the influence of the interview component in terms of student outcomes. A meta-analysis conducted by Goho and

(31)

Blackman (2006) demonstrated that admission interviews for health-related professional programs “weakly” predicted academic success but there was a “modest positive

correlation” between interview performance and clinical performance (pg. 339). Conclusions

The results of this study support the conclusion that the interview component as implemented by the UVU Department of Nursing resulted in a significant change in the rankings of student applicants, accepting some students into the nursing program who would not have qualified on academics alone and also eliminating other students who would have been admitted solely on academics. This suggests that the interview

component was able to effectively identify student applicants with the attributes thought to be essential by faculty in terms of contributing to success. These results confirm findings of Rosenberg, Perraud, and Willis (2007) that demonstrated the ability of admission interviews to identify applicants who appeared to be academically prepared but were not accepted due to non-academic issues measured in interviews. Lemay, Lockyer, Collin, and Brownell (2007) likewise demonstrated the ability of medical school admission interviews, specifically multiple mini-interviews to differentiate applicants in terms of non-academic qualities and attributes believed to be important in addition to academics.

The results of this preliminary study with its limited scope suggest that the initial implementation of the interview component into the UVU Nursing Program admission process has been successful. Further testing is recommended to determine its validity and reliability, and ultimately its ability to predict student success. The question which remains to be answered is if the interview component of the UVU Nursing Program

(32)

admission process is able to identify those students with attributes essential to their success not only in terms of NCLEX-RN passing but in attaining desirable levels of clinical competency as graduate nurses.

(33)

References

Blackman, I., Hall, M., & Darmawan, I. (2007). Undergraduate Nurse Variables that Predict Academic Achievement and Clinical Competence in Nursing.

International Education Journal, 8(2), 222-236.

Burgers, J., Grol, R., Zaat, J., Spies, T., Van der Bij, A., & Mokkink, H. (2003). Characteristics of effective clinical guidelines for general practice. British Journal of General Practice, 53, 15-19.

Carpio, B., & Brown, B. (1993). The admissions process of a bachelor of science in nursing program: initial reliability and validity of the personal interview. Canadian Journal of Nursing Research, 25(3), 41-52.

Ehrenfeld, M., & Tabak, N. (2000). Value of admission interviews in selecting of

undergraduate nursing students. Journal of Nursing Management, 8(2), 101-106. Eva, K., Rosenfeld, J., Reiter, H., & Norman, G. (2004). Original paper; An admissions

OSCE: the multiple mini-interview. Medical Education, 38(3), 314-326. Retrieved from EBSCOhost.

Eva, K.W., Reiter, H. I., Rosenfeld, J. & Norman, G.R. (2001). The relationship between interviewers’ characteristics and ratings assigned during a multiple

mini-interview. Academic Medicine, 79 (6), 602-609.

Farmer, S. (2011). Follow-up report for the National League of Nursing Accrediting Commission. Utah Valley University, 1-233.

Fong, M. L., & Leton, J. B. (1978). "The group interview: a tool for nursing student selection. University of Hawaii." Journal of Nursing Education 17, 35-40. Foy, R., Maclennan, G., Grimshaw, J., Penney, G., Campbell, M., Grol, R. (2002).

Attributes of clinical recommendations that influence change in practice following audit and feedback. Journal of Clinical Epidemiology, 55, 717-722. Goho, J., & Blackman, A. (2006). The effectiveness of academic admission interviews:

an exploratory meta-analysis. Medical Teacher, 28(4), 335-340.

Gorman, D., Monigatti, J., & Poole, P. (2008). On the case for an interview in medical student selection. Internal Medicine Journal, 38(8), 621-623.

Grol, R., Dalhuizen, J., Thomas, S., Veld, C., Rutten, G., & Mokkink, H. (1998). Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. British Medical Journal, 317, 858-861.

(34)

Grol, R. & Wensing, M. (2005). Effective implementation: a model. In Grol, R., Wensing, M., & Eccles, M. (Eds.). Improving patient care; the implementation of

change in clinical practice, (pp. 41- 58). Oxford: Elsevier.

Grol, R. M., Bosch, M. C., Hulscher, M. L., Eccles, M. P., & Wensing, M. (2007). Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Quarterly, 85(1), 93-138.

Grossbach, A., & Kuncel, N. (2011). The predictive validity of nursing admission measures for performance on the National Council Licensure Examination: a meta-analysis. Journal of Professional Nursing: Official Journal of the American Association of Colleges of Nursing, 27(2), 124-128.

Harris, S. & Owen, C. (2007). Discerning quality: using the multiple mini-interview in student selection for the Australian National University Medical School. Medical Education, 41, 234-241.

Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. (2001). Retrieved from

http://www.nap.edu/html/quality_chasm/reportbrief.pdf

Institute of Medicine. Future of Nursing: Leading Change, Advancing Health. (2010). Retrieved from http://www.iom.edu/Reports/2010/The-Future-of-Nursing-Leading-Change-Advancing-Health.aspx

Karstadt, L. (2007). Education matters. Requirements of the 'would be' nurse. British Journal of Nursing, 16(20), 1253.

Lemay, J., Lockyer, J., Collin, V., & Brownell, A. (2007). Assessment of non-cognitive traits through the admissions multiple mini-interview. Medical Education, 41(6), 573-579.

McBurney, S., & Carty, E. (2009). Promising practices. Using multiple mini-interviews to assess nursing school applicants. Canadian Nurse, 105(1), 8-10.

McNelis, A. M., Wellman, D. S., Krothe, J. S., Hrisomalos, D. D., McElveen, J. L., South, R. J. (2010). Revision and evaluation of the Indiana University School of Nursing baccalaureate admission process. Journal of Professional Nursing, 26(3), 188-195.

Newton, S., Smith, L., & Moore, G. (2007). Baccalaureate nursing program admission policies: promoting success or facilitating failure? Journal of Nursing Education, 46(10), 439-444.

(35)

Rosenberg, L., Perraud, S., & Willis, L. (2007). Educational innovations. The value of admission interviews in selecting accelerated second-degree baccalaureate nursing students. Journal of Nursing Education, 46(9), 413-416.

Sadler, J. (2004). Descriptions of caring uncovered in students' baccalaureate program admission essays. International Journal for Human Caring, 8(3), 37-46. Salvatori, P. (2001). Reliability and validity of admissions tools used to select students

for the health professions. Advances In Health Sciences Education: Theory And Practice, 6(2), 159-175.

Sayles, S., Shelton, D., & Powell, H. (2003). Predictors of success in nursing education. ABNF Journal, 14(6), 116-120.

Soule, E. (1995). Interviewing 101. The Journal of College Admission, 149, 8-11. Stansfield, R., & Kreiter, C. (2007). Conditional reliability of admissions interview

ratings: extreme ratings are the most informative. Medical Education, 41(1), 32-38.

Timer, J. & Clauson, M. (2010). The use of selective admissions tools to predict

students’ success in an advanced standing baccalaureate nursing program. Nurse Education Today, doi: 10.1016/j.nedt.2010.10.15.

Trice, L., & Foster, P. (2008). Improving nursing school diversity through use of a group admission interview. AORN Journal, 87(3), 522.

Utterback, A. S. (1984). College Admissions Face to Face: Prepare Yourself for Interviews and Campus Visits. Retrieved from EBSCOhost.

(36)

Appendix

(37)

Appendix A

(38)

Appendix B Faculty Survey Valued Characteristic Attributes of the Characteristic 1 = unimportant 2 = of little importance 3 = moderately important 4 = important 5 = very important Critical Criteria (Place an X if yes) Comments 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

(39)

Appendix C

UVU Nursing Program Admission Interview Process Feedback Survey In order to ensure anonymity, please do not include any personal identification

information on this survey form. Your responses will be kept confidential and will not be linked to your interview results. Please score the statements below by circling the

appropriate number, using the following scale: 4 = Strongly agree

3 = Agree 2 = Disagree

1 = Strongly disagree

1. The email communication I received regarding the interview process clearly stated where and when my interviews would occur.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

2. The location for the interview process was appropriate.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

3. The interview process appeared to be well organized.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

4. Nursing Department staff members who facilitated during the interview process were helpful and welcoming.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

5. Instructions I received during the interview process were clearly written.

Strongly agree Agree Disagree Strongly disagree

(40)

6. The interviewers and staff were receptive to answering my questions during the process.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

7. I was treated fairly by the interviewers.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

8. The interview questions were clear.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

9. The interview process provided an adequate opportunity to represent myself to the nursing department.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

10. The likelihood of my being accepted into the nursing program is higher due to the interview process.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

11. I believe that interviews should be one of the criteria used to determine admittance into the nursing program.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

12. I prepared myself adequately for the interview process.

Strongly agree Agree Disagree Strongly disagree

(41)

13. My preparation for the interview process included seeking information about what to expect at a professional job interview.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

14. I learned something new about myself through this experience.

Strongly agree Agree Disagree Strongly disagree

4 3 2 1

Please provide any comments regarding your experience with the interview process that you think we should know:

(42)

© 2012

Dale Moser Maughan ALL RIGHTS RESERVED

http://www.nap.edu/html/quality_chasm/reportbrief.pdf Leading-Change-Advancing-H

References

Related documents

estimate di¤erences in the use of temporary employment contracts for …rms in industries with intrinsically higher dependence on external …nance, or lower access to trade

feasibility of implementing programs aimed at increasing retailer compliance with sales- to-minors laws. In contrast, they point to the availability of several proven tobacco

4b (again, vehicle icons are not drawn to scale). This area was approximately 2. We analyzed scenarios with different vehicle densities per square kilometer, as specified in Table

Evergreen (or auto-renewal) clauses are in a state of constant evolution as litigation over their mechanics motivates more detailed clauses. The issuer does not issue

For the poorest farmers in eastern India, then, the benefits of groundwater irrigation have come through three routes: in large part, through purchased pump irrigation and, in a

Madeleine’s “belief” that she is Carlotta Valdez, her death and rebirth as Judy (and then Madeleine again) and Scottie’s mental rebirth after his breakdown.. All of these

(i) Explain why more carbon dioxide is produced when the complete homogenate is incubated with just glucose or pyruvate than when cyanide is