Effective use of interpreters by family nurse practitioner
students: Is didactic curriculum enough?
Susanne J. Phillips,MSN, RN, FNP-BC (Associate Professor)1, Desiree Lie,MEd, MD (Professor)2, Jennifer Encinas,
BA (Statistician)3,Carol Sue Ahearn,BSN, RN (Director)3, & Susan Tiso,MN, RN, FNP-BC (Associate Professor)1 1Program in Nursing Science, University of California, Irvine, California
2Department of Family Medicine, School of Medicine, University of California, Irvine, California 3Clinical Skills Center, School of Medicine, University of California, Irvine, California
Keywords
Education; nurse practitioners; students; communication.
Correspondence
Susanne J. Phillips, MSN, RN, FNP-BC, Program in Nursing Science, University of California, 209 Irvine Hall, Irvine, CA 92697.
Tel: 949-824-4274; Fax: 949-824-0470; E-mail: sjphilli@uci.edu Received: June 2009; accepted: January 2010 doi: 10.1111/j.1745-7599.2011.00612.x Abstract
Purpose:Nurse practitioners (NPs) care for patients with limited English pro-ficiency (LEP). However, NP education for improving communication in inter-preted encounters is not well reported. We report a single school study using standardized encounters within a clinical practice examination (CPX) to assess the adequacy of current curriculum.
Data sources: Entering family NP (FNP) students (n = 26) participated in a baseline CPX case. They were assessed by standardized patients using the validated Interpreter Impact Rating Scale (IIRS) and Physician-Patient Interac-tion (PPI) scale, and by interpreters using the Interpreter Scale (IS).The case was re-administered to 31 graduating students following completion of exist-ing curriculum. Primary outcome was aggregate change in skills comprisexist-ing global IIRS, PPI and IS scores. Pre- and post-performance data were available for one class of 10 students. Secondary outcome was change in skill scores for this class.
Conclusions: Mean aggregate global scores showed no significant improve-ment between scores at entry and graduation. For 10 students with pre- and post-performance data, there was no improvement in skill scores for any mea-sure. Skill assessed on one measure worsened.
Implications for practice:FNP students show no improvement in skills in working with interpreters with the current curriculum. An enhanced curricu-lum is needed.
Nurse practitioners (NPs) are important providers of primary care to underserved and ethnically diverse populations in the United States (U.S. Government Accountability Office [US GAO], 2008). With an increas-ing number of lincreas-inguistically and culturally diverse pa-tients seeking primary care within the U.S. healthcare system (Hayakawa, 1982; U.S. Census Bureau, 2000, 2004), there is a growing need for healthcare providers to be formally trained in working effectively with med-ical interpreters (Civil Rights Act, 1964; Grubbs, Chen, Bindman, Vittinghoff, & Fernandez, 2006; U.S. Depart-ment of Health and Human Services, Office of Minority Health, 2000). Studies have shown that the use of med-ical interpreters in the healthcare setting reduces health disparities (Flores, 2005; Karliner, Perez-Stable, &
Gilden-gorin, 2004). Cultural and ethnic competency standards for NPs emphasize the need to be sensitive to and in-crease communication with ethnically diverse popula-tions, highlighting the need to provide such training in NP programs (U.S. Department of Health and Human Services [US DHHS], 2002).
Professional nursing practice is founded on the prin-ciples of maintaining respect for the cultural, racial, and ethnic diversity of patients (American Nurses As-sociation, 2004). In 1990, the National Organization of Nurse Practitioner Faculties and the American Associa-tion of Colleges of Nursing developed domains and core competencies of NP practice in response to federal concerns for improving quality of educational pro-grams supported by Title VIII, Nursing Workforce
Development (Department of Health and Human Services, 2002). Included in the competencies are essen-tial behaviors that must be demonstrated upon gradua-tion from an NP educagradua-tional program. Two of these es-sential behaviors, defined within Domain VII: Cultural Competence (US DHHS, 2002), require the NP to demon-strate culturally sensitive care and to assist patients of di-verse cultures to access quality care. Appropriate use of medical interpreters assists NPs in achieving this standard. Despite the description of formal competencies for the provision of culturally and ethnically appropriate care in NP education, there is a lack of research in the nursing lit-erature that assesses the need for and the effectiveness of formalized curricula in working effectively with medical interpreters. Until recently, validated assessment tools for evaluating these specific communication skills have been lacking, limiting the ability to conduct pre- to postassess-ment of skills around newly introduced curricula.
With recent access to validated assessment tools for rating encounters involving interpreters, we con-ducted a needs assessment to examine skills of family nurse practitioner (FNP) students for working effectively with interpreters at entry into the program versus at graduation, to determine the effectiveness of our cur-rent didactic curriculum and to assess the need for curricular enhancement. We hypothesized that with-out a skills-based competency component in the current didactic curriculum, over a 2-year program, FNP students would not demonstrate improvement in these skills as as-sessed by standardized patients (SPs) and standardized in-terpreters (SIs). Our study, conducted from 2006 to 2008, was approved by the Institution’s Review Board.
Methods
Curriculum
The FNP program at the University of California, Irvine, was, until 2009, comprised of a postmaster’s certificate track and MSN/FNP program in partnership with the Cal-ifornia State University at Fullerton (nb: in 2009 it be-came an independent academic program at UCI). Each year, since 1995, 7–15 students were enrolled in the pro-gram. The curriculum requirements are 45 semester units in the postmaster’s track and 64 semester units in the MSN/FNP track. Included within the total units is 720 h of clinical internship. Assessment of FNP student skills is conducted by a variety of methods, including written ex-aminations, case presentations, direct observation of per-formance, a baseline skills-appraisal assessment (SKA), a mid-program objective structured clinical examination (OSCE), and a final multistation clinical practice exami-nation (CPX) conducted prior to graduation. At the time
Table 1 Student classes for data analysis
Characteristics Total,n(%)
Student gender –
Male 4, (7.0)
Female 53, (93.0)
Student class size by graduating year –
2006 12
2007 7
2008∗ 13
2009 15
Student level (aggregate) –
FNP1 26
FNP2 31
∗Subgroup with complete pre/post data.
of the study, there was no formal skills-based or clini-cal curriculum for working with interpreters. FNP stu-dents received a didactic session lasting 2.5 h in their ”Vulnerable Populations” course that covered content re-lated to culture, communication, and use of interpreters, including legal requirements for the use of interpreters. The curriculum, designed by the course instructor, in-cluded learning activities such as assigned reading, lec-ture/discussion, role-playing and observation, and video case presentations with critique, which covered impact of body language, cultural sensitivity, and styles of commu-nication. Because of the demographics of patients served in our local community with limited English proficiency (predominantly Latino with Vietnamese and other Asian subgroups), and the low level of Spanish and other lan-guage fluency among our FNP students, a curriculum to teach skills for working with interpreters was considered a significant program need.
Study participants
Participants were FNP students in four successive classes (see Table 1) attending our FNP Program who ma-triculated in the year 2004–2008, some of whom gradu-ated in 2006–2008. Students were assessed in a baseline and a graduation CPX and their skills for working with interpreters rated by trained SPs and SIs who performed in a standardized case. Because of the small numbers of students (range 7–15 in each class), aggregate data from four successive classes were used.
Interpreter CPX station
The 15-min interpreter station was administered within a multistation OSCE-type CPX. The OSCE is a set of stan-dardized clinical encounters using trained SPs to rate stu-dents in their clinical skills and competencies. Skills that
are assessed using OSCEs include history taking, commu-nication, physical examination, and counseling (Harden & Gleeson, 1979; Van der Vleuten & Swanson, 1990). The interpreter case involved a monolingual Spanish-speaking patient requiring counseling in smoking cessa-tion. Spanish language was chosen because of local com-munity demographics. To prepare for their roles, both the SPs and the SIs (actors who were bilingual in Span-ish and EnglSpan-ish) received four 2-h training sessions as a pair by English-speaking trainers. The training sessions included learning and rehearsing the case study, role-playing, and review of recorded encounters. One SP–SI pair was used to standardize case performance by having them rate videotaped encounters prior to the examina-tion. Extensive anchors were developed for the behaviors being rated and used for training. While training, trainers directly observed and critiqued the performance of the SPs and interpreters both individually and as a pair in their dual purpose roles as case actors and student raters. All raters were required to consistently score recorded training encounters with an error rate of no more than one error per encounter (Lie et al., 2007, 2009).
Following the CPX, SPs completed both the Interpreter Impact Rating Scale (IIRS) and Patient–Physician action Scale (PPI). Interpreters completed only the Inter-preter Scale (IS). FNP students completed a self-rating of their performance immediately after the encounter out-side the room. Over the 2-year study, none of the FNP students had Spanish fluency sufficient to conduct med-ical interviews in Spanish, by self-report. Because some FNP students had limited Spanish proficiency, the station instruction specified that students were being assessed for their ability to perform in an interpreted encounter and students should not use Spanish to conduct the interview even if they had some Spanish proficiency.
Outcome measures-rating scales
The IIRS is a validated scale assessing communication in an interpreted encounter from the patient’s perspec-tive (Lie et al., 2007). It consists of six items, including eye contact, addressing patient concerns, acknowledging patient beliefs, addressing patient in first person, seating at comfortable distance, and nonverbal language, describ-ing verbal and nonverbal behaviors. A 5-point Likert scale (from 1 = marginal/low to 5 = outstanding performance) was used to rate each behavior. The global measure rating overall satisfaction of the patient (item 7 on the IIRS) was found to be predictive of performance (Lie et al., 2007) and thus used for our study.
The IS is a similar validated measure, rating provider communication performance in an interpreted encounter from the interpreter’s perspective, using a similar 5-point
Likert scale for each behavior (Lie et al., 2009). It consists of 12 behavior items, with the first four rating the ability of the provider to ”set the stage” for the encounter and the remaining eight items rating communication verbal and nonverbal behaviors, including seating, eye contact, not interrupting, clarifying statements, correcting mis-takes, use of first person for patient, summarizing and closing the interview, and staying on track. The global measure (item 13 on the IS) of overall satisfaction was found to be predictive of overall performance in an in-terpreted encounter and was thus used for our study (Lie et al., 2009).
The PPI, performed by the SP, is a validated seven-item rating scale using a 6-point Likert scale (ranging from 1=
unacceptableto 6 = outstanding) to measure the compe-tency of communication in the context of professional behavior (Makoul, 2001). The seven items are:appeared professionally competent, effectively gathered information, lis-tened actively, established personal rapport, appropriately ex-plored my perspective, addressed my feelings,andmet my needs. The mean of the summed score of the PPI (maximum of 7) was used to rate overall communication skills in the FNP students.
For the self-assessment, FNP students were asked to rate their own skills for 11 items on a 6-point Likert scale (from 1=very poorto 6=excellent). The behaviors they self-rated included ability to involve the interpreter, elicit patient’s cultural beliefs, work effectively with the inter-preter, focus on the patient, assess accuracy of interpreta-tion, set boundaries, and the perceived satisfaction of the patient and interpreter with their communication skills (Lie et al., 2007). Their overall self-assessed skill for in-teracting with the patient (”How satisfied do you think the patient was with how you involved the interpreter during this encounter?”) was used as a comparison with the patient (IIRS 7) and interpreter (IS 13) measures.
Descriptive statistics were used. We calculated the mean score for each measure (IIRS, IS, self and the mean of summed PPI scores) at program entry and just prior to graduation for the CPX interpreter station. We first com-pared the mean scores for the aggregate of entering ver-sus graduating students. We then performed a subgroup analysis for the single FNP class (class of 2008) for which we had both entering and graduating data (n =10), to assess if our aggregated results were supported by those from a single longitudinal cohort followed from enroll-ment to graduation.
Statistical analysis was performed using the software package JMP version 7.0.1 (SAS Institute Inc., Cary, NC). A one-way analysis of variance with similar group sizes was conducted to determine if differences existed within the groups being compared. The Student’st-test was used for comparisons of each pair. A value ofp<.05 was used
as the standard for statistical significance for paired com-parisons.
Because this was a needs assessment study, FNP stu-dents were not given feedback by faculty observers about their individual performance after the baseline CPX sta-tion. They also did not receive additional training based on their performance, for interacting with interpreters.
Results
Over the 2 years of the study, complete skills data were available for an aggregate of 26 entering and 31 gradu-ating FNP students (Table 1) who matriculated between 2004 and 2008. Data for 5 student encounters in the in-terpreter stations for 2006 are missing as a result of tech-nical problems with audio recording. These students did not differ in mean age or experience from students in-cluded in this study.
Of FNP students whose scores were included for anal-ysis, 90% were female. Mean age was 39.7 (range 29–55 years). Both baseline and graduating information was available for one class of 13 FNP students matriculating in 2006 and graduating in 2008, with a mean age of 43.5 years. However for this class, two students graduated early and one student extended her program of study, leaving 10 students with both pre- and post-training data for the CPX interpreter station in this cohort.
Aggregate data for entering and graduating classes
Overall performance as assessed by the SPs on the IIRS global score was average, with a mean score of 3.35 for the entering class versus a lower score of 2.87 of 5 for the graduating class (difference of –0.48 points,p=.054; see Table 2). Performance as assessed by the SI using the IS global score improved from a mean of 3.31 for the
en-Table 2 Aggregate comparison-–skill scores for FNPs 2006–2008
FNP1 FNP2 Mean Difference
(n=26) (n=31) Entering-graduating p-value
Scale Total,n(%) Mean, SD (95% CI)
IIRS 3.35 (0.89) 2.87 (0.92) −0.47 (−0.01; 0.96) .0542 IS 3.31 (0.74) 3.77 (0.67) +0.46 (0.09; 0.84) .0152∗ Self 5.38 (0.57) 5.42 (0.56) +0.04 (−0.27; 0.34) .8187 PPI 4.78 (0.91) 4.90 (0.85) +0.12 (−0.35; 0.59) .6144 ∗p<.05.
IIRS=Interpreter Impact Rating Scale, scored on 5-point Likert scale (1=
marginal/lowto 5=outstanding).
IS=Interpreter Scale, scored on 5-point Likert scale (1=marginal/lowto
5=outstanding).
Self=Self-rating of perceived patient satisfaction with provider, score on 6-point rating scale (1=very poorto 6=excellent).
PPI=summed score of 7 items with maximum score of 7.
Table 3 Pre to post skills scores for single FNP class of 2008
Pre Post Mean difference
(n=10) (n=10) entering-graduating p-value
Scale Total,n(%) Mean,SD (95% CI)
IIRS 3.90 (0.74) 2.90 (0.57) −1.00 (0.38; 1.62) .0032∗ IS 3.20 (0.63) 3.50 (0.53) +0.30 (−0.25; 0.85) .2643 Self 5.40 (0.52) 5.30 (0.48) −0.10 (−0.37; 0.57) .6601 PPI 5.34 (0.49) 5.34 (0.21) 0.00 (−0.35; 0.35) 1.0000 ∗p<0.005.
IIRS=Interpreter Impact Rating Scale, scored on 5-point Likert scale (1=
marginal/lowto 5=outstanding).
IS=Interpreter Scale, scored on 5-point Likert scale (1=marginal/lowto
5=outstanding).
Self=Self-rating of perceived patient satisfaction with provider, score on 6-point rating scale (1=very poorto 6=excellent).
PPI=summed score of seven items with maximum score of 7.
tering to 3.77 of 5 for the graduating class (difference of+0.47 points, statistically significant atp=.015). The PPI rating by patients remained unchanged with a mean score of 4.78 of 7 at entry and a graduation mean score of 4.90 (difference+0.12,p=.614). Self-assessed perceived patient satisfaction with the encounter remained high at entry and graduation (5.38 and 5.42 of 6, respectively, difference+0.03,p=.819).
Subgroup analysis of one student cohort
For the 10 FNP students who had complete data at en-try and at graduation, performance as rated by the SP on the global IIRS item fell from a mean of 3.90 at entry to 2.90 at graduation (mean difference−1.00, significant at
p=.003; see Table 3). SI rating improved slightly from a mean global score of 3.20 to 3.50 at graduation but the difference of+0.3 points was not significant (p= .26). PPI scores remained similar at a mean of 5.40 at entry and 5.30 at graduation of 7 (difference of+0.1, not sta-tistically significant). PPI scores remained unchanged at 5.34 of 7 at both entry and graduation for this cohort. Further analysis of individual items of the IIRS identi-fied no particular behaviors out of six individual items rated, which accounted for the drop in score for the IIRS. In addition, no particular student accounted for the drop in IIRS global score between matriculation and gradua-tion. Apart from one student who improved in the IIRS 7 score, all others had scores that remained unchanged or fell between matriculation and graduation (data not shown).
Discussion
Is didactic curricular content alone adequate to improve FNP students’ competency for working effectively with
medical interpreters? To answer this question, we as-sessed the skills of FNP students in one U.S. FNP pro-gram. We used a combination of two global communi-cation measures that have been developed and validated for this purpose: the IIRS (Lie et al., 2007) and the IS (Lie et al., 2009); a general communication skill mea-sure, the PPI (Makoul, 2001); and student self-ratings (Lie et al., 2007) to provide multiple perspectives on com-munication skills between matriculation and graduation. The study was conducted in the absence of a skills-based curriculum for effective use of medical interpreters. We documented a lack of improvement in all the skills as-sessed between matriculation and graduation with some decrease in skills.
The trend for deterioration of skills instead of improve-ment in both the aggregate data of the entering and graduation groups, and the individual class of 2008, as assessed by the SP and SI were of significant concern. There are several possible explanations for this obser-vation. First, after their baseline assessment in the in-terpreter skill station, FNP students were not informed of their skill deficits and had no opportunity to correct mistakes or improve their skills specific to interpreted encounters. Second, during their internships in clinical practices, they may have observed behaviors inconsistent with appropriate skills for working with interpreters, and picked up “bad habits,” which were not corrected or were reinforced. Of interest in this study, their PPI scores re-mained high and unchanged over 2 years, suggesting that skill deficits were specific to interpreted encounters and did not necessarily apply to communication skills in gen-eral. As well, and of potentially high significance, their self-assessed measure of perceived patient satisfaction re-mained high at baseline and graduation, suggesting that they were unaware of the relatively low scores from the interpreter CPX station.
There are several strengths of our study. We used val-idated scales (IIRS and IS) for measuring communica-tion in the interpreted encounter. We standardized the training of SPs and SIs and their scoring accuracy. We used rigorous training methods for assessment and ex-amined specific behaviors amenable to individual teach-ing intervention. We were able to compare performance in the skills for effective use of medical interpreters, with generic communication skills using the PPI. We did not introduce any new curricula for the skill set during the study.
This study had some limitations. First, we had a small number of participants and used aggregate data from four different classes during the timeframe of the study, to maximize our statistical power. Pre/post data were avail-able for only one cohort of 10 FNP students. The impact of student Spanish fluency on performance in the station
was not assessed in this study, and may have been a con-founding factor, making it harder for the partially pro-ficient student to perform well in the station. However, from direct observation of the DVDs, we found no evi-dence that any student spoke in Spanish during the en-counters. The behavior scales had been validated in med-ical students and may not apply to FNP students, but we have found no study or report that suggests that skills for working effectively with interpreters should be dif-ferent among these difdif-ferent health professional groups. As well, our results are based on standardized encounters in a high stakes examination rather than actual clinical encounters. Lastly, the data are limited to Spanish speak-ing monolspeak-ingual patients only and generalization to other languages cannot be made.
Implications
Program-based implications
As a result of our findings, we plan to enhance cur-rent didactic teaching methods and learning strategies with interactive learning tools and workshops. Addition-ally, faculty and SPs will provide feedback to students at their baseline skills examination. We will require that students view and critique their own taped performance with faculty, to identify learning deficits and strengths for improvement. Our long-term goal is to demonstrate an improvement in these skills with training using the same scales. Of greatest importance, we hope to demon-strate that these skills for working effectively with in-terpreters ultimately translate into more effective patient care outcomes, including adherence to clinician advice, patient satisfaction, and improved quality of care. This study presents a small but much needed first step to link health professions education and quality of care, on the continuum from knowledge to competency and profi-ciency, to improved patient outcomes.
General implications
Our study has implications for NP education in general. Needs assessment is the basis of curriculum development. In the topic area of cultural competence, NP programs should develop skill-specific measures to identify their trainees’ learning needs before introducing new curric-ula. Our findings in learning to work effectively with in-terpreters represent one example of this recommended approach. Learning outcomes should then be tracked us-ing validated tools to assure that learnus-ing has indeed occurred, and the efficacy of teaching methods should be continually assessed through trainee performance and evaluations.
Acknowledgments
This project was supported in part by a grant from the National Institutes of Health (NIH), National Heart, Lung and Blood Institute, award number K07 HL079256–01, RFA-HL-04–012, (2004–2009) and from the Association of American Medical Colleges (AAMC) grant initiative “Enhancing Cultural Competence in Medical Schools” (2005–2008) supported by the California Endowment. The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH or AAMC. The Student Skills Training Center at the UCI School of Medicine provided invaluable support and expertise in data collection and case implementation.
References
American Nurses Association. (2004).Nursing: Scope and standards of practice(1st ed.). Silver Spring, MD: The Publishing Program of ANA.
Civil Rights Act of 1964, July 2. (1964). (Document Number PL 88–352), 88th Congress, H. R. 7152. Retrieved April 3, 2008, from http://usinfo.state. gov/usa/infousa/laws/majorlaw/civilr19.htm
Flores, G. (2005). The impact of medical interpreter services on the quality of health care: A systematic review.Medical Care Research Review,62, 255–299.
Grubbs, V., Chen, A. H., Bindman, A. B., Vittinghoff, E., & Fernandez, A. (2006). Effect of awareness of language law on language access in the health care setting.Journal of General Internal Medicine,21, 683–688. Harden, R. M. & Gleeson, F. A. (1979). Assessment of clinical competence
using an objective structured clinical examination (OSCE).Medical Education,13(1), 41–54.
Hayakawa, S. I. (1982). English as our official language: Why official English?
Retrieved April 3, 2008, from http://www.proenglish.org/issues/ offeng/index.html
Karliner, L. S., Perez-Stable, E. J., & Gildengorin, G. (2004). The language divide: The importance of training in the use of interpreters for outpatient practice.Journal of General Internal medicine,19, 175–183.
Lie, D., Bereknyei, S., Braddock, C., Encinas, J., Ahearn, S., & Boker, J. (2009). Assessing medical students’ skills in working with interpreters during patient encounters: A validation study of the interpreter scale. Academic Medicine,84(5), 643–650.
Lie, D., Boker, J., Bereknyei, S., Ahearn, S., Fesko, C., & Lenahan, P. (2007). Validating measures of third year medical students’ use of interpreters by standardized patients and faculty observers.Journal of General Internal Medicine,22(Suppl. 2), 336–340.
Makoul G. (2001). Essential elements of communication in medical encounters: The Kalamazoo consensus statement.Academic Medicine,76, 390–393.
U. S. Census Bureau. (2000).Language use and English-speaking ability, 1–3. Retrieved April 3, 2008, from http://www.census.gov/prod/2003 pubs/c2kbr-29.pdf
U.S. Census Bureau. (2004).Percent of people 5 years and over who speak English less than “very well.”Retrieved April 3, 2008, from http://factfinder.census. gov/home/saff/main.html? lang=en& ts
U.S. Department of Health and Human Services, Office of Minority Health. (2000).National standards of cultural and linguistically appropriate services in health care. Retrieved April 3, 2008, from http://www.omhrc.gov/ templates/content.aspx?ID=87
U.S. Department of Health and Human Services. (2002).Nurse practitioner primary care competencies in specialty areas. (HRSA 00–0532P). Rockville, MD: National Organization of Nurse Practitioner Faculties and American Association of Colleges of Nursing.
U.S. Government Accountability Office. (2008).Primary care professionals: Recent supply trends, projections, and valuation of services. Testimony by Steinwald, A.B. to Committee on Health, Education, Labor, and Pensions, U.S. Senate. Washington, DC: Author. (GAO Publication No. 08–472T, pp. 1–18). Van Der Vleuten, C. P. & Swanson, D. B. (1990). Assessment of clinical skills
with standardized patients: The state of the art.Teaching and Learning in Medicine,2, 58–76.
