doi:10.1016/j.jemermed.2004.07.012
Education
THE UCSD RESEARCH ASSOCIATE PROGRAM: A RECIPE FOR
SUCCESSFULLY INTEGRATING UNDERGRADUATES WITH EMERGENCY
MEDICINE RESEARCH
Daniel P. Davis, MD, Jennifer C. Poste and Donna Kelly, RN
Department of Emergency Medicine, University of California San Diego Medical Center, San Diego, California
Reprint Address:Daniel P. Davis,MD, Department of Emergency Medicine, UCSD Medical Center, 200 W. Arbor Drive, #8676,
San Diego, CA 92103-8676
e Abstract—Previous reports have documented the suc-cessful integration of undergraduates into the Emergency Department (ED) to assist in data collection for various research projects, with resultant improvements in depart-mental academic productivity. These reports have not de-tailed the exact procedures required to institute such a program. Over 100 undergraduates from three institutions currently participate in the UCSD Research Associate Pro-gram. Here we document our experience with the inception of such a program and define the various components required to successfully conduct an ED undergraduate re-search associate program, defining the roles of key person-nel, detailing the administrative component, discussing the selection of appropriate research projects and the design of data and instruction sheets, addressing issues related to training and didactics, and suggesting strategies for recruit-ment and upkeep of interest. © 2005 Elsevier Inc. e Keywords—research associate program; undergradu-ates; emergency department; research
INTRODUCTION
The inception of a research associate program (RAP) that integrates undergraduates into the Emergency
De-partment (ED) requires a considerable investment of time and effort to ensure its long-term success (1,2). However, making this investment up front can result in an invaluable resource for conducting clinical re-search (1–3). Hollander et al. document an increase in their annual publication rate from 2 to 20 after insti-tution of the Academic Associate Program (4). Fur-thermore, students participating in such programs re-ceive multiple benefits, including a quality clinical experience, exposure to clinical research, mentorship, and letters of recommendation for professional school applications (1,2,4).
The purpose of this report is to document our experience initiating such a program, focusing on per-sonnel, administrative factors, appropriate research projects, training issues, recruitment strategies, course content, quality assurance, and maintaining enthusi-asm among students and faculty.
PERSONNEL
Dedicated, enthusiastic personnel are one of the most important components to the successful implementa-tion of an undergraduate RAP. Key personnel for our program include a director, course administrator, nurse coordinator, and student liaison. The RAP di-Presented at the Society for Academic Emergency Medicine
Annual Meeting, Boston, Massachusetts, May 2003.
Educationis coordinated by Stephen R. Hayden,MD, of the University of California San Diego Medical Center, San
Diego, California
RECEIVED: 31 July 2003; FINAL SUBMISSION RECEIVED: 18 June 2004;
ACCEPTED: 30 July 2004
0736-4679/05 $–see front matter
rector should be a faculty physician with an interest in both undergraduate education and clinical research. This individual is ultimately responsible for the qual-ity of the experience for the participating undergrad-uates as well as the appropriateness and successful completion of all research projects. In addition, the RAP director serves as the liaison to other faculty in the department and helps recruit funding and foster collaborative relationships with outside investigators. Finally, the RAP director coordinates didactic sessions for the participants, assuring their continual involve-ment and integrating them into the departinvolve-ment and its ongoing clinical research. Thus, it is important that the RAP director be given protected time to invest in the inception and maintenance of the program. The equiv-alent of approximately 1–2 shifts per month seems to be appropriate to initiate and maintain a successful program.
The course administrator has a variety of responsi-bilities, including oversight of the “housekeeping” aspects of the program, support for the RAP director, and communication with the participants. For partici-pating undergraduates, the course administrator serves as the initial contact person, generally via e-mail or telephone. Once prospective students have committed to the program, the course administrator ensures that each successfully completes the initial health screen-ing, orientation, and Health Insurance Portability and Accountability Act (HIPAA) compliance question-naires. Once students are enrolled in the program, the course administrator is responsible for scheduling stu-dent shifts, maintaining adequate research supplies, keeping a log book of shifts for each research associ-ate, and scheduling didactic sessions. These responsi-bilities equate to approximately a one-half full-time employee (FTE).
The nurse coordinator performs several important tasks in our RAP model. First, this individual acts as a liaison to the nursing staff, ensuring the relatively seamless integration of research associates into the ED milieu. The nurse coordinator can also provide direct supervision of research associates and serve as a re-source during their shifts. In our program, the nurse coordinator also supervises the productivity and qual-ity of the data being collected and enters data for certain projects into electronic spreadsheets. The qual-ity assurance for data is explored in more depth below. The nursing responsibilities mandate approximately a one-quarter FTE.
The student liaison should be enrolled at one of the universities involved in the program. The specific du-ties of the student liaison include: publicizing the program at various campus locations and websites, serving as a resource to the program director and nurse
coordinator, and acting as a peer advocate for other research associates. This individual should be encour-aged to attend planning meetings for the RAP and for specific projects being considered for inclusion in the program.
ADMINISTRATION
Several options exist to provide liability coverage for participating undergraduates. If the program exists only as a formal university course, liability coverage may be available through the undergraduate campus. Alternatively, the ED may be able to extend coverage to research associates as an extension of existing pol-icies. In our program the hospital volunteer office was able to provide coverage and coordinate hospital ori-entation and health screening as well as provide park-ing permits for each participant. This seems to be an appropriate option for most hospitals, as research as-sociates generally do not receive monetary compensa-tion for their service.
Orientation to the hospital, ED, and each research project is required before allowing research associates to participate in data collection. In our program the hospital orientation is performed by the volunteer of-fice, whereas a separate orientation is held by the program director for the ED and individual projects. Research associate involvement in the program neces-sitates that confidential patient information be man-aged, requiring that hospital and HIPAA requirements be fulfilled, with appropriate documentation required for most Investigational Review Boards. Because re-search associates have access to sensitive health in-formation, both the hospital and RAP orientations cover the relevant ethical and legal issues in some detail.
Because it is not always possible to assemble all research associates at short notice, e-mail orientations and user-friendly instruction sheets can substitute for group orientations. A group e-mail account is impor-tant not only for performing these communications but also for maintaining a “paper trail” for permanent documentation. E-mail can also be used to distribute the program syllabus or information package. Access codes to hospital medical records and databases should be strictly monitored but are an essential com-ponent to complete data collection. Lastly, establish-ing a uniform also can be an important aspect of the program for easy identification of research associates and to foster unity. In our program, lab coats embroi-dered with “Research Associate” serve this purpose. Nametags are a requirement in most hospitals and
allow research associates to gain access to the ED and other relevant hospital areas.
RESEARCH PROJECTS
Selecting appropriate research projects for the RAP is of critical importance. The types of project can be placed into three general categories, depending upon the type of data being collected. Research associates may serve as “scribes” to record data collected by physicians and nurses, ensuring that data sheets are filled out completely in a timely fashion; this works best for more complex data requiring some level of clinical sophistication beyond that of an undergraduate student. A second project type utilizes research asso-ciates to interview patients and record their answers directly on data sheets; psychosocial assessments and patient questionnaires are ideal in this regard. Finally, research associates may undergo special training to make simple clinical assessments and abstract data from the medical record. Examples include the ab-straction of data obtained from monitors during ad-vanced procedures and more sophisticated neurocog-nitive testing. The selection of appropriate projects for a RAP should take the other ongoing projects into account, factoring in the expected number of patients to be enrolled and the time burden for each as well as the variety of patients and procedures experienced by the research associates. The majority of projects should be viewed as convenience samples, because research associate coverage is unlikely to be continu-ous due to schedule constraints, final examinations, and vacations.
Design of an appropriate data sheet and instruction sheet is also vital to avoid confusion and assure accu-rate data collection. Ideally, the data sheets for differ-ent projects are constructed in a standardized format, with sections for demographic data, information from the medical record, and questions for the patient, physicians, and nurses. This allows research associates to quickly and easily adapt to the various studies conducted as part of the program. The option for an electronic or scanned data sheet exists; however, part of our quality assurance process includes screening of a percentage of charts before entry of data into a spreadsheet. The percentage of charts audited varies from 20% to 100%, depending upon the complexity of information that research associates are asked to record and the number of patients enrolled. Instruction sheets are also a key component to assuring complete-ness and ease of data collection. Each is formatted to be project specific and outline the steps required for patient enrollment and data collection. In addition,
each instruction sheet contains an overview of the study to educate the research associates and increase their investment in the project. Again, the format should be standardized to facilitate its use.
TRAINING
Multiple training models can be used, either separately or in combination, to orient participants to the general approach for data collection and on the specific issues for individual projects. These include (a) didactic ses-sions, (b) apprenticeships with experienced research associates, (c) e-mail instructions, and (d) videos, CD-ROMs, and web-based education modules. Didactic sessions assure a standardized training environment and allow for questions and demonstrations. They require that every participant attend one of the ses-sions, however, which can translate into a significant time investment for the program director and require some advanced notice. Apprenticeships with experi-enced research associates require less time from pro-gram personnel; in addition, there is no substitute for individualized, on-site instruction. The disadvantage of apprenticeships concerns the lack of standardization and the possibility that bad habits will be perpetuated. E-mail works well for simple studies or for refining a particular protocol but should not be used as a sole method for orienting new research associates. E-mail is ideal for addressing general trends that require correction, as it allows rapid notification of all re-search associates and creates a “paper trail” for doc-umentation. Videos, CD-ROMs, and web-based edu-cation modules offer flexibility with regard to scheduling and allow demonstrations and examples. Finally, each institution may also have unique require-ments for attendance of a hospital orientation session or completion of an online HIPAA training course.
During orientation sessions, the individual projects are summarized, eligibility criteria for each project defined, the general approach to patient enrollment described, and data sheets reviewed. Consent proce-dures are also carefully explained for studies in which research associates obtain consent vs. those in which physician involvement is required. Each project is then covered in some detail and the specific expecta-tions of participants defined. Requiring research asso-ciates to sign the data forms they complete and audit-ing a defined percentage of charts can improve accountability and assure quality data collection. Care should be taken to avoid the impression by research associates that their individual performance in the program is tied to the number of subjects enrolled, as this introduces the possibility of coercion. That
the vast majority of RAP projects are ultimately convenience samples should be emphasized during orientation.
RECRUITMENT
Several strategies exist to maximize involvement in the program and achieve near-complete coverage of the ED. We were able to recruit over 30 students in the first month of the program and had over 100 students from three different undergraduate institutions on our roster within the first 6 months. Pre-health clubs on university campuses can be targeted through bulletin board postings and e-mail groups. In particular, the American Medical Student Association (AMSA) has pre-medical chapters at many undergraduate cam-puses. Current research associates can be asked to attend club meetings to publicize the program and answer questions. Pre-health advisory offices can also refer students for participation and post advertise-ments for the program. If course credit is offered to participants, the university course catalog can include a RAP listing. Other locations to post program an-nouncements include internship offices, career centers, volunteer offices, and bulletin boards around campus, emphasizing the life science areas of campus. In ad-dition, the campus newspaper may be interested in an article about the program and can publish announce-ments in the classified section. Ultimately, word of mouth may serve as the best recruitment tool.
COURSEWORK
Offering course credit to participants may be possible, either as a research elective or externship. Alterna-tively, the program can exist as a course in research methodologies, as has been described elsewhere (1,4). Didactic sessions may be offered weekly, biweekly, or monthly on a variety of topics relevant to medicine and clinical research. Emphasis on the individual stud-ies being performed will not only provide relevance to the lessons being taught but will improve research associate investment in the projects. The hours re-quired for course credit, both in the classroom and the ED, and the number of units offered should be clearly defined and can be discussed with the institution’s pre-health advisory office or the department through which credit will be given. Options for assigning grades include a “Pass/No Pass” system or the assign-ment of formal grades. Some combination of a re-search paper, formal tests, and an assessment of ED performance can be used for evaluation. In our
pro-gram, paper topics reflect the actual studies being performed in the ED at that time, with “dummy data” supplied for interpretation; alternatively, students may report data reflecting their expected results or perform a comprehensive literature review on the topic.
MONITORING
Monitoring the quality and completeness of collected data is essential, both to identify target areas for additional training and to report in published studies. Sample quality indicators include the total percentage of patients meeting inclusion criteria approached by research associates, the number of eligible patients enrolled, and the percentage of data points correctly entered onto data sheets and into spreadsheets. In addition, auditing a certain percentage of charts is recommended as a validation technique if data can be retrospectively abstracted from the medical record. To establish the efficacy of the program, patient recruit-ment during research associate shifts can be compared to enrollment before the program or during periods when research associates are not working. Logbooks must be maintained to determine how many patients are being approached for inclusion in various studies. Some documentation as to the reasons for potential subject refusals also should be maintained. A ques-tionnaire to assess the experience of research associ-ates can be valuable for documentation purposes and to identify areas for improvement to assure a quality experience for participants.
INTEREST AND ENTHUSIASM
Several mechanisms exist to maintain the enthusiasm of research associates. Establishing the importance of the research being performed and their vital role in the process is a key strategy to develop interest in the program. In addition, feeling as though they are part of a larger entity can increase research associate commit-ment and longevity. This can be accomplished through regular meetings and didactic sessions, frequent e-mail communication, and occasional social events at local restaurants and beaches. Inviting research asso-ciates to departmental conferences and grand rounds can also accomplish these goals. Finally, extending other opportunities to research associates can enhance their experience and retain veteran research associates. This may include more substantial involvement in particular projects, either those being performed in the ED as part of the program or other studies conducted by departmental personnel. In addition, creating grant
opportunities for participants and inviting interested research associates to regional and national research meetings are invaluable opportunities for undergradu-ates interested in medicine. Finally, opportunities to enhance medical school applications, including didac-tic sessions on the admission process, letters of rec-ommendation, and individualized counseling, are in-valuable to pre-medical students. It is also possible that such a program will lure physicians-in-training into a research career. Future studies should follow research associates longitudinally to determine the impact of the RAP on their career aspirations.
CONCLUSIONS
Programs that integrate undergraduates into the ED to assist with data collection are ideal strategies for en-rolling patients in studies without affecting patient
flow or adding to emergency physician workload. By investing time and energy up front, an optimal pro-gram can be created that ensures a steady supply of interested students, maintains participant enthusiasm, and ensures quality data collection.
REFERENCES
1. Hollander JE, Valentine SM, Brogan GX Jr. Academic associate program: integrating clinical emergency medicine research with undergraduate education. Acad Emerg Med 1997;4:225–30. 2. Bradley K, Osborn HH, Tang M. College research associates: a
program to increase emergency medicine clinical research produc-tivity. Ann Emerg Med 1996;28:328 –33.
3. Hollander JE, Reinhard SS, Shofer FS. Clinical research assistants in the emergency department: can volunteers be relied upon? (ab-stract) Acad Emerg Med 1998;5:533.
4. Hollander JE, Singer AJ. An innovative strategy for conducting clinical research: the academic associate program. Acad Emerg Med 2002;9:134 –7.
