Faculty and Student Evaluations of a Web Based Nursing Program
G
GeeoorrggeeFF..SShhuusstteerr Associate Professor of Nursing
College of Nursing, University of New Mexico G
GlloorriiaaBBiirrkkhhoollzz Professor of Nursing
College of Nursing, University of New Mexico L
LiinnddaaPPeettrrii Project Manager
College of Nursing, University of New Mexico
Work Funded by HRSA grant # 1D11 HP 00706-01 0
The College of Nursing at the University of New Mexico developed it’s first fully web based course in 1999 and by fall, 2001 all eight of the RN-BSN courses in the program were offered as fully web based courses using the Web-CT courseware platform. This RN-BSN program provides online courses to a largely rural state with a significant Hispanic and American Indian population. The program itself progressed from individual courses offered during the first semester to a fully web based program. This progression is common among nursing programs with distance outreach commitments given the current level of internet and web based software technology. The number of fully web based programs continues to grow as Colleges, whose missions include distance education, implement web based courses and programs as the means for fulfilling their mission. These new programs require periodic systematic evaluation, especially when they involve new technology and faculty who have not taught web based courses. The purpose of this program evaluation was to collect faculty and student data that would provide the basis for specific course and program revisions.
It is common for individual nursing faculty to evaluate their own courses, often using whatever methods are available. These course evaluations may include a variety of anecdotal, informal, and formal means, but these evaluations are done on an individual course by course basis. Results from many of these individual evaluations are found in the nursing education literature. Much less common is the systematic, multi-method evaluation of an RN-BSN web based program by faculty and students; very few of these more comprehensive evaluations are found in the literature. Before the completion of this systematic evaluation, our faculty collected individual course evaluation data using the Instructor and Course
Evaluation System (ICES). However, like many other fully web based programs; there was no systematic program or standardized evaluation. Individual ICES course evaluations required each course faculty to select 20-24 specific questions from a pool of 600 potential questions making cross course comparisons for program evaluation impossible. Faculty recognized these problems and also desired to address other course related questions including: the effectiveness of web based course design in general and
individual courses in particular; how congruent faculty and student perceptions were related to web based course content; the course teaching process; and what web based course experiences RN-BSN students liked and disliked. These questions became the basis for the specific aims of this evaluation. These three aims were to:
1. Identify, describe and analyze the strengths and weaknesses of course design and the teaching process;
2. Compare and contrast faculty and student perceptions about course design and the teaching process;
3. Identify, describe, and analyze what RN-BSN students liked and disliked about the web course design and the web based teaching process on a course by course and program wide basis.
Literature Review
RN–BSN students. Traditional baccalaureate nursing education is taught on-campus, but this tradition is changing as increasing emphasis in BSN programs is placed on distance education. Distance education is typically necessary when the education involves associate degree students who desire a baccalaureate degree but are also needed in the workforce because of the current nursing shortage (Armstrong, Gessner, & Cooper, 2000). Institutions with distance education programs are generally responding to the needs of working students, many of whom live and work in rural areas. This need for distance education is based on the type of students who enroll in these courses. Fulmer, Hazzard, Jones, and Keene (1992) stated these are students who could not otherwise enroll in these courses. For nursing, this frequently means students who are women dividing their time among the demands of family, work, and courses.
Web based distance education. Studies of learning outcomes comparing on-site and earlier distance education technologies such as Interactive Television by Billings (1994) and Shuster and Learn (2003) suggest distance students have equivalent or better knowledge gains and satisfaction with their courses. Leasure, Davis and Thievon (2000) compared traditional face-to-face and Web based sections of a nursing research course and found no significant differences in examination scores. A study by Atack (2003) had similar results. Buckley (2003) evaluated the three semester transition of a baccalaureate program nutrition course from traditional face-to-face, to Web enhanced and then to a fully Web based course. Buckley found no difference among the courses in student learning outcomes but the Web based course received the lowest mean course evaluation.
Satisfaction and dissatisfaction with web sites and courses. Zhang and von Dorn (2000) looked at web sites in general to identify factors that satisfy or dissatisfy users. They found that web site-user
interactions where users felt cognitively or emotionally involved are important sources of satisfaction; while web sites with navigation and technical difficulties are important sources of user dissatisfaction. A course design that is simple, clear, and easy to navigate is important to students (Atack, 2003; Forman, Nyatanga and Rich (2002). One source of dissatisfaction according to Billings, Connors and Skiba (2001) is that many nursing students believe Web based learning consumes more time because they don’t differentiate between course work and study. Students see learning as more time consuming because of the additional search time for course materials. Regardless of course site design or longer learning times; students value prompt faculty responses (Koeckeritz, Malkiewicz and Henderson, 2002)
Methodology
This evaluation was done with funding from Health Resources and Services Administration as part of a larger project. Faculty and students (N=227) systematically evaluated the course content, processes and resources for the eight courses in the RN to BSN program using quantitative and qualitative methods. Students currently in the program, graduates, and students who have paused out were individually
interviewed and completed standardized evaluation tools. Focus groups were also conducted with current students in the program. Faculty and a faculty emeritus peer reviewer completed an online instruction questionnaire developed by the Partnerships for Training Project funded by the Robert Wood Johnson Foundation.
Sample. The study sample included students (N= 108) who were currently enrolled in the fall, 2003 fully web based courses, graduates (N= 104), pause out students who had not taken a course in three
consecutive semesters (N= 7) and faculty (N=8). Subjects were solicited by letter and online mailings, with follow-up phone calls. Response rates varied from 20% for pause out students to 88% for faculty.
Design. Both qualitative and quantitative methods were used for data collection. Data were collected by grant and technical support personnel using five different types of data collection: student interviews, student focus groups, standardized web based surveys, self evaluations and faculty peer evaluations. Qualitative interview data were obtained using semi-structured questions developed by grant faculty and library and technical personnel. Quantitative data from two standardized questionnaires were obtained from the Instructor and Course Evaluation System (ICES) and the American Association of Colleges of Nursing Educational Benchmarking Institute (EBI) BSN Nursing Program Exit questionnaires. The ICES, EBI data and other technical evaluations were collected by links within the RN to BSN web based courses. The eight courses which constitute the RN to BSN web based program were self evaluated using the Robert Wood Johnson Partnerships for Training Learner-Centered Online Instruction Evaluation Instrument (http://pftweb.org/index.htm) by the faculty instructors and by a peer emeritus faculty with expertise in course design and teaching. The faculty peer also had access to all faculty and student postings and discussions in all of the courses for the entire semester.
Results
Results are summarized based on the three aims of the evaluation:
1. Course design and the teaching process: Strengths and weaknesses. Students noted that course design varied from course to course among different individual faculty. Students found these variances in course design and organization confusing when they started new courses or moved between courses during the semester. Students also felt faculty did not make course expectations clear.
Students noted both strengths and weaknesses in the teaching processes among different courses. Strengths of the course process included the quality of technical help from faculty and technical support. Students felt their work was important to faculty and reflected in faculty-student interactions.
In contrast, the quality of course related activities were uneven among different courses. Course activities greatly varied from the very practical and relevant to dreadful. Overall, students rated faculty as effective teachers. Despite faculty efforts, students did not feel very connected to other students nor did they consider that important. However, students indicated discussions were the key teaching strategy related to feelings of connectedness: when discussions went well students felt much more connected.
2. Course design and teaching process: Comparing and contrasting faculty and student perceptions. Findings suggest that students and faculty have similar views regarding technical and faculty expertise, anytime-anywhere web convenience, and the usefulness of research and written assignments. However, faculty and students had differing viewpoints regarding the time burden, usefulness of web discussions, the value of building a learning community and the clarity of communications and faculty organization. The faculty peer evaluator agreed with the students based on evaluation of course by course content and her evaluation of semester long student discussions, posted questions, and faculty responses and course postings.
3. Course design and teaching process: What RN-BSN students liked and disliked. Students liked the fact that faculty valued their effort. Students liked assignments that allowed choice, written
assignments and clinical assignments. Student reactions to course discussions were mixed: some liked them and some did not.
1. Use an androgogical approach- treat them like adult learners during the semester course process.
2. Improve consistency across courses in both course design and the process and set-up for taking quizzes and tests.
3. Faculty need to remember “my internet services are limited” when designing course materials.
Summary
The use of a systematic, multi-method approach enabled faculty to identify areas of strength and weakness within the RN-BSN program. As a result of this evaluation course design is being
standardized. The college also presented a series of faculty workshops on web based design, teaching process, and the development and facilitation of student discussions. Finally, a web based course faculty handbook with guidelines, recommendations and resources is being developed for both faculty who are currently teaching web based courses and new faculty teaching on the Web for the first time.
References
Armstrong, M., Gessner, B., & Cooper, S. (2000). POTS, PANS, and PEARLs: The nursing profession's rich history with distance education for a new century of nursing. Journal of Continuing
Education in Nursing, 31(2), 63.
Atack, L. (2003). Becoming a web-based learner: Registered nurses' experiences. Journal of Advanced Nursing, 44(3), 289-297.
Billings, D., Durham, J., Finke, L., Boland, D., Manz, B., & Smith, S. (1994). Collaboration in distance education between nursing schools and hospitals. Holistic Nursing Practice, 8(3), 64-70. Billings, D., Connors, H., & Skiba, D. (2001). Benchmarking best practices in web-based nursing
courses. Advances in Nursing Science, 23(3), 41-52.
Buckley, K. (2003). Evaluation of classroom-based, web-enhanced, and web-based distance learning nutrition courses for undergraduate nursing. Journal of Nursing Education, 42(8), 367-370. Cobb, K., Billings, D., Mays, R., & Canty-Mitchell, J. (2001). Peer review of teaching in web-based
courses in nursing. Nurse Educator, 26(6), 274-279.
Cragg, C., Plotnikoff, R., Hugo, K., & Casey, A. (2001). Perspective transformation in RN-to-BSN distance education. Journal of Nursing Education, 40(7), 317-322.
Forman, D., Nyatanga, L., & Rich, T. (2002). E-learning and educational diversity. Nurse Education Today, 22(1), 76-82.
Koeckeritz, J., Malkiewicz, J., & Henderson, A. (2002). The seven principles of good practice: Applications for online education in nursing. Nurse Educator, 27(6), 283-287.
Leasure, A., Davis, L., & Thievon, S. (2000). Comparison of student outcomes and preferences in a traditional vs. world wide web-based baccalaureate nursing research course. Journal of Nursing Education, 39(4), 149-154.
Shuster, G., Learn, C., & Duncan, R. (2003). A strategy for involving on-campus and distance students in a nursing research course. Journal of Continuing Education in Nursing, 34(3), 108-115.
Wang, A. Y., & Newlin, M. H. (2000). Characteristics of students who enroll and succeed in psychology web-based classes. Journal of Educational Psychology, 92(1), 137-143.
Zhang, P., & von Dran, G. (2000). Satisfiers and dissatisfiers: A two-factor model for website design and evaluation. Journal of the American Society for Information Science, 51(14), 1253-1268.
Biographical Sketch
George F Shuster is currently an Associate Professor at the College of Nursing, University of New Mexico (UNM). He has taught a number of outreach courses in a number of different formats including fully web based courses, Interactive Television courses, and on-ground outreach courses at both UNM and at the University of Virginia. He has taught, published and presented at both the national and
international levels. This current work is funded by the Department of Health and Human Services Basic Nurse Education and Practice Program. 1D11-HP-00706-01 3 year grant: June, 2003-June, 2006. Address: UNM College of Nursing, MSC09 5350, 1
University of New Mexico 86131-0001
E-mail: [email protected]Phone: 505-272-5612 Fax: 505-272-8901
Gloria Birkholz is currently a Professor at the College of Nursing, University of New Mexico (UNM) She has taught by various distance education methodologies including on-the-ground outreach, video, interactive television and fully web based courses. She has taught, published and presented at both the national and international levels. This current work is funded by the Department of Health and Human Services Basic Nurse Education and Practice Program. 1D11-HP-00706-01 3 year grant: June, 2003-June, 2006.
Address:
E-mail: [email protected] Phone 505-272-8862
Fax 505-272-8901
