2019 Volume 8 Number 3, 39-56
Midterm evaluation of the implementation of the first phase
of the practice teaching program: Field studies using CIPP
model
Torio, Myla Zenaida C.
Philippine Normal University, Manila, Philippines ([email protected])
Torio, Von Anthony G.
Philippine Normal University, Manila, Philippines ([email protected])
Received: 28 January 2019 Revised: 23 April 2019 Accepted: 17 May 2019
Available Online: 31 July 2019 DOI: 10.5861/ijrse.2019.4001
ISSN: 2243-7703 Online ISSN: 2243-7711
OPEN ACCESS
Abstract
Program evaluation is an inevitable requisite to quality assurance in higher education.
Ensuring the public that programs offered by an institution of higher learning is a must and
should be observed to maintain the standards and meet the demands of the public. In teacher
education, practice teaching program is a common feature of teacher education programs in
the country. Practice Teaching is commonly the terminal program undergone by pre-service
teachers and is considered to be one of the most crucial programs of soon-to-be teachers. It is
the aim of the current study to evaluate the midterm implementation of the practice teaching
program in an institution with the aim of looking for areas of improvement and best practices.
The study involved the use of Context-Input-Process-Product (CIPP) as an evaluation model
to achieve its aim. Evaluators of the program include 1 administrator, 4 supervising
instructors, 14 student-teacher leaders and 640 student teachers. The results of the study
revealed that all areas of evaluation had an overall mean equivalent to an interpretation of
very good. The context area received the highest evaluation (Mean = 3.99
0.18) while the
input area had the lowest evaluation (Mean = 3.82
0.23). The results of the study provided
inputs to better examine the practices in the program in the quest for further improvements. The
results of the program served as an input for the members of the institution to recalibrate their
efforts towards better implementation of the program.
Torio, M. Z. C., & Torio, V. A. G.
Midterm evaluation of the implementation of the first phase of the practice teaching
program: Field studies using CIPP model
1. Introduction
Institutions of learning particularly those for teacher-training are conscious about the delivery of quality programs that will contribute to the frontline workforce knowledge in education. The conscious effort of institutions is also guided by what teachers and stakeholders in public schools will expect teacher education graduates as they progress in their ladder in the teaching profession as guided by the Philippine Professional Standards for Teachers (PPST). Program evaluation is part of the quality assurance mechanisms of institutions to ensure the public of the quality of programs offered not only at the unit level but also at an institutional or even national level.
One of the culminating programs of a teacher-training program is that of their practice teaching. The practice teaching program is a formal way of the gradual introduction of the takers of the teacher-education program to the teaching profession. It usually is the terminal program of the course and is usually taken during the students’ senior level of a four-year degree program. It is thus a significant step to evaluate this program offering to assure the public of the quality of the program offering of the institution and their products as well.
Several institutions in the country offering teacher-training are considered as centers of excellence and development by the Commission on Higher Education (CHED). One of the higher education institutions of learning in the country is recognized as the country’s national center for teacher education. CHED mandates institutions of higher learning to adopt the framework of Outcomes-based-education. In an effort to live up with the expectation of delivering quality programs, the institution introduced an Outcomes-based Education aligned set of programs such as the practice teaching program. It is aimed that at the end of the program, the teacher will be equipped with the skills necessary to face the learners of Education 4.0.
Zhang et al. (2011) cited that there are about 26 evaluation approaches that may be utilized in evaluating projects. These approaches according to them may be classified into five: 1) pseudo-evaluations, 2) quasi-evaluation, 3) improvement-and-accountability-oriented evaluation, 4) social agenda and advocacy, and 5) eclectic evaluation. Pseudo-evaluations are those that are often associated with political objectives. Five approaches fall into the pseudo-evaluation and the rest fall into the remaining categories. Fourteen (14) approaches fall under quasi-evaluation studies and are focused on answering a particular question or utilized a specific methodological approach. The Improvement-accountability group is concerned about the identification of “merit and worth of the project”. The social agenda/advocacy category focuses on “pursuing social justice”. The last category is the eclectic category which selectively draws from all the given approaches to serve the needs of a particular group of users described by Stufflebeam and Shinkfield (2007).
The Context-Input-Process-Product (CIPP) approach of Stufflebeam was identified to be best fit for the program to be evaluated. This evaluation method is classified under the third category, the improvement/accountability group of approaches. The strength of the CIPP model is that it is designed to “systematically guide both evaluators and stakeholders in posing relevant questions and conducting assessments at the beginning of a project (context and input evaluation), and its end (product evaluation)” (Zhang et al., 2011). Galvin (1983) cited that CIPP was considered to be the evaluation model of preference among the members of the American Society for Training and Development.
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model programs are evaluated. In order for the public to be assured that programs are of high quality, the results of the evaluation must be made public. It is with this light that this study came about. The following are the purposes of the study as well as the research questions:
1.1 Evaluation Purpose and Questions
General Objective - The main objective of the study is to evaluate the PT 01 program offering of the the identified teacher education institution.
Specific Questions - In addressing the main objective of the study, the following specific questions were aimed to be addressed:
Are the key components of the program in place?
Is there a “right” mix of activities in the program?
How well is the program working?
Where there changes in the student teachers’ knowledge, attitude, values and ethics at the end of the program?
1.2 The program (Program Features and Background)
The identified teacher education institution launched innovations in its program offering in the advent of the Outcomes-based Education (OBE) popularized by the Commission on Higher Education (CHED). One of the innovations in the delivery of teacher education curricula of the university is to have two tiers of practice teaching courses instead of the regular six 1 unit courses of Field Study and 6 units of practice teaching. The practice teaching program now includes Practice teaching 01 and 02.
The first takers of the curricular offering have just graduated last year, School Year 2017-2018. This school year 2018 - 2019 is the second year of the implementation of the program. PT 01 is one of the courses that the students need to complete before they can proceed to the final curricular offering in their program, PT 02 and a no-credit requirement, the finishing school. Most of the students complete the PT 01 course on-campus and some of them are given an opportunity to be deployed at an earlier time.
1.3 Conceptual Framework
In evaluating the practice teaching program, the Stufflebeam’s Context – Input – Process – Product (CIPP) model was utilized. In this model, the terms refer to all four aspects of the program to be evaluated. The figure that follows refers to the conceptual framework of the study.
Torio, M. Z. C., & Torio, V. A. G.
The CIPP model of Stufflebeam is described as “a comprehensive framework for conducting formative and summative evaluations of projects, personnel, products, organizations, and evaluation systems” (Stufflebeam & Shinkfield, 2007). The model includes an evaluation of the context of the program. CIPP is considered to be one of the popular evaluation means for educational programs and projects (Zhang, Griffith, et al., 2009; Zhang, Zeller, et al., 2008). Several studies were conducted utilizing CIPP model for several purposes: Felix (1979) used the approach to evaluate and improve instruction; Nicholson (1989) used the model to evaluate reading instruction; Matthews and Hudson (2001) came up with guidelines for parent training projects. Chien, Lee and Cheng (2007) cited that the model was used to develop Taiwan’s national educational indicator system. Osokoya and Adekunle (2007) cited that they used the model to assess the trainability of enrollees in a particular Nigerian school. Ariawan, Sanjaya, and Divayana (2016) used the CIPP model to evaluate the implementation of their practice teaching program at Ganesha University.
The four components of evaluation (CIPP) - The context includes all concerns of the program that can give a clear picture of the program. The components of this aspect are as follows: Vision/Mission of the institution and the program; program implementation and regulation, different regulatory and institutional references including documents from the Commission on Higher Education (CHED), the Implementation of Outcomes-based Education by the institution; and the Philippine Professional Standards for Teachers (PPST).
The second aspect of the program is the Input. The input refers to concrete and abstract contributions of the institution as it implements the program. The components of this aspect include: the syllabus of the program; the human resources; the infrastructures and facilities; partnership with other institutions; the program of activities; and the materials/time components.
The third aspect is the implementation process. The implementation process includes the different steps and procedures that are undertaken in the conduct of the program. The components of this aspect include: the implementation process; the evaluation process and the mentoring mechanism. The last aspect is the product. The product refers to the culmination of the entire program. In order to determine the value of the conduct of the program, the product only has one component, the impact of the implementation.
Frye and Hemmer (2012) cited the following as common evaluation questions linked to the CIPP model:
Context Questions
What is necessary or useful: In other words, what are the educational needs? What are the impediments to meeting necessary or useful needs?
What pertinent expertise, services, or other assets are available?
What relevant opportunities (e.g., funding opportunities, administrative support) exist?
Input Questions
What are the potential approaches to meeting the identified educational need?
How feasible is each of the identified approaches, given the specific educational context of the need? How cost-effective each identified approach, given the specific educational context of the need?
Process Questions
How was the program actually implemented, compared to the plan? How is/was the program implementation documented?
Are/were program activities on schedule? If not, why?
Is/was the program running on budget? If it is/was over or under the planned budget, why? Is/was the program running efficiently? If not, why?
Can/did participants accept and carry out their roles?
What implementation problems have been/were encountered? How well are/were the implementation problems addressed?
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model Product Questions
What positive outcomes of the program can be identified? What negative outcomes of the program can be identified? Were the intended outcomes of the program realized? Were there unintended outcomes, either positive or negative? What are the short-term implications of the program outcomes? What are the long-term implications of program outcomes? What impacts of the program are observed?
How effective was the program? How sustainable is the program?
How sustainable are the intended and positive program outcomes?
How easily can the program elements be adopted by other educators with similar needs?
Another version of the questions for the four components of the CIPP model was adopted in the study. The questions are from that of Patil and Kalekar (2015). The questions are all reflected in the instruments used in the study and may be located in Appendix A to C.
2. Methods
2.1 Evaluation Model
The evaluation model used in this study was that of the CIPP model. The aspects of the CIPP model were considered as independent areas to be evaluated. Each area has specific indicators and relevant documents. The aspects of the evaluation model are given in the table that follows:
Table 1
Aspects of evaluation
Component Code Aspect
A. Context A.1 Vision and Mission of the Program and the Institution
A.2 Program Implementation Regulations
A.3 PSG – CHED, OBTEC, PPST
B. Input B.1 Syllabus
B.2 Human Resources
B.3 Infrastructures and Facilities
B.4 Partnership
B.5 Program of Activities
B.6 Materials / Time
C. Process C.1 Implementation Process
C.2 Implementation
C.3 Evaluation
C.4 Mentoring Mechanism
D. Product D.1 Impact of the implementation
2.2 Research Instruments
There are four main instruments used in this study: 1) the Rating scale instrument for Midterm Evaluation of the PT 01 Program, herein referred to as (RIMEPT); 2) the questionnaire for stakeholders of the PT01 Program (QSPT); 3) the Interview protocol for the stakeholders of the PT01 Program (IPSPT); and the 4) validation instrument of the PT01 instruments (VIPT).
Torio, M. Z. C., & Torio, V. A. G.
for Context, 8 for Input, 6 for Process and 5 for Product. This instrument is to be rated using a scale of 1 to 5, with 5 being the highest, by selected stakeholders. A copy of the instrument is included as Appendix A. An electronic form of the instrument was produced for data collection. The electronic form was sent to the students and responses were automatically gathered online. This instrument is designed to be answered by the Student-teachers.
Questionnaire for stakeholders of the PT01 Program (QSPT) - The QSPT is an open-ended type of questionnaire where stakeholders were asked to respond to four big questions under the CIPP model. The four questions include: Context: Are the key components of the program in place? Input: is there a “right” mix of activities in the program? Process: How well is the program working? and Product: Were there changes in the student-teacher’s knowledge, attitude, values and ethics at the end of the program? Guide questions under the main question were included to further probe stakeholders’ response. There are two versions of this instrument: version 1 is for students and version 2 is for the supervising instructors.
Interview protocol for the stakeholders of the PT01 Program (IPSPT) - The interview protocol reflects the same content as that of the SIMEPT and IPSPT. The difference is with that of the data collection procedure. The same set of 24 questions is reflected under four of the main research questions of the study.
Validation instrument of the PT01 instruments (VIPT) - This instrument was used to determine whether the three instruments, SIMEPT, QSPT and IPSPT are able to satisfactorily address the objectives of the study. The objectives are: 1) determine the acceptability of the adapted questions for the midterm evaluation for the PT 01: field studies program; 2) evaluate the clarity of the questions included in each of the four areas: Context, Input, Process, and Product; and 3) evaluate the plausibility of the adapted items in terms of addressing the different aspects (i.e., CIPP) of the program.
Population and Sample or Stakeholder Engagement - In identifying the samples of the study, the proponent tried to target 100% participation of the identified population of stakeholders. However, the availability of the respondents limited their number to a certain extent. The stakeholders who are part of the program who participated in the study are as follows: 1 Practice Teaching coordinator, 4 supervising instructors (SI), 14 Student teaching leaders, and 504 student-teachers (ST). The table that follows reflects the populations and samples considered in the study:
Table 2
Subjects of the study
Stakeholder Population Sample Percentage (%)
Administrator of the Program 2 (Institute Director and Practice teaching Coordinator)
1 50
Supervising Instructors (SI) 22 4 18
Student-Teacher (ST) leaders 14 14 100
Primary Stakeholder: Student-Teachers (ST)
640 504 78.75
Note. Female: 373, Male: 131.
Program Evaluation Design Matrix - In evaluating the program, the following table gives the different evaluation aspects and the corresponding questions, the information needed, the sources of information, the data collection procedures and the expected outputs. The table gives an idea of how the research procedures will be carried out (see Table 3).
3. Results and Discussion
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model
Table 3
Program evaluation design matrix
Evaluation Aspects and Questions Information Needed Sources of Information (Who: Sample) Sources of Information (How: Instrument) Data Collection Procedure Expected Output Context
Process Question 1: Are the key components of the program in place?
Components of the program Availability of the components Director of the institute Program Coordinator Supervisor Students Student Teaching Handbook / Manual Questionnaire, Survey Interview Questionnaire: 1 representative per group of subjects (12 groups in all), 3 supervisors
Survey (samples taken from the population who responded to the online survey) Interview (1 Coordinator, 1 Director, 1 supervisor, 2 students) Accomplished Survey forms (online) Interview Transcripts Coded Summary of Responses in the Questionnaire Input
Process Question 2: Is there a “right” mix of activities in the program?
Activities of the Program
Process
Process Question 3: How well is the program working?
Evaluation of the program (pre- and post)
Product
Outcome Question: Were there changes in the student teacher’s knowledge, attitude, values and ethics at the end of the program?
Evaluation of Students’ KSAVE (pre- and post)
3.1 Overall Evaluation of the Program
The following interpretation table is used to convert the quantitative evaluation data of the students’ evaluation to qualitative interpretation:
Table 4
Interpretation table
Quantitative range Interpretation
4.50 – 5.00 Outstanding
3.50 – 4.49 Very good
2.50 – 3.49 Good
1.50 – 2.49 Poor
1.00 – 1.49 Needs Improvement
A total of four aspects as reflected in the CIPP model were evaluated by the primary stakeholders, the students of the program. The four components being Context, Input, Process, and Product. The overall results of the evaluation of the different aspects of the program by the students as primary stakeholders of the program is given as follows:
Table 5
Overall evaluation of the program
Aspect Quantitative Result Interpretation
1. Context 3.99 ± 0.18 Very Good
2. Input 3.82 ± 0.23 Very Good
3. Process 3.97 ± 0.18 Very Good
4. Product 3.97 ± 0.04 Very Good
Note. N=504.
Torio, M. Z. C., & Torio, V. A. G.
Input was found to be evaluated the lowest (Mean = 3.82 ± 0.23). The values do not have significant differences with each other.The result of the evaluation agrees with the study of Ariawan et al. (2016) where Context was also rated to be the highest while the lowest was that of process. The evaluation reveals that somehow, the primary stakeholders are more satisfied with that of the process and product of the practice teaching program than that of the input they have received.
The results of the evaluation revealed that there still are a lot that can be done to further improve the program. While the program passed the evaluation of the primary stakeholders, the result of the evaluation also revealed that further examination of the aspects may be done to identify weak points and work on that area. Although there is not much difference in the evaluation of the CIPP component, the interesting result that the input component was evaluated least needs to be given a closer look. Other components also require a closer look to further examine the strengths and weaknesses of the program. The following are the specific evaluation results of the different aspects of the program:
3.2 Context Evaluation (Are the key components of the program in place?)
The details of the evaluation of this aspect include a presentation of the results for all five validated benchmarked indicators. The same rating instrument is used with 5 being the highest rating and 1 as the lowest.
Table 6
Evaluation results for context
Component Quantitative Result Interpretation
1) How well were you made aware of the aims of the practice teaching program and how they relate to the institute’s aims?
4.18 ± 0.69 Very Good
2) How well do you think are the aims of your practice teaching program related to the aims?
4.14 ± 0.68 Very Good
3) How do you rate your degree of involvement in the activities of the program? How well are the activities related to the aims?
4.04 ± 0.74 Very Good
4) How well is the institute able to cater to social needs of the student-teachers?
3.82 ± 0.92 Very Good
5) How well were you given an opportunity to experience community and extension work in the practice teaching program?
3.79 ± 0.82 Very Good
Note. N=504.
The evaluation scores of the components of the context do not significantly vary with each other. All components were also rated to be very good. The first component got the highest evaluation mark. However, it is noteworthy to look at the last component which has something to do with community and extension work. This can be understood as the program that the students are under is actually an on-campus component of the program. In hindsight, the practice teaching program really gives limited exposure for students to do extension work as majority of the work given to students in the program relates to that of classroom teaching.
The response of the supervisors and student-teachers agree with the quantitative results of the evaluation. Here are some of the responses of the Supervising Instructor (SI) and students in the questionnaire:
SI 1: “I believe the mission, vision and goals of the school are aligned with the program. I think the objectives of the program are derived from PPST as indicated in the PT 01 Evaluation for
demo teaching. So far, the only activity happening related to the PT 01 students is their practice teaching. There are CTs who treat students as a friend, daughter or son.”
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model classroom but as well exceeds beyond that.”
The lowest evaluation given to extension work and school community linkage was also validated by the response in the questionnaires:
SI 1: “I have no information of PT 01 students joining any extension work of the school.”
3.3 Input Evaluation (Is there a “right mix” of activities in the program?)
The second aspect of the CIPP model is the input aspect. A total of eight validated benchmark indicators are under this aspect. The table that follows, Table 7, reflects the results of the evaluation.
Table 7
Evaluation results for input
Component Quantitative Result Interpretation
1) How well did the program consider your entering ability as a student-teacher determined?
3.96 ± 0.68 Very Good
2) How well was the institute able to consider your learning skills as student-teachers?
4.04 ± 0.72 Very Good
3) How well were you able to link theory and practice in your practice teaching program?
3.76 ± 0.73 Very Good
4) How well were you able to know the resources and equipment that are in place and available in your program?
3.63 ± 0.88 Very Good
5) How do you rate the state of the laboratories and library? 3.60 ± 0.94 Very Good 6) How strong are the teaching skills of the supervising instructors? 4.07 ± 0.74 Very Good 7) How do you rate the knowledge, skills, and attitudes related to
subjects do supervising instructors and student-teachers have?
4.03 ± 0.67 Very Good
8) How conducive is the classroom and school environment? 3.51 ± 0.96 Very Good
Note. N=504.
This aspect of the CIPP model was rated the least among the four indicators. The evaluation of this aspect revealed that the teaching skills of the teachers (Mean = 4.07 ± 0.74) was recognized by the respondents to be the strongest area of the Input area. However, the component that was rated to be the least has something to do with the classroom and school environment (Mean = 3.51 ± 0.96). Laboratories and library (Mean = 3.60 ± 0.94 was second from the bottom. This means that the school really has to look at improving the facilities of the school in order for us to advance in this area of evaluation. Facilities are essential components of the effective delivery of the program. A closer examination of the program will give insight as to how the administration can better give focus on improving the concrete foundations of the school.
The following statements from the responses of the students in the questionnaire validated the results:
ST 02: “The ability of the students in ITL are very competent and critical yet the ability of the students in the early off-site schools is very alarming in nature. The students in off-site campus are very diverse.”
ST 13: “The teaching skills of the teachers are well defined and slightly respected.”
The concerns about infrastructure were validated by the following statements from the other stakeholders:
ST 13: “Resources and equipment is so "LOW" and facilities are at some point can be close to worst.”
ST 12: “I could say that the laboratories are not well maintained and they probably don't have laboratories as for the library I also don’t know if they have one.”
Torio, M. Z. C., & Torio, V. A. G.
ST 08: “The classrooms are well-maintained and well-equipped with aircon, fans and even TV. However, not all rooms are properly ventilated which also affects the conduciveness of the learning environment.”
3.4 Process Evaluation (How well is the program working?)
The third aspect of the evaluation is the process. It has a total of six validated benchmark indicators. The process aspect has something to do with procedures and steps that are inherent in the conduct and delivery of the program. The results are given in table 8.
This aspect of the CIPP model was rated to be very good. The components do not significantly differ from each other. The highest evaluation was given by the students to the continuous evaluation of the teaching-learning (Mean = 4.14 ± 0.75). This has positive implications in the conduct of the program as this means that the administration is really conscious about the quality of the delivery of the program. The table is given as follows.
Table 8
Evaluation results for process
Component Quantitative Result Interpretation
1) How transparent and collaborative are the administrative processes of the institute?
3.78 ± 0.87 Very Good
2) How well has ICT been employed in various school practices? 3.72 ± 0.91 Very Good 3) How actively student-teachers and supervising instructors
participate in different activities?
4.12 ± 0.72 Very Good
4) Is communication effective? 3.96 ± 0.87 Very Good
5) Is teaching-learning continuously evaluated? 4.14 ± 0.75 Very Good
6) How important other activities are apart from teaching/learning for the institute?
4.09 ± 0.74 Very Good
Note. N=504.
The weakest component of the program relates to the utilization of ICT in the classroom (Mean = 3.72 ± 0.91). As with the previous aspect, this component also has something to do with infrastructure. This means that the concern on facilities and infrastructure is a recurring concern in the evaluation process. The results of the evaluation are validated by the results of the response of the respondents on the questionnaire:
SI 01: “It shows PT 01 students that teachers do more than just teaching inside the classroom.”
ST 01: “I will be confident in saying that the program is working or flowing very well. The school administration has done a great job into creating a program for us practice teachers. There were various school activities that helped us a fun experience and a meaningful one as well for the teachers as well as the students are very engaging. Because of us communicating with one another helped us to continuously experience the teaching - learning that we are undergoing.”
ST 04: “Communication with the administration is so easy by the use of emerging technology such as social media. The use of ICT is not an option because there are lack of connection (data) in the institution's building and grounds. I think it’s is the main concern of every PT because technology is part of the 21st century learning. In the other hand, teaching-learning is continuously evaluated since supervisors are every approachable to help in terms of evaluation.”
3.5 Product Evaluation (Where there changes in the student’s knowledge, attitude, values and ethics at the end
of the program?)
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model validated benchmark indicators. Table 9 below shows the results of details of the evaluation.
Table 9
Evaluation results for product
Component Quantitative Result Interpretation
1) How do you rate the achievements of the institute in co-curricular and extracurricular activities?
3.99 ± 0.73 Very Good
2) How well are assessment strategies and procedures used for different components of the institute?
3.96 ± 0.69 Very Good
3) How well were you, as a student-teacher, in the program use what you have learned?
3.97 ± 0.75 Very Good
4) Were there official report of each activity? 3.91 ± 0.81 Very Good
5) How well have the supervising instructors’ and institute’s reputation improved?
4.03 ± 0.74 Very Good
Note. N=504.
The last aspect was also evaluated very good overall. The component with the best rating is that of the improvement in the teacher’s and school’s reputation (Mean = 4.03 ± 0.74). Although not significantly low, the component with the least rating is that of the official report of the activities (Mean = 3.91 ± 0.81). This low evaluation on the official report is understood as the students are not given a copy of completed reports of the activities. There is no means for the students to access the report of the activities and will naturally rate this aspect low. This somehow gives an idea that reports prepared on activities be communicated to students as well.
The results of the quantitative evaluation are supported by the qualitative results. Here are direct quotes from the responses of the stakeholders:
SI 01: “PT 01 students use their knowledge from the different subject offerings in their curriculum. They realized how important the content subjects and the PCK courses.”
ST 01: “There are lots of changes for us student teachers because we were able to first hand experience on what its like to be a teacher. Some things that we were able to change was our communication skills, our classroom management skills, our attitude towards other people and our approach inside the class. We were able to learn to be flexible meaning that we should adjust and adapt what is there for us to use so that every minute of the child's learning is worthwhile. All of our learning in our 4 years of stay in PNU was practiced and trained in PT 01. All activities had reports and were properly recorded.”
ST 03: “There is a change on student teacher's knowledge, attitude, values and ethics. For knowledge, our supervisor gave some point on the classroom management, for the attitude of the teacher, the teacher must be enthusiastic while discussing at the front and ethics, we must employ the margin. Every day we need to report to our supervisor on the happenings about the PT
experience and we learn a lot from it.”
ST 06: “Having our PT01 here in _____ really helped us gain more knowledge, skills, values, and strategies. Putting us in a real classroom set-up gave us the things we need to learn as a teacher. The students are our evidences. Based on the way that we have learned here, their excellence and achievement academically and in co-curricular and extracurricular activities, _____ have really molded great students and great teachers. The reputation has improved because I believe that PTs have been a good reflection of what _____ should be.”
4. Conclusion and implications
Torio, M. Z. C., & Torio, V. A. G.
institution offers, evaluation studies such as this one are necessary. This study evaluated the conduct of one of the most crucial components of the pre-service teaching program of an institution, the practice teaching program. The CIPP model was utilized because of its merits as an improvement/accountability approach of evaluation.
The results of the study revealed the answers to all four specific questions of the study. First, we can say that the key components of the program are in place given that the evaluation of the primary stakeholders of the program is “very good”. This is validated by the inputs given by other stakeholders, vis-à-vis, the supervisors and the leaders of the different groups in their responses in the questionnaires. The second question was also satisfactorily addressed, that is, we can say that there is a “right” mix of activities in the program since the quantitative result may be interpreted to be “very good”. We can also say that the stakeholders are satisfied with the way that the program is running. This is supported both by the quantitative response given by 504 respondents in the survey and the qualitative data gathered through the questionnaire. Finally, we can say that there were changes in the student’s knowledge, values and ethics because of the complementary results of the students’ quantitative evaluation of the program and the qualitative remarks of other stakeholders of the program.
The results of the evaluation revealed in general that all four aspects of evaluation were evaluated to be very good by the primary stakeholders of the program, the students: Context (Mean = 3.99 ± 0.18); Input (Mean = 3.82 ± 0.23); Process (Mean = 3.97 ± 0.18); Product (Mean = 3.97 ± 0.04).The overall results of the evaluation made by the students revealed that the input was the weakest aspect. A closer look at the components of the different aspects showed that infrastructures and facilities were rated to be the weakest areas of the program. While the program was evaluated to be good by the primary stakeholders, there remains to be a big room for improvement in the program. None of the components reached the outstanding mark.
4.1 Implications of the study
The conduct of evaluation studies such as this study is essential in the improvements that can be brought about via recommendations as part of the formal process of scrutinizing the program. The CIPP model provides a comprehensive model of evaluating the entirety of the program with the aim of identifying weaknesses and strengths. The results of the evaluation yielded “very good” evaluation in all areas. This may appear to be acceptable to the eyes of those whose aim is simply to judge the implication of the program as carried out in its initial phase. For those implementing the program, the evaluation “very good” means that there are still areas that can be improved. There is still sufficient room to grow as there still is a higher mark to reach, the “outstanding” mark.
The CIPP component that got the highest rating is that of the “Context”. A closer examination of this area can help the administrators as well as other stakeholders to consider these good practices and use the evaluation as a reference for better practices in the future. The top indicator of the Context component involves the capability of the program to inform the student teachers about the aims of the program and their relation to the aims of the institute. The second top indicator in this area has something to do with the awareness and participation of the students in the program activities and how these activities relate to the aims of the program. It is a good start to know that students are aware of the aims of the program and their alignment with different activities. Administrators of programs should really ensure that students are made aware of their aims so that students will not get lost in the implementation process. The details of the evaluation can help give direction to the administrators of the program the right path to follow.
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model classroom and the school environment?” This should concern the administrators as they should be made aware that the environment plays a significant role in the conduct of the practice teaching program.
A deeper examination of this indicator will lead possible areas for improvement related to school environment. What makes the stakeholders evaluate this area as the least? Another area of concern includes the resources of the school, the ability of the students to link theory and practice and an evaluation of their entering ability. One particular question is, “How well did the program consider your entering ability as a student-teacher determined?” This can help the administrators in their retrospection of the program particularly concerning the idea of analyzing the entering ability of the student-teachers of the program. It has many possible implications. This may mean that student-teachers are conscious as to how they are initially evaluated for later placement. They may also be interested to know how well they were prepared by their respective programs before entering the final phase of their teacher education program.
In order to complete the evaluation cycle and to further enhance the program, the following are the recommendations of the proponents: 1) Continue the evaluation process by proceeding to an end-of-the-program evaluation or final evaluation of the program; 2) Involve additional stakeholders of the program including basic education students handled by the pre-service teachers, the parents and the community members; 3) Consider improving the instrument to further address other components of the program that may not be covered by the initial evaluation; 4) Consider the evaluation of other components of the pre-service teaching program in general to help identify areas of improvement; and 5) Provide recommendation to the administration in relation to the results of the program particularly about the concerns on infrastructures and facilities.
Acknowledgement: The authors would like to recognize the inputs of the director of the institute Prof. Maria Ruth M. Regalado, all the supervising instructors, faculty members of the Institute of Teaching and Learning and all the student teachers who gave their consent in the conduct of the study. Special gratitude is also expressed to the authors’ family members who served as inspiration in the continuous quest for wisdom in the teaching profession. The authors would also like to bring back all the glory to the God almighty.
5. References
Ariawan, I. P. W., Sanjaya, D. B., & Divayana, D. G. H. (2016). An evaluation of the implementation of practice teaching program for prospective teachers at Ganesha university of education based on CIPP-Forward Chaining. International Journal of Advanced Research in Artificial Intelligence, 5(2), 1-5.
Chien, M., Lee, C., & Cheng, Y. (2007). The construction of Taiwan’s educational indicator system: Experiences and implications. Educational Research for Policy & Practices, 6(3), 249-259.
https://doi.org/10.1007/s10671-007-9026-x
Department of Education – Teacher Education Council. (2017). Philippine Professional Standards for Teachers. Felix, J. L. (1979). Research and evaluation to improve instruction the Cincinnati strategy. Educational
Evaluation & Policy Analysis, 1(2), 57–62. https://doi.org/10.3102/01623737001002057
Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE Guide No. 67.
Medical Teacher, 34, 288-299. https://doi.org/10.3109/0142159X.2012.668637
Galvin, J. C. (1983). What can trainers learn from educators about evaluating management training? Training
and Development Journal, 37(8), 52–57.
Langmeyer, D. B. (2008). Developing evaluation questions. Retrieved from http://www.archrespite.org/archfs13.html
Matthews, J. M., & Hudson, A. M. (2001). Guidelines for evaluating parent training projects. Family Relations, 50(1), 77-86. https://doi.org/10.1111/j.1741-3729.2001.00077.x
Nicholson, T. (1989). Using the CIPP model to evaluate reading instruction. Journal of Reading, 32(4), 312-318. Osokoya, M., & Adekunle, A. (2007). Evaluating the trainability of enrollees of the Leventis foundation (Nigeria)
Torio, M. Z. C., & Torio, V. A. G.
Patil, Y., & Kalekar, S. (2015). CIPP Model for School Evaluation. Scholarly Research Journal for Humanity Science & English language, 2(10), 2615-2619.
Redhakrishna, R. B., & Relado, R. Z. (2009). A framework to link evaluation questions to program questions. Retrieved from: https://www.joe.org/joe/2009june/tt2.php
Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications. San Francisco, CA; Jossey-Bass.
Zhang, G., Griffith, R., Metcalf, D., Zeller, N., Misulis, K., Shea, D., & Williams, J. (2009). Assessing service learning of a service-learning program in teacher education using mixed-methods research. Paper presented at the American Education Research and Association Annual Conference, San Diego, CA. Zhang, G., Zeller, N., Shea, C., Griffith, R., Metcalf, D., Misulis, K., Williams, J., & Knights, S. (2008). A 360
assessment of the multidimensional effects of a service-learning program in teacher education using mixed-methods research. Paper presented at the 8th International Research Conference on
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model Appendix A
Rating Scale Instrument for Midterm Evaluation of the PT 01 Program (RIMEPT)
Please rate the following items in terms of the ability of the PT 01 program to address the following questions. Use the scale 1 – 5, with 5 being the highest for your evaluation. Put a tick mark on the corresponding column of your response.
Question / Criteria 1 2 3 4 5
Context
1) How well were you made aware of the aims of the practice teaching program and how they relate to the institute’s aims?
2) How well do you think are the aims of your practice teaching program related to the aims?
3) How do you rate your degree of involvement in the activities of the program? How well are the activities related to the aims?
4) How well is the institute able to cater to social needs of the student-teachers? 5) How well were you given an opportunity to experience community and extension work in the practice teaching program?
Input
1) How well did the program consider your entering ability as a student-teacher determined?
2) How well was the institute able to consider your learning skills as student-teachers? 3) How well were you able to link theory and practice in your practice teaching program? 4)How well were you able to know the resources and equipment that are in place and available in your program?
5) How do you rate the state of the laboratories and library? 6) How strong are the teaching skills of the supervising instructors?
7) How do you rate the knowledge, skills, and attitudes related to subjects do supervising instructors and student-teachers have?
8) How conducive is the classroom and school environment? Process
1) How transparent and collaborative are the administrative processes of the institute? 2) How well has ICT been employed in various school practices?
3) How actively student-teachers and supervising instructors participate in different activities?
4) Is communication effective?
5) Is teaching-learning continuously evaluated?
6) How important are other activities apart from teaching/learning for the institute? Product
1) How do you rate the achievements of the institute in co-curricular and extracurricular activities?
2) How well are assessment strategies and procedures used for different components of the institute?
3) How well were you, as a student-teacher, in the program use what you have learned? 4) Were there official report of each activity?
5) How well have the supervising instructors’ and institute’s reputation improved?
Torio, M. Z. C., & Torio, V. A. G. Appendix B
Questionnaire for Stakeholders of the PT 01 Program (QSPT)
(Version for Supervising Instructors)Dear PT 01 Stakeholders,
Good day!
Please accomplish the following evaluation questionnaire which will help us evaluate the implementation of the Practice Teaching program. There are two parts in this survey questionnaire: Part 1. Basic Information and Part II. Written Evaluation of the program. The second part includes the guide questions that will help you evaluate the practice teaching program.
Thanks!
Myla Zenaida C. Torio
EVALUATION QUESTIONNAIRE Part I. Basic Information:
Name (Surname first): Role in the program:
Citizenship: Gender:
Department/Unit Email address:
Part II. Written Evaluation of the program:
CONTEXT: Are the key components of the program in place? Guide Questions: 1. Are the aims of the instate suitable to the practice teaching program?
2. Are the objectives of the practice teaching program derived from the aims of the institute?
3. What are the activities provided in the program? Are they in accordance with the aims of the institute? 4. Is the institute able to cater to social needs of the student-teachers?
5. What are the links between school community and extension work?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________
INPUT: Is there a “right” mix of activities in the program? Guide Questions: 1. What is the entering ability of the students?
2. What are the learning skills of the students? 3. What is the theory/practice balance?
4. What resources and equipment are in place and available? 5. Are laboratories and library well maintained?
6. How strong are the teaching skills of teachers?
7. What knowledge skills and attitudes related to subjects do teachers and students have? 8. How conducive is the classroom and school environment?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________
PROCESS: How well is the program working? Guide Questions: 1. Are the administrative processes transparent and collaborative? 2. Has ICT been deployed in various school practices?
3. How actively students and teachers participate in different activities? 4. Is communication effective?
5. Is teaching-learning continuously evaluated?
6. How important other activities are apart from teaching learning for the school?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________
PRODUCT: Were there changes in the student teacher’s knowledge, attitude, values and ethics at the end of the program? Guide Questions: 1. What are the achievements of the school in co-curricular and extracurricular activities?
2. What assessment strategies and procedures are used for different components of school? 3. How do students use what they have learned?
4. Is there official report of each activity?
5. Have the teachers and school’s reputation improved?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ Overall Evaluation of the Program:
Evaluation of the implementation of the first phase of the practice teaching program: Using CIPP model Appendix C
Questionnaire for Stakeholders of the PT 01 Program (QSPT)
(Version for Student teachers)Dear PT 01 Stakeholders,
Good day!
Please accomplish the following evaluation questionnaire which will help us evaluate the implementation of the Practice Teaching program. There are two parts in this survey questionnaire: Part 1. Basic Information and Part II. Written Evaluation of the program. The second part includes the guide questions that will help you evaluate the practice teaching program.
Thanks!
Myla Zenaida C. Torio
EVALUATION QUESTIONNAIRE Part I. Basic Information:
Name (Surname first): Role in the program:
Citizenship: Gender:
Department/Unit Email address:
Part II. Written Evaluation of the program:
CONTEXT: Are the key components of the program in place? Guide Questions:
1. Were you made aware of the aims of the practice teaching program and how they relate to the institute’s aims? 2. Do you think the aims of your practice teaching program are related to the aims?
3. What are the activities where you were involved? Are the activities related to the aims? 4. Is the institute able to cater to social needs of the student-teachers?
5. Were you given an opportunity to experience community and extension work in the practice teaching program?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________
INPUT: Is there a “right” mix of activities in the program? Guide Questions: 1. Was your entering ability as a student-teacher determined?
2. Were your learning skills as student-teachers initially determined?
3. Where you able to link theory and practice in your practice teaching program?
4. Were you able to know the resources and equipment that are in place and available in your program? 5. Are laboratories and library well maintained?
6. How strong are the teaching skills of the supervising instructors?
7. What knowledge, skills, and attitudes related to subjects do supervising instructors and student-teachers have? 8. How conducive is the classroom and school environment?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________
PROCESS: How well is the program working? Guide Questions:
1. Are the administrative processes of the institute transparent and collaborative? 2. Has ICT been employed in various school practices?
3. How actively student-teachers and supervising instructors participate in different activities? 4. Is communication effective?
5. Is teaching-learning continuously evaluated?
6. How important other activities are apart from teaching/learning for the institute?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________
PRODUCT: Were there changes in the student teacher’s knowledge, attitude, values and ethics at the end of the program? Guide Questions: 1. What are the achievements of the institute in co-curricular and extracurricular activities?
2. What assessment strategies and procedures are used for different components of the institute? 3. How do you as a student-teacher in the program use what you have learned?
4. Is there official report of each activity?
5. Have the supervising instructors’ and institute’s reputation improved?
____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ Overall Evaluation of the Program:
Torio, M. Z. C., & Torio, V. A. G. Appendix D
Interview Protocol for the Stakeholders of the PT01 Program (IPSPT)
Please answer the following items using the context of the PT 01 program. The following are the research questions and the corresponding probing questions:
Interviewee’s Basic Information
Name of Subject: ____________________________________________ Role in the program: _________________________________________
Research and Probing Questions
Research Question 1: CONTEXT: Are the key components of the program in place? R.Q.1. Probing Questions:
1.1. Are the administrative processes transparent and collaborative? 1.2. Has ICT been deployed in various school practices?
1.3. How actively students and teachers participate in different activities? 1.4. Is communication effective?
1.5. Is teaching-learning continuously evaluated?
1.6. How important other activities are apart from teaching learning for the school?
Research Question 2: INPUT: Is there a “right” mix of activities in the program? R.Q.2. Probing Questions:
2.1. What is the entering ability of the students? 2.2. What are the learning skills of the students? 2.3. What is the theory/practice balance?
2.4. What resources and equipment are in place and available? 2.5. Are laboratories and library well maintained?
2.6. How strong are the teaching skills of teachers?
2.7. What knowledge skills and attitudes related to subjects do teachers and students have? 2.8. How conducive is the classroom and school environment is?
Research Question 3: PROCESS: How well is the program working? R.Q.3. Probing Questions:
3.1. Are the administrative processes transparent and collaborative? 3.2. Has ICT been deployed in various school practices?
3.3. How actively students and teachers participate in different activities? 3.4. Is communication effective?
3.5. Is teaching-learning continuously evaluated?
3.6. How important other activities are apart from teaching learning for the school?
Research Question 4: PRODUCT: Were there changes in the student teacher’s knowledge, Attitude, values and ethics at the end of the program?
R.Q.4. Probing Questions:
4.1. What are the achievements of the school in co-curricular and extracurricular activities? 4.2. What assessment strategies and procedures are used for different components of school? 4.3. How do students use what they have learned?
4.4. Is there official report of each activity?
