Although instructional systems are validated prior to implementation, students may still have difficulty with the instruction during day-to-day system operation. Possible causes of student problems are:
·Instructors do not follow the POI or course syllabus.
·The developed course is different from the course that is actually implement ed. ·Resources required to support, operate, and maintain the system are inadequate. ·Instructional materials are not correlated.
·Students do not meet course prerequisites.
Periodic internal evaluations may identify weaknesses (problems) as well as strengths of the instructional system.
Data Collection
Several methods of collecting internal evaluation data are listed:Data Collection Method Purpose
Review Course Control Documents
· To determine if there are any discrepancies between the planned course and the course that was actually implemented.
and support) are available and adequately maintained.
·To ensure that equipment (instructional, support, and test) and supplies are available.
·To ensure that human resources
(instructional developers, instructors, students, and mainte nance personnel) are available.
·To ensure that there is adequate time (adequate course length, sufficient time to maintain the course).
·To ensure that funds are adequate to sup- port, operate, and maintain the course.
Visit Instructional Facilities · To evaluate the quality of
implemented instruction (ensure that the visit is long enough to ensure observation of representative instruction).
·To check equipment, instructional media, training aids and devices for
condition, operation, and appropriateness.
·To check instructional literature such as study guides and workbooks for quality and availability.
Evaluate Instructor Performance · To check if instructor follows the plan of instruction, uses instructional media properly, responds to student needs, and is qualified to teach. ·To check instructor evaluation forms to
determine if noted weaknesses have been corrected.
Monitor Measurement Program · To check the measurement program
for compromise. If a test has been compromised,
it cannot provide useful feedback. ·To monitor the measurement program to
ensure quality.
·To evaluate instruction in terms of student performance. Use
performance mea sures to determine students' achieve ment of objectives.
Conducting an
Internal
Evaluation
Collect sufficient internal evaluation data for the analysis. Insufficient data may skew the analysis results, possibly leading to incorrect decisions being made. Job aids can be used to gather internal evaluation data. An example of a job aid is provided below.
Check Data Source
· Does the POI/course syllabus reflect the operational course? · Is the POI/course syllabus current and accurate?
· Does the POI/course syllabus provide adequate guidance? · Do the lesson plan and POI/course syllabus agree?
· Is the lesson plan current and accurate?
· Do instructional materials support the lesson plan and POI? · Do instructional facilities meet system requirements? · Do support facilities meet system requirements? · Does training equipment meet system requirements? · Is the instructional equipment adequately maintained? · Does support equipment meet system requirements? · Are instructors teaching according to the lesson plan?
· Are instructors oriented and trained to execute the courses, i.e., have they been given the "big picture"?
· Are they adequately trained?
· Do tests adequately measure the objectives? · Is the test data thoroughly analyzed? · Can improvement be made in the course?
PERIOD_____________________________________________________________________ DATE_________________________________
INSTRUCTOR_________________________________________________________STUDENT_____________________________ ____ ______
One way instruction is improved is by sampling student reaction to the instruction. Please answer the following questions. 1. Prior to this instruction, my experience in this area was
__________ extensive __________ moderate __________ little or none
2. Did your knowledge of the subject increase as a result of the instruction? __________ yes
__________ no
3. If your knowledge increased as a result of the instruction, to what extent did it increase? __________ not applicable (my knowledge didn't increase)
__________ slightly __________ moderately __________ extremely
4. Based on my experience, the level of instruction was __________ too advanced
__________ about right __________ too elementary 5. The organization of the instruction was
__________ very helpful __________ helpful _________ not very helpful
6. The lecture outline (main points of instruction) was __________ very helpful
__________ helpful __________ not very helpful 7. Audiovisual aids were
__________ of great value __________ valuable __________ of little or no value
__________ not used, but could have helped __________ not used and not needed 8. Answers to student questions were
__________ meaningful __________ somewhat helpful __________ not helpful
__________ not applicable (no questions asked) 9. Should the subject matter covered be changed?
__________ yes (please explain below) __________ no
10. Should the method of instruction be changed? __________ yes (please explain below) __________ no
11. Overall, the instruction was __________ outstanding __________ good __________ fair __________ poor
12. Instruments (including tests) to evaluate student performance were __________ outstanding
__________ good __________ fair __________ poor
Data Analysis
Before beginning analysis of the data, ensure that:·Data have been collected from each component of the instructional system.
·Adequate data samples are collected in order to validate the reliability of the findings. Following are some methods of analyzing the internal evaluation data.
·Compare the instructional standard with the POI/course syllabus to determine if the requirements of the standard are being met.
·Compare POI/course syllabus with operational course to determine if the planned and operational courses are the same.
·Review POI/course syllabus, lesson plan, and instructional material to determine if they are current, adequate, and in agreement.
·Compare stated resource requirements with actual resources to determine if adequate resources are available to support, operate, and maintain the instructional system. ·Review records to determine if instructors are qualified to teach the course.
·Review test data to ensure that students are meeting course objectives. ·Analyze test data to determine if test items are valid and reliable.