Context and participants

The research was conducted at polytechnic TVET colleges in the Addis Ababa administrative region that have been promoted from college level to polytechnic status mainly based on the relative qualifications of the teaching staff and resource endowment. Four of the six government polytechnic TVET colleges in the region were selected for this study. The selected polytechnic colleges have large numbers of teachers and students and provide a range of programs at all levels of training compared to the other non-polytechnic TVET colleges. This makes it possible to contact students attending training from level III to level V because level V training is only provided in polytechnic colleges. The polytechnic colleges have been providing training for many years and are equipped with better training resources than other TVET colleges and institutions in the region. Data were collected with the permission of the respective college administrations.

Participants included TVET teachers, students, employed TVET graduates, and job supervisors. Teachers who were willing to participate were approached through training coordinators in the TVET colleges. A total of 160 questionnaires were distributed to TVET teachers in the four polytechnic colleges, of which 126 (79%) were returned. However, 13 questionnaires were discarded from the study because the questionnaires were either incomplete or filled in inaccurately. Students who were available in the colleges during data collection periods and volunteered to participate were asked by teachers assigned for data

collection purposes to fill in the questionnaire. TVET graduate employees and job supervisors were approached through employees working in the organizations recruited for data collection purposes. The participants were 113 TVET teachers, 353 students, 87 employed graduates, and 20 job supervisors.

Most of the teachers (78%), students (58%), graduates (76%) and job supervisors (100%) were male. About 68% of the teacher participants had over six years of teaching experience in competence-based programs. Most of the teachers (78%) provided training at all TVET training levels; 74% were first degree holders and 21% were Master’s degree holders. Teacher, student and graduate participants came from both soft skill occupations such as business and hotel/tourism and hard skill occupations such as construction, automotive, electrical, manufacturing and information technology. All the supervisors had supervisory experience of more than five years with academic qualifications ranging from a two-year college diploma to a Master’s degree.

4.3.3 Instruments

Based on the comprehensive CBVE model, statements reflecting the CBET principles were developed as a basis to measure the realization of CBET in the TVET programs. The participants (teachers, students, and employed TVET graduates) rated the statements in questionnaires using a six-point Likert-type scale with response categories ranging from strongly disagree=1 to strongly agree=6 (scale is the same for all tables). Three separate questionnaires were prepared for teachers, students and employed graduates. In this study, CBE realization was examined using ten CBE principles. The self-reflection and self-steering principles were combined because they are very much related to each other. The principle on professional core problems could be addressed through the qualification principle because the qualification profile principle is a reflection of the principle of professional core problems. The teacher, student and graduate questionnaires had separate sections for each of the eight CBET principles, with statements that are believed to measure the aspects of a particular principle. The statements on the aspects of the CBE principles are the same for teachers, students and graduates. For example, questionnaire items such as “The graduate (qualification) profile fully reflects the critical competences of my area of training” and “In my area of training, knowledge, skills and attitudes are taught (learned) separately” were included to measure qualification profile and knowledge, skill and attitude integration principles respectively. Questionnaire items such as “Assessment includes students

demonstrating competence through performance” and “Learning activities are determined by discussion/negotiation with students” were included to measure assessment and learning activity principles respectively.

The graduate questionnaire includes open-ended questions such as “What do you want to be in the future?” and “In what career development activities are you engaged for your career scheme?” The purpose of the open-ended items was to have an overview of what graduates were doing towards their future career development after employment. The employed graduates questionnaire also incorporated statements on factors that influence their job performance, focusing on the influence of education and training. Compiled from the literature (Khaya, 2007; Kim & Jogaratnam, 2010; Pavin & Kabir, 2011; Saeed et al., 2013), four factors (independent variables) believed to influence employees’ job performance (dependent variable) were examined for this study.

 Education and training (measured by eight items) includes items such as “I got the appropriate practical experience in the college” and “I sufficiently developed the skill to work with others in teams”.

 Job conditions: measured by seven items such as “The job I am assigned is related to my specialization” and “The work environment is suitable for applying my competence”.  Organizational conditions measured by seven items such as “Management support and

encouragement is motivating” and “My salary is commensurate with my level of competence”.

 Social factors include three items such as “The perception of my community towards TVET is favorable” and “Family responsibility has made me late for work”.

 Graduate job performance (dependent variable) was measured by nine items such as “I complete my work on time” and “I am satisfied by my accomplishments on the job”. The questionnaire for job supervisors consisted of three main parts: graduates’ job placement (4 listed alternatives), possible competence-related gaps observed in graduates (12 listed items), and job supervisors’ ratings of employed graduates’ performance. Job supervisors’ ratings of graduates’ job performance consists of seven items such as “Graduates complete tasks on time” and “We are satisfied with their job accomplishments”. These are the same as the questions for the employed TVET graduates but expressed differently.

4.3.4 Data analysis

First we tested the reliability of items using Cronbach’s alpha to measure the internal consistency of the scale. Most of the values of Cronbach’s alpha were above 0.7, except a few items which were > 0.6 < 0.7. Following the suggestion of George and Mallery (2003), Cronbach’s alpha values of less than 0.6 were excluded from the ANOVA analysis. Second, mean and standard deviation were used to summarize and describe the teachers’, students’ and graduates’ responses on the realization of the principles of CBET (Table 4.1). Third, ANOVA and post hoc Tukey tests were performed to find out whether the observations of teachers, students and graduates on CBET principles were significantly different from one another (see Table 4.2). The average of the measurement scale (3.5) was taken as a threshold for rating the realization level of CBET and comparing the ratings of graduates and job supervisors. A one-way ANOVA was also conducted to see if there was a statistically significant difference between employed TVET graduates’ and job supervisors’ ratings of graduate job performance. A Spearman’s Rank Correlation (rs) was run to examine the relationship between the various factors and the job performance of employed TVET graduates (Table 4.7).