Performance differences across schools can be at least partly related to differences in curricula. However, it is impossible to determine a causal impact of the curriculum on performance using only PISA data. The comparison between two study programmes will always be confounded by differences between students, teachers and schools that cannot be captured by questionnaires; even figures that account for socio-economic background or gender cannot be interpreted causally.
In most countries, there is a major distinction between vocational or pre-vocational study programmes and general study programmes. Generally, only a minority of 15-year-olds in each country is enrolled in vocational study programmes; the exceptions are Serbia, Croatia, Austria, Montenegro, Slovenia and Italy, where a majority of 15-year-olds students is enrolled in such programmes (Table V.4.2).
How are study programmes related to the unique aspects of problem-solving performance? This “relative performance in problem solving” of each study programme can be estimated by comparing the performance of students in each study programme only to students who share their same proficiency in mathematics, reading and science. Such a comparison can show whether good or poor performance in a subject is reflected in equally good or poor performance in problem solving; or, conversely, whether there is a specific advantage in problem solving for students in a particular type of study programme.
Figure V.4.2 shows that, in 4 of 31 countries and economies, namely Shanghai-China, Turkey, the United Arab Emirates and Malaysia, students in vocational study programmes have significantly better performance in problem solving than students with comparable performance in mathematics, reading and science who are in general study programmes. In all of these cases, the advantage of students in vocational programmes corresponds to at least 12 score points on the problem-solving scale. In all of these countries and economies, with the exception of the United Arab Emirates, more than one in eight students (more than 12.5%) attend vocational study programmes. Meanwhile, in the Russian Federation and Germany, students in vocational study programmes have significantly lower performance in problem solving than students with comparable performance in mathematics, reading and science. The gap between the two groups of students exceeds 24 score points on the problem-solving scale. In both countries, however, fewer than 5% of students are enrolled in a vocational study programme (Tables V.4.2 and V.4.4).
•
Figure V.4.1•
performance variation unique to problem solving As a percentage
of the total variation in performance 30 20 10 0 10 20 30 40 50
Variation between schools
(38.3%) Variation within schools (61.0%) Variation shared
with performance in mathematics Variation unique to problem solving
Note: The figure shows the components of the performance variation in problem solving for the OECD average. Source: OECD, PISA 2012 Database, Table V.4.1.
25.6%
12.7%
40.4%
20.6%
Figure V.4.3 uses the national classification of study programmes to highlight education tracks where students have significantly better performance in problem solving than students with comparable performance in mathematics, reading and science in their country who are enrolled in different study programmes.
Many of the differences in relative performance across study programmes concern countries or economies with overall weaker-than-expected performance in problem solving (see Figure V.2.15 and Table V.2.6); in these cases, a “relatively strong” programme may constitute an exception to the overall weakness. Students enrolled in general study tracks that prepare for higher education in Germany (Gymnasium) and in Hungary (Gimnázium), for instance, show stronger performance in problem solving, on average, than other German or Hungarian students with similar scores in mathematics, reading and science. While, overall, students in Germany and Hungary perform below students from other countries with similar performance in core subjects, this finding suggest that students outside of these general study tracks account for most of this negative result. In other countries, students from specific vocational programmes score higher than other students in their country who are similarly proficient in mathematics, reading and science. Such is the case for students in the vocational upper secondary programmes in the Flemish and German-speaking Communities of Belgium: they tend to score 8 and 25 points, respectively, above their expected level when compared to all Belgian students of similar proficiency in core subjects. Similarly, in Portugal, students in the professional upper secondary track score 17 points above their expected level. The performance gap in problem solving between students in the professional track and students in the general track is in this case smaller in problem solving than in mathematics, reading and science (Table V.4.5).
Fewer significant differences can be observed among countries whose students, overall, are relatively strong in problem solving when compared with students in other countries with similar proficiency in mathematics, reading and science.
•
Figure V.4.2•
relative performance in problem solving among students in vocational and pre-vocational tracks Difference in problem-solving performance between students in vocational or pre-vocational programmes
and students in general programmes with similar performance in mathematics, reading and science
1 2http://dx.doi.org/10.1787/888933003611 Score-point difference 30 20 10 0 -10 -20 -30 -40 -50
Note: Statistically significant differences are marked in a darker tone (see Annex A3).
Countries and economies are ranked in descending order of the score-point difference between students in vocational/pre-vocational programmes and those in general programmes with similar performance in mathematics, reading and science.
Source: OECD, PISA 2012 Database, Tables V.4.2 and V.4.4.
Percentage of students in pre-vocational or vocational programmes Students in vocational/pre-vocational programmes
perform above their expected level in problem solving
Students in vocational/pre-vocational programmes perform below their expected level in problem solving
Shanghai-China Tu rkey United Arab Emirates K or ea Mala ysia Italy France Colombia Chinese T aipei Belgium Por tug al Japan Slo vak R epublic Montenegr o Cr oat ia Mac ao-China Netherlands Chile Serbia
England (United Kingdom)
OECD a
ver
ag
e
A
ustralia Ireland Austr
ia Slo venia Cz ech R epublic Bulg ar ia Spain Urugua y Hung ar y German y Russian F ederat ion 21.2 38.1 2.7 19.9 13.3 51.5 15.3 25.2 34.5 44.0 16.7 24.2 8.2 66.0 70.1 1.6 22.2 2.8 74.4 1.2 15.4 10.9 0.8 69.3 53.2 31.0 40.8 0.8 1.4 14.3 2.0 4.1
•
Figure V.4.3 [Part 1/2]•
relative performance in problem solving, by education track Education tracks with a relative
strength in problem solving
Education tracks whose students’ performance in problem solving is in line with their performance
in mathematics, reading and science Education tracks with a relative weakness in problem solving Numbers in parentheses indicate the proportion of 15-year-olds in the study programme
OECD
australia General lower secondary (75.4%); Lower secondary with some vocational subjects (5.3%); General upper secondary (13.5%); Upper secondary with some vocational subjects (4.1%); Vocational upper secondary (1.5%)
austria Charter schools
(Statutschulen) (0.3%) Pre-vocational transition year (Polytechnische Schule) and lower secondary (Hauptschule) (14.6%); General lower and upper secondary leading to university entrance qualifications (AHS) (25.7%); Vocational school for apprentices (Berufsschule) (15.4%); Intermediate technical and vocational school (BMS) (11.7%); College for higher vocational education (BHS) (32.4%)
belgium Vocational upper secondary (Fl: TSO, KSO, BSO) (29.1%); Lower secondary (Ger.) (0.1%); Vocational upper secondary (Ger.) (0.2%)
Lower secondary (Fl.) (1.5%); General upper secondary (Fl.: ASO) (24.3%); Lower secondary (Fr.) (5.3%); General upper secondary (Fr.) (24.9%); Vocational upper secondary (Fr.) (10.5%); General upper secondary (Ger.) (0.4%); Vocational upper secondary, part-time programmes (Fl.,Fr.,Ger.) (0.5%); Special education (Fl.,Fr.,Ger.) (3.1%)
chile Lower secondary (5.5%); Upper secondary, first cycle (87.8%); General upper secondary, second cycle (3.9%); Vocational upper secondary, second cycle (2.8%)
czech republic Basic school (47.1%) General lower and upper secondary (Gymnasium) (19.3%); Vocational upper secondary with school-leaving exam (21.9%); Vocational upper secondary without school-leaving exam (8.4%); Special education (2.8%)
denmark Upper secondary (0.5%) Primary and lower secondary (88.3%); Continuation school
(11.2%)
Estonia Lower secondary (98.1%) General upper secondary (1.5%)
france Lower secondary (27.3%); Special education (lower secondary) (2.5%); General upper secondary (57.4%); Technical upper secondary (11.0%); Professional upper secondary (1.8%)
Germany General lower secondary with access to general upper secondary (Gymnasium) (36.1%)
Special education (2.8%); General lower secondary without access to general upper secondary (Hauptschule) (15.5%); General lower secondary without access to general upper secondary (Realschule) (33.5%); General upper secondary (Gymnasium) (0.8%); Comprehensive lower secondary (Integrative
Gesamtschule) (9.3%)
Pre-vocational and vocational (Übergangsjahr, Berufsschule,
Berufsfachschule) (2.0%)
hungary General upper secondary
(Gimnázium) (38.2%) Vocational upper secondary with access to post-secondary and tertiary (36.2%); Vocational upper secondary without access to post-secondary and tertiary (14.3%)
Primary school (11.3%)
ireland Transition year programme (24.3%) Applied upper secondary (Leaving certificate applied) (0.8%); General upper secondary (Leaving certificate) (7.4%); Vocational upper secondary (Leaving certificate vocational) (5.1%)
Lower secondary (Junior certificate) (62.4%)
italy Scientific, classical, social science, scientific-technological, linguistic, artistic, music and performing arts high schools (45.9%); Technical institute (29.0%); Vocational institutes (service industry, industry, arts and crafts workers) (17.0%); Vocational training, vocational schools of Bolzano and Trento provinces (5.5%)
Lower secondary (2.6%)
Japan General upper secondary (74.4%); Vocational upper secondary
(24.2%)
korea Lower secondary (5.9%); General upper secondary (74.2%);
Vocational upper secondary (19.9%)
netherlands Practical preparation for labour market (PRO) (2.5%); Pre-vocational secondary, years 1 and 2 (VMBO 1 & 2) (2.4%); Pre-vocational secondary, years 3 and 4, basic track (VMBO BB) (8.4%); Pre-vocational secondary, years 3 and 4, middle management track (VMBO KB) (11.4%); Pre-vocational secondary, years 3 and 4, theoretical and mixed track (VMBO GL/TL) (24.4%); Senior general secondary education (HAVO), leading to university of applied sciences (25.9%); Pre- university (VWO) (25.1%)
Portugal Professional upper secondary
(7.2%) Lower secondary (35.6%); General upper secondary (47.7%); Vocational training (CEF - Curso de Educação e Formação) (9.3%)
Slovak republic Specialised upper secondary
with school-leaving exam (26.1%) General lower secondary (41.6%); Special education (1.2%); General lower and upper secondary (Gymnasium) (22.9%); Specialised upper secondary without school-leaving exam (ISCED 3C) (8.2%)
Note: Numbers in parentheses indicate the proportion of 15-year-olds in the study programme; percentages may not add up to 100 within each country/economy because
of rounding and of rare programmes for which results are not reported. Only countries/economies with results reported for more than one study programme are included in this figure. The middle column includes all programmes for which relative performance in problem solving is not statistically different from 0 (see Annex A3). In Belgium, the information about study programmes in variable PROGN was combined with information about regions to identify education tracks: “Fl.” refers to the Flemish Community, “Fr.” to the French Community, and “Ger.” to the German-speaking Community; results for “Part-time vocational” programmes and “Special education” programmes are reported at the national level. In Germany, students in schools with multiple study programmes are classified according to their specific education track.
•
Figure V.4.3 [Part 2/2]•
relative performance in problem solving, by education track
Education tracks with a relative strength in problem solving
Education tracks whose students’ performance in problem solving is in line with their performance
in mathematics, reading and science Education tracks with a relative weakness in problem solving Numbers in parentheses indicate the proportion of 15-year-olds in the study programme
OECD
Slovenia Technical upper secondary
(38.3%) General upper secondary (technical gymnasiums) (7.6%); Basic (elementary) education (5.4%) General upper secondary (general and classical gymnasiums) (33.8%); Vocational programmes of medium duration (13.8%); Vocational programmes of short duration (1.1%)
Spain Lower secondary (99.2%); Initial vocational qualification programme
(0.8%)
Sweden General, compulsory basic (97.8%); General upper secondary
(1.8%)
turkey Anatolian vocational high school (5.7%); Technical high school (1.5%); Anatolian technical high school (2.5%)
Primary school (2.7%); General, science, and social sciences high school (32.2%); Anatolian high school (22.5%); Vocational high school (24.7%); Multi programme high school (3.7%)
Anatolian teacher training high school (4.5%)
England (united kingdom) General upper secondary, compulsory (Students studying mostly toward GCSE) (97.7%); Vocational upper secondary, compulsory (Students studying mostly towards a level 1 Diploma) (0.9%)
General upper secondary, post- compulsory (Students studying mostly for AS or A Levels) (1.1%)
Par
tners
bulgaria General upper secondary,
specialised (47.6%) General upper secondary, non-specialised (6.7%); Vocational upper secondary (40.8%) Lower secondary (4.8%)
colombia General upper secondary (35.7%); Vocational upper secondary
(25.2%) Lower secondary (39.1%)
croatia Gymnasium (29.9%); Four year vocational programmes (46.7%); Vocational programmes for industry (6.5%); Vocational programmes for crafts (15.2%); Lower qualification vocational programmes (0.8%)
macao-china Lower secondary (54.9%); General upper secondary (43.5%); Pre-vocational or vocational upper secondary (1.6%)
malaysia Vocational upper secondary
(13.3%) Arts upper secondary (44.8%); Religious secondary (3.3%); Lower secondary (4.0%) Science upper secondary (34.6%)
montenegro General upper secondary school or gymnasium (33.6%); Four-year vocational secondary (60.0%); Three-year vocational secondary (6.0%);
russian federation General upper secondary
(13.4%) Lower secondary (82.5%); Vocational upper secondary (technikum, college, etc.) (2.2%) Vocational upper secondary (professional schools, etc.) (1.9%)
Serbia Arts upper secondary (1.6%) General upper secondary (Gymnasium) (24.0%); Technical upper secondary (30.3%); Vocational technical upper secondary (6.5%); Medical upper secondary (9.3%); Economic upper secondary (18.8%); Vocational economic upper secondary (3.0%); Agricultural upper secondary (4.2%)
Shanghai-china Vocational upper secondary
(19.8%) General upper secondary (34.3%) General lower secondary (44.4%)
chinese taipei Junior high school (36.4%); Senior high school (29.1%); Vocational senior high school (30.6%); Five-year college (not including the last two years) (4.0%)
united arab Emirates Vocational secondary (2.7%) General lower secondary (15.0%); General upper secondary (82.3%)
uruguay General lower secondary (31.4%); Lower secondary with a technological component (5.3%); Lower Secondary with a very important technological component (2.9%); Vocational lower secondary (1.3%); General upper secondary (50.2%); Vocational upper secondary (more than one year) (1.3%)
Technical upper secondary (6.2%)
Note: Numbers in parentheses indicate the proportion of 15-year-olds in the study programme; percentages may not add up to 100 within each country/economy because
of rounding and of rare programmes for which results are not reported. Only countries/economies with results reported for more than one study programme are included in this figure. The middle column includes all programmes for which relative performance in problem solving is not statistically different from 0 (see Annex A3). In Belgium, the information about study programmes in variable PROGN was combined with information about regions to identify education tracks: “Fl.” refers to the Flemish Community, “Fr.” to the French Community, and “Ger.” to the German-speaking Community; results for “Part-time vocational” programmes and “Special education” programmes are reported at the national level. In Germany, students in schools with multiple study programmes are classified according to their specific education track.
Source: OECD, PISA 2012 Database, Table V.4.5.
12 http://dx.doi.org/10.1787/888933003611
Students in arts upper secondary programmes in Serbia seem to beat expectations by an even greater margin than other students in that country, but fewer than 2% of all 15-year-olds are in these programmes. In Italy, students who are held back in lower secondary education (about 2.6% of all 15-year-olds) are relatively weak in problem solving, even after accounting for differences in mathematics, reading and science performance. These students, therefore, do not seem to contribute to the overall (relative) strength of Italy’s students in problem solving.
Strong performance in problem solving among students in certain education tracks, relative to their performance in the other subjects assessed by PISA, can be interpreted in two ways. On the one hand, the curriculum and teaching practices
in these programmes may promote authentic learning, and equip students for tackling complex, real-life problems in contexts that they do not usually encounter at school. On the other hand, better-than-expected performance in problem solving may be an indication that in these programmes, students’ potential is not nurtured as much as it could be within the core academic subjects.