Teacher and Peer Effects in Higher Education: Evidence
from a French University
Thibault BRODATY
∗CREST-INSEE, Paris School of Economics† &Université Paris-Dauphine
Marc GURGAND
Paris School of Economics‡&CREST-INSEE
October 2008
Very Preliminary
Abstract
Using a quasi-random allocation of students to groups in a French University, we are able to estimate peer effects and teacher effects, with a specific attention to non-linear effects. We find that both teacher and peer effects are significant, but only the initially weakest students are influenced by the presence of high performing students. Other students are hardly affected. This implies that it is optimal to mix students. Moreover we show that teacher and peer effects are of comparable order of magnitude. It implies that having good peers is as important as having good teachers, and the other way round.
Keywords: Higher Education, Peer Effects, Teacher Effects, Random Assignement.
∗ Thibault Brodaty: kthibault.brodaty‘’at”ensae.fr; BCREST - LMI, 15 Boulevard Gabriel Péri, 92245
Malakoff Cedex, France. Tel: +33(0)1 41 17 59 02. Marc Gurgand: kgurgand‘’at”pse.ens.fr; BParis-Jourdan
Sciences Economiques, 48 Boulevard Jourdan, 75014 Paris, France. Tel: +33(0)1 43 13 63 05. We thank Paris-Dauphine University for giving us the access to the data, and Bernard Guillochon and Jean-Marie Janod for their help in collecting the data. We thank seminar participants at CREST and Paris-1 for useful suggestions and comments. The usual disclaimer applies.
†Université Paris-1 Panthéon-Sorbonne.
1
Introduction
Peer effects in education have received a great deal of attention among economists in the recent period. One reason is that, if peer effects are present, efficiency of schooling remediation policies is enhanced by a so-called social multiplier. But it is also decisive to the debated issue of school segregation. From a positive point of view, peer effects are important to models that explain stratification and selective entrance into schools. For instance, peer effects are the source of stratified equilibria in Epple and Romano’s model (1998), provided that high ability students value more sitting with high ability peers than do low ability students, a very specific structure. From a normative point of view, mixing classes has a distributional impact, which depends on the very shape of peer effects, because resorting students would create winners and losers. Mixing classes can also be desirable for efficiency reasons: it is generally efficient to generate heterogenous groups if peer effects are stronger for low ability students. As illustrated by Arnott and Rouse (1987), optimal decisions are very sensitive the educational production function if the social planner has to decide simultaneously over the allocation of students and resources. Therefore, the existence, importance and the details of the structure of peer effects are decisive to several major policy issues.
In this general context, higher education raises specific issues. It is most often much more selective than compulsory education: does the significance of peer effects explain this situation, and is it efficient that universities are strongly stratified? Also, most countries experience enlarg-ing access to higher education: does this have any visible impact over performance because of the change in peer environment ? Although there is a large recent empirical literature on peer effects in education, some of which is attentive to their very shape and possibly non-linear features, there is only limited evidence on higher education. Arcidiaccono et al. (2007), De Giorgi et al. (2006), De Paola and Scoppa (2007) find some effects, whereas Martins and Walker (2006) do not. These papers, however, do not document how peers may affect differently students of different initial ability. In contrast, this is considered in detail by Kermer and Levy (2003), Sacerdote (2001),
Winston and Zimmerman (2004) and Zimmerman (2003) who find some non-linear effects, but not entirely consistent ones. Furthermore, this series of papers is attentive to social interactions between roommates, not classmates, which may be less relevant to the organization of education.
In this paper, we estimate class level peer effects at a French University, with a focus on detecting non-linear effects. We allow characteristics of the distribution of peers (not just the mean) to have specific impacts and allow students of different initial ability to be affected dif-ferently. This adds to the current limited knowledge on this question. More specifically, this University happens to have experienced strong shifts in the ability of selected students over a short period of time. Did high level students suffer from weaker selection? And did (remaining) weaker students benefit from an even better environment? Naturally, the external validity of the peer effects estimated in this context is questionable. This University is a strongly selective one by French standards and one in which all teaching is provided in small classes, an exceptional situation. As a result, neither the technology nor the population is typical. However, this is an interesting laboratory to learn more about the very structure of peer effects. We complement this estimation with the measure of teacher effects: this provides a benchmark to judge the importance of peers.
As is well understood since Manski’s (1993) seminal paper, identification of peer effects is difficult. We do not attempt to estimate endogenous effects, that is effects on current behavior, as we only consider a reduced form and estimate the impact of predetermined ability measures. Even then, the challenge lies in separating the effect of peer characteristics from unobserved group effects. We will argue that, in this university, group formation is as good as random, and test for it. This allows straighforward identification.
The strucure of the paper is as follows. Section 2 presents to the institutional context and data, Section 3 introduces the model and discusses the identification strategy, Section 4 presents the empirical results and Section 5 concludes.
2
Institutional context and data
We are considering an undergraduate economics program of a French elite public University, with a yearly inflow of typically 700 to 800 students. Students are assigned to small groups of about 30, and all the teaching is given to the group. There is no lecture given to the whole cohort and the groups are fixed for the whole academic year. This is a very exceptional situation in the French system, and one that makes it possible to observe peer effects and teacher effects altogether in higher education.
We will consider the two first years of the program (”L1” and ”L2”) and use as outcomes end-year grades in the following 7 topics: Maths, Microeconomics (1st and 2d semester), Macroeco-nomics (1st and 2d semester), Statistics, Computer Science. Most teachers teach the same topic in several groups, and occasionally also teach several different topics. The grades in each of the 7 topics are based on a general exam that is common to all students. It is thus comparable accross groups. However, it is different every year, so that between year comparison may not be a source of identification. Because failing students are allowed to take a second chance in September, we will sometimes distinguish beween the June and the September results.
We are considering the years 2002 to 2006. The practice and legal status of this University is however specific and it has evolved over this period. In 2002, it would select students, although not under a fully recognized process. In 2003, in contrast, it was compelled by the educational authority, to welcome any student with a High school degree from its district. Finaly, since 2004, it can be openly selective. The consequence of this evolution on the recruitment is very impressive. Table 1 shows the High school performance of the inflow over those 5 years. The High school degree (Baccalaureat), that conditions entry into higher education, is delivered through a formal examination: the first 3 lines of the table give the share of the inflow whose average grade at this exam was 10-12, 12-14 or 14-20 out of 20 (the degree is not delivered, thus access to University barred, below 10). The share of weak students has decreased from 34% to 5%, whereas the share of very good students has increased from 18% to 67%. Year 2003 is clearly
a break in this trend, as the share of non-selected, thus on average weaker, students had to be larger. The share of high school repeaters follows a similar movement, and the share who is currently a repeater in the L1 level follows a lagged and somewhat attenuated movement. No such strong changes can be observed for the High school major (economics, maths or science), reflecting a willingness from the University to balance these populations.
In such a context, it is interesting to wonder about the social impact of such a policy, at least in relation to peer composition. Is there any visible gain - to the best students and to the weaker ones - at moving towards more homogeneous and high quality cohorts of students? As is well known, if there is non-linear peer influence, the peer composition of classes encompasses efficiency, not just distributional, effects. Naturally, we can only provide a very indicative assesment of the social desirability of such a move, as we have no information over the outcome of students outside this University.
We have collected data over all of the 2002 to 2006 inflow students in L1; we only follow the 2002 and 2003 cohorts in L2 (thus observed in 2003 and 2004). Some of the students asked for admission through the selective process: we then have full information about their High school performance. Some students were admitted outside the selective process: some did not submit a file (we thus have no High school information for them); others had submitted a file, were rejected based on it, but managed to be incorporated, something that occurrred frequently in 2003. The first panel of Table 2 shows the initial sample. In 2003, half of the students were incorporated although they did not pass the selection, but only 2% were in this situation in 2005 and 2006.
Part of the registered students (7% to 8% and almost 13% in 2003), did not take the exams. This situation is much rarer in L2. We excluded those non-participants from the sample, although it is not clear how long in the year they attended class, thus contributed to peer effects. The second panel describes the sample once non-participants are excluded. In L1, there is missing information on initial academic performance for the students that did not submit a file. There are 4% to 5% of them at the beginning of the period and only 1% to 2% at the end. In L2,
lagged performance is based on the grades obtained in L1: because a number of students step in directly from another university, this information is missing for them. This represents larger numbers: 10% to 20%. We have to drop those observations. As we will argue below that group formation is as good as random, this does not generate systematic bias in the measure of relative group quality. However, it generates measurement error on peer quality.
The final sample, with only participants to final exams and available information on initial academic quality, is described in the third panel of Table 2. We end up with 22 to 27 groups each year with 25-30 students per group, representing 3,457 students overall in L1 and 1,136 in L2. The evolving general quality of the cohorts is illustrated by the share of non-selected students, the grades and the percentage failed.
3
Econometric Framework
3.1
The model
In this section we present how we model students grades. Consider an individual i from cohort
c, in group g. Denoteyigmkc the grade of studenti, taught by teacher k in coursem. The grades
depend essentially on the quality of i, the quality of her peers in her group and her teacher in course m. Suppose that we observe a continuous quality index Ii for each individual i. We will
use this index in its initial continuous form and a qualitative transformation of it, based on its quantiles.
3.1.1 Model with the continuous quality index
The model is
yimgkc =a+αIi+βIg−i+γIi ∗Ig−i +δk+γmc+νi+ǫimgkc (1)
whereIi is the quality index of studenti. To facilitate the interpretation, the index is normalized
with a minimum equals to zero and a variance equals to one. I−i
g students,excluding her. δkis the effect of teacher k;γmc is the effect of coursemfor cohortc, it
takes into account the fact that exams difficulty and notation standards are potentially different between courses and cohorts; νi is an individual random effect reflecting the unobserved part of
student ability andǫimgkc is an iid shock.. Model parameters are easily interpreted. The effect of
a one standard deviation increase in I−i
g on the grade isβ+γIi. Thusβ is the effect on the worst
students (with Ii = 0) and γ tells us how this effect changes when students quality increases.
First note that the quality of students does not depend on the course: we suppose that students have the same ability in Maths, Microeconomics, Macroeconomics, Statistics and Com-puter Science. The goal of this asumption is to increase the length of the course dimension of the panel.
In this specification peer effects are totally taken into account by the average quality variables. As Arcidiacono et al. (2007) the grades of the other students in the group are not included, which means in the words of Manski (1993) that we don’t consider any endogenous peer effect. We therefore identify a reduced form that mixes the effects of exogenous and potential endogenous peer effects.
Teacher effects don’t depend on the cohort c, which means that the quality of the teach-ers is time constant. Therefore we don’t take into account potential increases of quality with experience, and teacher effects must be understood as an average quality over the period. More-over they don’t depend neither on the course m. In practice it implies that teachers who are present in different courses are considered as different teachers in each course. This asumption is stated because we don’t observe enough teacher/courses combinations. Therefore we can’t compare teachers between courses and we can only test the equality of teachers quality within each course. We can’t reject teacher effects if teachers dummies are significantly different within each course. Thus, a teacher is said to be better than another if, all else equal1
, the average grade of her group is significantly greater than the one of the other teacher. It is now clear that
1
in this work, the quality of a teacher is defined by her ability to prepare students for the current exams.
Finally, we suppose that all explanatory variables are independent of νi and ǫimgkc. This
hypothesis is justified on the grounds that allocation of students and teachers to groups is as good as random, as argued in the following section.
3.1.2 Model with quantiles of the quality index
The continuous index I can be discretized with respect to some percentiles to create a qualitative index with three categories, Low (L), Medium (M) and High (H). We will use the 25/75 and the 33/66 indexes. The model is
yimgkc =a+αL1{Ii=L}+αH1{Ii=H}+βLP −i gL +βHPgH−i +βLL1{Ii=L}∗P −i gL +βLH1{Ii=L}∗P −i gH+βHL1{Ii=H}∗P −i gL +βHH1{Ii=H}∗P −i gH +δk+γmc+νi+ǫimgkc (2)
where 1{Ii=L} and 1{Ii=H} are dummy variables indicating ifi’s quality is respectively Low and
High; PgL−i and PgH−i are respectively the proportions of Low and High quality students in her group, excluding her; Sacerdote (2001) uses the same three quantiles based categories and the same individual/group interactions.
The parameters corresponding to peer effects (βL,βH,βL,βLL,βLH,βHL,βHH) are not easily
interpretable under this form. But it is clear that the interactions between quality levels and the proportions of each level in the group allow to study the presence of non-linear peer effects, that is the effect of one level of student on another level. Peer effects are not supposed a priori
to be the same for all students. To facilitate the interpretation of the parameters, we show in Appendix A that if we replace in one group one percentage point of some quality students by one of some other ability, then the effect on a student of a particular quality can be expressed with model parameters. These expressions are presented in Table 3.
Both regressions considered in this work are linear random effects panel data models. They are estimated by Generalized Least Squares with robust standard errors.
3.2
Using Correspondence Analysis to construct quality indexes
For L2 students the choice of a measure of quality is straightforward. At the end of L1 the administration computes a total grade out of 200 based on exams and group specific grades. Students who have at least 100 have access to L2. Students below 100 in June can try in September to improve their score. These scores reflect students quality at the end of L1 and can thus be used to approximate their ability in L2.
For L1 students, we observe qualitative high school characteristics (High School examinations grade, High School Major, if they have repeated at least one grade in High School), if they have been selected and if they are currently repeating their L1. To take into account all the information, we have to summarize these variables into a single index. To do so, we use Multiple Correspondence Analysis, that allows to create a single continuous quality index for each student. Multiple Correspondence Analysis (MCA) is a statistical method widely used in Sociology, in Marketing Research and by french economists2
. Let’s describe very briefly the idea of this method3
. The primary goal of CA is to give a graphical representation of the relations between several qualitative variables. Suppose that we have K qualitative variables with mk categories,
k = 1...K. First transform these variables into PK
k=1mk dummies and consider the space in
which each dimension corresponds to one of these dummies (minus one for each variable). It is then possible to place each individual in that space, her coordinates being the zeros and the ones corresponding to the value of the dummies. From this representation we can calculate and place in the space the average point of each category, that is the representative individual of each category. If two categories are ‘’close” in that space, it means that the individuals who have
2
MCA has been first popularized in France by Jean-Paul Benzécri in the 1960’s, and is therefore largely taught in France.
3
one of the category have often the other one as well. The goal of MCA is to find a space with few dimensions (typically 2) such that the projection of all the points on it keeps the distance between points as much as possible. At the end of the process we can see on a two dimension graph the projections of the average point of each category and therefore evaluate the relations between variables. Moreover, each individual can also be projected on this low dimension space and has consequently a coordinate on its first axis. This coordinate is a continuous summary of each individual, and this is the variable we use to summarize the ability of each student in L1 after having achieved the MCA of all the qualitative characteristics previously listed.
3.3
Identification
3.3.1 Random assignment into groups
The main point concerning identification of peer effects is the question of random assignment into groups. Imagine for example that students self-select into groups and that this selection is based on ability. In this extreme example, good students are with good students and bad students with bad ones. Thus there is a positive correlation between own ability and peers ability. νi being in
both models (1) and (2) the unobserved part of student ability, there is in this case a positive correlation between νi and the average observed group quality, which upward biases peer effects
estimates. If the group assignment is random with respect to student quality, these effects don’t exist and estimates are unbiased.
Here is how assignment is carried out. Each year, some courses are compulsory (those that we analyse) and other ones are optional (that we don’t take into account). Students must choose a certain number of optional courses, and just before the beginning of the year, students give to the administration their preferred options. Then the administration makes groups trying to satisfy students preferences and the constraint that each group is compatible with 4 or 5 options. The groups are sequentially filled 5 by 5. Thus we see that, because the assignment is administrative, their can’t be any student self-selection into group, and we can therefore believe
that this process leads to a random assignment. In order to check statistically if this is really the case, we propose a way to test it. The idea is to use a random numbers generator to allocate students into groups and to test if the observed and simulated distributions of group quality can be considered as coming from the same distribution. If the answer is yes, it means that the real assignment process can’t be distinguished from an assigment based on a random generator. Consider for example the dummy variable that characterizes students who have repeated at least one grade in high school. We can calculate the proportion q of repeaters in each group and obtain a sample of observed proportions S={qg}g=1...G, where Gis the total number of groups.
When we simulate a first time the allocation of students into groups, we obbtain a first sample of simulated proportions S1∗ =
q∗1
g g=1...G. Repeating R times this simulation leads to a sample
of simulated proportions S∗ ={S∗
1, ..., SR∗}. The test compares the distributions ofS and S∗. If
we can’t reject that these two samples come from the same distribution, we will conclude that the real assignment process is random. We use the well known Kolmogorov-Smirnov two-sample test. The results of these tests are presented in Table 4 for L1 and Table 5 for L2. Concerning the L1, the tests are performed separately each year for all the students characteristics. For the L2 the tests are based on the discretized quality indexes. The numbers between brackets are the p-values of the Kolmogorov-Smirnov two-sample test. The results can’t be clearer: the tests never reject random assignment.
For the L2 an other point must be checked. In L2, students quality is based on the L1 total grade. Suppose that L2 and L1 groups are similar. It is clear that in this case potential L1 peer effects create a correlation between the average quality of the L2 group and the student’s own ability, and in particular with its unobserved part νi. Peer effects are therefore biased. If groups
are reallocated between L1 and L2 these effects don’t exist, and we checked that it is indeed the case (see Appendix B).
3.3.2 Heterogeneity of group quality
The quality of peer effects estimation depends primarily on the presence of very contrasted group composition in the data. Table 6 shows the distribution of the individual quality index. We pool the 2002 and 2003 cohorts on one hand and the 2004 to 2006 cohorts on the other hand, because each corresponds to a different regime with respect to the selection process. The mean value of the index in L1 is normalized to 0, so that changes in the quality of the students over those two subperiods is not visible in this table. We then compute the mean value of the index in each group: the between group variance is 0.042 for the 2002/03 period, representing 14% of the total variance. This implies that allocating individuals randomly in groups of 25 to 30 maintains significant contrasts, so that we can hope to estimate the impact of having different peer environment. The hypothesis that group means are equal is strongly rejected. For the 2004/06 subperiod, the between group variance is lower at 0.026: this is the direct result of an inflow of more homogeneous cohorts. As will appear later, this situation is less favorable for estimating precisely the impact of changing peer quality accross groups.
The L2 observations all belong to the 2002/03 cohorts, and have been selected through the L1 to L2 exam. Because the weakest students have made up their score in the September exam as compared to June, variances are generally smaller, to the point that between group differences are not very significant for September.
4
Empirical results
The main results of the paper are presented in Tables 8 and 9. Table 8 presents a simple specification with own quality (standardized), average group quality and own quality interacted with group quality, in order to capture, in a constrained way, possible non-linear effects. Quality is either the quality index based on High school performance for L1’s or L1 average grade for L2’s. The explained variable is the grade in each of the 7 topics listed above (grade out of
20), and controls are a dummy for each topic interacted with year, as well as teacher dummies. Separate pooled estimations are run for the years 2002/03 and for the years 2004/06 in L1; in L2, regression over grades after the June exam and grades after the additional September exam are shown.
Looking at L1’s in 2002/03, we can see that the direct effect of increasing either own quality or peer group average quality is positive and significant. The interaction, however, is significantly negative. At the average value of the own and group quality index (2.8), increasing own quality by one standard error, increases the grade by about 1 point (out of 20). In contrast, increasing group quality by the same amount (which is enormous because it is a class average) does not increase the outcome. The non-linearity is thus very strong, and peer effects are only positive for students below the average initial quality. Relying on this specitifcation, however, would imply negative peer effects for good students, which is not very sensible. A more flexible specification is considered below.
For L1’s from the following 2004/06 cohorts, significant peer effects are not found, although the point estimates are in a similar range. Our interpretation is that there is significantly less heterogeneity in this data, so that group peer composition is less constrasted and the tests have much less power, as indicated by the systematically larger standard errors.
Finally, the same 2002/03 cohorts now observed in L2 show comparable results as in L1, but only for the final September grades. This is the final grade, so the most complete measure of student outcome, especially for weaker students. The index is now simply based on the average grade at the end of L1. The sample average of this variable is 1.6 in September: it implies that an increase of one standard error of own quality at the average group composition increases grades by 1.6 (out of 20). A similar increase in group quality increases grades by 1.2 for the average student. Although still present, the non-linearity is less steep that in L1.
It is important that the 2002/03 L1 and L2 results point to the same direction, altough with different magnitudes, because the groups are completely reassigned: therefore, we have two independent group allocations that lead to similar peer effects. This indicates that findings are
robust.
Table 9 presents the results of a more flexible specifcation. Remember that we have classified all students into three parts based on the initial index : those who are among the 25% best in their whole cohort (”high”), those who are among the 25% weakest in their cohort (”low”), and the other 50% students (”medium”). We also define those 3 categories using a 33%-66% cutoff rule. In each class, we have computed the percentage of low, medium and high quality students. As indicated previously, a flexible specification, given the limited amount of data available, interacts the impact of each of those three percentages with the own quality of the student. This allows for the distribution of peer quality to matter, rather than just the mean, and for effects to be non-linear in the sense that peer quality does not affect different individuals the same way.
The corresponding coefficients of such a regression are not readily interpretable, because of normalizations. Rather, Table 9 presents the implied effects of making marginal changes in the peer composition on the results of each type of students, low, medium and high.
Looking at L1’s in 2002/03, we observe that peer effects affect significantlly the performance of students with low initial academic performance only. Reducing by 1 percentage point the share of low performing students, and replacing with high performing ones, increases their results by 0.047. In other words, substituting 1 student (about 3% of the group) from low to high increases the grade of low performing peers by about 0.15 points (out of 20). Shifting from medium to high and from low to medium is a less strong change: it has sometimes significant effects on low performing students, but of smaller magnitude. In contrast, medium and high performing students now simply do not seem to be affected by changes in the peer composition.
The corresponding effects using the 2004/06 cohorts are smaller and not significant. Re-member that the high-low-medium categories are based on quantiles in own cohort distribution. Therefore, a 25% ”low” level student in 2004 is better in absolute terms than a 25% ”low” level student in 2003, because the general quality of the cohort is itself lower in 2003. The low level students that are typically affected by the presence of good students in 2002/03 regression, may hardly be present among the low level students in 2004/06. This is another indication that peer
effects are really effective for the lowest part of the distribution.
At the L2 level, we find some significant impact of marginally changing peer composition from low to high and from medium to high, for the lowest performing students. Thus substituting one student would increase their grade by 0.1 to 0.15 (out of 20). Some effect on high performing students is found occasionally, which is consistent with the observation from Table 8, that non-linear effects, although present, are less strong in this sample.
Generally, these results point to two important features. First, having high quality peers instead of medium or low is a good, whereas having low quality peers instead of medium is not a bad. Changes in the lower part of the distribution do not seem to matter. Second, peer effects seem to affect mostly students with initial low performance. This implies that, for a given cohort of students, it is desirable to build heterogenous groups: it is beneficial to weak students and not detrimental to good ones.
Some of the empirical literature on non-linear peer effects is compatible with this finding: Sacerdote (2001) does find that top roommates is favorable bottom level students; less so to other top level students. Winston and Zimmerman (2004) also find that low and middle students are primarily affected by peer composition, but Zimmerman (2003) finds middle-level students to be mostly affected. Several of those results are not very robust, though: an interesting feature of our findings is that they are robust over different samples. Outside the higher education literature, results are quite different: Hoxby and Weingarth (2005) insist on the importance of limited heterogeneity, whereas Lavy et al. (2007) find that high acheiving peers have a good influence on rather good students and low acheiveing peers have negative effects on regular students. Quite different processes might be at work at different levels of the educational system. Indeed the context of an elite University, we do not expect low performing students to disrupt the class (Lazear, 2001), as may more likely happen at primary or secondary level.
Assessing the impact of selecting better and better students into the University is a more complicated matter, however. The good students do not get any clear benefit from having an even larger proportion of good student mates. As for weaker students, those who are lucky
enough to access this University under a more selective regime clearly benefit from being in even better classes. For those who are excluded, it is hard to figure out their loss without knowledge of their destination.
Table 10, 11, 12, 13 provide a summary of the teacher effects estimated in the model. All effects are identified within topics, so that we do not try to compare the efficiency of a teacher in maths and stats: the reason is that we don’t have a sufficient number of teacher that teach several topics at a time, to identify properly such parameters. The tables first present, for each topic and for each specification of the model a test of equality of teacher effects. Teacher impact is everywhere extremely significant. The table then displays some elements of the distribution of those effects: the impact of having the best rather than the worst teacher and the standard-error of the distribution of the effects. The strongest contrast is typicall arount 3 points in L1 and 2 points in L2, over a total of 20. But the standard error is more modestly between 0.5 and 1 points. As a comparison, imagine shifting 12% of a class from low performing students to high performing students, which is a standard contrast in the 2002/03 sample: according to previous estimates, this would raise the grade by about 0.5 points. In that sense, peer effects and teacher effects are of comparable order of magnitude, which implies that having good peers is as important as having good teachers, and the other way round.
5
Conclusion
In this paper, we estimate peer effects in higher education in a French University, with a strong emphasis on non-linear effects. Indeed, the details of the structure of peer effects, not just their existence, is important for economic analysis and policy recommandation, and there is only limited and sometimes inconsistent evidence on this in the literature. It is therefore urgent to provide additional results, in particular at the university level, where results found at lower levels are not expected to generalize.
following a rule that should not generate any grouping of students by academic quality. This is tested, and confirmed, using the group distribution of observed predetermined characteristics.
We find that peer effects are present, but are also very non-linear. First, not only does the mean value of peers matter, but also changes in their distribution. Recomposition taking place at the bottom of the distribution of qualities does not seem to matter much. What matters is really the share of students from the upper 25% or 33% of the whole cohort that is present in a class. Second, this peer composition seems to be important almost exclusively to the lowest performing students. As is well known, this implies that mixing students is efficient.
Our results are robust, in the sense that they are present in different samples of groups. They are not very precisely estimated, however, using cohorts that are quite homogenous to start with. Also, because of the specific pedagogical organization of this University, external validity should not be overclaimed. However, this very organization enables us to estimate peer effects in a context of stable social interactions in small classes, a situation where peer effects should be apparent if they can be important.
references
Arcidiaccono et al. (2007), "Estimating Spillovers using panel data, with an application to the classroom", mimeo.
Arnott and Rouse (1987), "Peer group effects and educational attainment", Journal of Public Eco-nomics, 32, 287-305.
De Giorgi et al. (2006), "Be careful of the books you reas as of the company you keep. Evidence on peer efects in educational chohices", IZA Discussion Papers 2833.
De Paola and Scoppa (2007), "Peer froup effects on the academic performance of Italian students", mimeo.
Epple and Romano (1998), "Competition between private and public schools, vouchers and peer group effects", American Economics Review, 88, 33-62.
Hoxby and Weingarth (2005), "Taking race out of the equation: school reassignment and the struc-ture of peer effects", mimeo.
Kermer and Levy (2003), "Peer effects and alcohol use among College students", mimeo.
Lavy et al. (2007), "Inside the black box of ability peer effects: evidence from variation in high and low achieversr in the classroom", mimeo.
Lazear (2001), "Education production",Quarterly Journal of Economics, 116, 777-803.
Manski (1993), "Identification and endogenous social effects: the reflection problem", Review of Economic Studies, 60, 531-542.
Martins and Walker (2006), "Student achievement and university classes: effects of attendance, size, peers and teachers", IZA Discussion Papers 2490.
Sacerdote (2001), "Peer effects with random assignement: results for Darthmouth roommates", Quarterly Journal of Economics, 116, 681-704.
Winston and Zimmerman (2004), "Peer effects in higher education", in Hoxby (ed.) Colleges choices: the economics of where to go, when to go and how to pay for it, Chicago University Press.
Table 1: Students Quality, by Cohort
2002 2003 2004 2005 2006
High School [10 12] 33.81 44.93 19.55 8.84 4.84
Examinations [12 14] 48.49 40.10 46.37 49.08 28.03
Grade (out of 20) [14 20] 17.69 14.98 34.08 42.08 67.13
Has repeated at least one Yes 11.53 16.97 11.10 7.39 4.67
grade in High School No 88.47 83.03 88.90 92.61 95.33
Economics 32.63 40.77 42.14 34.96 34.60
High School Major Science opt not Maths 29.23 25.46 27.87 35.49 33.39
Science opt Maths 38.14 33.78 29.99 29.55 32.01
Currently Yes 8.39 14.81 16.78 11.87 10.38
Repeating L1 No 91.61 85.19 83.22 88.13 89.62
Table 2: Sample Selection
L1 L2
02 03 04 05 06 03 04
Sample Size 870 738 849 817 635 748 627
Pre Sample Non Selected (%) 14.3 50.7 17.3 1.8 2.1 10.6 29.0
Non Participated (%) 8.2 12.7 7.7 6.0 6.9 1.5 2.1 Sample Size 799 644 784 768 591 737 614 Non Selected (%) 13.9 48.6 16.2 1.8 1.7 10.6 29.0 Non Participated (%) 0 0 0 0 0 0 0 Missing (%) 4.1 5.8 3.2 1.2 2.1 9.5 23.0 Initial # Groups 26 24 27 25 22 24 20
Sample Group Size 30.7 26.8 29.0 30.7 26.8 30.7 30.7 (2.2) (2.3) (1.3) (1.4) (1.9) (1.5) (1.3) Total June 105.5 97.9 100.6 117.3 118.6 141.3 138.8 Total Sept 107 101.2 103.8 119.6 121.2 142.9 137.4 Failed (%) 14.3 25.16 19.0 12.0 11.7 4.3 21.7 Sample Size 763 601 757 758 578 666 470 Non Selected (%) 10.0 45.1 13.7 0.7 0 11.7 37.9 Non Participated (%) 0 0 0 0 0 0 0 Missing (%) 0 0 0 0 0 0 0 Final # Groups 26 24 27 25 22 24 20
Sample Group Size 29.3 25.0 28.0 30.3 26.2 27.8 23.5 (2.5) (2.7) (1.5) (2.0) (2.2) (1.7) (1.6)
Total June 105.9 98.2 100.8 117.5 119.2 141.5 139.4
Total Sept 107.4 101.6 103.9 119.7 121.6 143.2 138.1
Failed (%) 13.8 23.8 18.8 11.7 11.3 4.8 20.2
Table 3: Interpreting peer effects parameters in the dis-cretized case
On Effect
1pt Low less Low (βH −βL) + (βLH −βLL)
1pt High more Medium βH −βL
High (βH −βL) + (βHH−βHL)
1pt Med less Low βH +βLH
1pt High more Medium βH
High βH +βHH
1pt Med less Low βL+βLL
1pt Low more Medium βL
High βL+βHL
Reading: The first line means that if one percentage point of Low students is replaced in some group by one of High students, then the effect on Low students of that group equals (βH−βL) + (βLH−βLL), where theβ’s are defined in (1).
Table 4: Kolmogorov-Smirnov Random Assignment Tests, L1 2002 2003 2004 2005 2006 Selected Yes 0.18 (0.42) 0.13 (0.79) − (−) − (−) − (−) No 0.15 (0.63) 0.14 (0.77) − (−) − (−) − (−) High School [10 12] 0.07 (1) 0.14 (0.78) 0.07 (1) 0.08 (1) 0.06 (1) Examinations [12 14] 0.06 (1) 0.15 (0.68) 0.13 (0.79) 0.12 (0.87) 0.24 (0.16) Grade (out of 20) [14 20] 0.14 (0.7) 0.07 (1) 0.12 (0.86) 0.08 (1) 0.18 (0.5)
Has repeated at least one Yes 0.13 (0.82) 0.15 (0.67) 0.14 (0.66) 0.15 (0.61) 0.06 (1)
grade in High School No 0.13 (0.77) 0.13 (0.85) 0.15 (0.58) 0.15 (0.62) 0.06 (1)
Economics 0.09 (0.99) 0.19 (0.39) 0.17 (0.42) 0.13 (0.83) 0.13 (0.83)
High School Major Science opt not Maths 0.12 (0.85) 0.12 (0.88) 0.11 (0.91) 0.07 (1) 0.13 (0.85)
Science opt Maths 0.13 (0.82) 0.17 (0.55) 0.09 (0.99) 0.14 (0.76) 0.1 (0.98)
Currently Yes 0.18 (0.37) 0.23 (0.17) 0.25 (0.08) 0.19 (0.37) 0.14 (0.8)
Repeating L1 No 0.19 (0.31) 0.24 (0.15) 0.22 (0.16) 0.17 (0.48) 0.16 (0.64) Note: Kolmogorov statistics and its p-value between brackets.
Table 5: Kolmogorov-Smirnov Random Assignment Tests, L2 June Sept 2003 2004 2003 2004 Low 0.14 (0.78) 0.19 (0.46) 0.11 (0.95) 0.15 (0.76) 25/75 Medium 0.1 (0.98) 0.16 (0.74) 0.15 (0.71) 0.18 (0.53) High 0.12 (0.91) 0.19 (0.49) 0.08 (1) 0.19 (0.52) Low 0.11 (0.93) 0.14 (0.83) 0.16 (0.57) 0.18 (0.58) 33/66 Medium 0.07 (1) 0.16 (0.74) 0.14 (0.77) 0.16 (0.73) High 0.12 (0.88) 0.1 (0.99) 0.11 (0.93) 0.12 (0.96) 50 Low 0.13 (0.82) 0.12 (0.94) 0.16 (0.61) 0.08 (1) High 0.15 (0.64) 0.09 (1) 0.15 (0.71) 0.18 (0.58)
Note: Kolmogorov statistics and its p-value between brackets.
Table 6: Group Quality Heterogeneity, Continuous Indexes
L1 L2
02/03 04/05/06 Total June Total Sept Mean 0 0 109.5 112.09 Quality Std 0.550 0.628 13.8 11.1 Index Min −1.539 −2.430 71 94 Max 0.634 0.481 163 163 Mean of Mean −0.015 0.001 109.3 112.0 Quality Std 0.211 0.165 4.3 2.5 Index by Min −0.429 −0.341 91.2 106.8 Group Max 0.339 0.403 117.1 117.1 Equality of F-test 4.32 2.02 2.4 1.36 Means p-value <0.0001 <0.0001 <0.0001 0.06 R2 0.1387 0.0680 0.0862 0.0508 Between Group 0.042 0.026 16.45 6.27 Variance
Reading: in the first block ‘’Quality Index”, we give summary statistics of the quality index for each cohort. In the second block we consider the mean of the quality index in each group and we give summary statistics of this distribution. Then we regress the quality index on group dummies and we test the equality of the coefficients of the dummies. We report finally theR2 of this regression and the between group
Table 7: Group Quality Heterogeneity, Discretized Indexes
L1 L2
25/75 36/66 Total June Total Sept 02/03 04/05/06 02/03 04/05/06 25/75 33/66 25/75 33/66 Mean 25.0 20.9 28.1 31.7 23.8 32.6 24.5 29.9 Std 14.4 11.3 15.6 14.1 15.5 15.5 9.7 10.5 Low Min 3.2 0 3.2 0 3.4 10.3 7.1 7.1 Max 54.1 41.3 61.5 59.3 90.0 95.0 43.4 55.5 Mean 61.5 59.2 47.4 38.7 52.1 35.1 51.1 37.3 Std 12.8 9.2 12.5 8.2 13.5 12.8 12.9 11.0 Medium Min 34.6 29.6 26.9 13.0 5.0 0 13.0 8.6 Max 87.0 81.4 80.6 58.6 77.7 59.2 75.0 57.6 Mean 13.3 19.7 24.4 29.5 24.0 32.1 24.3 32.7 Std 6.4 12.6 11.7 15.4 10.0 10.8 9.6 10.2 High Min 0 3.4 0 6.8 4.5 5.0 4.5 13.6 Max 29.0 70.3 48.3 74.0 48.2 55.1 48.2 55.1 Reading: in this table we compute the proportions of each level of quality in each group. We then report summary statistics of these distributions. 25/75 means that the quality index is dicretized at the 25th and 75th percentiles. Low means a quality below the 25th percentile, etc.
Table 8: Peer Effects Estimates with Continuous Quality Index
L1 L2
02/03 04/05/06 Total June Total Sept Quality Index 2.51∗ ∗∗ 1.84∗ 1.72∗ ∗∗ 2.43∗ ∗∗ (.42) (.88) (.45) (.39) Group Quality 1.45∗ ∗ 1.41 .46 1.99∗ ∗∗ (.48) (1.01) (.43) (.45) Quality Index −.51∗ ∗ −.35 −.04 −.54∗ * Group Quality (.15) (.23) (.16) (.23) Note: Standards errors between brackets. *, **, *** are significance levels at 5%, 1% and 0.1% respectively.
Table 9: Peer Effects Estimates with Discretized Quality Index
L1 L2
25/75 36/66 Total June Total Sept
On 02/03 04/05/06 02/03 04/05/06 25/75 33/66 25/75 33/66 Low .047∗ ∗ .010 .047∗ ∗∗ .003 .025∗ .024∗ ∗ .046∗ ∗ .038∗ ∗
(.019) (.015) (.011) (.009) (.011) (.009) (.016) (.012)
1pt Low less Medium −.015 −.006 .001 −.014∗ .013 .005 .020 .018
1pt High more (.013) (.008) (.009) (.007) (.009) (.010) (.011) (.011)
High −.031 −.008 −.020 −.010 .008 .011 .001 .019
(.026) (.011) (.012) (.007) (.012) (.010) (.015) (.011)
Low .014 −.011 .026∗ −.009 .026∗ .036∗ ∗ .042∗ ∗∗ .031∗ ∗
(.021) (.015) (.014) (.011) (.013) (.011) (.011) (.010)
1 pt Med less Medium −.021 −.006 −.008 −.017 .005 .016 .021∗ .021
1 pt High more (.013) (.007) (.009) (.010) (.009) (.011) (.009) (.011)
High −.018 .000 −.014 −.004 .013 .029∗ ∗ .017 .026∗
(.028) (.010) (.013) (.010) (.013) (.011) (.012) (.011)
Low −.033∗ ∗ −.021 −.020 −.013 .001 .012 −.003 −.006
(.010) (.014) (.011) (.010) (.007) (.007) (.011) (.010)
1 pt Med less Medium −.006 .000 −.010 −.002 −.007 .011 .001 .002
1 pt Low more (.008) (.008) (.008) (.010) (.008) (.010) (.008) (.010)
High .012 .009 .006 .006 .005 .018 .016 .007
(.014) (.015) (.011) (.012) (.011) (.010) (.011) (.010)
Note: Consider a Low Quality (<25th percentile) student in L1 in 2002 or 2003. If you replace in his group one percentage point of Low students by one of High Quality students, then the effect on the grade of that Low student is 0.047 points. Standards errors
Table 10: Teacher Effects Estimates with Discretized Quality Index
L1 L2
25/75 36/66 Total June Total Sept
02/03 04/05/06 02/03 04/05/06 25/75 33/66 25/75 33/66 Chi2 68.11 93.56 63.32 90.81 62.19 54.22 59.67 53.19 P-value <0.0001 <0.0001 <0.0001 <0.0001 0.0001 0.0006 0.0001 0.0008 Maths Max-Min 3.44 3.84 3.18 3.61 2.38 2.45 2.49 2.43 Std 0.81 0.78 0.78 0.77 0.68 0.65 0.67 0.64 Chi2 95.82 133.08 95.98 135.62 24.81 27.44 25.73 28.44 P-value <0.0001 <0.0001 <0.0001 <0.0001 0.0733 0.0369 0.0580 0.0280 Micro1 Max-Min 3.59 2.66 3.55 2.7 1.26 1.4 1.21 1.45 Std 1 0.87 1 0.87 0.36 0.38 0.37 0.4 Chi2 42.37 56.67 42.70 55.56 68.55 70.47 73.05 68.19 P-value 0.0006 <0.0001 0.0005 <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 Macro1 Max-Min 1.95 1.94 1.98 1.95 2.48 2.64 2.57 2.61 Std 0.53 0.52 0.54 0.53 0.64 0.67 0.66 0.67 Chi2 69.66 106.83 71.49 108.72 21.68 24.82 23.29 25.70 P-value <0.0001 <0.0001 <0.0001 <0.0001 0.1538 0.0731 0.1061 0.0584 Micro2 Max-Min 2.73 3.55 2.76 3.47 1.92 1.83 1.81 1.89 Std 0.82 1.05 0.84 1.06 0.44 0.48 0.45 0.49 Note: We test the equality of all teacher dummies within each course. ‘’Chi2” lines give the values of Chi2 statistics and the
Table 11: Teacher Effects Estimates with Discretized Quality Index, continued
L1 L2
25/75 36/66 Total June Total Sept
02/03 04/05/06 02/03 04/05/06 25/75 33/66 25/75 33/66 Chi2 93.46 183.99 91.83 182.63 49.32 52.12 46.24 51.57 P-value <0.0001 <0.0001 <0.0001 <0.0001 0.0011 0.0005 0.0028 0.0006 Macro2 Max-Min 3.75 2.82 3.77 2.8 3.53 3.61 3.54 3.53 Std 0.93 0.91 0.93 0.9 0.77 0.79 0.73 0.77 Chi2 76.62 94.42 74.06 94.40 44.37 40.09 43.37 40.37 P-value <0.0001 <0.0001 <0.0001 <0.0001 0.0047 0.0150 0.0063 0.0140 Stat Max-Min 2.46 3.44 3.07 3.41 2.22 2.17 2.22 2.16 Std 0.75 0.85 0.82 0.85 0.62 0.59 0.61 0.59 Chi2 54.41 173.34 55.73 173.64 80.81 84.32 82.47 84.47 Computer P-value 0.0004 <0.0001 0.0002 <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 Science Max-Min 2.57 5.6 2.48 5.63 3.92 4.05 4.53 4.26 Std 0.71 1.22 0.7 1.22 1.08 1.1 1.13 1.12 Note: We test the equality of all teacher dummies within each course. ‘’Chi2” lines give the values of Chi2 statistics and the p-value of the test just below. We then report some descriptive statistics (max-min and standard errors) of the distribution of teacher dummies.
Table 12: Teacher Effects Estimates with Continuous Quality Index
L1 L2
02/03 04/05/06 Total June Total Sept Chi2 68.27 93.65 63.77 107.44 P-value <0.0001 <0.0001 <0.0001 <0.0001 Maths Max-Min 3.52 3.86 2.34 2.92 Std 0.83 0.79 0.68 0.87 Chi2 93.42 132.33 26.76 32.43 P-value <0.0001 <0.0001 0.0442 0.0088 Micro1 Max-Min 3.5 2.66 1.39 1.49 Std 0.99 0.86 0.37 0.45 Chi2 41.37 56.74 66.67 73.10 P-value 0.0008 <0.0001 <0.0001 <0.0001 Macro1 Max-Min 1.83 1.86 2.41 2.79 Std 0.52 0.52 0.63 0.73 Chi2 69.91 107.48 23.91 28.49 P-value <0.0001 <0.0001 0.0914 0.0276 Micro2 Max-Min 2.73 3.56 1.94 2.06 Std 0.81 1.05 0.47 0.52
Reading: We test the equality of all teacher dummies within each course. ‘’Chi2” lines give the values of Chi2 statistics and the p-value of the test just below. We then report some descriptive statistics (max-min and standard errors) of the distribution of teacher dummies.
Table 13: Teacher Effects Estimates with Continuous Quality Index, Con-tinued
L1 L2
02/03 04/05/06 Total June Total Sept Chi2 94.24 186.11 59.08 61.56 P-value <0.0001 <0.0001 0.0001 <0.0001 Macro2 Max-Min 3.63 2.84 3.62 3.26 Std 0.92 0.91 0.83 0.82 Chi2 74.25 93.15 44.04 86.27 P-value <0.0001 <0.0001 0.0052 <0.0001 Stat Max-Min 2.89 3.44 2.31 3.4 Std 0.81 0.85 0.63 0.94 Chi2 54.35 172.98 85.39 89.82 Computer P-value 0.0004 <0.0001 <0.0001 <0.0001 Science Max-Min 2.68 5.64 4.02 3.94 Std 0.71 1.23 1.07 1.1
Reading: We test the equality of all teacher dummies within each course. ‘’Chi2” lines give the values of Chi2 statistics and the p-value of the test just below. We then report some descriptive statistics (max-min and standard errors) of the distribution of teacher dummies.
APPENDIX
A
Interpreting peer effects parameters
The full initial model is
y=α∗L1L+α∗M1M +α∗H1H +βL∗PL+βM∗ PM +βH∗PH
+βLL∗ 1LPL+βLM∗ 1LPM +βLH∗ 1LPH
+βM L∗ 1MPL+βM M∗ 1MPM +βM H∗ 1MPH
+βHL∗ 1HPL+βHM∗ 1HPM +βHH∗ 1HPH
But of course, 1M = 1−1L−1H and PM = 100−PL−PH. By substitution,
y=α∗M + 100βM∗ + 100βM M∗ + (α∗L−α∗M + 100βLM∗ −100βM M∗ )1L+ (αH∗ −α∗M + 100βHM∗ −100βM M∗ )1H + (βL∗ −βM∗ +βM L∗ −βM M∗ )PL+ (βH∗ −βM∗ +βM H∗ −βM M∗ )PH + (βLL∗ −βLM∗ +βM M∗ −βM L∗ )1LPL+ (βLH∗ −βM H∗ +βM M∗ −βLM∗ )1LPH + (βHL∗ −βHM∗ +βM M∗ −βM L∗ )1HPL+ (βHH∗ −βHM∗ +βM M∗ −βM H∗ )1HPH Thus, αL =α∗L−α ∗ M + 100β ∗ LM −100β ∗ M M αH =α∗H −α ∗ M + 100β ∗ HM−100β ∗ M M βL =βL∗ −β ∗ M +β ∗ M L−β ∗ M M βH =βH∗ −β ∗ M +β ∗ M H−β ∗ M M βLL =βLL∗ −β ∗ LM +β ∗ M M −β ∗ M L βLH =βLH∗ −β ∗ M H+β ∗ M M−β ∗ LM βHL =βHL∗ −β ∗ HM+β ∗ M M−β ∗ M L βHH =βHH∗ −β ∗ HM +β ∗ M M −β ∗ M H
Now consider a low quality student in some group. Her grade equals αL∗ + (βL∗ +βLL∗ )PL+ (βM∗ +β ∗ LM)PM + (βH∗ +β ∗ LH)PH
If one percentage point of low quality students is replaced by one point of High in this group (PL → PL−1 and PH → PH + 1) then the difference between the new and the previous grade
of that low student equals
βH∗ +βLH∗ −βL∗ −βLL∗ =βH −βL+βLH −βLL
B
Distribution of L2 groups conditional on L1 groups
Table 14: Distribution of Group L2 conditional on Group L1, 2003
Gr L1 Gr L2 freq Gr L1 Gr L2 freq Gr L1 Gr L2 freq
1 7 1 2 15 3 4 10 1 1 8 7 2 16 3 4 12 1 1 9 1 2 17 1 4 15 1 1 11 1 2 19 1 4 16 1 1 12 3 2 22 1 4 18 1 1 15 4 2 23 1 4 20 3 1 16 3 2 25 1 4 21 1 1 18 1 3 2 2 4 23 1 1 21 2 3 3 3 4 24 2 1 22 3 3 4 3 4 25 1 1 23 1 3 11 2 4 26 2 1 24 3 3 14 1 5 2 2 1 26 1 3 15 2 5 3 3 2 2 2 3 17 1 5 6 2 2 3 1 3 18 1 5 8 1 2 4 2 3 19 1 5 9 2 2 5 2 3 20 5 5 10 1 2 6 2 3 21 2 5 11 2 2 7 1 3 23 4 5 14 1 2 8 1 4 1 2 5 17 2 2 9 1 4 2 2 5 19 2 2 11 1 4 3 4 5 21 2 2 13 1 4 5 2 5 23 1 2 14 2 4 8 1 5 24 2
Reading: On the first line of the first column block, we read that only one student was in group 1 in L1 in 2002 and in group 7 in L2 in 2003. To save space we don’t print the whole contingency table.
Table 15: Distribution of Group L2 conditional on Group L1, 2004
Gr L1 Gr L2 freq Gr L1 Gr L2 freq Gr L1 Gr L2 freq
1 2 1 2 19 1 4 16 4 1 3 1 2 20 3 4 17 2 1 4 1 2 22 2 4 21 1 1 5 1 2 23 2 5 1 1 1 6 3 3 1 1 5 5 1 1 8 1 3 2 2 5 6 5 1 10 1 3 3 3 5 7 2 1 15 3 3 4 1 5 8 1 1 18 3 3 5 3 5 15 3 1 19 3 3 6 2 5 18 1 1 20 1 3 8 2 5 19 4 1 22 1 3 12 2 5 20 1 1 23 3 3 16 1 5 22 4 1 24 1 3 18 1 5 23 1 2 2 1 3 22 1 6 2 2 2 7 4 3 23 1 6 4 3 2 8 1 3 24 3 6 6 3 2 9 2 4 1 5 6 8 1 2 10 1 4 4 2 6 10 2 2 12 2 4 5 2 6 12 2 2 14 1 4 9 1 6 14 1 2 15 2 4 11 1 6 15 1 2 17 2 4 13 6 6 16 3 2 18 1 4 15 1 6 17 1
Reading: On the first line of the first column block, we read that only one student was in group 1 in L1 in 2003 and in group 2 in L2 in 2004. To save space we don’t print the whole contingency table.
