Online Appendix for
Explaining Charter School Effectiveness
Joshua D. Angrist Parag A. Pathak Christopher R. Walters∗ February 5, 2013
Data Appendix
The data used for this study come from charter school lottery records, student demographic and school attendance information in the Massachusetts Student Information Management System (SIMS), and test scores from the Massachusetts Comprehensive Assessment System (MCAS) database. This appendix describes each data source and details the procedures used to clean and match them. The steps used here are an updated version of the methods described in the data appendix to Angrist et al. (2010).
1
Data Sets
1.1 Charter School Entrance Lotteries Data description and sample restrictions
Our sample of applicants is obtained from records of lotteries held at 20 Massachusetts charter schools between 2002 and 2010. The participating schools and lottery years are listed in Table A1, along with schools eligible for the lottery study that did not contribute records. A total of 100 school-specific entry cohorts are included in the analysis. Lotteries at three schools contribute observations to both the middle and high school samples.
The raw lottery records typically include applicants’ names, dates of birth, contact infor-mation, and other information used to define lottery groups, such as sibling and out-of-area status. The first five rows in each panel of Table A7 show the sample restrictions we impose on the raw lottery records, separately by lottery cohort and school level. We exclude duplicate applicants and applicants listed as applying to the wrong entry grade. We also drop late appli-cants, out-of-area appliappli-cants, and sibling appliappli-cants, as these groups are typically not included in the standard lottery process. Imposing these restrictions reduces the number of middle school lottery records from 13,038 to 11,220 and reduces the number of high school records from 9,506 to 9,009.
Lottery offers ∗
Angrist: Department of Economics, Massachusetts Institute of Technology and NBER, E52-393, 50 Memorial Drive, Cambridge, MA 02142, email: [email protected]. Pathak: Department of Economics, Massachusetts Insti-tute of Technology and NBER, E52-391C, 50 Memorial Drive, Cambridge, MA 02142, email: [email protected]. Walters: Department of Economics, Massachusetts Institute of Technology, 50 Memorial Drive, Cambridge, MA 02142, email: [email protected].
In addition to the data described above, the lottery records also include information regarding offered seats. We used this information to reconstruct indicator variables for whether lottery participants received randomized offers. For most schools and years, we code the offer variable as one for applicants who received offered seats at any time after the lottery, including offers to waitlisted students. This definition corresponds to the “ever offer” instrument used by Ab-dulkadiro˘glu et al. (2011). For a few schools, information on waitlist offers was unavailable, but records were sufficient to determine the students who received offers on the day of the lottery. The offer variables for these schools are coded as one for the initially offered students and zero otherwise. The instrumentZi used in our analyses is one for any student who received an offer
from any school included in our lottery sample. Offer rates were 67 percent and 64 percent in our middle and high school samples, respectively.
1.2 Student Information Management System Data Data description
Our study uses SIMS data from the 2001-2002 school year through the 2010-2011 school year. Each year of data includes an October file and an end-of-year file. The SIMS records information on demographics and schools attended for all students in Massachusetts’ public schools. An observation in the SIMS refers to a student in a school in a year, though there are some student-school-year duplicates for students that switch grades or programs within a school and year.
Coding of demographics and attendance
The SIMS variables used in our analysis include grade, year, name, town of residence, date of birth, sex, race, special education and limited English proficiency status, free or reduced price lunch, and school attended. We constructed a wide-format data set that captures demographic and attendance information for every student in each year in which he or she is present in Massachusetts’ public schools. This file uses information from the longest-attended school in the first calendar year spent in each grade. Attendance ties were broken at random; this affects only 0.007 percent of records. Students classified as SPED, LEP, or free/reduced price lunch in any record within a school-year-grade retain that designation for the entire school-year-grade.
We measure charter school attendance in calendar years. A student is coded as attending a charter school in a particular year when there is any SIMS record reporting charter attendance in that year. Students who attend more than one charter school within a year are assigned to the charter they attended longest.
1.3 Massachusetts Comprehensive Assessment System Data Data description and sample restrictions
We use MCAS data from the 2001-2002 school year through the 2010-2011 school year. Each observation in the MCAS database corresponds to a student’s test results in a particular grade and year. We use math and English Language Arts (ELA) tests in grades 3 through 8 and 10, as well as Writing Topic and Writing Composition scores in grades 4, 7, and 10. The test score variables are standardized to have mean zero and standard deviation one within a subject-grade-year in Massachusetts. Repetitions of the same test subject and grade are dropped. In cases with multiple records within a year and grade, ties are broken at random; this affected 0.10 percent of MCAS records.
In the lottery-based middle school analysis, all post-lottery test scores through 8th grade are used as outcomes. High school outcomes are from 10th grade. The most recent pre-lottery score in a subject defines a student’s baseline score. For the observational analysis, outcome grades are 5th through 8th for middle school 10th for high school; baseline scores are from 4th grade for middle school and 7th or 8th grade for high school.
2
Matching Data Sets
2.1 Match from the MCAS to the SIMS
The processed SIMS and MCAS files were merged by grade, year, and a state student identifier known as the SASID. Scores that could not be matched to the SIMS were dropped. This restricted eliminated 0.7 percent of MCAS scores statewide.
2.2 Match from the Lottery Records to the State Database Match procedure
Lottery records were matched to the state SIMS/MCAS database by name, application year, and application grade. In some cases, this procedure did not produce a unique match. We accepted some matches based on fewer criteria where the information on grade, year, and town of residence seemed to make sense.
Match success rate
Our matching procedure successfully located most applicants in the SIMS database. Table A8 reports cohort-specific match rates from the lottery records to the combined SIMS/MCAS file, separately for middle and high school. The overall match rates for middle and high school were 92.1 percent and 93.7 percent, respectively. Table A8 also reports separate match rates for offered and non-offered students. In middle school, offered students were slightly more likely to be matched (94.0 percent compared to 89.4 percent). Offered and non-offered applicants to charter high schools were matched to the SIMS at almost similar rates (94.0 percent compared to 93.3 percent).
3
Construction of the Outcome Data Sets
3.1 Lottery Sample Further sample restrictions
Once matched to the SIMS, each student is associated with a unique SASID; at this point, we can therefore determine which students applied to multiple schools in our lottery sample. Following the match, we reshape the lottery data set to contain a single record for each student. If students applied in more than one year to lotteries at a particular school level (middle or high), we keep only the records associated with their first year of application. In our basic lottery analyses, we also exclude students without baseline demographics in the SIMS; in effect, this rule limits the sample to students in Massachusetts’ public schools at baseline. Rows 6-9 in each panel of Table A7 report the impact of these restrictions on sample sizes for middle and
high school. The set of matched first-time applicants with baseline demographics includes 7,530 middle school students and 5,260 high school students.
Final set of outcomes and students
To generate the middle school analysis file, the matched lottery/SIMS/MCAS file is reshaped to long format, with each observation referring to a test score outcome for a student in a particular year. The high school analysis file uses only 10th grade outcomes, so it includes a single observation for each student. Table A9 summarizes the analysis files for middle and high school. Columns (1) and (2) list the application and outcome grades for each cohort, and column (3) lists the number of applicants satisfying the sample restrictions from Table A7. In middle school, 7,307 of 7,530 students contribute at least one test score to the analysis. In high school, 4,025 of 5,260 students have at least one score. Middle school applicants contribute different numbers of scores to the analysis depending on their years and grades of application; math and ELA tests were not given in every middle school grade until 2006, and some cohorts are not observed through 8th grade. Table A10 lists the grades and years in which math and ELA subjects were administered. As shown in columns (5) through (8) of Table A9, we find 16,543 out of 18,798 expected scores for middle school math, 16,285 of 18,515 for middle school ELA, 4,047 of 5,260 for high school math, and 4,100 of 5,260 for high school ELA. These outcomes are used to produce the 2SLS estimates reported in Tables 4 and 5.
3.2 Observational Sample
To produce the analysis file used for the observational analysis, we begin with the matched SIMS/MCAS state database. As described in Section V, we define cells based on baseline school, baseline year, race, and sex, separately for middle school and high school. We then count the number of students in each cell who go on to spend time in eligible charter schools and regular public schools in the relevant range of grades (5th through 8th for middle school and 10th for high school). Observations in cells that do not include at least one student who attends eligible charter schools and one student who attends regular public schools are dropped. We then produce a long format data file containing the full set of test score outcomes for the remaining sample of matched students at the relevant school level, as well as variables counting years of attendance at each eligible charter school. This file is used to produce the observational estimates. Our matching procedure excludes 23 percent of students who attend eligible charter schools in middle or high school.
Survey Appendix
This appendix describes the information collected in our survey of school administrators. The survey responses were used to construct the variables used in tables 2, 8, and A6. The survey was organized in six sections: general school structure, school philosophy and curriculum, areas of emphasis, classroom setting and practices, school day and policies, and teacher profile. We next describe the format and content of each section of the survey.
General School Structure
Survey respondents were asked to answer the following questions about school structure [possible responses in brackets]:
• How did your charter school originate? Check all that apply. [Founded as charter, Charter restart of a traditional public school, Another conversion model of an existing public school or a previously operating charter, Other]
• Who founded your school? Check all that apply [Parents, Teachers, Business leaders, Phi-lanthropist, Managment company, Nonprofit organization, Community members, Other] • Since the initial approval of your charter, has this group changed? If yes, please explain
briefly. [Yes, No]
• Please indicate sources of funding. Check all that apply. [Federal, State, Foundations, Individuals, Corporations, Other]
School Philosophy and Curriculum
Survey respondents were asked to rate their schools’ adherence to a variety of instructional ap-proaches, with the prompt: “Please indicate thedegree to which the educational experience at your school reflects the following approaches to teaching and learning.” For each area, respon-dents checked boxes corresponding to numbers from 1 to 5, with 1 indicating “not at all,” 3 indicating “somewhat,” and 5 indicating “strongly.” An additional box corresponded to “don’t know.” The approaches were as follows: Common core values (unique to your school, NOT Common Core Standards), No Excuses, traditional reading and math skills, college prepara-tion, preparation for specific careers, STEM (Science, Technology, Engineering, Mathematics), Leadership. In addition, the prompt for “preparation for specific careers” included a request to “pleast list relevant careers, if any.” A final prompt asked respondents to list any philosophy or curricular focus not mentioned on the survey.
Areas of Emphasis
Survey respondents were asked to rate the emphasis placed on a variety of areas, with the prompt: “Please indicate theextentto which the educational program at your schoolemphasizes
the following principles.” For each area, respondents checked boxes corresponding to numbers from 1 to 5, with 1 indicating “not at all,” 3 indicating “somewhat,” and 5 indicating “strongly.” An additional box corresponded to “don’t know.” The areas were as follows: Cultural awareness, strict adherence to a set of school-wide standards and practices, social and physical well-being, individually-tailored instruction, discipline and comportment, speech and writing development,
measureable results (for example, gains on state achievement tests), qualitative achievement (for example, leadership, creativity, and community involvement).
Classroom Setting and Practices
Survey respondents were asked to rate their schools’ use of a variety of classroom practices, with the prompt: “Please indicate theextent to which the following settings and practices arepresent
in classrooms at your school.” For each practice, respondents checked boxes corresponding to numbers from 1 to 5, with 1 indicating “not at all,” 3 indicating “somewhat,” and 5 indicating “strongly.” An additional box corresponded to “don’t know.” The practices were as follows: Group projects, cold calling, checks for understanding (informal tests to gauge understanding during the lesson), DEAR or SSR (Drop Everything and Read/Sustained Silent Reading), read-ing aloud, math drills, college icons in the classroom (e.g. banner with name of teacher’s college, past students’ graduation years and college outcomes), teacher autonomy.
School Day and Policies
Survey respondents were asked to answer the following questions about other school practices [possible responses in brackets when provided]:
• What time do instructional activities begin each day for the typical student? • When do instructional activities end each day for the typical student?
• Do certain students have longer or shorter school days (i.e. extended days for struggling students)? [Yes, No]. If Yes, please explain.
• How many days are in your school year for a typical student (excluding optional Satur-days)?
• Are students required to wear uniforms or follow a dress code? [Yes, No] • Do parents sign a commitment contract? [Yes, No]
• Do students sign a commitment contract? [Yes, No]
• Which of the following disciplinary actions are used at your school? Please check all that apply. [Detention, Positive behavior support, In school suspension, Counseling session, Be-havior improvement plan, Saturday school, Merit/demerit System, Timeout/quiet room, Other]
• Are your students eligible for rewards for achievement or good behavior (e.g. small pay-ments or redeemable points)? [Yes, No] If yes, please describe briefly.
• Who attends Saturday school? Check all that apply. [All students, Students in certain grades, Students who need academic help, Students being disciplined, None]
• How often does the typical Saturday school student have school on Saturday? [Monthly or less, Bi-weekly, Weekly]
• What is the average length of a period of math instruction, and the number of periods per week?
• What is the average length of a period of reading instruction, and the number of periods per week?
• Is reading instruction based on an RTI (Response to Intervention) model? [Yes, No] • Is math instruction based on an RTI (Response to Intervention) model? [Yes, No]
• To what extent is MCAS preparation a part of your curriculum (check all that apply) [Special classes before exams, Regular weekly classes/study sessions, After-school instruc-tion for all students, After-school instrucinstruc-tion for struggling students, Classes or meetings for parents on nights/weekends]
• Which students receive tutoringduring the school day? Check all that apply. [All students, Students who struggle with class work, Students who struggle with MCAS, None, Other] • Which students receiveafter schooltutoring? Check all that apply. [All students, Students
who struggle with class work, Students who struggle with MCAS, None, Other]
• If your school offers tutoring, which of the following describe the tutors? Check all that apply. [Recruited from Teach for America or AmeriCorps, Community members, Parents, Teachers/staff, Local college students]
• (High Schools only) Does your school offer extra preparation for the SAT and AP exams? [Yes, No, Not applicable]
• Please estimate the hours per week teachers typically spend on (in classroom) instructional activities.
• Please estimate the hours per week teachers typically spend on instructional activities outside of the classroom (preparing lesson plans, tracking student performance).
Teacher Profile
Survey respondents were asked to rate the frequency with which their schools hire various types of teachers. Possible responses were “frequently,” “occasionally,” and “rarely or never.” The teacher types were as follows: Teach for America novices, Teach for America alumni, MATCH Teacher Residency, recent college graduates.
Survey respondents were also asked to answer the following questions about their schools’ teachers [possible responses in brackets when provided]:
• How are teachers hired? Check all that apply (please respond for full-time, year-round teachers). [On a contract of specific length, At-will]
• Please estimate your hiring acceptance rate (i.e. 50 accepted/200 applicants). • Has there ever been a teacher’s union active at your school? [Yes, No]
• Please indicate the performance incentives available at your school (check all that apply). [Merit pay, Higher salaries for hard-to-fill subjects, Yearly bonus, Within-school promotion, Recognition/non-monetary rewards, Other]
• How often do administrators and supervisors observe new teachers in the classroom to review their performance (periods/month)?
• How often do administrators and supervisors observeveteran teachers in the classroom to review their performance (periods/month)?
School Town Urban Grades
Eligible middle
Eligible
high Years in lottery study
(1) (2) (3) (4) (5) (6) (7)
Academy of the Pacific Rim Charter School Boston Yes 5-12 Yes 2005-2010
Advanced Math and Science Academy Charter School Marlborough 6-12 Yes
Barnstable Horace Mann Charter School Marstons Mills 4-5 Yes
Berkshire Arts and Technology Charter Public School Adams 6-12 Yes
Boston Collegiate Charter School Boston Yes 5-12 Yes Yes 2002-2010
Boston Preparatory Charter Public School Boston Yes 6-11 Yes 2005-2010
Cape Cod Lighthouse Charter School Orleans 6-8 Yes 2007-2010
Christa McAuliffe Regional Charter Public School Framingham Yes 6-8 Yes
City on a Hill Charter Public School Boston Yes 9-12 Yes 2002, 2004-2009
Codman Academy Charter Public School Boston Yes 9-12 Yes 2004, 2008-2009
Community Charter School of Cambridge Cambridge Yes 7-12 Yes
Dorchester Collegiate Academy Charter School Boston Yes 4-5 Yes
Edward Brooke Charter School Boston Yes K-8 Yes 2006-2009
Excel Academy Charter School Boston Yes 5-8 Yes 2008-2010
Four Rivers Charter Public School Greenfield 7-12 Yes Yes 2003-2010
Francis W Parker Charter Essential School Devins 7-12 Yes 2006-2010
Global Learning Charter Public School New Bedford Yes 5-12 Yes 2006-2007, 2009
Hampden Charter School of Science Chicopee Yes 6-10 Yes
Health Careers Academy Charter School Boston Yes 9-12 Yes
Innovation Academy Charter School Tyngsboro 5-11 Yes 2007-2010
KIPP Academy Lynn Lynn Yes 5-8 Yes 2005-2009
Marblehead Community Charter Public School Marblehead 4-8 Yes 2005-2007, 2010
MATCH Charter Public School Boston Yes 6-12 Yes Yes 2002-2010
New Leadership Charter School Springfield Yes 6-12 Yes
North Central Charter Essential School Fitchburg Yes 7-12 Yes
Phoenix Charter Academy Chelsea Yes 9-12 Yes
Pioneer Charter School of Science Everett Yes 7-11 Yes
Pioneer Valley Performing Arts Charter Public School South Hadley 7-12 Yes 2006-2010
Rising Tide Charter Public School Plymouth 5-8 Yes 2009
Roxbury Preparatory Charter School Boston Yes 6-8 Yes 2002-2010
Salem Academy Charter School Salem 6-12 Yes 2010
Smith Leadership Academy Boston Yes 6-8 Yes
Sturgis Charter Public School Hyannis 9-12 Yes 2004, 2006, 2008-2009
Table A1: Massachusetts Charter Schools Eligible for the Lottery Study
Notes: This table lists all charter schools in Massachusetts eligible for the lottery study. To be counted as eligible, a school must be open in the relevant years and meet the entry grade and student population restrictions required for inclusion in column (3) of Table 1.
All charters Urban charters
Non-urban
charters All charters Urban charters
Non-urban charters (1) (2) (3) (4) (5) (6) 0.023** 0.031** 0.007 0.000 -0.001 0.003 (0.010) (0.014) (0.009) (0.015) (0.017) (0.011) -0.011 -0.020 0.009 0.010 0.012 -0.016 (0.011) (0.016) (0.008) (0.017) (0.019) (0.016) -0.005 -0.005 -0.005 -0.004 -0.004 -0.001 (0.010) (0.013) (0.015) (0.010) (0.011) (0.032) 0.003 0.002 0.007 -0.002 -0.003 0.006 (0.004) (0.005) (0.008) (0.006) (0.007) (0.017) 0.010 0.003 0.024 0.010 0.015 -0.038 (0.014) (0.017) (0.025) (0.018) (0.019) (0.062) 0.007 0.013 -0.007 0.023 0.021 0.043 (0.012) (0.015) (0.016) (0.016) (0.017) (0.040) -0.004 -0.003 -0.006 -0.002 0.006 -0.080* (0.011) (0.013) (0.019) (0.014) (0.015) (0.047) 0.001 0.005 -0.007 0.009 0.009 0.000 (0.008) (0.011) (0.006) (0.006) (0.006) (0.000) -0.015 -0.023 0.003 -0.012 -0.023 0.107 (0.026) (0.033) (0.043) (0.034) (0.035) (0.101) -0.006 0.001 -0.020 -0.041 -0.038 -0.077 (0.027) (0.035) (0.039) (0.032) (0.034) (0.081)
p-value, from F-test 0.452 0.694 0.386 0.741 0.824 0.275
N 7060 4852 2208 4671 4104 567
* significant at 10%; ** significant at 5%; *** significant at 1%
White
Table A2: Covariate Balance
High school
Hispanic Black
Middle school
Baseline math score
Notes: This table reports coefficients from regressions of the variable in each row on an indicator variable equal to one if the student won the lottery. Regressions include risk set dummies and baseline grade dummies and exclude students with sibling priority and late applicants. Samples are restricted to students who have baseline demographics and test scores. F-tests are for the null hypothesis that the coefficients on winning the lottery in all regressions are all equal to zero.
Asian Female
Subsidized Lunch Special Education
Limited English Proficiency
Proportion of
non-offered with MCAS Differential
Proportion of
non-offered with MCAS Differential
Proportion of
non-offered with MCAS Differential
School level Subject (1) (2) (3) (4) (5) (6)
Middle Math 0.907 0.019*** 0.925 0.012 0.866 0.032*** (0.007) (0.008) (0.011) N 2933 7530 2055 5169 878 2361 ELA 0.903 0.024*** 0.918 0.018** 0.868 0.033*** (0.007) (0.008) (0.011) N 2933 7530 2055 5169 878 2361 High Math 0.753 0.009 0.750 0.004 0.800 0.059 (0.015) (0.016) (0.046) N 1829 5261 1704 4631 125 630 ELA 0.766 0.004 0.762 0.000 0.816 0.038 (0.015) (0.016) (0.046) N 1829 5261 1704 4631 125 630
* significant at 10%; ** significant at 5%; *** significant at 1%
Table A3: Attrition
Notes: This table reports coefficients from regressions of an indicator variable equal to one if a student has a follow-up test score on an indicator variable equal to one if the student won the lottery. Column (1) shows the fraction of non-offered students with follow-up scores, while column (2) shows the differential by offer status. Columns (3) and (4) show corresponding results for urban applicants, while columns (5) and (6) show results for non-urban applicants. Regressions include risk set dummies as well as demographic variables, year of birth dummies, year of baseline dummies, and baseline grade dummies. The sample is restricted to students who participated in an effective lottery from cohorts where we should observe follow-up scores.
Mean First Stage 2SLS Mean First Stage 2SLS Mean First Stage 2SLS Subject (1) (2) (3) (4) (5) (6) (7) (8) (9) Graduate on-time 0.673 0.300*** -0.098 0.663 0.289*** -0.031 0.790 0.933*** 0.053 (0.050) (0.072) (0.051) (0.023) (0.047) (0.094) N 0.751 0.304*** -0.024 0.747 0.288*** -0.025 0.801 0.970*** -0.006 (0.060) (0.081) (0.062) (0.087) (0.033) (0.087) N
*significant at 10%; **significant at 5%; ***significant at 1%
Table A4: Lottery Results for High School Graduation
All charter schools Urban charter schools Non-urban charter schools
2863 2649 214
Notes: This table reports 2SLS estimates of the effects of charter school attendance on high school graduation. The endogenous variable is a dummy for attending a charter school in the year after the lottery, and the instrument is a lottery offer dummy. Columns (1)-(2) show estimates for all charter high schools, columns (3)-(4) show estimates for urban charter high schools, and columns (5)-(6) show estimates for non-urban high schools. The "graduate on-time" outcome is a dummy for graduating in or before a student's projected graduation year assuming normal academic progress from baseline. The "graduate within two years" outcome is a dummy for graduating in or before the year after a student's projected graduation year. The urban and non-urban estimates for a given subject come from a single regression with two endogenous variables, using urban and non-urban offers as instruments. All models control for race, sex, special education, limited English proficiency, subsidized lunch status, and a female by minority dummy. Year of birth and risk set dummies are also included. Standard errors are robust to heteroskedasticity.
Graduate within two years
Lottery estimate Lottery sample
Non-lottery
sample Lottery estimate Lottery sample
Non-lottery sample
School level Subject (1) (2) (3) (4) (5) (6)
Middle Math 0.321*** 0.249*** -0.024* -0.123*** -0.015** -0.008 (0.031) (0.013) (0.014) (0.047) (0.007) (0.012) N 11941 4602 ELA 0.146*** 0.158*** -0.035*** -0.144*** -0.009 -0.013 (0.028) (0.010) (0.012) (0.039) (0.007) (0.011) N 11649 4636 High Math 0.339*** 0.322*** -0.009 -0.020 0.047*** -(0.077) (0.034) (0.018) (0.071) (0.017) N 3519 531 14881 ELA 0.264*** 0.260*** 0.100*** -0.046 0.081*** -(0.067) (0.018) (0.019) (0.059) (0.017) N 3567 536 14967
*significant at 10%; **significant at 5%; ***significant at 1%
Table A5: Comparison of Lottery and Observational Estimates for Eligible Charters
Urban Non-urban
Notes: This table reports estimates of the effects of years in charter schools on test scores. Eligible charters are schools with entry grades 4-7 (middle) or 9 (high), and that meet the other restrictions from Table 1. The sample is produced by matching charter students to students in traditional public schools on cells defined by sending school, baseline year, and baseline demographics (race, sex, limited English proficiency, special education status, and free lunch status). All models control for cell fixed effects, year effects, grade effects, and baseline test scores. Middle school regressions pool outcomes from 5th through 8th grade and cluster by student identifier as well
Observational estimates Observational estimates
131136 136046 8208 8018 248711 239288
Mean 2SLS Mean 2SLS
School level Outcome (1) (2) (3) (4)
Middle Total days suspended 0.537 0.710*** 0.080 -0.016
(0.080) (0.040) N 5123 2641 0.073 0.172*** 0.020 -0.002 (0.034) (0.016) N 5123 2641 0.464 0.538*** 0.060 -0.014 (0.067) (0.032) N 5123 2641 Days truant 0.594 0.128 0.235 -0.111 (0.208) (0.214) N 5123 2641
Total days attended 173 7.28*** 171 5.52***
(1.19) (1.59)
N 5175 2605
High Total days suspended 0.465 1.27*** 0.126 -0.100
(0.194) (0.080) N 3582 533 0.086 0.277*** 0.036 -0.024 (0.076) (0.023) N 3582 533 0.379 0.993*** 0.090 -0.075 (0.162) (0.073) N 3582 533 Days truant 5.69 -11.5 0.305 -0.370 (7.58) (1.36) N 3582 533
Total days attended 163 15.9*** 168 9.41*
(4.22) (5.47)
N 3647 530
Notes: This table reports 2SLS estimates of the effects of charter school attendance on disciplinary outcomes and attendance in the year after the lottery. Standard errors are clustered by school-grade-year.
*significant at 10%; **significant at 5%; ***significant at 1%
Urban Non-urban
Table A6: Effects on Discipline and Attendance
Days of in-school suspension Days of out-of-school suspension Days of in-school suspension Days of out-of-school suspension
2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 All lotteries
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
Total number of entry grade records 313 394 391 990 1578 2124 2132 2877 2239 13038
Excluding disqualified applicants 313 394 391 990 1577 2106 2115 2873 2225 12984
Excluding late applicants 313 391 390 972 1551 2046 2054 2829 2222 12768
Excluding applicants from outside of area 313 387 388 963 1540 2028 2041 2741 2202 12603
Excluding siblings 295 358 343 890 1378 1787 1801 2395 1973 11220
Excluding records not matched to the SIMS 267 311 305 838 1311 1710 1669 2095 1825 10331
Reshaping to one record per student 267 311 304 741 1115 1505 1424 1757 1568 8992
Excluding repeat applications 267 308 302 728 1093 1470 1360 1705 1497 8730
In Massachusetts public schools at baseline 201 228 223 603 924 1291 1195 1578 1287 7530
Excluding students without a test score 187 208 210 569 883 1219 1129 1475 1080 6960
Total number of entry grade records 775 717 1313 1219 1148 1411 1392 1531 - 9506
Excluding disqualified applicants 775 717 1309 1218 1146 1408 1391 1520 - 9484
Excluding late applicants 765 710 1280 1215 1138 1408 1372 1517 - 9405
Excluding applicants from outside of area 765 706 1278 1206 1134 1403 1372 1504 - 9368
Excluding siblings 732 677 1218 1165 1120 1362 1334 1401 - 9009
Excluding students not matched to the SIMS 645 614 1121 1074 1091 1306 1255 1321 - 8427
Reshaping to one record per student 573 614 895 852 834 936 863 937 - 6504
Excluding repeat applications 573 612 891 846 812 919 830 895 - 6378
In Massachusetts public schools at baseline 406 462 732 690 692 821 715 742 - 5260
Excluding students without a test score 328 358 583 519 567 659 574 537 - 4125
Table A7: Sample Restrictions for the Lottery Analysis Lottery cohort
Panel A. Middle School
Panel B. High School
Total Offered Not offered Lottery cohort (1) (2) (3) (4) 2002-2003 295 0.908 0.934 0.859 2003-2004 358 0.869 0.882 0.817 2004-2005 343 0.889 0.924 0.849 2005-2006 890 0.942 0.967 0.886 2006-2007 1378 0.951 0.962 0.933 2007-2008 1787 0.957 0.978 0.917 2008-2009 1801 0.927 0.958 0.881 2009-2010 2395 0.875 0.865 0.884 2010-2011 1973 0.925 0.950 0.901 All 11220 0.950 0.940 0.894 2002-2003 732 0.898 0.911 0.831 2003-2004 677 0.907 0.879 0.932 2004-2005 1218 0.922 0.934 0.893 2005-2006 1165 0.922 0.937 0.901 2006-2007 1120 0.974 0.977 0.971 2007-2008 1362 0.959 0.965 0.955 2008-2009 1334 0.941 0.939 0.951 2009-2010 1401 0.939 0.956 0.932 All 9009 0.937 0.940 0.933
Notes: This table summarizes the match from the lottery records to the SIMS data. The sample excludes disqualified applicants, late applicants, out-of-area applicants, and siblings.
Table A8: Match from Lottery Records to SIMS Number of
records
Fraction with SIMS match
Panel A. Middle School
Application grades Outcome grades Number of applicants Number with a test score Number of math scores expected Number of ELA scores expected Number of math scores observed Number of ELA scores observed Lottery cohort (1) (2) (3) (4) (5) (6) (7) (8) 2002-2003 5-6 6-8 201 187 402 290 351 253 2003-2004 5-7 6-8 228 208 510 418 433 356 2004-2005 5-7 6-8 223 210 619 547 542 472 2005-2006 4-7 4-8 603 569 2115 2115 1894 1894 2006-2007 4-7 4-8 924 883 3037 3037 2693 2700 2007-2008 4-7 4-8 1291 1219 4287 4280 3724 3701 2008-2009 5-7 5-8 1195 1129 3385 3385 2950 2956 2009-2010 5-7 5-8 1578 1475 3156 3156 2856 2850 2010-2011 4-7 4-8 1287 1427 1287 1287 1100 1103 All 4-7 4-8 7530 7307 18798 18515 16543 16285 2002-2003 5,9 10 406 328 406 406 327 328 2003-2004 5,7,9 10 462 258 462 462 352 356 2004-2005 7,9 10 732 583 732 732 569 579 2005-2006 7,9 10 690 519 690 690 507 514 2006-2007 9 10 692 567 692 692 561 562 2007-2008 9 10 821 659 821 821 637 657 2008-2009 9 10 715 574 715 715 564 570 2009-2010 9 10 742 537 742 742 530 534 All 5,7,9 10 5260 4025 5260 5260 4047 4100
Table A9: Outcome Data for the Lottery Analysis
Panel A. Middle School
Panel B. High School
Notes: This table summarizes observed test score outcomes for charter school lottery applicants. The sample is restricted to randomized applicants matched to baseline SIMS demographics. Expected test scores are post-lottery scores in grades 4-8 for middle school and grade 10 for high school that would be taken in Spring 2010 or earlier given normal academic progress after the lottery. Table A1 lists the schools participating in each cohort and their entry grades. Table A7 lists the availability of math and ELA tests by year.
4th grade 5th grade 6th grade 7th grade 8th grade 10th grade
Subject School year (1) (2) (3) (4) (5) (6)
Math 2001-2002 Yes Yes Yes Yes
2002-2003 Yes Yes Yes Yes
2003-2004 Yes Yes Yes Yes
2004-2005 Yes Yes Yes Yes
2005-2006 through 2010-2011 Yes Yes Yes Yes Yes Yes
ELA 2001-2002 Yes Yes Yes
2002-2003 Yes Yes Yes
2003-2004 Yes Yes Yes
2004-2005 Yes Yes Yes
2005-2006 through 2010-2011 Yes Yes Yes Yes Yes Yes Table A10: Availability of MCAS Math and ELA Tests by Year
References
[1] Abdulkadiro˘glu, A., Angrist, J., Dynarski, S., Kane, T., and Pathak, P. (2011). “Account-ability and flexibility in public schools: Evidence from Boston’s charters and pilots.” The Quarterly Journal of Economics, forthcoming.
[2] Angrist, J., Dynarski, S., Kane, T., Pathak, P., and Walters, C. (2010). “Inputs and impacts in charter schools: KIPP Lynn.”American Economic Review 100(2), 239-243.
