Assumptions for causality

unbiased. For unbiased estimation of causal treatment effects, Rubin (1986, 1990) and Rosenbaum & Rubin (1983) list two assumptions that research designs should meet: 1) the stable-unit-treatment value assumption (SUTVA) and 2) the strongly ignorable treatment

assignment assumption. Below are short descriptions of these assumptions and their implications for this study.

3.1.2.9.1 SUTVA. Rubin (1986, pg. 961) describes two conditions to be met for SUTVA: 1) “the value of 𝑌𝑌 for unit 𝑢𝑢 when exposed to treatment 𝑡𝑡 will be the same no matter what

mechanism is used to assign treatment 𝑡𝑡 to unit 𝑢𝑢” and 2) “the value of 𝑌𝑌 for unit 𝑢𝑢 when exposed to treatment 𝑡𝑡 will be the same no matter what treatments the other units receive.” Moreover, these two assumptions should hold for all 𝑢𝑢 = 1, … , 𝑁𝑁 and all 𝑡𝑡 = 1, … , 𝑇𝑇. Violations of SUTVA can happen in two ways: neighborhood effects and treatment group non-adherence. • Neighborhood effects. Violations of SUTVA can happen when study participants share the

same environment because the treatment received by some students/schools may affect the response given by other students/schools. In this study, treatment and comparison schools are all in one school district, and thus, they share the same BPS district and the neighborhood. Having the intervention take place in some schools may affect other schools through shared connections. For example, teachers in comparison schools may talk to teachers in treatment schools and learn about the services and resources that City Connects provides, to which they may participate independent of the City Connects (e.g. community partnerships with

organizations like Big Brothers Big Sisters would be available to anyone who applies). Likewise, students in treatment schools who have siblings, cousins, or close friends attending comparison schools may have a peer effect on one another, either positive or negative. Therefore, the treatment given to some schools may affect the treatment received by other schools. In addition, regardless of shared connections, it is very common for schools in BPS to have community partnerships with a few organizations or some type of support services available. Thus, adopting some of the same services that are also available through the City Connects can be considered as business as usual for schools in BPS. However, City

Connects provides a system that makes it possible to serve all students within a school. Each student is evaluated by trained site coordinators at least once a year and is provided with a targeted set of enrichment and prevention services based on students’ strengths and

weaknesses. Also, students are monitored throughout the year to assess the progress and if necessary are provided with new ones. Thus, even though non-City Connects schools may participate in some of the same services that are also available through City Connects, those services may not be as effective since the match between services and students’ needs, as well as the monitoring systems, are not in place as they are in the City Connects.

• Treatment group non-adherence. Students’ mobility between schools should be examined with respect to group non-adherence. The City Connects treatment does not impede students’ transfer from one school to another. Thus, transfers from City Connects schools to non-City Connects schools can occur and vice-versa. If students transferred from a non-City Connects school to a City Connects school, then these students will be flagged as pre-treatment

students for the period that they were in non-City Connects schools and then flagged as City Connects students once they were in one of the treatment schools. Since the sample

definition will require that City Connects students be enrolled in one of the treatment schools by first grade at the latest, the sample definition automatically excludes pre-treatment

students from the analysis. In the opposite case, students that started in a City Connects school but then transferred to a non-City Connects school will always be considered City Connects students in the context of the study and will therefore also be automatically excluded from the control sample. Because of these measures, the final analytic sample will be unlikely to include students who might pose a group non-adherence threat. Thus,

3.1.2.9.2 Strongly ignorable treatment assignment assumption. Rosenbaum and Rubin (1983) describe this assumption as the independence of treatment assignment and potential outcomes, given the observed and unobserved pre-treatment variables. In randomized studies, every individual has a chance of receiving each treatment and which treatment they are given does not depend on potential outcomes. Simply put, this implies that treatment assignment is strongly ignorable given a vector of pre-treatment variables. However, this assumption islikely violated when randomization is not used as the assignment mechanism. For this study, schools were assigned to treatment conditions as whole units, and so the assignment mechanism was non-random. Consequently, this study used propensity score weights to balance scores so as to estimate the probability of assignment to a treatment group given observed pre-treatment variables (Rosenbaum & Rubin, 1983b) (see section 3.3.2 for further details).

3.2 Data Description