Performance Management Framework
Guidelines and Technical Guide
About the DC Public Charter School Board
The DC Public Charter School Board (PCSB) currently oversees 53 schools on 99 campuses, which serve nearly 30,000 students from every ward of the city. The organization’s mission is to provide high quality public school options for District of Columbia students, families, and community through four functions:
• A comprehensive review application process—ensures that the PCSB only approves charter school applications that will prepare and train students for post secondary experiences and individual career paths;
• Effective oversight—holds schools to high standards for results, with extensive reviews and data collection, and makes oversight decisions with the best interests of students in mind;
• Meaningful support—provides clear feedback and increased oversight to low performing schools, and rewards consistently high-performing schools with more autonomy; and • Active engagement of stakeholders—solicits community input and strives to be responsive to
and transparent with all who are impacted by and impact the PCSB and public charter schools.
The PCSB’s vision is to lead the transformation of public education in DC and to serve as a national model for charter school authorizing and accountability.
At the heart of the organization’s core values is the belief that every child is entitled to a high quality education that will enable him or her to leave high school well-prepared for college and career. A mayor-appointed governing board of seven with a professional staff of 24 is responsible for the oversight and management of the organization’s mission and vision.
SECTION 1:ACADEMIC EVALUATION - COMMON ACADEMIC INDICATORS, MEASURES, AND
METRICS ... 5
Overview of academic framework ... 5
Common Indicators ... 5
Standard schools serving students within grades 3-12 ... 6
Elementary school and Middle school framework ... 6
High school framework ... 6
Summary of academic evaluation frameworks - common academic indicators and measures ... 7
Common Metrics ... 9
Performance bar metrics ... 9
Missing data ... 10
SECTION 2: ACADEMIC EVALUATION FOR NON-STANDARD SCHOOLS AND PROGRAMS ... 11
Non-standard schools ... 11
Early childhood and elementary schools under third grade (PS-2) framework ... 11
Elementary schools under fourth grade (K-3) ... 11
Adult education ... 11
Special education .………..……….…11
New schools ... 12
SECTION 3:MONITORING CHARTER SCHOOLS’ ACADEMIC PERFORMANCE ... 13
PMF Performance tiers ... 13
PCSB responses ... 13
Tier I ... 13
Tier II ... 13
Tier III ... 13
Candidates for Charter Revocation ... 14
SECTION 4:STRUCTURE OF NON-ACADEMIC EVALUATION FRAMEWORK - COMPLIANCE ... 15
Overview of framework ... 15
Compliance ... 15
SECTION 5: STRUCTURE OF NON-ACADEMIC EVALUATION FRAMEWORK - FINANCE ... 16
Overview of framework ... 16
Reporting ... 16
Assessing Fiscal Management ... 16
APPENDIX... 18
Median Growth Percentile Overview ... 19
Accountability Plan Guidance ... 20
Compliance Review ... 23
TECHNICAL GUIDE ... 25
Glossary of Measures ... 26
4
Performance Management Framework Overview
The School Reform Act (“SRA”) grants the D.C. Public Charter School Board authority to hold D.C. public charter schools accountable for fulfilling their duties and obligations under the Act. The PCSB has developed these Guidelines to outline the process by which it will evaluate the performance of the charter schools, including how the PCSB will ensure that each school complies with its charter agreement and applicable law and how the PCSB will track the progress of each school in meeting its student academic achievement expectations.
The PCSB’s performance management framework is divided into the academic elements of school performance directly related to student outcomes and the non-academic elements of school performance, which include finance and compliance.
Each year, pursuant to §38-1802.12 (a)(3) of the SRA, the PCSB will evaluate the academic performance of each public charter school. The overall assessment will determine whether a school is designated a high-performer, mid-performer, or low-performer. Closer scrutiny will be directed to those charter schools that are failing to meet the goals and student academic achievement expectations set forth in their charters. Schools that are persistently or significantly low performers may be candidates for charter revocation pursuant to the PCSB’s authority under § 38-1802.13 (a) of the SRA.
The framework also includes an evaluation of a school’s performance in non-academic areas, such as financial health and compliance. Each year, pursuant to §38-1802.12 (a)(3) of the SRA, each school will receive an evaluation of its performance in these areas. As a result of these reviews, schools that show signs of fiscal mismanagement and/or have violated applicable law or the terms of their charter
agreement may receive closer scrutiny, including the possibility of recommendations to improve their performance. Schools that continually violate material terms of their charter agreement or applicable law, including violations relating to the education of children with disabilities, may be candidates for charter revocation pursuant to the PCSB’s authority under § 38-1802.13 (a) of the SRA. Schools that engage in a pattern of non-adherence to generally accepted accounting principles, engage in a pattern of fiscal mismanagement, or are no longer economically viable will be candidates for charter revocation pursuant to the PCSB’s authority under § 38-1802.13 (b) of the SRA.
5
Academic evaluation - common academic indicators, measures, and metrics
Overview of academic frameworkTo assess a charter school’s academic performance, the PCSB has developed an academic evaluation framework comprised of indicators, measures, and metrics. This structure has been adapted from a report by the National Consensus Panel on Charter School Academic Quality.1 Indicators are defined as “general dimensions of academic quality or achievement.” Measures are defined as “general
instruments or means to assess performance in each area defined by an indicator.” Metrics are defined as “the calculation method or formula for a given measure.” As an example, a common indicator of student performance is academic achievement level, a common measure of achievement is
performance on statewide assessments like the DC-CAS, and a related performance metric is the percentage of students in a school who score at least Proficient on the assessment.
This section of the Guidelines focuses on indicators, measures, and metrics common across all schools within each grade-span.
Common Indicators
For the 2010-2011 school year the PCSB used four indicators to measure academic performance: (1) student progress, (2) student achievement, (3) gateway measures, and (4) leading indicators.
Figure 1.
The four core indicators of academic achievement apply to every school; however, different weights will be assigned to the indicators depending on the grade span of the school.
1
See “A Framework for Academic Quality,” National Consensus Panel on Charter School Academic Quality, June 2008, available at http://www.bcsq.org/downloads/BCSQ_Report.pdf. Student Achievement Gateway Measures Leading Indicators Student Progress Academic indicators
6 Standard schools serving students within grades 3-12
Elementary and middle school (3-8) framework
Figure 2.
The elementary school framework applies to schools with all or part of the 3-5 grade-span and that include both 3rd and 4th grades. Schools that have only fifth grade in the elementary school span and that span into the middle school range will be considered part of the middle school framework. The middle school framework applies to schools with all or part of the 6-8 grade-span and that include both 6th and 7th grades. Schools where 8th grade is the only middle school year and with all or part of the 9-12 grade-span are included in the high school scoring framework.
The elementary and middle school frameworks are designed to place an additional emphasis on student progress and student achievement. Gateway measures are designed to capture key subject area mastery, literacy at the elementary school level, and mathematics at the middle school level. High school (9-12) framework
Figure 3.
This framework applies to all schools with any grades in the 9th through 12th grade-span.
The high school framework places less emphasis on student progress. This is largely due to the fact that this indicator is comprised of a measure based on the DC-CAS, and at the high school level, only 10th grade students are tested. The balance is more heavily weighted on achievement than progress, which reflects the fact that, at the high school level, students have neared the end of their public school tenure and the rate of progress is less important than the overall level that has been reached. Similarly, the high school framework places more emphasis on gateway and post-secondary measures that are indicators of overall preparation for college and work-force readiness.
For a list of all common measures and metrics please see the Glossary of Measures in the Technical Guide. Student Achievement Gateway Measures Leading Indicators Elementary and middle schools
40% Student Achievement Gateway/Post Secondary Measures Leading Indicators High schools 15% 30% 25% Student Progress Student Progress 25% 15% 20% 30%
7
Table 1
Elementary and Middle School Framework
Student Progress
Median Growth Percentile1 – Reading 20% Median Growth Percentile1 – Math 20%
40%
Student Achievement2
DC-CAS Proficient + Advanced – Reading 10% 3rd-5th grades – 5%3
6th-8th grades – 5%3
DC-CAS Proficient + Advanced – Math 10% 3rd-5th grades – 5%3
6th-8th grades – 5%3
DC-CAS Advanced – Reading 2.5% 3rd-5th grades – 1.25%3
6th-8th grades – 1.25% 3
DC-CAS Advanced – Math 2.5% 3rd-5th grades – 1.25%3
6th-8th grades – 1.25%3
25%
Gateway4
DC-CAS 3rd grade reading proficiency 7.5% DC-CAS 8th grade math proficiency 7.5%
15% Leading Indicators Attendance 10% Re-enrollment 10% 20% 1
For more information about Median Growth Percentile please see Appendix A.
2
Elementary and middle grades will be shown separately if at least two grades within the 3rd-5th grade range and the 6th -8th grade range are represented. If a school does not have two grades within both ranges, the available grades will be combined, i.e., a school with 3-7th grade would include both 3-5th and 6-7th separately; a school with 3-6th would combine 3-6th into one score.
3
If elementary and middle grades are shown separately the points available will also be separated so that schools can earn up to 5% of the total points for reaching proficiency for each subject and grade range and up to 1.25% of the total points for advanced. If a school has ES/MS combined, the maximum points for proficiency in each subject will be 10% of the total points for proficiency and 2.5% of the total points for advanced.
4
If a school has both a 3rd and 8th grade that school will have two gateways, each worth 7.5% of the total points. If a school has only 3rd or 8th that school will have one gateway worth 15% of the total points.
8
Table 2
High School Framework
Student Progress
Median Growth Percentile – Reading 7.5% Median Growth Percentile – Math 7.5%
15%
Student Achievement
DC-CAS Proficient + Advanced – Reading 10% DC-CAS Proficient + Advanced – Math 10% DC-CAS Advanced – Reading 2.5% DC-CAS Advanced – Math 2.5%
Advanced Placement or International Baccalaureate 5%
30%
Gateway
Graduation Rate 7.5% PSAT (11th grade performance) 7.5% SAT/ACT (12th grade performance) 7.5% College Acceptance rate 7.5%
30%
Leading Indicators
Attendance 10% Re-enrollment 10% 9th grade credits on track 5%
25%
If a charter school does not contain the grade for which a common measure applies, the points
associated with that measure will be removed and the total possible points available will be adjusted. For example, a school that terminates in the 11th grade will not have graduation, SAT, AP, or College acceptance data. In this case, the school will be evaluated against the remaining 72.5 points.
9 To assess charter schools’ progress in meeting their goals and academic achievement expectations, the PCSB will use various metrics to assign points to each common measure. The overall percentage of possible points each school receives will determine whether a school is designated as a high-performer, mid-performer, or low-performer. Closer scrutiny will be directed to those charter schools that are failing to meet or in danger of failing to meet the goals and student academic achievement expectations set forth in their charters. Schools that are persistently low performers may be candidates for charter revocation pursuant to the PCSB’s authority under § 38-1802.13 (a) of the SRA.
Performance bar metrics
Most common measures are based on the percentage of students that meet or exceed a certain
performance bar. For example, with DC-CAS proficiency, schools are scored based on the percentage of students that meet or exceed the performance bar of “Proficient” on the DC-CAS. Each student either meets or does not meet the common measure; the metric is based on the percentage of all students that meet the standard.
The floor determines the minimum value for which any points will be awarded. Charter schools will receive no points for values that are at or below the floor. For example, the floor for re-enrollment is 55%. A school where 50% of its students re-enroll would not receive any points for the measure even though half of its eligible students re-enrolled in the school.
The target determines the value at which the maximum points for a common measure will be awarded. For example, the target for re-enrollment is 90%. A school where 92% of its students re-enroll would receive the full amount of points available for the measure even though all eligible students had not returned to the school.
The tables below summarize the performance bar metrics for the common measures.
Table 3
Elementary and Middle School Framework
Indicator PMF Metric Elementary (3-5) Floor Middle (6-8) Floor Target (ES and MS) Progress MGP-Reading 30 30 70 MGP-Math 30 30 70 Achievement
DC CAS Reading % proficient + advanced 22 25 100
DC CAS Math % proficient +advanced 10 18 100
Reading % advanced 0 0 25
Math % advanced 0 0 25
Gateway DC CAS 3rd Reading % proficient DC CAS 8th Math % proficient
17 – – 22 100 Leading Indicators Attendance 85 85 95 Re-Enrollment 55 55 90
10
Table 4
High School Framework
Indicator PMF Metric High School Floor High School Target Progress MGP-Reading 30 65 MGP-Math 30 65 Achievement
DC CAS Reading % proficient + advanced 24 100
DC CAS Math % proficient + advanced 20 100
DC CAS Reading % advanced 0 25
DC CAS Math % advanced 0 25
Advanced Placement and
International Baccalaureate 0 15
Gateway
Graduation Rate 82 100
PSAT Performance (11th grade) 0 50
SAT/ACT Performance (12th grade) 10 75
College Acceptance Rate 63 100
Leading Indicators
Attendance 85 95
Re-Enrollment 59 90
9th grade on track to graduate 33 100
Missing data
Most data required to calculate the score for each charter school’s common measures will be collected from third party data sources. The PCSB will rely on charter schools to provide data that are not available through third parties. When data for a common measure are not available due to issues beyond a school’s control, such as small sample sizes for growth measures that require statistical calculations, then the points associated with that measure will be removed and the total possible points available will be adjusted. When data for a common measure is missing because a charter school has not provided it to the PCSB in a timely manner, zero points will be awarded for the relevant measure.
11
Academic Evaluation for Non-Standard Schools and Programs
Non-standard schools
Non-standard schools include ones that do not fall into the 3-12 grade range or are elementary schools without at least two years of DC-CAS testing data. Non-standard schools will not receive a PMF Annual Performance Review for the 2010-2011 or 2011-2012 school years and will instead be evaluated with accountability plans. Schools that span the preschool through elementary or middle grades will receive both an accountability plan for any grades served below 3, as well as a PMF for the 3rd-8th grades. The accountability plan system is designed to measure and report the academic performance for students at either end of the educational spectrum, while allowing schools the autonomy to identify appropriate measures based on their unique populations. Each school’s accountability plan was developed in concert with PCSB staff andtheschool’s leadership. Schools were provided guidance on the minimum and maximum number of targets to include in the accountability plan as well as criteria in the identification of assessments and measures to determine performance (see Accountability Plan Guidance charts in Appendix B).
With the exception of schools undergoing charter review, schools and programs using accountability plans were not rated or tiered for the 2010-2011 school year. Accountability plan results are
determined and reported based on whether the school met or missed its established targets. The accountability plan system is used for the following types of schools and programs. Early childhood and elementary schools under third grade (PS-2) framework
Preschools and early years of elementary schools are in the non-standard framework because DC does not mandate a standardized assessment for students in these grades. Schools with both early childhood and elementary school grades but that do not exceed 2nd grade will be considered in the early
childhood framework and be evaluated using an accountability plan for the 2010-2011 and 2011-2012 school years.
Elementary schools under fourth grade (PS-3)
Schools that end in the third grade are separated from other elementary schools because they do not have at least two years of DC-CAS data to measure student growth in mathematics and reading sufficiently.
Adult education
Schools offering adult education and G.E.D. coursework are included in the non-standard framework because DC does not mandate a standardized assessment for students in these programs.
Special Education
For the 2010-2011 school year, the PCSB recognizes one charter school as a stand-alone special
education entity as 100% o of the school’s population is identified with significant physical and cognitive disabilities. All eligible students taking the state-wide assessment at this school are administered the DC
12 CAS Alternative Assessment Portfolio to measure their academic and developmental progress and performance.
New schools
In their first year of operation, charter schools will be held to a modified academic review. Data will be collected on all measures, where available. For schools administering the DC-CAS, student achievement will be measured by the common measures of Proficient and Advanced in reading and mathematics. Beginning in their second year, new charter schools will be held to the same performance framework as existing schools. The growth measure will be based upon two years of data only, and the graduation rate will be modified according to OSSE guidelines.
Accountability Plan Guidelines
13
Monitoring Charter Schools’ Academic Performance
Pursuant to the SRA §38-1802.13 (a), the PCSB has the discretion to revoke the charter of a school that is failing to meet its goals and student academic expectations. Based on the data collected and the metrics applied to a school’s academic performance measures as outlined in the previous sections, the PCSB will make an overall assessment of a school’s academic performance on an annual basis.
The PCSB assesses academic performance using PMF results for standard schools and accountability plan results for non-standard schools. Using the PMF, the PCSB will designate a school as a high-performer, mid-high-performer, or low-performer. Under the accountability plan system, a non-standard school was not tiered or rated for the 2010-2011 school year. Closer scrutiny will be directed to those charter schools that fail to meet student academic achievement expectations set forth in their charters. Standard and non-standard schools that are low performers may be candidates for charter revocation pursuant to the PCSB’s authority under the SRA.
PMF Performance tiers
Using a 100-point scale and based on the scores for the academic scoring screen, standard schools will be identified as Tier I (high-performers), Tier II (mid-performers), or Tier III (low-performers). Tier I schools earn at least 65% of the possible points. Tier II schools earn between 35% and 64% of the possible points. Tier III schools earn less than 35% of the possible points. A school must meet the threshold for points for each tier; points are not rounded up to the next whole number. The threshold points for identifying each tier will be set every few years and identified through an assessment of past overall school performance across all public charter schools.
PCSB responses Tier I
High performing schools will be publically recognized as such by the PCSB and will be exempt from Program Development Reviews (PDRs) unless conducted as part of the charter review or renewal cycle. Tier II
Mid-performing schools will be exempt from PDRs unless conducted as part of the charter review or renewal cycle.
Tier III
The PCSB will commence a full PDR for low-performing schools during the school year following the academic performance evaluation. The PCSB will suggest that low-performing schools develop a plan to target areas in need of improvement. Charter schools that are low performers may be candidates for charter revocation pursuant to the PCSB’s authority under the SRA.
Program Development Reviews are conducted at the campus level; multi-campus charter schools will receive a PDR for a Tier III campus, but would not undergo a PDR for a Tier I campus.
14 Candidates for Charter Revocation
Charter school may become candidates for revocation due to academic or non-academic issues. Under the PMF, candidacy for revocation due to academics will be based on severity, trajectory, or duration of low performance at one or more schools within an LEA. Based on PMF results, standard schools scoring below 20 percentage points in the most recent year, showing ≥ 5 percentage point decrease within Tier III from one year to the next , or performing in Tier III for three consecutive years will become
candidates for revocation.
Under the accountability plan system, non-standard schools that fail to attain the majority of the academic performance goals listed in its accountability plan or come within 90% of all missed academic performance goals on its accountability plan, perform within a minimum of 90% of its accountability plan attendance targets, or maintain sufficient enrollment levels to sustain the economic viability of the school will become candidates for revocation.
Any public charter school may be a candidate for revocation of its charter if it commits a violation of applicable law or a material violation of the conditions, terms, standards, or procedures set forth in the charter, including violations relating to the education of children with disabilities.
The PCSB retains the right to revoke a school’s charter at any time pursuant to its authority under the Act and the terms outlined in §38-1802.13 of the Act.
1
Poor academic performers are standard schools that 1) score 20% or below on applicable frameworks in the most recent year; 2)decrease in academic score within Tier III by 5 or more percentage points between two consecutive years,or 3) remain in Tier III for three consecutive years.
15
Structure of non-academic evaluation framework - compliance
Overview of framework
Schools that continually violate material terms of their charter agreement or applicable law, including the SRA, may be candidates for charter revocation pursuant to the PCSB’s authority under § 38-1802.13 (a) of the SRA. The PCSB may use its oversight authority under the SRA § 38-1802.11 (a)(2) to request documents and information from charter schools when it determines it is necessary to carry out its duties under the SRA.
Compliance
SRA §38-1802.04 outlines the duties and other requirements of public charter schools. As part of its oversight responsibilities, the PCSB will determine on an annual basis, in coordination with the charter schools, whether sufficient documentation is on file with the PCSB to ensure the charter schools are satisfying these duties and requirements. Further, the PCSB may require schools to provide documents supporting charter schools’ compliance with applicable law, including the Elementary and Secondary Education Act. These documents include, but are not limited to:
• Background check reports conducted on employees to ensure the charter school is maintaining the safety of all students attending the school.
• A report that documents that the charter school’s facilities comply with the applicable health and safety laws and regulations of the federal government and the District of Columbia, including the D.C. Fire Prevention Code.
• The school’s informal complaint resolution process and student handbook.
• Compliance with No Child Left Behind notifications and Highly Qualified Teacher requirements. To ensure charter schools are governed by a Board of Trustees in a manner consistent with the charter granted to them and in accordance with the requirements of the School Reform Act, the PCSB requests charter schools submit board meeting minutes on a quarterly basis and written notification of any board or key personnel changes as these changes occur. PCSB staff will verify that these documents are on file from the previous school year and request a copy of the current Board of Trustees roster.
• Pursuant to the School Reform Act §38-1802.11 (a), the PCSB has the duty to ensure public charter schools comply with the annual reporting requirement of SRA §38-1802.04 (c)(11). The PCSB may use the contents of the annual report to assess a charter school’s academic
achievement and the extent to which the school is meeting its mission and goals as stated in its charter agreement.
• Certain events may trigger the PCSB to request additional information from a charter school pursuant to its non-academic evaluation of the school. These events include turnover of key personnel identified in the charter agreement and changes in school enrollment and daily attendance rates as reported in the charter school’s annual report.
16
Section 5
Structure of non-academic evaluation framework - finance
Overview of frameworkThe PCSB must revoke the charter of a school when it determines that the charter school has engaged in a pattern of non-adherence to generally accepted accounting principles or a pattern of fiscal
mismanagement or is no longer economically viable. See SRA §38-1802.13 (b). Accordingly, the PCSB conducts an annual review under the PMF of a charter school’s financial performance. Pursuant to this mandate, the PCSB also routinely collects financial statements from schools including interim and audited financial statements and annual budgets.
Reporting
Charter schools submit financial reports to the PCSB through the Authorizers Oversight Information System (AOIS). The PCSB has established a schedule for the submission of financial reports, and schools initially will be evaluated based on the submission of requested documents by the designated due date. If a charter school does not submit requested financial documents on time and no extension has been granted by the PCSB, the charter school will be subject to a more extensive inquiry by the PCSB. When a public charter school has failed to submit its financial reports in a timely matter, the PCSB may initiate a site-based diagnostic review conducted by a certified public accountant (CPA). The purpose of the review will be to determine the general adequacy of a school’s financial management systems and record keeping mechanisms. The emphasis of the review will be on:
1. Verifying budget and cash flow projections and processes are in place.
2. Checking for evidence that the school regularly monitors actual revenues and expenditures against its budget and cash flow projections.
3. Verifying that the school is current in its payment of employee retirement, health insurance costs, and employee withholding taxes.
4. Verifying that there is a long term financial planning process that appropriately considers the impact of expanding facilities and increasing enrollment on cash flow.
5. Verifying that the school’s books and records are up to date. 6. Verifying that checking accounts are reconciled in a timely fashion.
7. Verifying that there are appropriate internal controls, particularly with regard to segregation of duties among employees.
8. Verifying that financial reports are provided to the school’s Board of Trustees at least quarterly. 9. Verifying that major financial commitments and decisions are approved by the school’s Board. 10. Verifying that operating account balances are sufficient to carry the school throughout the year. 11. Verifying that there is a demonstrated technology software package in place to keep accurate
financial records and generate reports.
12. Verifying that the accounting system is set up to track revenues and expenditures by funding streams, where required.
13. Verifying that the school maintains proper documentation of grants and donations. Assessing Fiscal Management
To track a school’s fiscal management, the PCSB uses a General Performance Assessment (GPA). Through the GPA, schools receive a score between 0 (low) and 4 (high) based on eight financial ratios calculated using school-provided financial data. After the ratios have been tabulated, the PCSB will
17 ratios, and an explanation of each ratio, is contained in the PCSB’s Fiscal Policy Handbook.
The PCSB Business Oversight Manager will follow-up with school leadership to inquire about a school’s unsatisfactory financial performance. Schools may be allowed additional time to comply and provide information to enhance the school’s financial picture. If charter schools do not provide adequate supplemental financial data to show that the school is fiscally managed, the PCSB may initiate a site-based diagnostic review conducted by a PCSB approved auditor or the DC Office of Integrity and Oversight. The purpose of the review will be to determine the general adequacy of a school’s financial management systems, record keeping mechanisms, and to ascertain the school’s short and long-term sustainability. The emphasis of the review will be the same as the initial diagnostic review performed by a CPA as described above.
Schools that engage in a pattern of non-adherence to generally accepted accounting principles, engage in a pattern of fiscal mismanagement, or are no longer economically viable will be candidates for charter revocation pursuant to the PCSB’s authority under § 38-1802.13 (b) of the SRA.
18
Appendix
19 Student Growth Percentiles – Individual Student Growth
Traditional presentations of students’ DC CAS scores reflect absolute achievement. These snapshots are useful for describing the performance level of students within a school for any given year but do little to explain the progress students are making.
In order to paint a more comprehensive picture of student and school performance, the PCSB, in conjunction with The Office of the State Superintendent, implemented a measure of student progress that compares changes in a student’s DC CAS scores to changes made by other students with similar score histories. A student growth percentile measures student progress by comparing one student’s progress from year to year to the progress of other students with similar DC CAS score histories. In this way, students are only compared based on their previous DC CAS score, not on demographic metrics (like race, gender, or socioeconomic status). This metric uses scores from all District students (including those at DCPS schools) to determine an academic peer group and calculate individual student growth percentiles.
Percentiles are commonly understood values that express the percentage of cases that fall below a certain score. For example:
• A student with a growth percentile of 77 in 6th grade mathematics grew as much or more than 77 percent of her academic peers (students with similar score histories) from the 5th grade DC CAS Mathematics to the 6th grade DC CAS Mathematics. Only 23 percent of her academic peers grew more in mathematics than she did.
or
• A student with a growth percentile of 34 in 8th grade reading grew as well or better than 34 percent of her academic peers (students with similar score histories) from the 7th grade DC CAS reading to the 8th grade DC CAS reading. This student grew less in reading than 66 percent of her academic peers.
Since this growth measure is assessing change in performance, it is possible for students to have
performed below the proficiency mark but to have shown a considerable amount of growth. This sort of occurrence could indicate that elements within a school’s program are working to help this student advance. Conversely, it is possible for students to be performing well above the proficiency mark but to have shown little growth. In this way, this growth measure serves to put DC CAS scores into greater context.
Student Growth Percentiles in the Aggregate – Median Growth Percentile
To represent the growth of a school, individual student growth percentiles can be aggregated. After significant research and consultation, the PCSB chose the Median Growth Percentile model as the most appropriate measure for reporting the growth of a school. This model finds the median (the middle number for scores arranged in order from least to greatest) of each school’s individual student growth percentiles. This is known as the median growth percentile for a school.
The DC PCSB will report student growth for DC CAS reading and mathematics for grades 4 through 8, and grade 10. Because the model requires data from at least two grade levels, students in grade 3 (the first testing grade) will not be included in the calculation.
20 Appendix B: Accountability Plan Guidance
Adult Education/GED Programs
Schools using the Adult Education/GED Framework must choose at least 5 targets (ideally no more than 9) from the Academic Indicators. Should a school choose to include more than 9 targets, the total number of targets must be odd.
GRADE
STUDENT PROGRESS
Min/Max Targets Criteria
No minimum/ No maximum • Can be curriculum based or standardized; PCSB preference is measures be standardized
• Measures are chosen from what schools already have in place from the 2010-2011 school year
• Targets must assess a cross section of students in programs
STUDENT ACHIEVEMENT
Min/Max Targets Criteria
No minimum/ No maximum • Can be curriculum based or standardized; PCSB preference is measures be standardized measures
• Measures are chosen from what schools already have in place from the 2010-2011 school year
• Targets must assess a cross section of students in programs
GATEWAY
Min/Max Targets Criteria
No minimum/ No maximum • Optional
LEADING INDICATORS
Min/Max Targets Criteria
No minimum/ No maximum Re-enrollment: • Optional Attendance:
21 Early Childhood Programs
Schools using the Early Childhood Framework (schools serving preschool and/or prekindergarten students) must choose at least 5 targets and can choose up to 9 from all of the Academic Indicators: Student Progress, Student Achievement, Gateway, Leading Indicators and Mission Specific.
STUDENT PROGRESS (PS-2)
Min/Max Targets Criteria
2 minimum/ 3 maximum • Schools can choose
a minimum of 2 targets and up to 3 targets
• Can be curriculum based or standardized; PCSB preference is K-2 measures be standardized
• Aligned to Early Learning Standards when possible (PS-PK) • Aligned to Common Core Standards when possible (K-2) • Measures are chosen from what schools already have in place
from the 2010-2011 school year
• Indicators/targets must assess all grades (PS-2). • Measures do not have to be the same for all grades
STUDENT ACHIEVEMENT (1st -2nd grade- K is optional)
Min/Max Targets Criteria
1 minimum/ 3 maximum • Schools can choose
a minimum of 1 target and up to 2 targets
• Can be curriculum based or standardized; PCSB preference measures be standardized measures
• Preschool and Prekindergarten students are not included • Aligned to Common Core Standards when possible (K-2)
• Measures are chosen from what schools already have in place for 2010-2011 school year
• Indicators/targets must assess grades 1-2 with K being optional • Measures do not have to be the same for each grade
LEADING INDICATORS
Min/Max targets Criteria
1 minimum/ 2 maximum • Schools can choose
a minimum of 1 target and up to 2 targets
Re-enrollment:
• Not required for PS-K only schools.
• Not required, but optional for continuous schools. (i.e., any school that serves post 2nd grade)
Attendance:
• PS-PK (non-compulsory) 88% target • K-2 (compulsory) 92% target
22 MISSION SPECIFIC (optional for K-2nd grade)
Min/Max Targets Criteria
No minimum/ 2 maximum • Schools can opt
not to include a target or to include up to 2 targets
• Optional for K-2
• Schools can choose the measure but should relate to each grade-span
• Schools may utilize surveys (e.g., character education surveys, parent satisfaction/involvement)
• Schools may include an environmental assessment (e.g., CLASS, ECERS, ELLCO)
23
INDICATOR DOCUMENTATION LOOK FOR RATIONALE
Enrollment of New Students
1. Fair enrollment process
Enrollment application; written lottery
procedures with dates for enrollment process.
Lottery cutoff date and date of lottery published in advance (if applicable); waiting list includes name of student, wait list number and current status.
Compliance with School Reform Act Section 38-1802.06.
Student Suspension and Expulsion
2. Notice and due process
Current year student handbook or other written document that outlines the school's discipline policy and procedures.
Clear explanation of infractions; consequences, rewards, and interventions; clearly outlined due process procedures (including an explanation of the basis for the action).
Compliance with School Reform Act Section 38-1802.06 (g); guidance for PCSB staff when contacted by parents.
Student Health Records
3. Health and safety of students
Option 1: Notice of assigned nurse on staff .
Option 1: Letter from DOH or
contracted agency. Compliance with School Reform Act Section 38-1802.04 (c)(4) and the Student Access to Treatment Act of 2007.
Option 2: Copy of staff certificate to administer medications.
Option 2: Ensure that there is at least one certificate from school staff who has been certified to administer medications from approved agency
Background Checks on Employees and Volunteers
4. Health and safety of students.
Current roster of all employees and volunteers (working greater than 10 hours at the school) with
indication of date background check conducted and that a copy of the report is on file.
Documentation includes employee or volunteer name, date of hire, date of background check, and company that conducted the background check.
Compliance with School Reform Act Section 38-1802.04 (c)(4). Employee Handbook 5. Employment policies and the protection of confidential information. Employee handbook or other written document on policies and
procedures governing employment at the school.
Policies guiding the safeguarding of confidential student data; compliance with relevant employment laws including policies on sexual harassment, equal opportunity hiring, drug-free workplace, and termination.
Compliance with School Reform Act Section 38-1802.04, FERPA, the Public Education Reform
Amendment Act of 2007, and applicable state and federal employment laws.
24
INDICATOR DOCUMENTATION LOOK FOR RATIONALE
Insurance 6. Appropriate insurance. Certification that appropriate levels of insurance have been secured.
General Liability, Directors and Officers Liability, Umbrella Coverage, Property/Lease
Insurance, Auto Liability Insurance, Workers Compensation.
Compliance with School Reform Act Section 38-1802.04 (b)(4). School Facility 7. Lease/Purcha se Agreement and certificate of occupancy. Certificate of occupancy.
Name and current address listed on the certificate of occupancy; enrollment projections for school year within maximum capacity in the certificate of occupancy.
Compliance with School Reform Act Section 38-1802.04 (b)(4) - a Certificate of Occupancy is required at opening and for relocation to a new facility.
Lease/Purchase Agreement.
Name and current address listed on the lease/purchase agreement.
Compliance with School Reform Act Section 38-1802.04 (b)(4) - lease or purchase agreement is required at opening, for relocation to a new facility, and for amendments to a lease once it expires.
No Child Left Behind (NCLB)
8. School quality and choice. Communication with parents on school's compliance with NCLB before September 1 or within 14 days of school AYP results.
Communication, where applicable, on AYP results, School
Improvement Status, school choice options, teacher qualifications.
Compliance with NCLB and ESEA guidance.
9. High quality teachers.
For Title I schools, current year teacher roster with grade and subject(s) taught, HQ status, and how the status was met (HOUSSE, Praxis, Degree, License/Certificate); action plans for all non-HQT staff.
Appropriate documentation for verification of HQT compliance; plan of action to assist teachers in meeting the requirements; appropriate staffing changes to ensure compliance with the law.
Compliance with NCLB and ESEA guidance to ensure that all elementary and secondary subject area teachers are highly qualified.
Board of Trustees
10.
Composition.
Board roster with names and titles.
Odd number of members greater than 3 but fewer than 15; majority DC residents; at least 2 parents of currently enrolled students.
Compliance with School Reform Act Section 38-1802.05 and the school’s articles of incorporation and bylaws.
11. Fiduciary
Duty. Board meeting minutes.
Decisions made by the Board are consistent with the charter granted to the school, the School Reform Act, and applicable law.
Compliance with School Reform Act Section 38-1802.05.
25
Performance Management Framework
26 Glossary of Measures
Listed below are brief descriptions of some of the common academic scoring measures the PCSB uses to track charter schools’ progress in meeting their goals and academic achievement
expectations.
Student Growth Measures
Student progress
Appears for 3rd -5th grades, 6th -8th grades, and High Schools
Median growth percentiles for mathematics and reading, a growth measure based on individual student DC-CAS performance. See Appendix A for a discussion of these growth measures. See documentation provided by OSSE and AIR for Technical Report and Business Rules.
Student Achievement Measures
DC-CAS Proficient & Advanced
Appears for 3rd -5th grades, 6th -8th grades, and High Schools
Percent of students scoring at least Proficient and those scoring Advanced on the DC-CAS for math and reading; both Proficient and Advanced measures are included for all grades in which DC-CAS testing occurs.
Advanced Placement
Appears for High Schools
Ratio of number of passing scores for the entire population to number of seniors, written as a percentage. Equivalent assessments, such as the International Baccalaureate exam, may also be substituted.
Gateway Measures
DC-CAS 3rd grade reading
Appears for 3rd -5th grades
Percent of students scoring at least Proficient in third grade reading on the DC-CAS DC-CAS 8th grade math Proficient
Appears for 6th -8th grades
Percent of students scoring at least Proficient in eighth grade mathematics on the DC-CAS Graduation rate
Appears for High Schools
Graduation rate based on the incoming 9th grade cohort, as defined by OSSE state-wide graduation standards.
PSAT 11th grade performance
Appears High Schools
Percentage of 11th grade students scoring at a level on-track for college readiness in the combined mathematics and verbal sections of the PSAT, defined to be a score of 80 or higher. SAT 12th grade performance
Appears for 3rd High Schools
Percentage of 12th grade students scoring at a level on-track for college readiness on the combined math and verbal sections of the SAT, defined to be a combined score of 800 or higher.
27 College Acceptance
Appears for High Schools
Percentage of seniors accepted to a college or university.
Leading Indicators
Attendance
Appears for 3rd -5th grades, 6th -8th grades, and High Schools
Ratio for the entire population, written as a percentage, of days present (inclusive of excused absences) to days enrolled.
Re-enrollment
Appears for 3rd -5th grades, 6th -8th grades, and High Schools Percentage of those eligible students who re-enroll.
9th grade credits
Appears for High Schools
Percentage of 9th grade students on-track to graduate in 4 years, based on OSSE guidelines for completed Carnegie Units and required courses.
28
Metrics and data sources for academic performance measures
Overview
The table below provides details about each of the measures included in the academic performance framework. Information is arranged as follows:
Indicator – identifies the academic indicator (progress, achievement, gateway, leading) to which each measure belongs
Measure –provides measure name and indicates which scoring frameworks include it
Description – provides brief summary of measure
Components, method, and additional notes - provides a more detailed description of all steps required to translate raw data into a measure calculation
Rationale – available rationale for why the metric has been designed as it has
Unless otherwise indicated, all measures are grouped by the grade span of the performance framework to which they apply. Grade-spans are as follows:
ES/MS: 3-8
HS: 9-12
Data sources and methodology
Indicator Measure Description Components, method, additional notes Rationale
Student Progress Median Growth Percentile (ES/MS and HS) Growth measure which tells how much change or growth there has been in
performance from year to year.
Components
• Median for 3rd-8th(or all grades within that range the school serves) grade student growth percentiles in reading
• Median for 3rd-8th grades (or all grades within that range the
school serves) student growth percentile in Mathematics References: • http://pcsb-pmf.wikispaces.com/Changes+to+the+2010-2011+PMF • http://www.duxbury.k12.ma.us/SPEDfiles/MCASStudentGrowt h%20%25%20.pdf • http://www.swcompcenter.org/educator_effectiveness2/The_ Colorado_Growth_Model.pdf
• For more information see the MGP Business Rules produced by
OSSE and AIR.
• A measure of student progress that compares changes in a student’s DC CAS scores to other students with similar score performance profiles.
• It tells how much change or
growth there has been in performance from year to year.
• Requirement of two years of
DCCAS results of each student.
Student Achievement DC-CAS Proficient in math and reading (ES/MS and HS) % of students scoring Proficient or above on the DC-CAS Components
• DC-CAS scores by grade for the most recent year
• Number of students tested for the most recent year
Method
Aligns with the AYP calculation rules, in which students in the Fully Academic Year status and ELL students residing in the US for more than 12 months are included.
29 100 * tested # Total SY in that student proficient Math of # Total OR 100 * tested # Total SY in that student proficient Reading of # Total Student Achievement DC-CAS Advanced in math and reading (ES/MS and HS) % of students scoring Advanced on the DC-CAS Components
• DC-CAS scores by grade for the most recent year
• Number of students tested for the most recent year
Method
•
Calculated as with DC-CAS proficient
100 * tested # Total SY in that student Advanced Reading of # Total OR 100 * tested # Total SY in that student Advanced Math of # Total
Aligns with the AYP calculation rules, in which students in the Fully Academic Year status and ELL students residing in the US for more than 12 months are included. Student Achievement Advanced Placement performance (HS) Number of passing exams per 100 12th grade students
Components
• # of passing AP or IB exams in the current school year • 12th grade enrollment in the most recent year
Method
100
*
students
grade
12th
of
#
exams)
IB
and
exams
AP
passing
of
#
100
*
tested
graders
12th
of
#
Total
grade
3rd
in the
proficient
reading
of
#
l
Additional notes• Passing exams are all those scoring 3 or above for AP and 4 or
above for IB in the current school year
• One student may contribute multiple passing exams
• 9-11 grade students may also contribute passing exams
• Any HS with the 12th grade that does not offer either AP or IB
• All passing exams contribute
to the score, regardless of who takes the test
• Expressing passing exams as a “per 100” score adjusts to reflect size of school and corresponds more closely to national reporting norms
• Using only 12th grade as the
denominator ensures all schools are counted equally regardless of what grades they offer
• Using full 12th grade class as
denominator enables measure to track passing scores without creating incentives for
30
Indicator Measure Description Components, method, additional notes Rationale
exams will be given 0 points on this metric.
schools to limit size of AP classes
Gateway DC-CAS Proficient
in 3rd grade reading (ES/MS) % of 3rd grade students scoring at least Proficient on DC-CAS reading Components
• 3rd grade DC-CAS scores
Method
100
*
tested
graders
3rd
of
#
Total
grade
3rd
in the
proficient
reading
of
#
Total
• Critical gateway year for
childhood literacy
Gateway DC-CAS Proficient
in 8th grade math (ES/MS) % of 8th grade students scoring at least Proficient on DC-CAS math in the most recent year Components
• 8th grade DC-CAS scores
Method 100 * tested graders 8th of # Total grade 8th in the proficient math of # Total
• Critical gateway year for
adolescent number skills
Gateway Graduation rate
(HS) Graduation rate of high school students based on OSSE standards Components
• Original 9th grade cohort size for current 12th grade
• # of graduates – self-reported and validated through transcript
reviews
• # of dropouts in the 12th, 11th, 10th , and 9th grades
Method 100 * period) years 4 latest for dropouts SY for the graduates of # ( diploma) regular a with SY a for seniors of # Total + Additional notes
Drop outs are those students who leave schools for following reasons: (a) Whereabouts unknown (b) Work, (c) Voluntary, (d) Adult Education, (e)Non-Attendee, (f) No Show
• Aligned with OSSE standards
Regular Diploma (OSSE) excluded the following:
(a) GED, (b) certificates of completion, (c) certificates of attendance, (d) or, any other diploma not fully aligned with the State’s academic records.
• Note: This definition may
change starting 2011-2012 http://newsroom.dc.gov/sho w.aspx?agency=seo§ion= 2&release=20924&year=2010 &file=file.aspx%2frelease%2f2 0924%2fDistrict_of_Columbia _Adjusted_Cohort_Graduatio n_Rate_Guidance_December_ 2010.pdf
31
Gateway 12 grade SAT
scores (HS) % of current 12 grade students scoring at least 800 on the SAT Components
• SAT/ACT data for current 12th grade • SAT/ACT data for previous year 11th grade
• Map of previous year 11th grade to current 12th grade • 12th grade enrollment in the most recent year • Additional notes
• Only current 12th graders are counted, but students scoring
above 800 in earlier grades and not in 12th grade will still contribute to score. In terms of ACT, students scoring above or equal to 16 are counted as on the numerator.
• Any HS with the 12th grade that doesn’t offer the SAT or ACT
exams will be given 0 point on this metric.
Method
• 11th and 12th grade students mapped together to determine
overlap 100 * students) grade 12th of # ( ACT) in M) (V hihger or 16 SAT in M) (V 800 least at scoring graders 12th current of # ( + + +
• College Board research indicates that a score of at least 800 on the math and verbal sections of the SAT predicts with high likelihood a GPA of at least 2.0 in
freshman year at community college or university
• For reference visit:
http://professionals.collegebo ard.com/profdownload/pdf/R N-30.pdf
• SAT-ACT concordance table: http://www.act.org/aap/conc ordance/
Gateway 11th grade PSAT
scores (HS) % of 11th grade students scoring at least 100 on the combined math and verbal portion of the PSAT
Components
• PSAT data for current 11th grade
• 11th grade enrollment in most recent year
• Any HS with the 11th grade that doesn’t offer the PSAT exam
will be given 0 points on this metric.
Additional Notes
• 9th and 10th grade PSAT scores do not count because they are
standardized to a different scale
Method 100 * students) grade 11th of # ( SAT) P on M) (V 80 least at scoring graders 11th current of # ( +
• 11th grade PSAT performance of at least 80 on the math and verbal sections is a predictor of SAT performance of at least 800 on the math and verbal sections.
• For reference visit:
http://professionals.collegebo ard.com/profdownload/PSAT-
32
Indicator Measure Description Components, method, additional notes Rationale
Gateway College Acceptance (HS) % of high school seniors accepted to a full-time college or university Components
• Unique college acceptances – self reported, verified by PCSB
acceptance letter review
• 12th grade enrollment in most recent year
Method 100 * seniors) of # ( ) university or college time full a to accepted seniors of (# • College acceptance is a necessary step to college enrollment and has been defined to be an important measure of student success
Leading Attendance (ES/MS) Total Daily Attendance rate for 3rd to 8th grade. Components • Attendance by grade-span Method 100 * enrolled) days (# absences) excused of # present days of (# +
• Aligns with the State calculation. Only 3rd through 8th graders are included.
Attendance (HS) Total Daily Attendance for 9th to 12th grade Components • Attendance by grade-span Method 100 * enrolled) days (# absences) excused of # present days of (# +
• Aligns with the State calculation. Only 9th through 12th graders are included.
Leading Re-enrollment
(ES/MS & HS)
Re-enrollment rate for those eligible to return # of students ineligible to reenroll are: (a) Students in the exit grade (b) students who moved out from DC. (c) students in the non standard school(EC, Adult
Components
• Previous fall enrollment by student id and grade (Audited Oct 5) • Current fall enrollment by student id and grade (Audited Oct 5)
Method
• Map students in previous fall and current fall to determine
overlap
• Determine which students in previous falls are ineligible to
re-enroll (those in their final year; moved out from DC; re-enrolled in Early Childhood, Adult and GED frameworks)
100 * enroll) to ineligible students of (# -Fall) previous in students of (# fall) current and Fall previous both attending Students of # (Total
• Used to assess student and parent satisfaction and drop-out rate
• Excluded are from early
childhood and non-GED Adult Ed. because students are likely to leave for reasons other than dissatisfaction with school
33 and GED)
Leading 9th grade credits
(HS) Percent of 9th grade students receiving enough credits to be on track to meet OSSE graduation requirements in 4 years Components
• Review of all 9th grade transcripts – on-site review
Method 100 * students) grade 9th of # (Total ) years) 4 in ts requiremen graduation LEA meet track to on be to credits enough receiving students grade 9th of # (
• OSSE requirements: Students must pass 6 credits or the
equivalent, consisting of Algebra I, an English course, a Science course, a Social Studies course that is among World History, US History, US Government or DC History, and two additional elective courses.
• LEA requirements vary by LEA.
• Some schools have adopted different internal standards for credit than OSSE
