1
Proposed Scope of Study
Introduction
This proposal outlines the process for reviewing Fairfax County Public School’s Advanced Academic Program in order to assist the School Board in future program planning. Three key beliefs drive the development of this proposal and the review suggested: 1) the fundamental role of the program review is to provide information that can be used to improve and enhance FCPS’ Advanced Academic Programs, 2) the use of multiple data sources helps to illuminate the complexity and salience of program issues that need to be considered, and 3) the program review should be a collaborative enterprise among various stakeholders in the district and gifted education experts.
Board Requests
At the February 11, 2013 Work Session, the Board requested the following issues be addressed in the AAP review:
• Request #1: Historical account of the identification practices (including assessments) that encompasses the significant increases in AAP eligibility
• Request #2: Full spectrum of all levels of services, not just centers
• Request #3: Recommendations for examining consistency of implementation across all schools
• Request #4: Examine AAP delivery and its connection with IBMYP and middle school honors
• Request #5: Critical mass analysis and connection to delivery methodology and quality of staffing
• Request #6: Information on AAP teacher certifications, both FCPS and best practices, and number of staff certified
• Request #7: Information on AAP services in neighborhood schools, including transportation costs
• Request #8: Impact of Young Scholars program on eligibility, implementation of
advanced math across county, use of external assessments for eligibility and successful programs used in other jurisdictions
• Request #9: Determination of alternative certification approaches
• Request #10: Use of surveys of stakeholders to help ground study
2
Guiding Questions
The following guiding questions will provide a framework for the program review:
• To what extent is FCPS practice in the identified focus area aligned with best practices in the field of gifted education?
Investigation of this question will focus on assessing the congruence of FCPS practices with National Association for Gifted Children (NAGC) program standards.
• To what extent is FCPS practice in the identified focus area perceived to be effective by relevant stakeholders?
Investigation of this question will focus on assessing the perceptions of administrators, parents, students, and teachers on benefits and liabilities of FCPS practices in the identified focus area.
• What are the FCPS strengths and areas for improvement in the identified focus area?
What are the recommendations for improvement and expansion?
Investigation of this question will focus on the results of a discrepancy analysis and multiple data that is collected and analyzed.
These questions will be addressed for each of the Focus Areas identified below:
Focus Area #1: FCPS Identification Procedures
To what extent are FCPS identification practices aligned with best practices in the field of gifted education?
1. Report on the identification procedures used for student eligibility in FCPS Advanced Academic Programs including:
i. Definition of Virginia criteria for identification practices and area(s) of student aptitude served
ii. Historical description of identification practices used in FCPS, changes in FCPS practices over time, and rationale for changes (Board request 1)
2. Report on the composition of current FCPS AAP student population (Board request 2; Board Request 8) including:
i. Disaggregated data by federal ethnicity reporting categories, level of services (level 1-4), and enrollment in Centers and Local Level IV programs
ii. Analysis of FCPS enrollment in comparison to a national sample
iii. Analysis of factors contributing to increased enrollment, including: Young Scholars program, test preparation industry, private testing, and out-of-district, private school, and home school student transfers into FCPS for AAP services
3
3. Report on results of discrepancy analysis of FCPS student identification practice alignment with NAGC best practice standards
To what extent are the identification procedures perceived to be effective by relevant stakeholders?
1. Report on stakeholder survey results
What are the strengths and areas for improvement in the area of identification? What are the recommendations for improvement and expansion?
1. Investigation of this question will focus on the discrepancy analysis and data that is collected and analyzed
Focus Area #2: Quality of Program Services
To what extent are the FCPS Advanced Academic Program services aligned with best practices in the field of gifted education?
1. Description of the FCPS Continuum of Services (Board request 2)
2. Review of Center program fidelity of implementation including (Board request 3):
i. Curriculum offerings, extra-curricular activities (Board request 3) ii. Qualifications of staff and teacher certification (Board request 5) iii. Grouping practices and critical mass (Board request 5)
3. Comparison of the FCPS center model with comparable districts by examining (Board request 8):
i. District demographics
ii. District service delivery model iii. District identification process
iv. District’s student population receiving AAP/gifted services
4. Report on results of discrepancy analysis of FCPS program services alignment with NAGC best practice standards
To what extent are the FCPS Advanced Academic Program services perceived to be effective by relevant stakeholders?
1. Report on stakeholder survey results
What are the strengths and areas for improvement in the area of Advanced Academic
4
Program services? What are the recommendations for improvement and expansion?
1. Investigation of this question will focus on the discrepancy analysis and data that is collected and analyzed
Focus Area #3: Curriculum and Instruction
To what extent is the FCPS Advanced Academic Program Curriculum and Instruction aligned with best practices in the field of gifted education?
1. Report on the FCPS AAP Curriculum and Instruction model including:
i. Description of FCPS Advanced Academic Curriculum Framework (Board request 2) ii. Description of IBMYP (International Baccalaureate Middle Years Program) and
alignment with the Level IV curriculum (Board request 4)
iii. Comparison of Middle School Honors curriculum and Level IV center curriculum (Board request 4)
To what extent is the FCPS Advanced Academic Program Curriculum and Instruction perceived to be effective by relevant stakeholders?
1. Report on survey results
What are the strengths and areas for improvement in the FCPS Advanced Academic Program Curriculum and Instruction? What are the recommendations for improvement and
expansion?
1. Investigation of this question will focus on the discrepancy analysis and data that is collected and analyzed
Focus Area #4: Teacher Certification and Professional Development
To what extent is FCPS Advanced Academic teacher certification and professional development aligned with best practices in the field of gifted education?
1. Report on the FCPS AAP teacher certification and professional development process including:
i. Data on the number of staff certified that teach in a Center or Local Level 4 program (Board request 6)
ii. Description of paths to gifted and advanced academic endorsement iii. Explanation of alternative certification approaches (Board request 9) iv. Description of training opportunities available to teachers
To what extent is the FCPS Advanced Academic teacher certification and professional
5
development perceived to be effective by relevant stakeholders?
1. Report on survey results
What are the strengths and areas for improvement in the FCPS Advanced Academic Program Curriculum, Planning, and Instruction? What are the recommendations for improvement and expansion?
1. Investigation of this question will focus on the discrepancy analysis and data that is collected and analyzed
6
Methodology
Sampling
The sampling plan for the program review will focus on data collection efforts in grades three through eight within selected elementary and middle school sites that operate as Centers for Advanced Academic education in Fairfax County Public Schools. Surveys will be conducted in these schools at selected levels of instruction. In order to assess the differences between students receiving Local Level 4 services and Center services, non-center schools with Local Level 4 services will be included in the study. Schools will be selected from different geographic areas of the county. The identity of the schools will be protected through a coding process.
Instrumentation
Data collected to address Guiding Question #1 will involve conducting a discrepancy analysis between the NAGC gifted program standards and FCPS Advanced Academic Programs to determine the alignment of best practices in the four areas of interest:
Identification/Assessment, Curriculum and Instruction, Programming, and Professional
Development/Certification. Areas of strength as well as areas in need of improvement will be determined by this analysis.
Data used to address Guiding Question #2 will result from a random sample survey of parents, students, teachers, and administrators.
Data collected to investigate Guiding Questions #3 would involve both empirical and perceptual sources. Onsite visits to a sample of schools designated in the review plan would be conducted that would include: 1) classroom observations and 2) review of program materials that
describe and define the current model of operation, including the identification system, and the curriculum used.
Finally, a triangulation of all data sources will be used to make valid inferences about the nature and scope of program strengths and weaknesses and recommendations to be suggested for an action plan.
In order to conduct the program review described, the evaluation will require the following instruments and protocols:
NAGC/CEC Standards Comparison
An analysis will determine whether or not the NAGC standards are being met by the FCPS Advanced Academic program. Areas of focus are identification/assessment, curriculum and instruction, certification/professional development, and programming.
7
Survey Questionnaire
A mail/self-administered survey will be sent to a random sample of students and parents in grades 3 – 8 receiving AAP services. A survey will also be sent to a random sample of Level IV teachers and administrators.
The Classroom Observation Scale-Revised (COS-R)
The program review will pilot the use of this 25-item instrument to assess the extent to which teachers are employing practices that differentiate instruction for advanced learners in their teaching. It has been used in several studies, with strong technical adequacy (.82 for inter-rater reliability). Validity was established by an expert panel.
Data Analysis and Interpretation
Both quantitative and qualitative analysis will be used in the analysis. Descriptive statistics such as means, standard deviations, frequencies, and percentages will be used to present the
classroom observation data. Data will be aggregated across school sites and classrooms.
Content analysis and identification of patterns or themes will be used to report findings.
Discrepancy analysis will be employed to assess alignment with the national program standards.
The findings will be reported by research question as well as by data source. Conclusions drawn regarding these questions will be based on the support available across data sources.
The review will also attempt to develop a plan of action to accompany the recommendations, suggesting annual goals across three years to upgrade/expand the program successfully.
Major Tasks and Timeline for the Work
The major tasks in the program review include the following, along with a relevant timeline:
1. Review all relevant program material………...………March 18, 2013 2. Survey relevant stakeholders (e.g., AAP students, parents, teachers)………March 22, 2013 3. Conduct classroom observations at targeted schools….………..April 15, 2013 4. Determine the extent to which best practices are being employed in the program, using
the NAGC Program Standards……….May 5, 2013 5. Develop an overall program review report that considers multiple data sources (surveys,
classroom observations, and program documents) to ascertain strengths and
weaknesses of the program……….June 7, 2013
8
6. Develop a set of recommendations for next steps in program development and action planning………..June 7, 2013 7. Present findings to the Board of Education/relevant groups……….June 10, 2013
Personnel
In order to carry out the scope of the program review within the time frame outlined in this proposal, FCPS staff will require technical assistance from external consultants with expertise in the field of gifted education.
