Background
Initiation of the Performance Review
Motivation: The Florida Legislature’s Office of Program Policy and Government Accountability (OPPAGA) sought this Performance Review to help provide recommendations for addressing the Orange County School Districts’ current and long range problems as well as provide other school districts with a model for
self-assessment.
Purpose: The purposes of OPPAGA in requesting this Performance Review are to identify ways that the Orange County School District can:
• Save funds,
• Improve management, and
• Increase efficiency and effectiveness.
Management Auditor: KPMG Peat Marwick LLP, an independent management consulting firm, conducted this
review, using a methodology refined during reviews of numerous school systems throughout the country. KPMG was selected through a competitive process from several other firms submitting proposals.
Background
Background: Orange County Public Schools (OCPS) is a growing system, with growing resource requirements:
• OCPS has grown faster than most other Florida school systems, growing approximately 4 percent annually since 1993-94.
• Enrollment growth is expected to continue at a similar rate into the foreseeable future.
O r a n g e C o u n t y E n r o l l m e n t G r o w th 1 1 4 , 3 7 7 1 1 8 , 6 6 6 1 2 3 , 2 3 7 1 2 9 , 1 3 7 1 0 5 , 0 0 0 1 1 0 , 0 0 0 1 1 5 , 0 0 0 1 2 0 , 0 0 0 1 2 5 , 0 0 0 1 3 0 , 0 0 0 1 9 9 3 - 9 4 1 9 9 4 - 9 5 1 9 9 5 - 9 6 1 9 9 6 - 9 7 S o u r c e : F l o r i d a D e p a r t m e n t o f E d u c a t i o n
• Per student, OCPS’ expenditures have grown by 5.5 percent between 1993-94 and 1995-96.
• As can be seen in the graph at right, OCPS expenditures per student were less than the peer and state averages.
• Peer spending grew 6 percent and state spending per student increased 6.4 percent between 1993-94 and 1995-96.
Expenditures per Student
$4,581 $4,631 $4,831 $4,760 $4,918 $5,047 $4,724 $4,879 $5,026 $4,300 $4,400 $4,500 $4,600 $4,700 $4,800 $4,900 $5,000 $5,100 1993-94 1994-95 1995-96 Orange Peer Average State Average
Background
Background: OCPS students achieve comparable state averages on the 1996-97 High School Competency Test (HSCT): Reading 78% 77% 70% 72% 74% 76% 78% 80%
Orange State Average
M a t h e m a t i c s 7 6 % 7 5 % 7 0 % 7 2 % 7 4 % 7 6 % 7 8 % 8 0 % O r a n g e S t a t e A v e r a g e
Source: Florida Department of Education
Background: Slightly more OCPS graduates pursue higher education than the state average:
Percent of Graduates Entering College
63% 61% 60% 60% 61% 61% 62% 62% 63% 63%
Scope, Limitations and Audit Methodology
Study scope included:
• School District Organization and Management
• Functional Analyses:
– Educational Service Delivery – Personnel Management – Community Involvement – Business Services
– Asset and Risk Management – Financial Management – Purchasing *
– Facilities Use and Management – Safety and Security
– Food Service – Transportation *
– Other Areas for Review: Information Services
* A recently conducted performance review exists that KPMG feels is representative. Therefore, KPMG did not conduct a detailed analysis in this area. Limitations: This study’s scope did not include:
• Classroom instruction, school-site operations or student performance.
• Performance of individual employees or managers.
• Reviews of financial controls or of financial/grant compliance.
While information received from OCPS management and staff, and that received from other school systems was reviewed for reasonableness, KPMG did not audit information received to verify its accuracy.
Scope, Limitations and Audit Methodology
Performance Review Methodology
KPMG’s Performance Review methodology included:
• Interviews:
– Board of Education members.
– Superintendent, Associate and Assistant Superintendents, and selected other School System managers – School system staff in each functional area reviewed.
• Focus groups conducted to obtain perspectives, concerns, priorities, and suggestions from: – Principals.
– Teachers and teacher association representatives. – Parent association representatives.
– Students.
– Community input sessions.
• Benchmarking - comparisons of expenditures, operating efficiency (e.g., workload per staff member), service level policy, processes, best practices, and other operational results.
• Customer and employee surveys - to help gauge customer satisfaction, identify concerns, and solicit ideas.
• Observation of operations, including visits to school and administrative sites, technology demonstrations.
• Analysis - analysis and synthesis of data to identify trends, patterns, issues and solutions.
Focus Group Purpose and Methodology
Purpose: KPMG conducted focus groups with various stakeholders to obtain detailed feedback on the Orange County Public School’s strategies, performance, and methods. This section summarizes focus group results. In addition, focus group comments and results are reflected in each appropriate functional analysis and in
accompanying recommendations. The following focus groups were conducted:
• Principals (randomly selected)
• Teachers (randomly selected)
• Students (randomly selected from high school student governments)
• Parents (randomly selected from Parent Teacher Associations)
• Community input sessions (publicly advertised)
Methodology: KPMG randomly selected principals, teachers, parents, and students to invite to focus groups, and conducted each focus group in a structured format. Participants were promised anonymity to assure free and open discussions.
Principal Focus Group Results
Fifteen elementary, middle, and high school principals were randomly selected by KPMG and provided the following input.
Orange County School District Strengths:
• The principals’ leadership support team provides adequate and timely leadership support.
• Clearly defined curriculum benchmarks.
• The transportation system is more responsive to school’s concerns and needs.
• Personnel Services provides assistance and guidance to new teachers and up-to-date information on certification requirements.
• Good support of student services such as Student Resource Officers (SROs), Alternative Program (Excel), and the expulsion process.
• The Community Relations Department is helpful and supportive.
Areas for Improvement:
• Need to recruit more certified ESE teachers.
• Decrease the length of time it takes to hire new teachers.
• Improve pupil assignment and placement.
• Increase technology support to schools.
• Improve the equity of technology among schools by providing all schools with equal access to technology and provide timely technology support in areas of trouble shooting, and training by allocating more funding to hire technology specialists.
Teacher Focus Group Results
Twelve elementary, middle, and high school teachers were randomly selected by KPMG to participate in an employee focus group.
Orange County Public School Strengths:
• We’ve always had wonderful things [in-service] provided to us by OCPS— except for the paperwork.
• As our cultural diversity grows, we’re seeing new things. They [SROs] serve as role models, and are helping us deal with issues.
• Internal communication is good— principals put staff development opportunities in your box, Orange Peal and other publications...I think they do an excellent job of identifying news.
• Program specialists link the district to the schools.
• Continue to look for savings and put them classroom [i.e.; $1,000 Superintendents grant].
• Course offerings to students— they do a great job.
• There is a lot of desire to have input (with new Superintendent).
• Central office is helpful and friendly.
• Leadership is trying to look for new styles...techniques of teaching.
• Trend for OCPS is toward decentralization; I like the fact that things are being pushed out from one building.
• Technology positions that are there to help teachers with the Internet (at central office).
Teacher Focus Group Results
Areas for Improvement:
• Education Leadership Center is more plush than schools. If this is set up for the kids, you’d see the kids in the best buildings, and what is left over for the administrators.
• Public perception is bad.
• We need money for classroom support— there isn’t enough for resources and materials per student.
• The District is too big to manage and we get lost in system.
• Old ways [teaching methods] are still around— there is no method for getting rid of teachers who won’t change or get them to change to new teaching methods.
• The District needs to be stronger in knowing a way to hold teachers accountable in the classroom.
• My school is 72 years old— it’s a “historic site” and they can only do certain things. We’re deprived of technological advancements.
• We do a poor job of tracking where kids are grouped or how well they do. We teach the best and bury the rest.
• Pupil assignment issues— boundaries are constantly moving.
• Class size is a major issue.
• Year after year, teachers didn’t know if they were going to have a job in August. I wish we knew what was going on.
• Poor top-down communication.
• Frequent movement of principals.
• We need to support people [technology] that know what they are doing, and pay them accordingly.
Student Focus Group Results
KPMG randomly selected seven high school students to participate in a student focus group.
Orange County School District Strengths:
• Principals are visible and approachable.
• Excellent Food Services: “Great value. They feed us well. They make good stuff.”
• Technology is excellent and available to student body.
• Ethic composition of some schools foster diverse educational opportunities.
• Competent teachers: “I go to class excited, because they know more than me.”
• Feel safe and secure in classrooms and on school campuses.
Areas of Improvement:
• Some teachers do not know how to handle discipline.
• Inequity in distribution of textbooks. Accelerated classes have an abundance of books; however, shortage of books for average classes prevents students from taking books home.
• Overcrowding in schools: limited locker space, lack of parking places, and reduced recreational areas because portables are taking up space.
• Many schools need new roofs.
• Inequities in distribution of technology among schools.
• Lack of technology standardization.
• Teachers lack expertise to integrate technology into classroom curriculum.
Parent Focus Group Results
KPMG randomly selected 15 Parent Teacher Association (PTA) members to participate in a parent focus group.
Orange County Public School Strengths:
• The majority of teachers and administrators are excellent even though we are well below the national average in salaries.
• I have worked in two surrounding counties and Orange County by far surpasses either in offering staff development to its employees.
• We have excellent technical preparation programs and many are pilot programs.
• School buses are clean and in good operational condition.
• I like that school buses have radios on them to communicate their needs.
• Computers are available in kindergarten through high school.
• There is technology funding available for each school and there is Internet access available in elementary schools.
• Mid-Florida Tech has an excellent technology vision.
• Food managers seem to be well trained, and there is good support from the Food Services Department.
• I like the newer elementary schools with a permanent core and flexible number of high quality portables reached by covered raised hallways.
• Orange County encourages support from the community.
Parent Focus Group Results
Areas for Improvement:
• There needs to be consistency in courses regardless of who is teaching the class.
• Allow teachers to expand on classroom instruction without principals micromanaging them.
• The District needs to develop a method of curriculum development that promotes ownership at the classroom teaching level.
• Train teachers to better recognize learning disabilities and allow aids to help those students in class instead of singling them out and having them leave class
• Need more staff in the Personnel Department to assist schools with staffing needs.
• The District needs to dispel the myth that School to Work is for low academic students by creating an honor roll or honor’s society so that these students can begin to receive respect from their peers.
• Get rid of portables and use funds to add on classrooms.
• There needs to be more bus stops because many children walk too far alone before and after-school.
• There needs to be monitors on each bus to prevent dangerous situations.
• Need more crossing guards and school zone warning signs at major intersections.
• Lack full-time security guards during school hours.
• Create a forum to bring parents and the District together to address student crime, gangs, drugs, etc.
• Provide support groups and training for exceptional education parents
• Offer computer classes to parents so that they can learn along with their children and create a parent computer take-home program.
• Increase business and community involvement in technology decisions.
• Need to develop ways for exceptional education parents and foreign speaking parents to participate in activities with schools as a whole (e.g., open houses, PTA, SAC, etc.).
Community Input Sessions
KPMG held two public forums to provide the Orange County community with the opportunity to provide its opinions concerning the performance of the Orange County School District. Fourteen participants provided input on the District’s strengths and areas for improvement. The community input sessions were December 16, 1997 at Howard Elementary School and January 22, 1998 at Southwest Middle School.
Orange County Public School Strengths:
• Teachers and support staff are great.
• The training available to teachers is good.
• Tech prep is developing well for greater student access.
• OCPS is on the cutting edge of instructional technology.
• Fully networked schools and up-to-date computers and software.
• Streamlined ways to communicate via e-mail and the Internet.
• The OCPS Websites for Strategic Planning and the Curriculum Frameworks
• Willingness to support community/business partnerships in education.
• The Hearing and Vision Department is very good.
• The Accelerated Reading Program is great.
Community Input Sessions
Areas for Improvement:
• Class sizes are too large.
• Encourage good teachers to mentor new teachers.
• Provide adequate training and curriculum development of teachers.
• Need more staff development to train teachers how to effectively integrate technology into the curriculum.”
• Insufficient training of teachers who have exceptional students mainstreamed into their classes.
• Need more computers in the classrooms and more schools need to be retrofitted.
• Need to fully integrate technology into every subject.
• Too much waste in finance and budgeting - too many chiefs with too little experience in the field they are managing.
• Contract out food services.
• Serve real food not fast food and provide better menus
• Plan for new schools before existing schools are overflowing!
• If you build schools cheaply then you get cheap schools - design more efficient and cost effective schools.”
• Ask developers to work with the District to provide schools - they don’t have to be the antagonists!”
• Facilities - Our children cannot wait for committees to decide that something must be done - do it right the first time for our children’s sake.
• Use long range planning of facilities in cooperation with county developers for optimum placement of new facilities as growth occurs.
• Use land banking for sites of future developments.
• Allow the community to work with the School Board Site Selection Committee for locating new schools - the community can be the best support.”
Community Input Sessions
• Standardize the fire alarm systems.
• Increase lighting at school properties to prevent crimes.
• Not enough cleaning, painting, debris removal, and housekeeping of facilities.
• Need more money and people to keep up properties.
• Inadequate training of bus drives to effectively handle students with behavioral problems.
• Transportation schedules should not determine school times-high schools and elementary schools start too early because of busing.
• Need monitors and/or cameras on buses.
• More businesses need to be involved with student internships.
• Student internships need to relate to future employment goals.
• Increase the visibility and awareness of the TAES and vocational program to businesses through increased marketing.
Benchmarking Candidates
KPMG selected the peer districts used for comparison based on demographics, revenues and expenditures, and enrollment size. Additionally, OCPS leaders provided input as to whom they considered peers based upon
experience and knowledge of Florida’s school districts. This information assisted KPMG select OCPS’ Florida peer districts. KPMG selected Fairfax County, Virginia because of its reputation as a district leader in the development and integration of best practices into management. In selected situations some benchmark districts were chosen because of their reputation as leaders in an area based on availability of data.
1996-97 1995-96
Peer Districts Total Staff Students Total Schools Revenues Expenditures
Orange 15,356 129,137 138 $ 761,625,000 $ 707,641,946 Broward 20,373 215,557 188 1,345,596,000 1,208,907,179 Duval 11,575 126,116 145 673,672,000 611,811,792 Hillsborough 17,650 147,897 172 876,838,000 824,125,937 Palm Beach 14,876 137,663 146 927,518,000 820,990,051 Pinellas 12,933 107,077 126 654,473,000 572,144,215 Peer Average 15,481 146,862 155 895,619,400 769,432,000 State Average 3,702 33,424 38 200,668,657 173,268,582 Fairfax, VA 16,554 143,040 204 870,717,093 856,560,887
Source: Profiles of Florida School Districts 1995-96, Financial Data Statistical Report, Florida Department of Education, August 1997. Profiles of Florida School Districts 1996-97, Student and Staff Data, Florida Department of Education, February 1998. Fairfax County Public Schools 1998 Approved Budget.
Customer Survey Results
One of the objectives of KPMG’s performance review was to determine how well existing central office services meet the needs of customers. To accomplish this objective, KPMG issued customer satisfaction surveys to customers that closely dealt with Facilities Services and Exceptional Education.
Survey Survey Recipients Number of Responses Response
Rate
Exceptional Education
150 to Exceptional Education teachers, 50 principals, and 50 regular education
teachers *
111 44%
Facilities Services All principals and assistant principals (340) 186 55%
* Recipients for the Exceptional Education survey were randomly selected.
These departments were selected by KPMG as areas in which the best opportunities to improve were suspected or in which a significant gap in expectations was apparent following our introductory interviews and document reviews.
The survey format was multiple choice, with one open-ended question that requested suggestions for improvement. Questions addressed core performance characteristics and asked respondents to define the service levels they expect
versus the levels they actually receive.
Employee Input
Job Activity Questionnaires (JAQs)
KPMG distributed approximately 650 JAQs to Orange County Public Schools central office staff to gather input from Orange County personnel. KPMG received back 392 responses, or 60 percent of distributed JAQs. Several common themes emerged from the JAQs:
Strengths:
• Highly qualified and motivated staff.
• Teamwork and collaborative atmosphere.
• Goals, objectives, policies, and procedures are clearly defined.
• Many staff indicated they are clear on their individual responsibilities and roles.
• Many staff indicated little duplication of functions and activities.
• Training and staff development.
Opportunities for improvement:
• Lack of strategic plans and planning process.
• Policies constantly change, making it difficult to stay abreast of current procedures.
• Organization structure:
− Confusion as to other units responsibilities and scope of services
− Workload exceeds organizational capacity
• Lack of communication.
