Education Theses, Dissertations and Projects School of Education
2015
A Program Evaluation on Implementing
Investigations in Number, Data, and Space® in
Three Elementary Schools
Alexandra Leigh Smith Gardner-Webb University
Follow this and additional works at:http://digitalcommons.gardner-webb.edu/education_etd
Part of theElementary Education and Teaching Commons
This Dissertation is brought to you for free and open access by the School of Education at Digital Commons @ Gardner-Webb University. It has been accepted for inclusion in Education Theses, Dissertations and Projects by an authorized administrator of Digital Commons @ Gardner-Webb University. For more information, please [email protected].
Recommended Citation
Smith, Alexandra Leigh, "A Program Evaluation on Implementing Investigations in Number, Data, and Space® in Three Elementary Schools" (2015). Education Theses, Dissertations and Projects. Paper 126.
By Leigh Smith
An Applied Dissertation Submitted to the Gardner-Webb University School of Education
in Partial Fulfillment of the Requirements for the Degree of Doctor of Education
Gardner Webb University 2015
All rights reserved
INFORMATION TO ALL USERS
The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion.
All rights reserved.
This work is protected against unauthorized copying under Title 17, United States Code Microform Edition © ProQuest LLC.
ProQuest LLC.
789 East Eisenhower Parkway P.O. Box 1346
Ann Arbor, MI 48106 - 1346 ProQuest 10023693
ii
listed below. It was submitted to the Gardner-Webb School of Education and approved in partial fulfillment of the requirements for the degree of Doctor of Education at Gardner-Webb University.
__________________________________ ________________________ E. Ray Dockery, Ed.D. Date
Committee Chair
_________________________________ ________________________
Steven Bingham, Ed.D. Date
Committee Member
_________________________________ ________________________
Dixie Abernathy, Ed.D. Date
Committee Member
_________________________________ ________________________
Jeffrey Rogers, Ph.D. Date
Dean of the Gayle Bolt Price School of Graduate Studies
iii Abstract
A Program Evaluation on Implementing Investigations in Number, Data, and Space® in Three Title I Elementary Schools. Smith, Leigh, 2015: Applied Dissertation, Gardner-Webb University, Investigations/Elementary School/Title I/Mathematics Programs This applied dissertation was designed to provide perceptual teacher data as well as summative testing data to educational leaders concerning the effects of implementing Investigations in Number, Data, and Space® (Investigations) in three Title I elementary school settings, two Title I schools, and one non-Title I school. Data collected during this dissertation will be of use to educational stakeholders in selecting mathematics programs for elementary age students.
The purpose of this applied dissertation was to assess the effects of the Investigations program utilizing Stufflebeam’s CIPP program evaluation model. End-of-grade math test data for third, fourth, and fifth grade from the 2010-2011 to 2013-2014 school years in a southeastern school district were analyzed along with teacher perceptual data.
Teacher perceptions of the effectiveness of Investigations were measured by a survey developed by the researcher. Specific process and product research questions asked, “What are the teacher perceptions about the impact of Investigations on student achievement,” “What were the unanticipated effects of the Investigations program on student academic development,” and “What are the teacher perceptions about any unanticipated effects of Investigations on student academic development?”
The survey data indicate that more than half of the teachers in the researched school district believed their opinions were not used in the selection of materials to implement Balanced Active Math strategies and the trainings offered did not adequately prepare them to deliver the Investigations program. All three schools dropped in proficiency following Investigations implementation in the 2012-2013 school year and increases in proficiency rates in the second year of implementation.
iv
Table of Contents
Page
Chapter 1: Introduction ...1
Statement of the Problem ...1
The Topic ...6
The Research Problem ...6
Background and Justification ...7
Theoretical Framework ...8
Deficiencies in the Evidence ...9
Audience ...9
Definitions of Terms ...9
Purpose of the Study ...10
Research Questions ...10
Process Evaluation Questions ...11
Product Evaluation Questions ...11
Chapter 2: Literature Review ...12
Importance of Periodic Assessment and Evaluation ...12
Program Evaluation ...13
History of Mathematics Reform ...16
Cooperative and Constructivist Instructional Design ...19
Proponents of DI ...25
Saxon Math ...29
Reform-Based Mathematical Programs ...33
Math Anxiety ...36
Chapter 3: Methodology ...41
Participants ...41
Instruments ...42
Procedures ...43
Process Evaluation Questions ...44
Product Evaluation Questions ...44
Limitations ...46
Delimitations ...47
Chapter 4: Results ...48
Introduction ...48
Teacher Perception of Investigations Math Program ...49
Process Evaluation Survey Statement Findings ...49
Math Proficiency Rate Differences from State for 2010-2011 through 2013-2014 ...63
Chapter 5: Discussion ...65
Introduction ...65
Process Evaluation Questions ...65
Product Evaluation Questions ...67
Conclusions ...69
Process Evaluation Questions ...69
Product Evaluation Questions ...70
Implications...71
Recommendations ...71
v
References ...74
Appendix Investigations Implementation Survey ...81
Tables 1 Math Proficiency—Overall ...3
2 Math Proficiency—Subgroups...4
3 Math Proficiency—Grade Level ...5
4 Percent Proficient on ABC EOG Tests—Mathematics 2011-2012 ...7
5 Percent Proficient on ABC EOG Tests—Subgroups—Mathematics 2011-2012 ...8
6 Lower and Upper Mean Limits for the Interpretation of Degree of Agreement with the 11 Statements ...49
7 Means and Interpretation for Each of the Items on the Survey for Entire Sample...51
8 Teacher Responses to Statement, “The training I received prior to implementing Investigations program was sufficient for effectively using the program.” ...52
9 Teacher Responses to Statement, “The materials I received to teach the Investigations program were appropriate and adequate.” ...53
10 Teacher Responses to Statement, “Classroom teachers were given an opportunity to view components of the Investigations program prior to implementation.” ...55
11 Teacher Responses to Statement, “Classroom teachers were given an opportunity to express their preference for the math program to use for implementation of Balanced Active Math strategies.” ...55
12 Teacher Responses to Statement, “Opportunity was given to ask questions and express concerns before, during, and after implementation.” ...57
13 Teacher Responses to Statement, “Opportunity was given to express concerns, make suggestions, or ask questions during the implementation process.” ...57
14 Teacher Responses to Statement, “Materials and activities used in implementation of Investigations are appropriate for the age of students I teach.” ...58
15 Teacher Responses to Statement, “Adjustments to the delivery of Investigations were made in my classroom after district-wide implementation.” ...60
16 Teacher Responses to Statement, “Implementation of Investigations has made a positive impact on student achievement in my classroom.”...60
17 Teacher Responses to Statement, “Implementation of Investigations has made a negative impact on student achievement in my classroom.” ...62
18 Teacher Responses to Statement, “Students are comfortable using strategies taught in Investigations in my classroom.” ...62
19 School Differences from State in Math Proficiency Rates from 2010-2011 through 2013-2014 ...64
vi Figure
School Differences from State in Math Proficiency Rates from 2010-2011
Chapter 1: Introduction Statement of the Problem
A Nation at Risk was first published in 1983 by the United States Department of
Education (National Commission on Educational Excellence, 1983). This report called American educators to awaken from a slumber of mediocrity concerning the quality of education provided to students from kindergarten to twelfth grade. Educators were not preparing students who successfully graduated high school, were literate, and able to perform adequately in the American workforce. Math data collected in the report showed that in 1983, only four of every 20 students were proficient in math. More recently, a 25-year review of the progress of American education was published in 2008. In A Nation
Accountable: Twenty-five Years after a Nation at Risk, eight of 20 students were
considered proficient in math (United States Department of Education, 1997). According to data analyzed by the Trends in International Mathematics and Science Study (TIMSS) collected by the National Center for Education Statistics (2011), student achievement data on standardized tests in fourth and eighth grade had increased from previous data collection years of 1995 and 2007. Even with an increase from previous years in proficiency, the United States Department of Education Secretary Arne Duncan stated that “While student achievement is up since 2009 in mathematics it’s clear that achievement is not accelerating fast enough for our nation’s children to compete in the knowledge economy of the 21st Century” (National Center for Education Statistics, 2011, p. 42).
National Assessment of Educational Progress (NAEP) data are collected periodically as an assessment tool to measure American student progress in various academic areas such as reading, mathematics, writing, science, and the arts. After
significant NAEP gains in the 1990s, particularly in mathematics, the 2011 results
continue a pattern of modest progress (NBC News Services, 2011). In examining the test scores assessed by NAEP, only 40% of fourth graders and 35% of eighth graders reached proficiency according to NAEP standards.
The landmark publication, A Nation at Risk, first collected education data in 1983 (National Commission on Educational Excellence, 1983). Sponsored by the U.S.
Department of Education, the report found disturbing data concerning the performance of American schools. Reassessing school performance in 1997, the commission gathered and examined data in the following areas: curriculum content, standards and
expectations, time, teacher quality, and leadership and financial support (United States Department of Education, 1997). Results in student proficiencies are reported to have only made slight gains in proficiencies since 1983. Of children born in 1983, only 20% would be proficient in reading and 4% would be proficient in math. Compared to students born in 2007, 7% of students were proficient readers and 8% of students were considered proficient in mathematics.
Comparison of American student proficiencies to their international counterparts was also discussed at length in this report. According to the study, international students are outperforming American students in both reading and mathematics. When data were collected and reexamined in 1997, the United States was found to have slipped to tenth place in the number of high school graduates it produces each year (United States Department of Education, 1997).
One important point to be made is that the United States Department of Education, publisher of A Nation at Risk, included the total student population that is testing in American schools. Students in American schools are tested regardless of
poverty level, special needs status, or limited English proficient. At best, it is unclear whether international schools that are compared to American schools include such student groups in their testing. In one southeastern state, the number of students who participated in end-of-grade (EOG) or end-of-course (EOC) tests who had a disability totaled over 14% of students tested for the 2009-2010 school year (North Carolina Department of Public Instruction, Accountability Services, 2011).
On the local level, the researched school district’s mathematics data reported through the EOG tests reflect the following student proficiencies from the 2010-2011 school year.
Table 1
Math Proficiency—Overall
Researched School District Grade 3 Grade 4 Grade 5
Math proficiency 80.0% 81.8% 80.1%
Student performance broken down by gender and ethnicity who passed math and standardized tests reveal that student performance in minority subgroups is lower than that of White students. Students with Disabilities group has the lowest proficiency scores of any group listed.
Table 2
Math Proficiency—Subgroups
Researched School District
Male Female White Black Hispanic Economically
Disadvantaged Students with Disabilities
Passed Math 62.3% 66.3% 71.3% 44.7% 55.2% 53.1% 22.8% The overall purpose of the research school district is to provide rigor, relevance, and relationships leading to a student who is globally competitive. The mission of the district is to provide quality educational opportunities to ensure student success and a lifetime of learning. The researched school district put forth strategic goals for the years 2011-2014. First on the list of goals was to have students demonstrate competency in reading, math, and science. In 2012, district-wide data indicated that 80% of students were proficient on EOG testing in mathematics for Grades 3-8. The goal by 2014 was 86%. Proficiency goals are of special interest to the researcher as they provide a context of school system priorities and implementation of math initiatives such as Investigations in Number, Data, and Space® (Investigations). While the combined mathematics proficiency stated above may not raise red flags to those outside of education, it is important to note that individual schools have varying proficiency scores. This is
especially true of schools that receive federal funding through the Title I program. In the district, schools that have an overall average of 65% of its students who receive free or reduced lunch qualify for additional funding. In the researcher’s school, 2011-2012 EOG data report great disparity from the overall district proficiency. Research school
Table 3
Math Proficiency—Grade Level
Grade 3 Grade 4 Grade 5 Overall Proficiency
62.6% 70.3% 64.4% 65.6%
Note. Information obtained from 2011-2012 North Carolina School Report Card.
Student demographics are also of interest to the researcher. The district is among the top 10 school districts in the state according to student population. Of these students, more than 3,990 students receive special education services and 1,235 receive second language programs.
The researched school district employs over 3,500 people to educate and serve more than 30,000 students in Grades Prekindergarten through 12. In total, 55 schools serve the needs of students; 30 are elementary and primary, two are intermediate, 11 are middle, 10 are high, one is a special needs school, and one is an alternative behavior school. Elementary schools serve Grades Kindergarten through 5. Primary schools serve K-2 students, while intermediate schools serve Grades 3-5. Middle schools serve
students in Grades 6-8, while high school finishes the spectrum from Grades 9-12. Student ethnicity is varied in this school district with 64.7% of students being Caucasian, 20.3% being African-American, and 9.5% being Hispanic. The researched district also provides free or reduced meal benefits to 59% of its students as well as transports 7,360 elementary students via school buses.
Students had an attendance rate of 95.2% during the 2011-2012 school year as well as a graduation rate of 78.8% in 2012. Special education services are provided to 9.1% of the student population. Students identified as Academically Gifted and Talented
make up 8.2% of the population. Additionally, 27 Advanced Placement courses are offered in the researched school district with a total Advanced Placement enrollment of more than 3,200 students.
Increases and decreases in math achievement scores can be related to a myriad of possible causes. One such reason could be the inclusion of more students with
disabilities in the testing program. From the testing year 2010-2011, in North Carolina alone, more than 33,000 students with disabilities participated in the testing program in Grades 3-10 (North Carolina Department of Public Instruction, Accountability Services 2011). Students who have diagnosed differences in learning are held to the same
standard as their nondisabled peers. Student test scores may not be as high as their peers therefore reflecting poorer performance when looking at the total number of students who are considered proficient.
The Topic
The topic of this dissertation was a program evaluation that examined aspects of the implementation of Investigations into the researched school district as the stand-alone math instructional tool. The effects of this implementation on EOG or EOG scores given in May 2014 were examined. Staff of three elementary schools in the researched school district were given a survey to ascertain their perceptions of the implementation of Investigations. Two of the research schools are considered Title I schools, and one is considered a non-Title I school.
The Research Problem
Math achievement scores have been decreasing in the researched school district for several years. This trend mirrors slow increases in student proficiency as well as state and national decline in overall math scores.
To remedy this effect, instructional specialists in the researched school district have mandated implementation of Investigations in all of its elementary schools. This study conducted a program evaluation to determine the impact of the implementation of Investigations on EOG test scores as well as teacher perceptual data of the effects of implementation.
Background and Justification
Information gathered from the 2011-2012 North Carolina Report Card indicated students in the researched school district are being outperformed by their peers across the state of North Carolina. The tables below indicate that in Grades 3-8, the researched district is below the state average in math performance on EOG tests as well as in reading. When examining data based on subgroups of students at the district level, the research district underperformed in the following subgroups: All, Male, Female, White, Black, Asian, Economically Disadvantaged, Not Economically Disadvantaged, and Students with Disabilities.
Table 4
Percent Proficient on ABC EOG Tests—Mathematics2011-2012
Grade District State
Third 80.0% 82.8%
Fourth 81.8% 85.1%
Table 5
Percent Proficient on ABC EOG Tests—Subgroups—Mathematics 2011-2012
All Male Female White Black Hispanic
District 64.3% 62.3% 66.3% 71.3% 44.7% 55.2%
State 67.5% 65.0% 70.1% 79.3% 49.4% 55.1%
Theoretical Framework
“Evaluation is a very young discipline—although a very old practice,” said Scriven (1996) in describing the history of program evaluation (p. 393). Organizational stakeholders and decision makers want and need to verify that programs are
accomplishing their stated purposes. To that end, questions must be asked from various contexts of the implementation process to evaluate effectiveness. Processes, procedures, and outcomes must all be inspected by examiners. Program evaluation can also judge the merit or worth of something (Scriven, 1991).
Program evaluations can involve ongoing monitoring of programs or one-time evaluations of students of processes, outcomes, and program impact (Stufflebeam & Shinkfield, 2007). Formative evaluations can help to strengthen or improve a program and help to determine what works best in an organization. Additionally, formative evaluations can help provide feedback for improvement while shedding light on any negative results of an implemented program.
On the other hand, summative evaluation examines the overall quality and outcome of a program. Summative evaluations are designed for decision-making purposes to ascertain if a program has met its planned outcomes. Both formative and summative evaluations are needed in the development of a product or service
(Stufflebeam & Shinkfield, 2007).
Once program evaluations have been conducted, assessors can also use results to plan effective staff development for areas of programming that need to be strengthened (Centers for Disease Control, 2010). Furthermore, results from program evaluations can also help to celebrate successes within the program itself as well as strengthen the program design through rigorous examination (Alleghany Evaluation Specialists, 2014). Deficiencies in the Evidence
To date, no assessment of the Investigations mathematics program has been conducted as it impacts the North Carolina EOG test scores. This study examined the effect as well as assessed teacher perceptions of the planning and implementation process of Investigations into teacher classrooms.
Audience
Practitioners in education in the elementary school setting as well as curriculum leaders in the district will benefit from the results of the study. The results of this study could be helpful to schools and districts in implementing future mathematics programs. It is also important to consider teacher viewpoints of the effectiveness of mandated educational programs such as Investigations before, during, and after implementation. Definition of Terms
Investigations. The instructional program used by the researched school district to instruct students in Grades K-5 in mathematics instruction.
EOG tests. The assessments students in North Carolina take in May of every academic year in order to assess their proficiency in reading and mathematics in Grades 3-8.
follows a constructivist approach whereby students build upon their own level of understanding with mathematical foundations often in cooperative groups where more than one method can be used to derive answers. Students often work in small cooperative groups during this time.
Title I schools. Schools that receive additional federal money through the Department of Education based on the number of students who qualify in each school district for free and reduced lunch. Each school district can set its own threshold for schools qualifying for these funds.
Context, Input, Process, and Product (CIPP). The program evaluation model developed by Stufflebeam (2011) to help the public and private sector evaluate the effectiveness of programs based on four quadrants.
Purpose of the Study
The purpose of this study was to determine the impact of Investigations on academic achievement, teacher perceptions of the implementation, and effectiveness upon EOG tests scores at the researched site. The implementation of the Investigations program has been mandated as the stand-alone math strategy and resource to be used by elementary school teachers within the school district where the study was located. This study also examined teacher perceptions of the implementation of Investigations using the CIPP model of program evaluation.
Research Questions
This program evaluation utilized Stufflebeam’s CIPP model to determine the impact of implementation of Investigations on student achievement at three elementary schools in the southeastern United States as measured by achievement data in the third grade. Teacher perceptions were measured by a survey developed by the researcher.
Research questions addressed in this program evaluation center on the Process and Product components of the CIPP program evaluation model.
Process Evaluation Questions
Were the various components of Investigations implemented as they were originally intended?
a. What are the teachers’ perceptions about the implementation of strategies and activities within Investigations?
b. How did teachers have an opportunity to ask questions and voice concerns during the implementation stage?
c. How were any program adjustments made by teachers during implementation?
Product Evaluation Questions
1. Based on EOG data from 2010-2011 through 2013-2014, what impact did Investigations have on student achievement through proficiency scores in Grades 3-5?
a. What are the teacher perceptions about the impact of Investigations on student achievement?
2. What were any unanticipated effects of the Investigations program on student academic development?
3. What are teacher perceptions about any unanticipated effects of Investigations on student academic development?
To answer these questions, the researcher conducted a quantitative study using EOG assessment scores from the third grade from the 2012-2013 school year. The usage of Investigations in teacher classrooms is mandated for all elementary teachers who teach
Grades K-5.
Chapter 2: Literature Review Importance of Periodic Assessment and Evaluation
In the world of education, periodic assessments can often look like teachers giving students common assessments which cover curriculum that has been previously taught in the classroom. Periodic assessments are important to teachers and managers as they can provide regularly scheduled feedback to improve performance (Thompson, 2011). In a similar manner, the evaluation of programs can also provide crucial
feedback to program managers. In both the academic and nonacademic worlds, program evaluation has proven to be a valuable tool in strengthening the quality of existing programs (Behavioral and Social Science Volunteer Program, 2012). Benefits to conducting periodic assessments and evaluations include evidence of effectiveness and justify the need for more support of the program, increasing the program’s ability to contribute to the knowledge of the field, improving upon skills and quality of the program, streamlining services, and promoting the effectiveness of the program (Behavioral and Social Science Volunteer Program, 2012).
Effective program evaluation also collects, provides, and analyzes data that can be used to learn about the program itself and any strengths and limitations (Centers for Disease Control, 2010). Routine program evaluation can offer improved documentation, learning opportunities, and common understanding about functions of a program that are successful and those that are not. Evaluations can also promote accountability within institutions, solidify requests for increased funding, identify ineffective practices, and produce credibility to outside agencies (Pell Institute, 2014).
Within the educational setting, conducting frequent assessments and evaluations is one of the methods in which educators can keep up with the ever-changing and multifaceted endeavor of teaching. Leaders in the field must be able to confront and challenge current methods and be prepared to change various parts of a program to develop more effective implementation (Ross, 2010). Leaders must also be versed in multiple approaches to evaluation in order to accommodate varied sources of data and purposes of evaluation.
Effective assessments and evaluations can be designed as formative or summative. Formative evaluation provides information that forms and refines the program. A review of practices and their interpretation is one crucial step in formative evaluation (CDC, 2010). Data collection and research on staffing, training, materials, and implementation processes can provide managers and educators with information to improve upon the program. If provided in a timely manner, the data can facilitate making corrections and adjustments that can refocus a program.
Summative evaluation can be most helpful in clearly defining the benefits created by a program as well as the costs and conditions necessary for maximum effectiveness. When using summative evaluation, educators and managers must look at their
measurement and data collection tools to ensure the correct tools are used to report the most effective data.
Program Evaluation
The field of program evaluation began in the mid-1900s as a way to “judge the worth or merit of something or the product of the process” (Scriven, 1991, p. 139). Defined by seven time periods of evolution, program evaluation has been used by
In determining effectiveness of new ideas in the workplace, program evaluations have been used to obtain reliable, valid, and credible data (Scriven, 1991) in order to judge the performance of programs. Hogan (2007) described five evaluation approaches to program evaluation that are in current use by practitioners. Objectives-orientated approaches focus on setting clear goals and objectives of the given program and describe the degree to which the goals have been attained. The recognized pioneer in this
approach is Ralph Tyler. Tyler (1949) stated that the goals and objectives of any program must be defined in order for evaluation to take place. While considered to be the pioneer, Tyler was not without his critics. Critics claimed that the selection of objectives for evaluation was faulty as not all objectives could be evaluated. In addition, selecting objectives to be evaluated was also open to bias (Stufflebeam & Shinkfield, 1985).
Expertise-oriented program evaluation is the most widely used and oldest method to judge an institution, activity, or program. A panel of judges or experts evaluates a program and makes recommendations based on their perceptions and opinions. A formal and informal review of internal systems can be used in this approach. Critics of this approach claim that judgments made by experts are biased and not based on program objectives (Worthen, Sanders, & Fitzpatrick, 2004).
Judicial processes are used when utilizing the adversary-oriented approach. Pros and cons of any issue are debated by two teams who then defend their positions in a public debate until an agreement can be made on a common position. Within this evaluation system, hearings, prosecutions, juries, charges, and rebuttals are integral components. By using this approach, positive and negative viewpoints are brought into the open; however, the truth is believed to emerge from a hard but fair fight (Worthen et
al., 2004).
Experience plays a huge part in the participant-oriented approach which values firsthand experiences with activities as well as the importance of the participant in the process. The least powerful stakeholders are used in this process from start to finish wherein the evaluator and stakeholder work alongside each other as partners to solve problems.
Empowerment evaluation is considered a subclassification within participative-oriented evaluation. Using the empowerment approach, participants develop a clear purpose, identify program strengths and weaknesses to assess where the program currently stands, and plan for the future by establishing goals.
Based on the four aspects of training—context, input, reaction, and outcome—the CIRO model was proposed in 1970. Context evaluation identifies an organization’s training needs and the setting of goals and objectives. Input evaluation is focused on the design and delivery of the training activity. The CIRO model also takes into account the objectives and training materials that are to be used in the evaluation. Reaction
evaluation examines the quality of the trainees’ experiences, while outcome evaluation highlights achievements gained from the activity that is assessed at three levels:
immediate, intermediate, and ultimate evaluation.
The management-orientated approach was intended to be utilized by
organizational leaders in providing information to decision makers on the managerial level. Stufflebeam (2011) created the widely used management-orientated tool, CIPP model, to evaluate programs. This model is intended for service providers ranging from university administrators, physicians, and military leaders when conducting internal evaluations for examination of the social acceptability, cultural relativity, and technical
adequacy of programs (Stufflebeam & Shinkfield, 2007). Context evaluation investigates the program objectives to determine if they are acceptable given social, cultural, and technical characteristics. Input evaluation examines the intended content of the program. Process evaluation relates to what degree the program was delivered as it was planned. Finally, product evaluation assesses program outcomes.
CIPP evaluations are conducted to complement rather than supplant other reviews of existing programs (Stufflebeam, 2011). Throughout the evaluation process a meta-evaluation, or evaluation of an meta-evaluation, is completed. The model’s main theme is that the evaluation’s most important purpose is not to prove but to improve.
History of Mathematics Reform
As the 20th century dawned, American culture saw a change in its character. It was during this time that the works of Thorndike (1923) called upon school psychologists to make schools more efficient and effective in educating large populations of children (Ellis & Berry, 2005). Through his research, Thorndike believed that drill and repeated practice were the best mathematical practices to instruct children in mathematics.
Furthermore, Thorndike called mathematics a “hierarchy of mental habits or connection” (p. 52). As such, mathematics should be clearly taught through carefully planned
sequences with much repetition so as to enable learning.
The Progressive Education Association or (PEA) entered the American
educational forum in the early 1920s as a counter measure to Thorndike’s (1923) call for rote memorization of skills and the use of repetition. Influenced by the works of Dewey (1961), PEA favored providing direction to children learning activities. Learning then, according to the progressives, occurs when it is connected to student experiences and interests (Przychodzin, Marchand, Martella, & Azim, 2004). Student interests should
also be a factor in developing instructional practices in the classroom. Furthermore, progressives also called for the role of the teacher to change from a taskmaster to one of facilitator or guide.
Fast forward to the mid-20th century where the New Math phenomenon was developed out of concerns of the Russians launching the satellite, Sputnik, into space prior to the Americans. In response, Congress created the National Science Foundation (NSF) in 1950 to help promote science education in the United States. From the funding provided through the NSF, numerous projects set their sights on overhauling mathematics education. Developed programs contained strategies such as usage of student
manipulatives, intensive in-service workshops for teachers, the development of textbooks heavily influenced by early constructivist thoughts, and the creation of Advanced
Placement testing by the College Entrance Examination Board for advanced students with mathematical aptitudes. These new approaches failed to gain pervasive success in American classrooms; however, they were beneficial to the next generation of educators as they laid the groundwork for future reform (Klein, 2003).
As a pushback against the New Math movement, Back-to-Basics was launched in the early 1970s. Proponents called for the simplification and orderly development of mathematical skills. This movement was connected closely with the competency test movement in American education in the 1970s and 1980s. Modest improvements were seen in test scores; however, critics espoused that the Thorndike-like math textbooks did little to prepare students for higher levels of cognition and understanding (Wilson, 2003).
In review, many of the revisions of mathematics education formulated over the past century have been created within the procedural-formalist paradigm (Ellis & Berry, 2005). The procedural-formalist paradigm asserts that mathematics is a set of organized
facts, skills, and procedures that exist apart from human experience which in turn make it difficult to learn. In stark contrast, the cognitive-cultural paradigm believes that all students can learn interconnected concepts that come from human experience as long as they are presented in a culturally relevant way.
Sensing a need to influence change in American mathematics, the National Council of Teachers of Mathematics (NCTM, 1989) published updated standards. With the implementation of these new goals, students would be able to apply knowledge to new situations, explain mathematical arguments, and make sense of conceptual connections.
Burrill (1998) explored implications of the NCTM standards on mathematics curriculum reform by reviewing the changes that have occurred in mathematics
education, myths about mathematics, and the mathematics that children like to do, and where we are headed given the tremendous changes in technological advances of the 20th century (Jackson, 1998). Burrill saw the need to create a curriculum that flows from various grade levels into one coherent whole in which students are expected to have a shared common knowledge base by a given grade level and in which teachers act on this expectation. A curriculum designed in this vein, Burrill argued, will reduce the emphasis on the repeat and remediation cycle in which today’s mathematics education is often embedded, allow for a broader and more useful base of mathematics to be explored in the classroom, and make mathematics consistent across grade levels nationally.
With the increase in usage of technology such as virtual classrooms and online tutoring, the questions of which of the programs will best serve students becomes more difficult. The technology of today provides numerous avenues for differentiating instruction that can both teach the logically sequenced set of mathematical skills as well
as provide for engaging group discussions and project-based learning opportunities for students (Kuhn & Dempsey, 2011). The push for an increase in technology integration will no doubt influence future reforms in mathematics.
Cooperative and Constructivist Instructional Design
People construct their own reality based on their experiences and knowledge. Constructivism provides an understanding of learning where individuals create new understandings or knowledge sets based on interaction with ideas, activities, and events in their daily lives. Teachers then serve as guides, co-explorers, and facilitators who encourage students to question, challenge, and formulate their own ideas, opinions, and conclusions (Cannela & Reif, 1994). It is important to note however that
“Constructivism is not a theory about teaching . . . it is a theory about knowledge and learning . . . the theory defines knowledge as temporary, developmental, socially and culturally mediated, and thus non-objective” (Brooks & Brooks, 1993, p. 8).
Dewey (1961) was a major force in progressive education in the United States in the early to mid-20th century. Dewey’s work led the way for other researchers such as Abraham Maslow, Carl Rogers, Lev Vigotsky, and Jean Piaget. All of these thinkers had their unique perspective of human development; however, they all shared Dewey’s belief that education naturally facilitates the developing tendencies and potential of each child (Matthews, 2003). From Dewey’s perspective, knowledge is not a representation of reality. The relationship between knowledge and reality is the result of individual and social experiences. Enriched experiences change people’s perception of right.
Classroom teachers understand this theory well as plan field trips for students to zoos, courthouses, capitals of states, and industries so they can experience and internalize life situations that may not be feasible under ordinary circumstances.
Self-direction in both adult and child learning was also of importance to Dewey (1961). He believed that active participation and self-direction were crucial to student success. Dewey believed the “contents of the child’s experience” are more important than the “subject-matter of the curriculum” (p. 342).
Piaget (1953), another pioneer of constructivist theory, centered his main focus on constructivism around how the individual builds knowledge. According to Piaget, the nature of knowledge should be studied empirically through experimentation of learners in their natural environments such as schools and homes. Humans cannot be given
information they immediately understand and use; instead humans must construct their own knowledge (Piaget, 1953). According to Piaget, three kinds of knowledge exist that are used to structure and build knowledge: physical, social, and logico-mathematical. Physical knowledge is knowledge of objects in external reality. Social knowledge includes knowledge of certain social norms such as Father’s Day or saying “good morning” under specific circumstances. Logico-mathematical knowledge contains relationships formed by each person.
In terms of mathematical theory and instruction, Kamii (1996) believed the traditional goal of memorizing facts in order to internalize sums is incorrect. Kamii asserted that mathematical sums must be internalized by each child on the inside. Methods of classroom practices should not then include superficial mastery of concepts through repetition and reinforcement from external sources. Instead, students should be exposed to numerical reasoning through daily life experiences, group games, and problem-solving discussions. Repetition is important; however, it should be
accomplished through games where students are motivated to learn arithmetic. Group work is also a foundational cornerstone of effective mathematics instruction as Piaget
(1971) pointed out that exchange of ideas and points of view are essential for intellectual and socio-moral development.
Leading Piaget student, Kamii (1994) believed so strongly in cooperative
instruction that she called for the end of teaching carrying and borrowing in first- through fourth-grade classrooms. In examination of schools through Hoover, Alabama, City Schools, Kamii (1996) compared the processes in which students answered algorithms in classrooms that did and did not teach the direct instruction (DI) approach of carrying and borrowing to procure sums. Students in each classroom were heterogeneously placed for ability. Two hundred and twenty students and their algorithms were examined during the study.
In summation of her research, Kamii (1994) asserted that students who used traditional algorithms to answer questions were more likely to answer the question incorrectly but could also not articulate how the numbers were related to each other and why they had to borrow and carry. Teaching algorithms is harmful, asserted Kamii (1996), because they do not allow for children to develop their own thinking. Algorithms remove the knowledge of place value children have already constructed, which in turn prohibits them from developing number sense (Kamii, 1996).
The concept of adaptation is also pivotal to the constructivist learning theory. Piaget (1971) believed that assimilation occurs when children bring new knowledge to their own schemas or experiences and accommodation occurs when children have to change their schema to accommodate the new knowledge or information. As the learner processes how to fit new information into existing memory files, this adjustment process occurs (Powell & Kalina, 2009).
approach to teaching and learning through student exploration of developing meaning of instructional materials. In her text, Duckworth asserted that students should be made to feel wonderful about their ideas and the process of developing them to their
understanding. Children should be exposed to a number of new ideas and theories to gain their attention whereby they can create their own understanding and meaning. A
predetermined pace of intellectual development is not found in children, declared
Duckworth, as students develop understanding based upon their experiences, actions, and connections.
Teachers, who are facilitators, present information to students; however, they do not assign meaning to the material. Students must be placed in situations where they develop their own understanding. As facilitators, teachers must present broad ideas to students, not just narrow goals and objectives (Meek, 1991). Teachers should also occasionally disagree with student viewpoints in order for a deeper level of thinking to occur by the student. These disagreements must be made respectfully; however, they can lead a student to accept another’s viewpoint as interesting and thereby increase
understanding of a topic.
To delve deeper, teachers must also refrain from providing hidden meanings or signals that could interfere with a student’s development of meaning and context of curriculum (Fusaro, 2014). As a result, teacher knowledge of the subject matter is increased as they try to take student thoughts and deepen them based on their own understanding.
Montessori schools bear the name of the first woman admitted to practice medicine in Italy, Maria Montessori. Trained as a physician, Montessori originally developed her program to assist children with various health disorders. These programs
challenged the traditional classroom model of students sitting at desks and memorizing facts. In Montessori models, students were given the opportunity for movement and interaction in a structured manner that supported students’ natural curiosity. Social skills, academic lessons, exercises in daily living, and concern for health, hygiene, and self-discipline were all fundamental components of the Montessori experience (Hedeen, 2005).
The cooperative aspect of the constructivist classroom is essential. Duff (2012) found that students prefer cooperative and constructivist components of instruction. Such components can include working in teams on an assignment or project and making team members accountable for the content and degree to which the project is completed. Duff conducted a survey of middle school students in which she asked the students if they felt their achievement scores would improve with more or less group work time. A total number of 15 students were in the sixth-grade classroom. Seven boys and eight girls ages 11-12 made up the study. According to questionnaire results, students believed their achievement would increase after group learning techniques in part due to their belief that the approaches used allowed them to understand the presented material better. Duff extrapolated that students benefited from cooperative learning strategies because they allowed students to make real world connections using learning styles and group work.
Student perception of effective math instruction techniques can also play a critical role in motivating students to participate fully in mathematics applications. The National Research Council—Mathematics Learning Study Committee released a report in 2001 that recommended teachers of mathematics use a mixed-methods approach to engage students in the five integrated competencies of conceptual understanding, strategic competency, adaptive reasoning, productive dispositions, and procedural fluency. By
considering learning styles, work completion rates, modes of expression, and student-centered techniques, students are predicted to be able to attend to tasks at a higher rate as well as process information to a higher extent. The council further recommended that a mixture of both DI and cooperative approaches be used to instruct students. Strategies such as collaborative group work, open-ended tasks, games, and student presentations are also predicted by the council to be effective for learners.
Researchers in parts of lower socioeconomic levels of Melbourne, Australia, have utilized student input to ferret out their preference of instructional activities in
mathematics classrooms. As part of the Task Types in Mathematics Learning (TTML) research project, researchers used student responses to help refine research questions to student surveys. Of the 12 students surveyed in this lower socioeconomic part of Melbourne, students created math stories in which their ideal classrooms were designed so that children would work together, play games, and move around. Of particular importance to students in the primary classrooms was the need to be outside and move while they were learning. Cooperative structures were also clearly preferred by students during the study. Seven students preferred working in groups or pairs, five students wanted to share their work with the rest of the class, and one student wanted to help younger students in their acquisition of mathematical concepts as well as to be able to sit and talk during math class (O’Shea, 2009). In creating their own learning experiences, students embarked on shared and cooperative strategies that as a whole group combine to form a “community of practice” (Lave & Wenger, 1991; Rogoff, Turkanis, & Bartlett, 2001).
Furthermore, students in this study stated they disliked conventional types of lessons that included sample problems posted on the board with instructions for students
to copy down in their notebooks, rote memorization of simple math facts, and an overabundance of worksheets.
Proponents of DI
Influenced by traditional models of mathematics education, the procedural- formalist paradigm asserts that mathematics is a group of logically organized skills, procedures, and facts that have been refined over centuries. Human experience does not factor into this mental model and therefore increases the difficulty of learning this material as students would not be influenced by life experiences. This viewpoint guided the works of back-to-basic advocates as well as those of psychologist Edward Thorndike in the early 20th century. Thorndike’s (1923) Stimulus-Response Bond theory had a penetrating influence on mathematics instruction. Thorndike and his believers felt that mathematics instruction was best internalized through drill and practice that is
deliberately sequenced and explicitly taught. Repetition and frequent practice are also hallmarks of the Stimulus-Response Bond Theory as is a student’s non-ability to reflect on mathematical concepts. Much of Thorndike’s data and philosophy shaped an entire generation of math students and teachers alike. Thorndike believed the learner is seemingly unable to formulate any thoughts of his or her own, let alone develop a new model of meaning.
To deliver this traditional view of delivering instruction, DI models have been developed and used in the classroom. DI models are highly segmented and sequenced and consist of design and effective presentation techniques (Stein, Silbert & Carnine, 1997). Four such DI programs—DISTAR Arithmetic I and II, Corrective Mathematics, and Connecting Math Concepts—were examined by researchers from Eastern
effectiveness of DI programs being used in United States schools from 1990 to 2004. All four programs examined were organized through the usage of formats, tasks, and tracks. As defined by Engelmann and Carnine (1975) identified programs consisted of major skills or strategies. The purpose of each track is to teach students to solve simple, written story problems independently. In all four programs, students must have prerequisite skills for success in completion of each track. Student success can be seen through formatted and successive lessons. Furthermore, tasks are created by inserting a new set of numbers into a pattern in which the wording is unchanged.
The DI approach, according to Przychodzin et al. (2004), aligns to the principles for improving math instruction as set forth by NCTM (1989). The first principle
validated by this meta-analysis is the Equity Principle that sets high expectations for all students through strong support systems. DI math programs utilize flexible skill groups which are based on current levels of performance using frequent progress monitoring. The Curriculum Principle asserts mathematical tasks must be a collection of activities that are coherent and articulated across grade levels. Advocates of DI advance this principle in designing strands of instruction that are organized around big ideas. Such math programs are also designed to guide students in the acquisition of basic
mathematical concepts and operations.
Third, NCTM’s (2006) Teaching Principle believes the needs of learners should be well understood by teachers as well as how to challenge and support these needs. Intensive preservice training is provided for teachers prior to the implementation of the four researched models. The Learning Principle claimed students must learn math skills with understanding, actively building new knowledge from prior knowledge and
day. Teachers using the DI approach are encouraged to spend 30-35 minutes on daily group instruction along with guided practice and modeling. Students then spend 20-30 minutes completing independent seatwork. DI math programs also offer teachers predesigned instructional formats to use in teacher preparation of lessons so as to minimize the amount of time the teacher needs to spend in lesson plan development. Fifth, NCTM’s Assessment Principle states that assessment should furnish useful information to both teachers and students. DI programs provide frequent in-program mastery tests to allow teachers to tailor instructional planning to student needs. Various forms of assessments are used in DI programs such as mastery tests, fact games, and take-home assignments.
The final principle from NCTM, Technology Principle, states that technology should influence and enhance skills that are taught to students. Technology, according to NCTM, should be embedded in the math program rather than being a supplemental element. Calculator usage is the predominant method of technology delivery in the four models of DI examined in the meta-analysis.
Managing DI initiatives can be time-consuming and radical in its change on mental models for teachers as well as administrators. Hill and MacMillan (2004)
professed that the implementation of DI models is essential to school success in the wake of federal and state mandates such as No Child Left Behind. In order to guide and successfully implement DI instruction, teachers and administrators must understand the essential components of the approach. Hill and Macmillan defined DI has having been based on the theory that instruction erases student misinterpretations and can improve and accelerate learning. Correct and immediate feedback is also a feature of this model where students are not allowed to learn concepts, facts, and skills in an incorrect manner.
Repeated presentation of tasks throughout and at the culmination of the lesson, called the firming cycle, is needed to ensure that students have a solid mastery of the taught skills (Kameenui & Simmons, 2008).
Within this approach, positive reinforcement and immediate feedback are considered to be hallmarks of the program and teacher toolkit. Feedback given to
students is essential for success and specifically targeted to the student and task. In short, students are not expected to guide their own learning and feedback. Instead, it is
presented in an explicit manner and directly reinforced. Subsequent activities such as concrete learning activities as well as shaping and scaffolding take the place of student-driven learning.
Because of its specific and sequential delivery of instruction education,
professionals who work with students with learning struggles and disabilities often tout the DI approach, however Hill and MacMillan (2004) sited that the DI approach can be used with diverse levels of student learning abilities such as those who speak English as a second language. Since its branding in the 1960s, compelling research supports that if students are taught basic skills in a clear and direct format that is strategic in design, they will learn to read and process mathematically (Martells, MacMillan, & Slocum, 2004). It is important to note that researchers point out that DI is one of the processes that promote academic achievement in students in reading and mathematics.
DI formats can be applied to any age student and in numerous instructional contexts. One such context is the application of DI to students with learning difficulties. Researchers at Al-Balqa Applied University in Jordan have examined the effect of DI on math achievement in fourth- and fifth -grade students with learning disabilities
Sixty students in fourth- and fifth-grade mathematics classes who attended special education classes in the resource setting were selected via a random sample through learning centers within the city of Amman, Jordan. Students were randomly assigned to experimental and control groups. Two tests were administered to students that measured student mathematical achievement, usage of mathematical skills in everyday life, and student attitudes towards mathematics.
Results indicated that a statistically significant difference existed among
achievement scores of the experimental and control subjects on mathematics posttests. Experimental students received training on basic math skills using DI strategy, whereas the control groups were taught using more cooperative measures of student grouping. Pretest and posttest mean scores of students in the experimental group were higher than those of the control group. The experimental group pretest scores were M=16.80, and posttest scores were M=40.73. Control group pretest scores were M=15.93, and posttest scores M=22.70.
Abdulhameed and Al-Makahleh (2011) pointed to the format of DI to its success across international borders and cognitive ability groups. Subsequently, the researchers claimed these steps benefited students in the experimental DI group: measuring student performance directly, accurate goals are set for students, tasks are arranged sequentially and systematically, adequate time is set for each task, feedback is provided for students, DI is provided to all students, student performance is displayed in suitable graphical forms, and appropriate problem-solving forms are matched to specific skills.
Saxon Math
Saxon Math is a DI mathematics program that has been developed for students in Grades K-12. Saxon Math programs are based on Gagne’s (1965) theory of cumulative
learning that stated that skills can be broken down into simple skills and then broken down further into even simpler tasks. Intellectual skills, research states, are organized into an arrangement that reveals prerequisite relationships among them (Gagne & Briggs, 1974). Anderson’s (2007) ACT theory explains development of experience in three stages: cognitive, associative, and autonomous. In the cognitive stage, learners memorize and rehearse facts related to a particular skill. During the associative stage, learners can detect errors and misconceptions through practice and feedback. Finally, during the autonomous stage, learners have practiced a skill so that it becomes routine therefore decreasing the amount of working memory needed to perform the skill.
Saxon publishers have created their math programs around incremental
instruction, continual practice, and cumulative assessment that are dispersed across the span of a school year. Based on the studies from Brophy and Everston (1976),
incremental steps must be taken in order to teach new information to students. Likewise, effective skill development requires additive skill distribution throughout the school year. Research studies praised by Saxon publishers have seen that students who are taught with a curriculum that uses consistent and continual math review have greater math
achievement than those taught with a mass approach (Good & Grouws, 1979).
Cumulative assessment is also a critical component of the Saxon Math program. According to Fuchs (1995), periodic assessments can enhance instruction by monitoring student learning, evaluating instructional programs, and revealing student remediation needs. Routine classroom assessments that are designed to be a part of the instructional program instead of an interruption are also recommended from NCTM (1989).
Research studies conducted by PRES Associates in South Carolina, California, Georgia, and Texas from 2005-2007, all examined longitudinal state achievement test
data to document the effectiveness of Saxon’s elementary and middle schools over time (Resendez & Azin, 2007). Performance was measured using student-level achievement data that compared users of Saxon Math to students who used other math curricula during the same years. In South Carolina, using the Palmetto Achievement Challenge Test, researchers found that students who used Saxon Math from third through fifth grade had an increase in scale scores from 306.8 to 502.9 by the end of the fifth-grade year. Scale scores of middle school students also increased from 590.8 at the end of the sixth-grade year to 782.0 at the end of the eighth-grade year.
Oklahoma City Public Schools Planning, Research, and Evaluation Department also conducted quasi-experimental studies on the effectiveness of Saxon Math. Using achievement data from the Iowa Tests of Basic Skills, students were compared against a matched sample of students who were in the control group that used other mathematics curricula (Nguyen & Elam, 1993). In the study, students were matched by grade level, previous test scores on the Iowa Basic Skills Test, socioeconomic status, race, and gender. Composite performance of students in the Saxon Math classrooms had scale scores of 55.0, while the students with other math curricula had a composite scale score of 52.59.
Researchers from the National Center for Education Evaluation and Regional Assistance (2013), which is a part of the Institute of Education Sciences, examined how four math curricula affected student’s achievement scores from first to second grades (Agodini & Harris, 2010). This study began in 2009 and concluded in 2010 while examining four math curricula that were balanced between constructivism and DI formats. Two constructivist approaches, Investigations and Math Expressions, were compared against the DI formats of Saxon Math and Scott Foresman-Addison Wesley
Mathematics, which was later renamed enVision.
Of importance to researchers in this study were the effects of switching math curricula for students between different theoretical types of curricula as well as student achievement. Publishers who were selected to be a part of the design had to apply and submit proposals. Programs were selected according to appropriateness for usage in a Title I school, capacity to train the number of teachers to be used for the study, quality of training and materials, empirical evidence of effectiveness, and research support for the conceptual foundations of the curriculum. The four programs listed above differ in mathematical emphasis in areas such as cognitive demand on the student, frequency and length of repeated practice and routine as well as the teacher’s role, pathway for learning, and support for teachers. While all curricula have varied amounts of each activity, stark differences do exist. Saxon Math for example contains 0% of students doing math or actively engaged in math problems, versus Investigations which has 40% of students using active engagement each day. Conversely, Saxon Math is heavy in repetition with procedures for completing math problems each day, 95% of tasks each day, while Investigations only spends 60% of its daily allotted math activities to procedures.
In this study, 111 schools from 12 districts across the United States enrolled in the study and agreed to participate for at least 1 year. In the second year of the study, only 58 schools participated. Random assignment was used to assign a curriculum to each school. Following assignment, publishers from each curriculum made presentations and delivered needed materials to the school staff.
To assess the outcomes of the curricula, researchers administered the Early Childhood Longitudinal Study-Kindergarten (ECLS-K) Class of 1998-1999 assessment. The ECLS-K assessment is given to students on an individual basis and is norm tested
nationally. Both open-ended and multiple-choice questions which measure number sense, operations and measurement, geometry and spatial sense, data analysis, statistics, probability, and algebraic patterns were used in the assessment.
Examination of data from the study indicated that after 2 years, Math Expressions, Saxon, and SFAW/envision all outperform Investigations in both first and second grades. Students using Investigations for 2 years had an average score of 65.5, whereas Math Expressions, Saxon, and SFAW/enVision represented scores of 69.8, 69.2, and 69.2, respectively.
Reform-Based Mathematical Programs
Instructional math programs have been caught between the so-called Math Wars waging in the United States since the 1989 publication of Curriculum and Evaluation
Standards for School Mathematics by NCTM (1989). This publication called for new
methods in mathematics instruction, often called reform mathematics, to be used in American classrooms. Reform mathematics differs from traditional mathematical practices of highly sequenced instruction with attention to algorithms and rote memorization of facts.
Exposing students to carefully paced instruction is essential for an adequate foundation to lifelong learning. Foundational skills are often taught in the home and then transferred to the kindergarten classroom in the K-12 spectrum of learning. Students with mathematical difficulties are typically deficient in three areas of mathematics: long-term memory retrieval; ability to solve word problems; and the ability to organize, monitor, and evaluate information (Mercer & Miller, 1997). In kindergarten, student acquisition of the ability to gain knowledge of number sense is a crucial step in order to provide students with the necessary requisite skills for future learning.
One study examined the use of Investigations to research its effects on the
acquisition of number sense among kindergarten students (Agodini & Harris, 2010). The Investigations unit is organized into six units that provide units of study that focus on the math strands of number sense, data analysis, and geometry. Units of study can vary in length from 3-8 weeks in duration. Sample topics of instruction include collecting, counting for ourselves and others, counting and measuring, collecting, pattern trains, and hopscotch path, among others.
Twenty-three students in the study were given various forms of assessment including the Stanford Achievement Test-10 (SAT10) as well as a set of Early Numeracy-Curriculum Based Measures (EN-CBM). The SAT is a norm-references achievement test given in group settings. The subsets of tests from the EN-CBMs
performed on students in this study included Oral Counting Fluency, Counting From, and Number Identification. Research results were examined using a Pearson Chi-Square analysis that indicated there were no statistically significant differences in gender and ethnicity. Based on these findings, researchers support the usage of carefully sequenced activities with the use of explicit instruction combined with student practice in order to affect student achievement.
In order to determine adequate mathematics instructional resources and practices, Slavin and Lake (2009) completed a meta-analysis of reviews produced at Johns Hopkins University and the University of York that are part of the Best Evidence Encyclopedia (BEE). The BEE uses research methods similar to those of the What Works
Clearinghouse yet are broader in focus and not contained to measures that are inherent in research treatments. Five researchers studied thousands of studies and found a total of more than 400 that met the inclusion standards for publication. Reviews suggest that
strategies likely to improve student performance are those that increase student active participation and focus in the classroom, help students to learn about their mathematical thinking, and improve the quality of daily mathematical instruction.
Best evidence synthesis (Slavin, 1986) seeks to apply consistent, well-justified standards to identify meaningful information from experimental studies that are unbiased and provide meaningful information. From this analysis and review of literature and research studies, the following conclusions were found:
1. Cooperative learning programs are consistently affiliated with the most positive learning outcomes. Programs that teach metacognitive strategies also have noted positive effect sizes.
2. Traditional computer-assisted instruction programs produced positive effects in math.
In review of mathematics data and traditional mathematics instruction models, Kohn (1999) continued to be an outspoken critic of traditional DI techniques. Kohn asserted that there exists no differences between first-grade classrooms and high school algebra classrooms where the teacher demonstrates the correct way to complete the algorithm and assigns a plethora of examples of the same problem whereby students should imitate the method in which they were shown. The transition model of
information leads this type of classroom where students are given facts and procedures by the textbook and the teacher. Students have little, if any, decision about diverse strategies they may be able to formulate to produce the correct answer. Kohn proclaimed one consequence of this drill and kill method is a society where students casually explain that they hate math and lack any skills in the area.