Sept 2010

### Mathematics Placement Follow-Up Study

Placement Recommendations After Changes to Branching Profile And Subsequent Mathematics Course Performance Outcomes Controlling for Students Placed at Higher Mathematics Course Levels

In Order to Generate Educational Background Predictor Variables

Overview: As a follow-up to a previous study, the Office of Institutional Research once again
examined the relationship between placement recommendations generated in the year before
(August 16th_{, 2008 through August 15}th_{, 2009) and after (August 16}th_{, 2009 through August 15}th_{, }
2010) changes were made to the mathematics test branching profile and subsequent
mathematics course outcomes. However, in the current study the Office of Institutional
Research controlled for students assessed after changes were made to the mathematics test
branching profile who received higher course placement recommendations in order to collect
data to validate educational background predictor variables.

Methodology: In prior conversations with the Mathematics Department, the then Senior Research Analyst discussed allowing students who generated assessment test scores in proximity to faculty identified mathematics course cut-scores to receive placement recommendations for the next higher mathematics course. In allowing students who scored slightly below existing mathematics course cut-scores to access the next higher level course in the mathematics course sequence, the intent was to capture additional data to produce more accurate educational background predictor variables. As mandated by the California Code of Regulations Title 5, § 55521(a), California Community Colleges must use multiple measures (other than two or more highly correlated instruments) to generate student placement recommendations. Capturing additional data was intended to strengthen this process, affording researchers with the opportunity to collect and analyze sufficient data to determine multiple measures that predicted optimal course success and non-success.

Table 1 on the following page identifies faculty determined mathematics course placement cut-score ranges prior to and after changes to cut-cut-score placement recommendations were implemented. In comparing previous and current cut-score ranges, table 2 identifies the specific cut score ranges by assessment test that were impacted, how these changes effected course placement recommendations, and the number and percentage of students within each score range that were affected. For the purposes of this follow-up study, the number and percentage of students who were affected by cut-score changes reflect only students who subsequently earned a grade on record in a mathematics course within a year of generating a mathematics assessment placement recommendation (see the initial study (Math Placement Study – Sep10.doc) for details about the methodology and determination of placement recommendations and relationship to subsequent course-taking behavior).

### 2

Table 1: Faculty Determined Mathematics Course Placement Cut-Score RangesPrior to and After Changes to Cut-Score Placement Ranges Faculty Determined

Mathematics Course Placement Cut-Score Ranges

College Level

Math Test Elementary Algebra Test Arithmetic Test Prior

Range Current Range Range Prior Current Range Range Prior Current Range

MATH-61 101.80 to 120.00 101.80 to 120.00 MATH-31 72.70 to 101.79 72.70 to 101.79 MATH-25 28.00 to 72.69 28.00 to 72.69 Branched to Take Elementary Algebra 20.00 to 27.99 20.00 to 27.99 Branched to Take College Level Math

111.74 to 120.00 105.60 to 120.00 MATH-425 73.40 to 111.73 68.70 to 105.59 MATH-410 34.90 to 73.39 32.30 to 68.69 Branched to Take Arithmetic Test 20.00 to 34.89 20.00 to 32.39 Branched to Take Elementary Algebra 78.50 to 120.00 73.20 to 120.00 MATH-520 53.70 to 78.49 47.20 to 73.19 MATH-510 28.90 to 53.69 27.20 to 47.19 MATH-504 20.00 to 28.89 20.00 to 27.19

Table 2: Cut-Cut Changes by Assessment Test - Changes to Mathematics Course Placement Recommendation and Number of Effected Students

Assessment Test Effected Cut-Score Range Formerly Placed Students Into. . . Currently Student Effected N %

Elementary Algebra 105.60 to 111.73 MATH-425 Take CLM 26 2.7

Elementary Algebra 68.70 to 73.39 MATH-410 MATH-425 59 6.1

Elementary Algebra 32.30 to 34.89 Take Arith Test MATH-410 36 3.7

Arithmetic Test 73.20 to 78.49 MATH-520 Take EA n/a n/a

Arithmetic Test 47.20 to 53.69 MATH-510 MATH-520 17 1.8

Arithmetic Test 27.20 to 28.89 MATH-504 MATH-510 3 0.3

TOTAL 141 14.6

In examining data displayed in table 2, the 26 students who generated an Elementary Algebra (EA) assessment test score in the 105.60 to 111.73 range all subsequently took the College Level Math (CLM) test and generated MATH-25 course placement recommendations. Under the prior branching profile, these students would have received MATH-425 placement recommendations. The table similarly identifies students who scored in the 68.70 to 73.39 range on the EA test (MATH-425 course placement recommendation; formerly would have received a MATH-410 placement recommendation); the 32.30 to 34.89 range on the EA test

(MATH-410 course placement recommendation; formerly would have received a
recommendation to take the Arithmetic test); the 47.20 to 53.69 range on the Arithmetic test
(MATH-520 course placement recommendation; formerly would have received a MATH-510
placement recommendation); and the 27.20 to 28.89 range on the Arithmetic test (MATH-510
course placement recommendation; formerly would have received a MATH-504 placement
recommendation). Students who scored in the 73.20 to 78.49 range on the Arithmetic test were
not affected; in making changes to the test branching profile, no students were identified who
started on the Arithmetic test (the previous study identified that 99.3% of students started with
the Elementary Algebra test after changes were made to the mathematics test branching profile
on August 16th_{, 2009) and received an Arithmetic score that would have branched them to take }
the Elementary Algebra test.

Findings: Controlling for the 141 enrollments that would have resulted in lower mathematic course placement recommendation if previous cut-score ranges had been applied, table 3 on the following page provides a detailed analysis of course success and retention rates by specific course placement recommendation, identifying statistically significant success and retention rate differences between groups at the course level. Even after controlling for the 141 enrollments that might have resulted in students being placed into higher mathematics courses, outcomes closely parallel the findings indentified in the initial study. Statistically significant differences were noted in the same areas identified in the initial study (e.g., MATH-504 retention rate; MATH-410 success rate; aggregate math success and retention rates). Table 4 compares findings from the initial study to the current study.

### 4

Table 3: Mathematics Course Performance Outcomes of Students Who AssessedBeforeand After Changes to the Branching Profile by Course Level – Controlling for Students Placed at Higher Course Levels in Order to Generate Educational Background Predictor Variables Assessment Group by Math

Course Placement Recommendation Level

Overall Math Course Performance

Outcomes

Course Performance Outcomes in: Lower

Math Courses Recommended Math Course Math Courses Higher

N Succ Ret N Succ Ret N Succ Ret N Succ Ret

MATH-504

Assessed Before Changes 27 44.4 85.2* 8 62.5 75.0 19 36.8 89.5

Assessed After Changes 30 36.7 60.0 11 18.2 54.5 19 47.4 63.2

MATH-510

Assessed Before Changes 264 60.6 86.7 11 63.6 90.9 220 62.7 85.9 33 45.5 90.9

Assessed After Changes 68 60.3 83.8 2 100.0 100.0 64 59.4 82.8 2 50.0 100.0

MATH-520

Assessed Before Changes 378 69.3 90.2 44 75.0 90.9 301 70.1 90.0 33 54.5 90.9

Assessed After Changes 27 66.7 77.8 5 60.0 60.0 22 68.2 81.8 0 n/a n/a

MATH-410

Assessed Before Changes 233 65.7* 88.4 40 82.5 92.5 177 60.5* 87.0* 16 81.3 93.8 Assessed After Changes 342 45.9 79.2 86 66.3 87.2 243 38.7 76.5 13 46.2 76.9 MATH-425

Assessed Before Changes 277 65.0 87.7 37 75.7 91.9 223 62.3 87.0 17 76.5 88.2 Assessed After Changes 307 65.1 86.9 61 72.1 90.2 236 63.6 86.9 10 60.0 70.0 MATH-25

Assessed Before Changes 62 79.0 95.2 17 70.6 94.1 39 82.1 94.9 6 83.3 100.0

Assessed After Changes 37 75.7 94.6 4 75.0 75.0 29 72.4 96.6 4 100.0 100.0

MATH-31

Assessed Before Changes 11 90.9 90.9 4 75.0 75.0 4 100.0 100.0 3 100.0 100.0

Assessed After Changes 13 84.6 84.6 6 66.7 66.7 6 100.0 100.0 1 100.0 100.0

MATH-61

Assessed Before Changes 1 100.0 100.0 1 100.0 100.0 0 n/a n/a

Assessed After Changes 3 100.0 100.0 0 n/a n/a 3 100.0 100.0

TOTAL

Assessed Before Changes 1,253 66.0* 88.7* 154 76.0 91.6 972 65.4* 88.0* 127 58.3 91.3*

Assessed After Changes 827 56.7 82.6 164 68.9 86.6 614 53.6 82.2 49 57.1 73.5

Table 4: Mathematics Course Performance Outcomes of Students Who Assessed Beforeand After Changes to the Branching Profile by Course Level – Comparison of Findings From Initial Study and Current Study That Controls for Potential Higher Course

Placement Recommendations Assessment Group by Math

Course Placement Recommendation Level

Findings from

Initial Study Findings From Current Study

N Succ Ret N Succ Ret

MATH-504

Assessed Before Changes 27 44.4 85.2* 27 44.4 85.2*

Assessed After Changes 30 36.7 60.0 30 36.7 60.0

MATH-510

Assessed Before Changes 264 60.6 86.7 264 60.6 86.7

Assessed After Changes 71 62.0 84.5 68 60.3 83.8

MATH-520

Assessed Before Changes 378 69.3 90.2 378 69.3 90.2

Assessed After Changes 44 65.9 81.8 27 66.7 77.8

MATH-410

Assessed Before Changes 233 65.7* 88.4 233 65.7* 88.4

Assessed After Changes 378 45.5 77.2 342 45.9 79.2

MATH-425

Assessed Before Changes 277 65.0 87.7 277 65.0 87.7

Assessed After Changes 366 63.1 85.8 307 65.1 86.9

MATH-25

Assessed Before Changes 62 79.0 95.2 62 79.0 95.2

Assessed After Changes 63 76.2 93.7 37 75.7 94.6

MATH-31

Assessed Before Changes 11 90.9 90.9 11 90.9 90.9

Assessed After Changes 13 84.6 84.6 13 84.6 84.6

MATH-61

Assessed Before Changes 1 100.0 100.0 1 100.0 100.0

Assessed After Changes 3 100.0 100.0 3 100.0 100.0

TOTAL

Assessed Before Changes 1,253 66.0* 88.7* 1,253 66.0* 88.7*

Assessed After Changes 968 56.7 81.9 827 56.7 82.6