Chapter 4: Development and validation of a mathematical-number sense web-based
4.3.2 Procedures and material
4.3.3.1 Measures Phase 1
Students' number specific abilities were assessed with three tests: Dot Task, Dot Matching, and Number Line. As part of general cognitive abilities, speed of processing was assessed with Reaction Time task; and visuo-spatial working memory was assessed with the Corsi Tapping block task. Mathematical skills were assessed with a timed task of arithmetic (Problem Verification Task); as well as a series of tests containing addition, multiplication, and subtraction. The battery also contained two mathematical Stroop tasks.
1. Dot Task (Halberda et al., 2008). This computerised task administered via laptop assessed the specific numerical ability to estimate large numerosity. Stimuli and parameters have been provided by the author of the task, Dr. Justin Halberda. The trials consisted of arrays of yellow and blue dots mixed together flashing on the screen for 400 ms (see Appendix 3 for a sample trial). There were 55 trials with ratios ranging between 0.33 and 0.66; 115 trials had ratios ranging between 0.66 and 0.83 and in 80 trials ratios ranged between 0.83 and 0.89.The trials were administered in random order. The task required to judge which of the two arrays had more dots. Responses were given by pressing the "F" key for more yellow, "J" for more blue. To facilitate response, the keys on the keyboard were colour coded. After response, the next trial was shown by pressing the space-bar. The 250 trials were divided in 5 blocks of 50 trials. At the end of each block it was possible to pause the test and take a break from it. If the students wanted to continue they had to press the space-bar to start a new block of trials. The Weber Fraction measure derived from this assessment was the acuity of the ANS (Approximate Number System) (see Halberda et al., 2008 method section). Cronbach alpha in this test was .89 with split half of .74.
2. Number Line (Opfer & Siegler, 2003). The task assessed estimation of numerical magnitudes and was in pen and paper format. Participants were shown an A4 paper with a line 25 cm long drawn in the middle with the extremes marked 0 and 100 (sample in Appendix - 4). The task required to estimate the position of a series of integers along the line. The 20 numbers to be estimate were presented with same order to all participants as follow: 25, 75, 50, 20, 5, 11, 17, 22, 29, 33, 40, 44, 95, 57, 60, 65, 71, 78, 83, 86. The choice of the range 0-100 was driven to make sure that the task was suitable for 16 year-olds and that students were able to perform the task within this range. Administration mode and stimuli were the same used in the assessment of the US twins of the Western Reserve Reading Project Math. This would have given comparable data for the Number Line in different twin samples. In the instructions given prior to the test trials, students were shown an A4 sheet displaying the line divided in 10 portions (Appendix - 5) and did one practice by estimating the number 50. Scores were the mean absolute difference between the correct location of the number on the line and the point of estimation made by participants. Cronbach alpha in this assessment was .75 with split half .29. The low split half correlation was probably due to the presentation order. The second half of the trials contained more numbers with larger magnitudes compared to the first half. Magnitude comparison in the higher range can be more sensitive to fluctuation in estimation as the numbers are represented "compressed". The presentation order was modified in the final version of the task, with large and small magnitudes randomly and evenly allocated. This improved the split half of the final version to .87 (section 4.3.3.2).
3. Dot Matching task. This computerised task assessed estimation of small and large numerosities. The task was available online at the website of our collaborators in Hong Kong: http://lab.kctam.com/stroop. The 36 stimuli consisted of arrays of dots presented with a number on the side (see Appendix - 6 for sample trial). Numbers and dots ranged between 1 and 9. Half of the trials were congruent (the number matched the dots) and half incongruent. Half of the trials had the numbers of the right-side and the dots on the left-side. The program also varied the spread and layout of the dots across participants. Students had to judge if the number matched the dots in the array. Responses were given by pressing the "J" for a match, "F" if there was no match between dots and numbers. For left-handed participants the keys
were reversed. To avoid confusion with the responding keys, an A4 sheet with a reminder of the response keys was placed next to the screen. After response the next trial followed without delay. Time out for response was 8 seconds, but students were encouraged to answer as quickly as possible. To start the test students had to provide their date of birth and handedness, after which the program generated a unique log- in. There was a set of on screen instruction and 4 practice trials with feedback. The practice could be repeated if needed. One point was awarded for each correct response, incorrect answers received zero points. The first 4 test trials allowed to adjust to the task therefore they did not concur to the calculation of the final score. Maximum score for this task was 32. Reaction time and accuracy on response were recorded. However, due to the slow internet connection and slow download times experienced during the administration of the task, reaction time was not used in the analyses. For the purpose of the analyses, proportion of correct answers computed as the total of correct answers divided by 32, was derived for each participant. Cronbach alpha in this assessment was .93 with split half .82.
4. Reaction Time (Deary et al., 2001). This task was a measure of general cognitive ability - speed of processing. It was programmed in E-Prime 2 following the procedure described in Deary et al. (2001) and was administered via laptop. The trials consisted of a single number presented in the middle of the screen. The computer program generated random sequences of the numbers 1, 2, 3 and 4 repeating them 10 times each (for a total of 40 trials) with a random interval of 1, 2 and 3 seconds; the 10 trials displaying the same number had an equal number of 1 second, 2 seconds and 3 seconds intervals. The task required to press as fast and accurately as possible the key corresponding to the number appearing on the screen. The responding keys were: X = 1, C = 2, B = 3, N = 4. In order to facilitate response the responding keys had stickers with the corresponding numbers. Reaction time and accuracy were recorded. Each correct answer was given one point, incorrect answers scored zero. To correct for speed-accuracy trade off effects in the analysis we used Efficiency (median Response time/proportion of correct response). Cronbach alpha in this assessment was .96 with split half .94.
5. Corsi Tapping Block (Corsi, 1972; Pagulayan et al., 2006). This test assessed non- verbal visuo-spatial short term memory. During phase 1 the test was administered using the Corsi Block apparatus. This consisted of a 25cm X 20cm wooden block with nine 3.1cm cubes placed as in Figure 4.1. The procedure and apparatus for this task are described in Pagulayan et al. (2006).
Briefly, the experimenter tapped the cubes in predetermined patterns. The patterns were the same for all participants. Students had to reproduce the patterns or sequences by tapping the cubes back in the same order. The list of sequences and administration test rules are shown in Table 4.1. The number of cubes tapped increased up to a maximum of nine or until participants failed to correctly reproduce 5 sequences with the same number of tapping. Each level had a maximum of 5 sequences. One point score was assigned for each correct sequence. Maximum score for this test was 30. Cronbach alpha in this assessment was .75 with split half .40.
9 9 6 6 6 5 5 5 3 3 1 1 1 8 8 4 4 4 2 2 7 7 7 7 7 7
Figure 4.1: Corsi Block apparatus
The numbers are not visible to participants. They can be seen only by the experimenter and are used to determine the pattern of tapping.
Table 4.1
Corsi Block tapping sequence - Phase 1 of testing
Level 4 Level 5 Level 6 Level 7 Level 8 Level 9 4 sequences 5 sequences 6 sequences 7 sequences 8 sequences 9 sequences
3 2 4 9 3 4 8 7 5 5 6 8 1 9 4 5 4 1 6 3 9 7 1 4 7 2 8 3 6 9 3 7 4 1 9 2 5 6 8
1 7 6 5 8 1 5 3 6 4 1 6 7 8 2 7 1 9 3 4 6 2 5 9 3 7 2 1 8 4 7 5 2 8 6 4 1 9 3
3 5 7 1 6 4 5 2 9 2 5 9 6 3 1 9 5 3 6 8 1 7 7 6 9 3 1 5 4 8 9 5 3 7 1 2 6 4 8
6 8 4 9 3 1 7 2 4 7 9 2 5 1 3 2 6 8 1 4 9 5 4 2 1 3 9 5 7 8 6 3 7 4 2 5 1 9 8
(5 2 6 4) (4 9 3 6 2) (8 6 1 4 3 9) (2 9 1 7 4 8 3) (9 3 1 4 2 6 5 7) (5 8 6 2 9 1 4 3 7)
The first level starts with a sequence of 4 numbers/items. This is an appropriate level of difficulty for 16 year-olds. Each level is successfully completed if 4 sequences are reproduced correctly. Five points score are awarded for each level successfully completed. The sequences in brackets are administered only if one of the first 4 is reproduced incorrectly. If all the first 4 sequences are successfully reproduced, one point for the sequence is bracket is credited. The test is discontinued after failure to reproduce all the 5 sequences in one level.
6. Problem Verification Task: This test was a timed task assessing mathematical fluency. It was programmed in E-Prime 2 following the procedure described in Murphy & Mazzocco (2008) and it was administered via laptop. The stimuli consisted of 88 arithmetic problems as follow: 14 additions (7 small double-digit and 7 large- three or four digits operands); 14 subtractions (10 small single and double-digit operands, 4 large three or four-digit); 14 multiplication (7 small single digit, 7 large- two and three-digit operands); 14 divisions (7 small one or two-digits, and 7 large two and three-digits); 32 fraction (12 additions and 20 equations). The problems were presented on screen in a fixed sequence together with a proposed answer. The task required to judge whether the answer was correct or not; the proposed answers were correct half of the times. Response was given by pressing the keys "T" for correct, "Y" incorrect, "U", don't know. In addition, participants used the same keys to rate how confident they were about their response: "T" confident, "Y" not confident, "U", don't know. The responding keys were colour coded to facilitate response: T was green, Y was red and U yellow. An A4 sheet with the coloured letters T, Y, U and their meaning for response was placed next to screen as reminder. After response was given the next trial followed without delay. One point was given for correct responses, the incorrect and "don't know" answers received zero point. The ratings on the confidence of response were coded as 1 = confident, 2 = not confident, 3 =
don't know. The task had on screen instructions and two practice trials. The program recorded accuracy and reaction time on correct responses. Accuracy was used for the analyses. Cronbach alpha in this assessment was .91 with split half .84.
7. Arithmetic Problems. This test assessed mathematical fluency. It was composed of 3 separate subtests: 20 single digits addition and multiplication problems, 20 subtractions (10 single digits and 10 with 1 double digit operand). The tasks were available online on the website: http://lab.kctam.com/stroop. The arithmetic problems were presented sequentially on screen together with a proposed answer. Students had to judge whether the answer was correct or not, within 8 seconds time limit. Right handed participants responded by pressing "J" for correct, "F" for incorrect. The keys were reversed for left-handed participants to allow correct responses with the dominant hand. Each sub-test started with its own on-line instructions followed by 4 practice trials. To avoid confusion with the keys to be pressed, a reminder with the responding keys was placed next to the screen. Accuracy and reaction time on response were recorded. The analyses were carried out on accuracy scores.
8. Numerical Stroop tasks - Physical and Numerical Comparison tests were taken from the website: http://lab.kctam.com/stroop. The stimuli consisted of two double digit numbers presented next to each other in the middle of the screen. The font-size of one of the numbers in the pair was always bigger than the other. The task involved identifying a larger number in the pair, according to its size in the physical comparison; or in numerical value (regardless the size) in the numerical comparison. Response was given by pressing "J" if the numbers larger in magnitude or larger in physical size were on the right side of the screen. Participants pressed "F" if the numbers larger in size or in magnitudes were on the left side of the screen. The keys were reversed for left-handed participants. Time out for response was 8 seconds. Both tests started with their own instructions followed by 4 practice trials; participants completed 52 numerical and 52 physical comparisons. Accuracy and reaction time on response were recorded.