Simultaneous Technique for Acuity and Readiness Testing (START): Further Concurrent Validation of an Aid for Developmental Surveillance

(1)

Simultaneous

Technique

for

Acuity

and

Readiness

Testing

(START):

Further

Concurrent

Validation

of

an

Aid

for

Developmental

Surveillance

Raymond

A.

Sturner,

MD*;

Sandra

C.

Funk,

PhD;

and

James

A.

Green,

PhD

ABSTRACT.

Study objective. A brief (8-minute)

pro-cedure, now called Simultaneous Technique for Acuity

and Readiness Testing or START, has been shown to be

efficacious for predicting developmental outcomes and a cost-effective screen for visual acuity. The objective of the two studies reported here was to examine the ability of this procedure to predict concurrent developmental out-come by using a new simplified scoring system.

Design.

A prospective design was used. Subjects were

screened using START, and then samples were stratified on the basis of developmental screening results (START in study 1 and the revised Denver Developmental Screen-ing Test and a shortened version of the Minnesota Child Development Inventory in study 2) into subsamples (n =

118 and 120) which were administered the standard cri-terion test (McCarthy Scales of Children’s Abilities in one cohort and the Stanford-Binet in the other).

Setting.

Prekindergarten

registration

for a rural

school

system in North Carolina.

Subjects. Two county-wide cohorts of preschool

chil-dren (n = 352 and 362).

Measurements and main results. Results for prediction

of the McCarthy outcomes were as follows: sensitivity, 0.76;

specificity,

0.99; predictive

value,

0.81; underreferral,

1.3%; overreferral, 1.0%; and percent agreement, 98%. Pre-diction of Stanford-Binet results was as follows: sensi-tivity, 0.94;

specificity,

0.83; predictive value, 0.22;

under-referral,

0.3%;

overreferral,

16%;

and

percent

agreement,

84%. Most of the overreferrals for the Stanford-Binet were in the clinically important borderline category.

Conclusion. These results provide further support for

the concurrent validity of START. The results illustrate how routine health procedures can be restructured to ob-tam clinically useful data on specific child developmental functioning. Pediatrics 1994;93:82-.88; developmental

screening, preschool screening, child health maintenance.

ABBREVIATIONS. START, Simultaneous Technique for Acuity and Readiness Testing; CCI, General Cognitive Index; MSCA, Mc-Carthy Scales of Children’s Abilities; DDST-R, revised Denver De-velopmental Screening Test; S-MCDI, shortened version of Mm-nesota Child Development Inventory.

British

child

health

authorities

have

recently

ques-tioned

the

value

of repeated

developmental

screening

for

preschool

children.”2

Their

disillusionment

with

developmental

screening

appears

to be based

both

on

From the’ Duke University Medical Center, Durham, NC; The University of North Carolina at Chapel Hill; and §University of Connecticut, Storrs CT.

Received for publication Apr 6, 1992; accepted Jun 9, 1993.

Reprint requests to (R.A.S.) Child Development Unit, Dept of Pediatrics, Box 3364, Duke University Medical Center, Durham, NC 27710. PEDIATRICS (ISSN 0031 4005). Copyright © 1994 by the American Acad-emy of Pediatrics.

the

limited

validity

of the

tests

used

and

the

amount

of

time

and

effort

required.’3

These

authorities

are

now

championing

an

integrated

system

of

monitor-ing

and

promoting

the

child’s

developmental

progress

using

parental

concerns,

“opportunistic

ob-servation”4

of the

child’s

developmental

functioning,

and

occasional

developmental

screening

tests.

This

integrated

system,

termed

“developmental

surveil-lance,”2’3

has

also

been

suggested

for

American

pe-diatric

practice.5

Unfortunately,

while

pediatricians

have

long

claimed

to

conduct

developmental

screening

through

opportunistic

observations,

stud-ies

have

shown

that

even

experienced

pediatricians

are

not

reliably

able

to

detect

developmental

prob-lems.6’7

Our

report

of

the

clinical

utility

of

observa-tions

of

developmental

functioning

during

a

pre-school

vision

acuity

screen8

is

the

only

data-based

example

that

has

been

cited

in

support

of

“opportu-nistic

observation.”5

We

have

also

reported

on

the

utility

of

developmental

observations

made

during

a hearing

screening

test.9

In

this

report,

we

present

a

practical

scoring

system

for

the

vision

acuity/

developmental

screening

observation

and

test

ad-ministration,

along

with

concurrent

validation

infor-mation.

The

studies

reported

here

have

implications

both

for

the

validity

of

this

specific

screening

tech-nique

and

for

the

general

use

of developmental

sur-veillance

by

pediatricians.

The

American

Academy

of

Pediatrics

encourages

pediatricians

to

make

preschool

developmental

screening

as routine

as the

well-established

practice

of

preschool

vision

and

hearing

testing.

Their

policy

statement

reads:

The need for visual and auditory examination to identify deficits in these areas is readily accepted. Use of these ex-aminations is standard practice. However, use of preschool examinations for developmental problems, potential learn-ing disabilities, and mental retardation needs to be

encour-ed’#{176}

The

literature

suggests

that

the

use

of

developmen-tal screening

tests

in pediatric

practice

is related

more

to feasibility

than

to the

willingness

of practitioners

to

participate.

One

survey

revealed

that

while

almost

all

(97%)

pediatricians

agreed

on

the

need

for

develop-mental

screening,

formal

testing

was

usually

omitted

because

of perceived

and/or

real

time

limitations.”

The

method

we

call

simultaneous

screening

for

child

health

and

development

makes

developmental

screening

as feasible

as standard

preschool

vision

and

hearing

screening

because

observations

of

the

child

during

the

hearing

and

vision

examination

provide

at Viet Nam:AAP Sponsored on September 1, 2020

www.aappublications.org/news

(2)

12% Pass

S Mean

McCarthy General Cognitive Index

Stanford-Binet Intelligence Test

developmental

data

without

adding

time

to

the

ex-amination

itself.

The

vision/development

test

which

we

now

call

Simultaneous

Technique

for

Acuity

and

Readiness

Testing,

or

START,

has

been

shown

to be

more

cost-effective

than

standard

preschool

vision

testing

even

when

one

is not

interested

in the

devel-opmental

data

it generates.

This

is mainly

because

the

brief

(8-minute)

vision

test

results

in half

the

number

of untestable

cases

of standard

vision

screening

(So-ciety

to

Prevent

Blindness

protocol).

Visual

acuity

data

from

the

two

screening

tests

were

quite

similar,

and

both

agreed

well

with

data

from

an

additional

screening

by

a

pediatric

ophthalmologist.

Further-more,

the

START

method

avoids

compromise

in

pre-cision

of acuity

measure

(ie,

cruder

picture

type

tar-gets

for

all

children)

typically

made

for

preschool

children

to enhance

testability.

This

may

explain

why

the

START

test

failed

some

children

who

passed

on

the

pediatric

ophthalmologist’s

Allen

card

screen

but

were

found

to

have

mild

acuity

problems

requiring

treatment

when

subjected

to

complete

ophthalmol-ogy

examination.’2

START

has

also

undergone

a preliminary

concur-rent

validation

study

in

which

it

was

shown

that

START

may

be

as

predictive

of

standard

develop-mental

diagnostic

test

results

as any

other

screening

procedures,

even

those

with

extensive

time

require-ments.8

However,

this

preliminary

study

used

dis-criminant

analysis

for

scoring.

If START

is to be

prac-tical

for

clinical

use,

a simple

scoring

procedure

is

needed.

The

goal

of the

studies

reported

here,

there-fore,

was

to develop

a practical

scoring

system

with

concurrent

validation

using

two

well-established

di-agnostic

tests

as

criteria-the

Stanford-Binet

Intelli-gence

Test’3

and

the

McCarthy

Scales

of

Children’s

Abilities.’4

The

Stanford-Binet

is the

most

time-honored

of

in-telligence

tests.’5

The

McCarthy

General

Cognitive

In-dex

(GCI)

is closely

but

is thought

to

be sensitive

to a wider

range

of learning

problems.’8”9

We

have

found

that

the

McCarthy

GCI

is a good

pre-dictor

of

concurrent

kindergarten

performance2#{176}

as

well

as

later

school

outcomes.2’

These

tests

are

im-portant

outcome

criteria

because

they

are

typically

used

for

further

assessment

of

children

of

this

age

who

are

suspected

of

having

cognitive

difficulties,

and

they

provide

scores

usually

required

for

place-ment

in special

educational

programs.

Use

of two

dif-ferent

criteria

minimizes

the

likelihood

that

the

rela-tionship

between

the

screening

test

and

the

criterion

test

is idiosyncratic.

In

an

area

such

as

child

devel-opment,

where

indirect

measures

such

as

these

tests

must

often

substitute

for

a more

direct

measure

(eg,

an autopsy

result

in traditional

medicine),

use

of more

than

one

criterion

test

is likely

to yield

a more

robust

approximation

of

the

“real

world”

outcome,

ie,

va-lidity.

The

START

screening

procedure

is of particular

in-terest

at present

because

of the

mandate

(Public

Law

99-457)

to

identify

developmental

problems

prior

to

school

entry.22

Now

children

need

not

wait

until

school

entry

to receive

intervention

services

when

di-agnostic

test

results

indicate

that

they

qualify.

START

is one

tool

that

can

help

identify

these

children.

Design Overview

METHODS

Two studies were conducted in 2 successive years in a rural

county of North Carolina where, each year in February or March, children who are to begin kindergarten the following fall are urged to attend a combined health screening and school registra-tion week. The START protocol was administered during this combined screening/registration week (see Figure). In each year children were recalled for criterion testing (McCarthy in study I and Stanford-Binet in study II) within a 2-month time window. Because of logistical problems, the children could not complete the criterion testing on the same day as screening. The 2-month time window was believed to be a practical time period to complete criterion testing including rescheduling “no shows” and was be-lieved to be within the definition of “concurrent” because most standardized tests for children of this age consider children with birthdays within 3 months to be the same age for scoring pur-poses. Because it was not practical to complete criterion testing on the county-wide sample, we chose a stratification procedure de-signed to oversample children with low scores on developmental screening tests to minimize the possibility of overlooking positive cases. The stratification procedure required that all strata of screening scores be sampled so that the resulting sample would be representative of the original large sample and rates of prediction could be calculated based on a projection or a statistical recreation of the original sample. This statistical procedure involved weight-ing each strata by the inverse of its proportional representation in the original sample (eg, if one half of a group was selected, sub-jects in that group would receive a weight of 2 in statistical analy-sis). In study I raw scores on the START procedure itself were used as the developmental at-risk index for stratification selection. This procedure maximizes the opportunity to explore screening test errors that could result from low scores assigned to normal chil-dren. In study II, two independent developmental screening meas-ures were administered during the screening week (of the subse-quent year), in addition to the START protocol, and were used for selecting the stratified sample. Use of independent selector screen-ing tests allowed us to detect potential START errors due to in-sensitivity since it is possible that a true-positive case could have

County-Wide Screening

. Study Instrument

. Instruments Used to Stratify

Stratification Procedure

Criterion Testing

Calculate Indices Based on Weighted or Projected Population

Study

1

START START

All Low 50% Intermed

#{149}>1SD

<Mean to Mean

Study

2

START DDST-R; S-MCDI

All Low

. DDST-R = Fall

. S-MCDI = >1 SD

<Mean

40% Intermed

. DDST-R = Quest . S-MCDI = 1 SD <Mean to Mean

1 1 % Pass

. DDST-R Pass

. S-MCDI Mean

Figure. Design overview. Abbreviations: START, Simultaneous Technique for Acuity and Readiness Testing; DDST-R, revised Denver Developmental Screening Test; S-MCDI, shortened version of Minnesota Child Development Inventory.

at Viet Nam:AAP Sponsored on September 1, 2020

(3)

scored adequately on START and be excluded from stratification

recall

because of a sampling error. The use of two different strati-fication procedures provides added assurance against any distor-tion in representation due to systematic sampling bias related to development.

Subjects

The

samples for the studies consisted of approximately equal

numbers of boys (52%) and girls (48%) and more whites (62%)

than

nonwhites (38%). All income levels were represented, with 53% of the sample reporting incomes of $10 000 or more per year. Census data indicate that this county has somewhat higher income levels than the state as a whole and thus more closely approxi-mates national norms.

THE START PROTOCOL

The START test protocol has been described previously.8

Stand-ard preschool

vision screening procedures have been modified so

that information can be obtained about children’s understanding of vision test materials

and

so that behavioral responses can be systematically recorded. Contingencies built into the test allow for flexibility both in the eventual choice of vision testing materials (letter chart, E chart, picture cards) and in the number of items

administered

that will be difficult

for an individual child. During phase I of the testing, the “test selection phase,” the child’s knowledge ofletters and pictures of objects is assessed. The tester is

seated

at a small table and the child stands next to him!

her.

By assessing

the child’s

ability to correctly name picture cards representing common objects (some from Allen cards’s) and to point to letters of the alphabet, draw them in the air, and name

them,

the examiner is able to evaluate the child’s level of

developmental/readiness functioning and to choose the most ap-propriate set of materials for vision acuity screening.

The

actual procedure to be used to assess the child’s visual acuity is selected on the basis of the examiner’s overall judgment (during the test selection phase) as to the most precise (optically correct) method to which the child is able to respond. These pref-erertces, therefore, are always Snellen letter chart before Illiterate E chart, before Allen picture cards.24n The most ideal measures

(based on principles of visual acuity testing) tend to be the most demanding developmentally.

In phase 2, the vision testing procedure chosen in phase 1 is completed while the child’s behavioral reactions to the standard

instructions are rated as part of the developmental component of the test. During this phase the child stands at a line marked on the

floor 15 feet from the visual target. The examiner points to the visual acuity stimuli and the child attempts to identify the visual target while covering one of his/her eyes. At the end of the acuity

screen, the tester records the vision acuity screen results, the be-havior ratings, and the developmental test scores. Vision and developmental/readiness testing thus proceed simultaneously.

Data relating to development include the correctness of the

child’s

response

to each of the

vision testing approaches presented

during the test selection phase, as well as ratings of specific

be-haviors

observed

during the test administration phase (eg,

com-pleted

all

three commands related to moving to the vision testing

line), judgments of overall performance, and three ratings of the child’s reaction to the test situation (eg, child did not give up on any task, despite difficulties). The specific behavioral responses

rated

were selected because they appear to have requisite

func-tions similar to those critical for classroom work. In addition, the method of vision testing actually used is noted, along with addi-tional comments.

To illustrate, correct responses to vision testing include giving the exact name for the Allen cards and other picture cards that we have developed based on their suitability as vision targets and the appropriateness of the vocabulary data they provide (eg, basket and leaf). When letters are used, the child is first asked to pick out a sample letter shown from five letters on a page. This task is very easy for almost all preschool children. The child is then asked to

give the name of one letter (E). If the child does not know the name of the letter, he/she is offhandedly told that it is something he/she

will be

learning in school. The E matching game is then

demon-strated

by asking the child to hold a sample to and turn it in the

same direction as the examiner’s model. Ability to use a hand-pointing response rather than manipulating the sample is also

noted

for children who do not meet criterion for understanding E

matching. The letter T is then shown and the child is asked to find

the

letter and name it. “Drawing the letter T in the air” is then

demonstrated,

and child’s

ability to imitate the response (using a

finger to draw an imaginary letter in the air) is rated. This proce-dure continues with the letters H, P. and N, but the letter-drawing in the air request is not made for those letters for children who are unable to respond adequately to the letter T request. If a child is

able

to name four or more letters, a typical first-grade sentence is presented and the child’s ability to read any of the words is noted. This word-reading task is the only one that is irrelevant to the vision acuity testing process, and it is only rarely administered to preschool children since most children cannot yet recognize enough letters. It is included in the test because identification of

preschool children who already read is believed to be clinically useful.

During the test administration, or the second phase of the test,

two specific behaviors are rated: (1) whether the child completes all three demands in moving to the test administration line after they are presented in a standard manner and repeated (the corn-mands include taking the cup, going to stand on the line, and putting the cup over the child’s eye); and (2) whether the eyes are covered throughout the test (noting whether the child needs to be reminded to keep one eye covered). Three additional global be-havior ratings are made based on responses in both phase I and phase 2. These ratings include (1) whether or not the child is able to follow directions correctly without ever misunderstanding

them;

(2) whether

the child

needs to be asked to repeat a response because of lack of verbal clarity; and (3) whether the child works

well

and

hard

throughout without giving up on any task. In addition, the method of vision testing actually used is noted, along with additional comments.

TABLE 1. Study 1:Concurrent Validity for START Test With McCarthy GCI*

START

Score McCarthy GCI Score Total

<68 68-84 >84

Further testing needed (fail) (scores 0, 1)

Questionable (scores = 2, 3)

Pass (scores 4) Total

13 4 0 17

3 37 16 56

0 26 213 239

16 67 229 312

Indices for Prediction of Low Scorers (<68) on the M cCarthy CCI From Failures on the START Test

Sensitivity (co-positivity)

Specificity (co-negativity) Predictive value

0.76 0.99 0.81

Overreferral rate 1.0%

Underreferral rate 1.3%

Percent agreement 97.8%

* Abbreviations: START, Simultaneous Technique for Acuity and Readiness Testing; GCI, General

Cognitive Index.

at Viet Nam:AAP Sponsored on September 1, 2020

(4)

Indices for Prediction of Low Scorers (<68) on the Stanford-Binet From Failures on the START Test

* Abbreviation: START, Simultaneous Technique for Acuity and Readiness Testing.

TABLE 2. Definitions of Clinical Indices Criterion Test

Screening Test

Problem

Criterion Test

No Problem Total

Problem a b a + b

Noproblem c d c + d

Total a+c b+d a+b+c+d

Sensitivity (copositivity)* a/(a + c)

Specificity (conegativity)* d/(b + d)

Predictive value of a positive result a/(a + b)

Overreferral [b/(a + b + c + d)1 x 100

Percent agreement [c/(a + b + c + d)

* The terms co-positivity and co-negativity are sometimes preferred when the outcome criterion is

another test rather than a disease state.

TABLE 3. Study 2: Concurrent Validity for START Test With StanfordBinet*

START Score Stanford-Binet Total

<68 68-84 >84

Further testing needed (fail) (scores = 0, 1, 2)

Questionable (scores = 3) Pass (scores >4)

Total

16

1

0 17

41

26

17 84

17

64

177

258

74

91

194 359

Sensitivity (co-positivity) Specificity (co-negativity) Predictive value

Overreferral rate Underreferral rate Percent agreement

0.94 0.83 0.22 16.2%

0.3% 83.6%

Study 1: Concurrent Validation With the McCarthy Scales of Children’s Abilities

Procedure In the first study, START was administered to 352 preschool children in Person County, NC. Testers accompanied the children to the test sites while their parents stayed behind and completed questionnaires eliciting demographic information and their perception of their child’s abilities. The children received

hearing and speech/language screening in addition to START testing. The average time taken to complete and score the vision/ developmental screening procedure (START) for each child was 8 minutes (based on timing a random sample of 50 children).

START testers were college graduates who were recruited lo-cally and trained by an experienced tester. Training consisted of reviewing a manual and observing a demonstration on nursery school children of the appropriate age, followed by alternating as testers and observers for at least 7 children. After training,

reli-ability was assessed by testing at least 10 children while the most experienced tester/trainer served as the observer and criterion. All testers achieved reliabilities of .90 (Pearson r) agreement or better.

Six weeks later, a stratified sample of I I 8 children was selected to receive the McCarthy test (see Figure). The stratification proce-dure was designed to include all children scoring poorly (>1 SD below the mean) on either the cognitive or behavioral portions of the START, half of those scoring from I SD below the mean, to the mean and 12% of those scoring at or above the mean.

Criterion Test The McCarthy Scales of Children’s Abilities

(MSCA) is a standard diagnostic test of cognitive functioning for 2#{189}-to 8#{189}-year-old children.’4 The MSCA comprises five scales, three of which (Verbal, Quantitative, and Perceptual Performance) combine to yield a General Cognitive Index, or GCI. As noted earlier, concurrent validity studies have consistently shown that the GCI is moderately to highly correlated with IQ,16”7 and some studies suggest that the GCI is more sensitive to learning disabili-tie&8’19 although others fail to show any advantage.26’27 We have reported good prediction of concurrent#{176} and later school

out-comes by the MSCA.2 By second grade, 94% of children scoring less than 68 (<2 SD) on the preschool GCI and 74% of those scoring between 68 and 84 (<1 SD) on the GCI had failed a grade, been placed in a special education class, or scored in the bottom 20% on achievement tests.2’

Examiners who administered the MSCA all held academic de-grees in psychology and had clinical and academic training in test administration. All but one of the MSCA testers had completed a special training program including formal reliability checks in the use of the McCarthy test, followed by 1 year of clinical experience using the test nearly daily as part of a special North Carolina preschool program. The one examiner who had not received spe-cial training had extensive experience with the instrument. All testers were always blind to other test results.

Results Since a stratified sampling technique with unequal

pro-portions was used to select the sample, the resulting group could not be considered representative of the total population. In all analyses, subjects’ scores were weighted by a reciprocal function of the subject’s probability of selection to approximate the popu-lation distribution. Using the sample weightings, the average score on the MSCA was 93.6 with an SD of ±16.

Age-stratified item analyses were performed to determine op-timal item selection and scoring. A pass/fail criterion was estab-lished per age stratum for each item. The scoring system consisted of the number of items passed; the possible range of scores is 0 to

7. Cross-tabulation of the score with the GCI groupings (<68, 68 through 84, >84) indicated that the best prediction was obtained when the cutoff points were established at 0 to I point for a fail; 2 to 3 points for a questionable; and 4 or more points for a pass.

Total scores created for each child were highly predictive of the McCarthy CCI score, as seen in Table I. The sensitivity or

co-positivity, which reflects the proportion of children scoring poorly (<68) on the GCI who also failed the START test, was 0.76 (see Table 2 for definitions of all clinical indices). Specificity or co-negativity was 0.99, meaning that 99% of children scoring above

(5)

a positive result on the START test (ie, the proportion of those failing the START test who also scored poorly on the GCI) was 0.81. Underreferral was 1.3% and overreferral was 1.0%; percent agreement was 97.8%. The indices were calculated on the basis of a 2 SD cutoff score for comparison with most other validation studies in the literature and the clinical target population desig-nated for services by schools. However, the 3 X 3 tabulation depicted at the top of Table I also illustrates the extent to which screening results might be sensitive to the clinically significant borderline outcome group. It should be noted that all overreferred cases and the majority of the START questionable category cases were in that clinically significant McCarthy borderline (68 to 84) category.

Study 2: Concurrent Validation With The Stanford-Binet

Procedure This study was performed on a separate cohort of 362

preschool children in the same county in a different year. START was administered to all children by the trained, reliable testers described earlier. However, in this study, because of time con-straints, testers did not record vocabulary information on an item-by-item basis so only a total score was available for vocabulary information, requiring an adjustment of scoring. A stratified sub-sample was chosen using two independent tests of development (see Figure): the revised Denver Developmental Screening Test (DDST-R)2829 (administered by nurses who had demonstrated re-liability of >95% agreement) and a shortened version of the Mm-nesota Child Development Inventory (S-MCDI/’#{176}

(self-adminis-tered by parents).

These two independent tests of development were used to en-sure that the number of true-positives would be maximized. To avoid a sampling bias based on idiosyncrasies or unusual scores on the experimental procedure, the START test was not used as a selector for the subsample. The stratification procedure was as follows: (1) all children failing the DDST-R or those scoring low on the S-MCDI (>1 SD below the mean) were recalled; (2) 40% of those scoring intermediate or below normal on the S-MCDI (be-tween 1 SD below the mean and the mean) or those scoring in the “questionable” category on the DDST-R were recalled; and (3) 11% of children scoring above the mean on the S-MCDI and passing the DDST-R were recalled.

Criterion Test The Stanford-Binet Intelligence Test-1972

revision’3-is the descendent of the original tests used to measure intelligence. Scores on this test continue to be a recognized basis for classification of children for special educational purposes. A score of 2 SD below the mean (or 68) is used for the definition of mental retardation’5 and was therefore used here as an outcome criterion. Using the sample weightings, the average score on the Stanford-Binet was found to be 92.7 with an SD of ±17.4. A score of between I and 2 SD below the mean (or 69 to 84) is generally defined as borderline intelligence5 and was therefore selected as the borderline outcome for this study.

The Stanford-Binet was administered between 6 and 12 weeks after the initial screening to a total stratified sample of 120 chil-dren. Testers administering the Stanford-Binet all held academic degrees in psychology and had received academic and clinical training and supervision in the conduct of this test. No tester administered more than one single test to a given child and testers were always blind to results of other tests.

By current convention a parent’s report measure regarding adaptive functioning is also used to make the diagnosis of mental retardation. We did not use these measures because for research purposes we were not able to check the reliability of the parent report and our goal was to compare two types of child test-a screening measure and a standard diagnostic instrument.

Results With the Stanford-Binet as criterion and the scoring system of study 1, the analyses were performed on the weighted stratified sample. The cutoff score for classifying a START failure was adjusted slightly to compensate for differences in picture-card scoring. Sensitivity was 0.94 and specificity was 0.83; overreferral was 16.2%, with a rather low predictive value of a positive result (0.22). Yet the underreferral rate was very low (0.3%) and most overreferred cases were in the clinically important borderline cat-egory (see Table 3). The overall percent agreement was 84%.

DISCUSSION

The

initial

concurrent

validation

study

we

re-ported8

used

discriminant

analysis

to

generate

an

equation

for

optimal

prediction

of

group

classifica-tion

based

on

standard

diagnostic

developmental

as-sessment

(McCarthy).

However,

this

does

not

repre-sent

a

practical

scoring

method

for

clinicians.

The

simple

scoring

system

developed

in study

1 reported

here,

using

the

McCarthy

as criterion,

yielded

results

similar

to those

of the

initial

study.

A similar

scoring

procedure

(adjusted

for

differences

in coding

vocabu-lary

items)

with

a different

sampling

plan

and

a

dif-ferent

outcome

criterion

(Stanford-Binet)

in

study

2

produced

similar

classification

outcomes

but

with

a

higher

overreferral

rate.

However,

since

most

of the

overreferrals

were

in the

clinically

important

border-line

intelligence

group,

this

should

be

advantageous.

Taken

together,

the

results

of these

two

studies

pro-vide

further

support

for

the

concurrent

validity

of our

combined

vision

acuity/school

readiness

screening

procedure

now

called

“Simultaneous

Technique

for

Acuity

and

Readiness

Testing”

or

START.

As

we

noted

earlier,

the

time

requirement

appears

to be the

main

limiting

factor

to use

of developmental

screening

procedures

in pediatric

practice.

Since

the

START

procedure

actually

provides

a net

time

say-ings

(without

even

considering

the

developmental

data

it yields)

in a practice

already

offering

preschool

vision

acuity

screening,’2

it is uniquely

time-efficient.

Furthermore,

recent

review

of developmental

screen-ing

measures

for

possible

use

in

pediatric

practice

found

that

only

one

of the

“most

recommended”

mea-sures

for

preschoolers

was

as short

as 15 minutes,

and

none

of the

others

required

less

than

20 minutes

for

administration.3’

In addition,

none

of the

other

“most

recommended”

tests

provided

as

much

validation

and

reliability

data

or exhibited

predictive

indices

su-perior

to what

we

report

here.

The

initial

report

of

the

START

vision/

developmental

test

has

been

cited

by

others5

to

illus-trate

that

developmental

surveillance

using

opportu-nistic

observations

is a viable

alternative

to standard

developmental

screening

tests.

The

further

validation

studies

reported

here

provide

additional

support

for

developmental

surveillance.

It should

be

noted

that

the

prevalence

of low

McCarthy

and

Stanford-Binet

scores

were

somewhat

higher

in

our

region

(as

we

have

shown

in other

studies8)

than

in national

norm-ing

studies.

Replication

of concurrent

validation

stud-ies of START

in other

regions,

where

prevalence

rates

may

differ,

would

be

valuable.

However,

we

believe

that

enough

data

are

now

available

to

recommend

START

for

clinical

use.

Strict

adherence

to

the

protocol

described

in

the

manual,32

including

recommendations

for

training

and

reliability

checks,

is essential.

A failing

score

is

now

labeled

“Further

testing

needed,”

which

reflects

our

view

that

children

with

these

scores

should

be

referred

for

individualized

developmental/cognitive

evaluation

by

a child

psychologist

or

other

child

de-velopmental

specialist.

We

recommend

that

chil-dren

scoring

in the

questionable

child

has

(6)

adapted

to that

environment.

This

group

of children

does

not

always

require

immediate

referral

for

diag-nostic

testing,

but

that

option

should

be

seriously

considered

since

a

large

proportion

of

these

chil-dren

fall

into

the

borderline

group

in their

scores

on

diagnostic

tests.

The

START

procedure

is an

example

of objective

and

structured

opportunistic

observation,

not

global

impression.

While

most

pediatricians

claim

to use

ob-seryation

during

health

routines

to

identify

cases

of

developmental

delay,

these

global

observations

have

not

been

shown

to

be

reliable

indicators

of delayed

development.6’7

In contrast,

while

START

capitalizes

on

observations

made

during

a routine

health

pro-cedure,

the

observations

are

not

free-form

global

es-timates

but

specific

observations

of a carefully

struc-tured

and

field-tested

routine

carried

out

by

raters

found

to be

reliable

with

strict

adherence

to our

pro-tocol

for

administration

and

scoring.

We

believe

that

the

vision

acuity

test

is not

the

only

health

routine

that

can

be tapped

for

developmental/

behavioral

data.

It is our

conviction

that

nearly

every

health

routine

could

be

thoughtfully

restructured

to

highlight

potentially

clinically

useful

data

regarding

child

functioning.

We

have

previously

shown

that

ob-servations

during

a preschool

hearing

test

can

pro-vide

clinically

useful

developmental

data,9

and

we

are

currently

modifying

this

procedure

to assess

commu-nicative

functioning,

as

well

as

hearing

sensitivity.

With

reorganized

health

procedures

that

take

a

de-velopmental

approach,

it is likely

that

children

will

be

easier

to

examine

and

the

examinations,

therefore,

will

provide

more

reliable

health-related

data,

as

is

the

case

with

our

vision

acuity

procedure.

For

ex-ample,

the

physical

examination

of

a febrile

infant

could

be modified

to yield

data

or specific

behavioral

indicators

of

illness

rather

than

just

a

“gut

impres-sion”

of

whether

the

child

is

“toxic.”

Likewise,

in-fants’

and

parents’

reactions

to

health

maintenance

examination

procedures

might

provide

data

regard-ing

parent-child

interactions

and

infant

temperament

as

well

as

developmental

status.

Developmental

surveillance2’3’5

provides

an

oppor-tunity

for

pediatricians

to become

involved

in issues

of

child

functioning

and

become

more

expert

in

developmental/behavioral

diagnoses.

However,

it is

possible

that

this

term

might

be

used

for

purely

an-ecdotal

and

unreliable

assessments,

and

this

could

undermine

the

opportunity

for

pediatricians

to

be-come

credible

players

in helping

children

access

the

newly

available

developmental

and

family

interven-tion

services.

If developmental

surveillance

is to be a

viable

process,

pediatricians

will

need

an

organizing

framework

on

which

to anchor

their

“opportunistic

observations”4’5

(and

interviews).

Lack

of such

an

or-ganizing

framework

for

the

observations

could

be

compared

to

listening

for

cardiac

noise

without

an

orientation

to

the

framework

of S-I,

5-2,

etc.

START

screening

results

should

always

be

consid-ered

in the

light

of other

available

information

on

the

child,

and

especially

parent

concerns.

For

example,

a

questionable

result

in a child

whose

parent

has

con-cerns

should

certainly

lead

to further

evaluation

since

the

questionable

score

is less

than

reassuring.

In fact,

any

serious

concern

of the

parent

probably

deserves

more

than

a brief

screening,

if only

to

more

thor-oughly

understand

and

evaluate

the

specific

concern.

The

integration

of all

available

data

during

a health

checkup

(the

occasion

for

a vision

screen)

is an

ex-ample

of the

concept

of developmental

surveillance.

Within

this

context,

we

are

confident

that

START

will

be

a practical

aid

to pediatricians.

ACKNOWLEDGMENTS

This investigation was supported by grant MC-R-370427 from the Maternal and Child Health and Crippled Children’s Services Research Grants Program, Bureau of Community Health Services,

HRSA, PHS, DHHS.

Ms Nancy Carver and Ms Margaret Morris are acknowledged for their assistance as head screening testers. Mrs Madalou Wright helped with data tabulation and coordinated criterion testing. Dr Joanne Barton and Ms Marjory Albright provided critical assis-tance in the Denver screening and in coordination with the schools. The Person County Board of Education is also acknowl-edged for recruitment of parents and making space available for this joint project.

REFERENCES

1. Bain J. Developmental screening for preschool children: is it worth-while? I R Coil Gen Pract. 1989;39:133-137

2. Developmental surveillance. Lancet. 1986:950-951 .Editorial

3. Hutchison T, Nicoll A. Developmental screening and surveillance. Br

I

Hosp Med. 1988:22-29

4. Houston H, Davis RH. Opportunistic surveillance of child development in primary care: is it feasible? I R Coll Gen Pract. 1985;35:77-79 5. Dworkin PH. British and American recommendations for

developmen-tal monitoring: the role of surveillance. Pediatrics. 1989;84:1000 6. Bierman 1’ Connor A, Vaage M, et al. Pediatricians’ assessment of the

intelligence of two year olds and their mental test scores. Pediatrics. 1964;34:680

7. Korsch B, Cobb K, Ashe B. Pediatricians’ appraisals of patient’s intelli-gence. Pediatrics. 1967;27:990-1003

8. Sturner RA, Funk SC, Barton J, Sparrow 5, Frothingham TE. Simulta-neous screening for child health and development: a study of visual! developmental screening of preschool children. Pediatrics. 1980;65:614 9. Sturner RA, Green JA, Funk SC. Developmental functioning related to

preschool hearing screening. I Dcv Behav Pediatr. 1983;4:94-98

10. American Academy of Pediatrics, Committee on Children With

Dis-abilities. Screening for Developmental Disabilities. Pediatrics. 1986;78:

526-528

I I.Smith RD. The use of developmental screening tests by primary-care

pediatricians.

J

Pediatr. 1978;63:524

12. Sturner RA, Green JA, Funk SC, Jones C, Chandler A. A developmental approach to preschool vision screening. IPediatr Ophtha!mol Strabisinus. 1981;18:61-67

13. Terman LM, Merrill MA. Stanford Binet Intelligence Scale Manual for the

Third Revision Form L-M. Boston, MA: Houghton Mifflin; 1972

14. McCarthy D. A Manualfor the McCarthy Scales ofChildren’s Abilities. New York, NY: The Psychological Corporation; 1972

15. Sattler JM. Assessment of Children’s Intelligence. Philadelphia, PA: WB Saunders Co; 1974

16. Harrison KA, Lieve MJ. Correlational study of McCarthy, WISC, and Stanford-Binet scales. Percept Mot Skills. 1977;49:783-790

17. Davis EE, Walker C. McCarthy scales and WISC-R. Percept Mot Skills. 1977;44:966

18. Kaufman NL, Kaufman NL. Comparison of normal and minimally

brain dysfunctioned children on the McCarthy Scales of Children’s Abilities. I Cli,: Psycho!. 1974;30:69-72

19. Goh DS, Youngquist J, Lucenti J. Comparison of the McCarthy Scales of Children’s Abilities and the WISC-R. In: National Association of School Psychologists Convention Proceedings. Washington, DC: National Associa-tion of School Psychologists; 1978:239-240

20. Sturner RA, Funk SC, Green JA. Predicting kindergarten school

perfor-mance using the McCarthy Scales of Children Abilities. IPediatr Psycho!. 1984;9:495-503

21. Funk SC, Sturner RA, Green JA. Preschool prediction of early school

(7)

prior to school entry to achievement in kindergarten, first, and second

grades.

J

Sc/i Psycho!. 1986;24:181-194

22. PL 99-457. Education of the Handicapped Act Amendments of 1986 Title I Handicapped Infants and Toddlers. Washington, DC: House Congressional Board; 1986

23. Allen HF. A new picture series for pre-school vision testing. Am I

Ophthalmo!. 1957;44:38

24. Hughes WF. Office Management of Ocular Diseases. Chicago, IL: Yearbook Publishers; 1939

25. Brown MS. Vision screening of pre-school children. Clin Pediatr (P/iila). 1975;14:968

26. Naglieri JA. Comparison of McCarthy General Cognitive Index and

WISC-R IQ for educable mentally retarded, learning disabled, and nor-mal children. Psycho! Rep. 1980;47:591-596

27. Schmits DW, Flagg C, Simon J. Comparison of WISC-R and McCarthy scores of LD and regular classroom children. In: National Association of

School Psychologists Convention Proceedings. Washington, DC: National

Association of School Psychologists; 1978:256-257

28. Stumer RA, Horton M, Funk SC, Barton J, Frothingham TE, Cress JN. Adaptations of the Denver Developmental Screening Test: a study of

preschool screening. Pediatrics. 1982;69:346-350

29. Frankenburg WK, Fandel AW, Sciarillo W, Burgess D. The newly

ab-breviated and revised Denver Developmental Screening Test. I Pi’diatr. 1981;99:995-999

30. Sturner RA, Funk SC, Thomas PD, Green JA. An adaptation of the

Minnesota Child Development Inventory for preschool developmental

screening.

I

Pediatr Psycho!. 1982;7:295-306

31. Glascoe FP, Martin ED, Humphrey SA. Comparative review of

devel-opmental screening tests. Pediatrics. 1990;86:543-554

32. Sturner RA, Funk SC, Green JA. Simultaneous Screening for Acuity and Readiness Testing (START) Manual. Durham, NC: Duke University Medi-cal Center; 1990

BLACK

STUDENTS

BECOME

TARGETS

OF BIDDING

WAR

A financial-aid

bidding

war

for

top

black

students

has

broken

out

among

col-leges,

yielding

generous

scholarships

and

some

knotty

questions

of equity

in an era

of scarce

academic

resources.

Eager

to diversify

the

racial

profile

of their

student

bodies,

colleges

are

offering

unprecedented

sums

to

academically

talented

blacks,

admissions

officers

say.

Many

of the

awards

are

not

based

on

need

but

on

merit,

causing

some

colleges

to

questions

whether

the

trend

is diverting

aid

dollars

from

poorer

students.

“It’s

deeply

troubling

to me,”

says

Neil

Rudenstine,

Harvard

University’s

presi-dent.

“It’s

a situation

that

looks

like

it could

develop

into

a system

that

doesn’t

pay

attention

to the

real

needs

of families

and

students.”

Instead

of partial

scholarships,

awards

covering

all

costs-more

than

$20,000

at

some

private

school-are

becoming

more

common.

Putka C. Black students become targets of bidding war. The Wall Street Journal. October 7, 1992.

Noted

by

J.F.L.,

MD

(8)

1994;93;82

Pediatrics

Raymond A. Sturner, Sandra G. Funk and James A. Green

Concurrent Validation of an Aid for Developmental Surveillance

Simultaneous Technique for Acuity and Readiness Testing (START): Further

Services

Updated Information &

http://pediatrics.aappublications.org/content/93/1/82

including high resolution figures, can be found at:

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtml

entirety can be found online at:

Information about reproducing this article in parts (figures, tables) or in its

Reprints

(9)

1994;93;82

Pediatrics

Raymond A. Sturner, Sandra G. Funk and James A. Green

Concurrent Validation of an Aid for Developmental Surveillance

Simultaneous Technique for Acuity and Readiness Testing (START): Further

the World Wide Web at:

The online version of this article, along with updated information and services, is located on

been published continuously since 1948. Pediatrics is owned, published, and trademarked by the

Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it has

at Viet Nam:AAP Sponsored on September 1, 2020

http://www.aappublications.org/site/misc/Permissions.xhtml

http://www.aappublications.org/site/misc/reprints.xhtml