Eastern Illinois University Eastern Illinois University
The Keep
The Keep
Undergraduate Honors Theses Honors College
2014
Bilingual Narrative Assessment: Evaluation of the Kindergarten
Bilingual Narrative Assessment: Evaluation of the Kindergarten
Language Benchmark Assessment in Native Spanish-Speaking
Language Benchmark Assessment in Native Spanish-Speaking
Children
Children
Rebecca Hunt
Follow this and additional works at: https://thekeep.eiu.edu/honors_theses
Part of the Educational Assessment, Evaluation, and Research Commons, and the Language and Literacy Education Commons
Bilingual Narrative Assessment: Evaluation of the Kindergarten Language Benchmark Assessment in Native Spanish-Speaking Children
BY Rebecca Hunt
UNDERGRADUATE THESIS
Submitted in partial fulfillment of the requirement for obtaining UNDERGRADUATE DEPARTMENTAL HONORS
Department of Communication Disorders and Sciences along with the Honors College at EASTERN ILLINOIS UNIVERSITY
Charleston, Illinois
2014
I hereby recommend this thesis to be accepted as fulfilling the thesis requirement for obtaining Undergraduate Departmental Honors
s/ :J--/ 1J
' -� Date5/J_);y
7 1 Date..!;-- S- -/'(
Running head: BILINGUAL NARRATIVE ASSESSMENT
Bilingual Narrative Assessment:
Evaluation of the Kindergarten Language Benchmark Assessment in Native Spanish-Speaking Children
Rebecca Hunt Eastern Illinois University
I I B I LI N G U AL N A R RATIVE ASSESS M E NT
Abstract
The purpose of the current study was to evaluate the effectiveness of the Kindergarten Language Benchmark Assessment (KLBA) (Preschern & Konikoff, 20 1 3) in identifying at risk
narrative abilities in English Language Learners (ELLs) whose primary language is Spanish, as well as in monolingual English speakers. In addition, the study compared the effectiveness of the KLBA scoring system with the established measure of high point analysis for Spanish ELLs, and examined the performance of English monolinguals on the same assessment. The KLBA was administered in September, and again in January. The children' s narratives were scored using the KLBA measure, transcribed, and coded for elements and narrative pattern using high point analysis. Participants included nine children, three in a Spanish group and six in an English comparison group. Six of the nine participants, four English and two Spanish, passed the KLBA during both administrations. High point analysis results revealed that none of the participants in either group produced an age-appropriate narrative. The children who produced the least sophisticated narrative patterns failed the KLBA. Results suggested that the KLBA Narrative Story Retell Subtest effectively identified highly at-risk narratives, but did not consistently identify all at risk narratives. With further modification, the KLBA could be an effective screening tool for narrative language abilities in both monolingual and bilingual populations.
Ill B I L I N G U A L N A R RATIVE ASSESS M E NT
Acknowledgments
There are many people who have supported me in my undergraduate research endeavors, and who made this proj ect possible. First, I would like to thank Naomi Konikoff and Jennifer
Preschem for allowing me to study their new language benchmark. I would also like to thank the
parents of the study' s subjects for allowing their children to participate. Lastly, I would especially like to thank Dr. Angela Anthony for extending an invitation to take part in this exciting research, and for offering guidance and support throughout the process.
IV B I LI N G U A L N A R RATIVE ASS ESSM ENT
Table of Contents
Abstract . . . . .II
Acknowledgements . . . .. . . .. . . .. . . .III Table of Contents . . . .IV CHAPTER
I. Introduction . . . 1
II. Review of the Literature . . . .4
Response to Intervention . . . 4
Emergent Literacy and Narrative . . . ... . . 5
Narratives . . . 5
Eliciting and Assessing Narratives . . . 7
Typical Narrative Development. . . 10
Narrative Language in Bilingual Development . . . 1 1 Exposure to L 1 and L2 . . . 1 2 Linguistic and Cultural Differences . . . 1 5 Bilingual Impacts on Elicitation and Assessment. . . ... . . 1 7 Rationale for Study . . . 1 9 Research Questions . . . ... . . ... . . 20 III. Methods . . . .... . . .... . . ... . . 2 1 Research Design . . . 2 1 Participants . . . 22 Procedures . . . 22 Reliability . . . ... . . 23
v B I L I N G U A L N A R RATIVE ASSESS M E NT
Data Analysis . . . 23
IV. Results . . . 24
Kindergarten Language Benchmark Assessment Group Results . . . 24
High Point Analysis Group Results . . . 26
Individual Comparison Results . . . 29
V. Discussion . . . 3 1 Kindergarten Language Benchmark Assessment. . . 3 1 High Point Analysis . . . 33
Limitations/Future Research . . . 35
Conclusion . . . 36
VI. References . . . 3 7 APPENDICES A. Tables . . . .. . . .i
B. Consent Form Cover Letter . . . iv
C. Consent Form . . . .. v
B I LI N G U A L NAR RATIVE ASS ESS M E N T
Chapter 1 Introduction
1
The Hispanic population in the United States (U. S.) has been steadily i ncreasing since the 1 980s. From 1 980 to 2008, this population i ncreased from 6.4% to 1 5 .4%. This trend is
expected to co ntinue, with a projected 21 % o f the total U . S. population being Hispanic by 2025 (NCES, 20 1 0). According to the National Center for Education Statistics (NCES, 20 1 0), the number o f Hispanic children i n U.S. schools increased by as much as 38% in certai n regio ns between 2000 and 2008 . Approximately 23 . 1 % o f students attending American schools in 20 1 0 were Hispanic (NCES, 20 1 0). Also in 2010 , approximately 7.2 million Hispanic children from elementary to high school did not speak English as their primary language (NCES, 20 1 0). This increase in Hispanic population will mean that more bilingual children or English Language Learners (ELL) will be attending U. S. schools (Gildersleeve-Neumann, Kester, Davis, & Pefia,
2008). These statistics suggest a growing need to understand bilingual language development. One aspect of bilingual development that should be co nsidered i n language assessment is literacy and narratives. An oral narrative is a story consisting o f at least two related independent clauses separated by a "temporal juncture" in which a past experience is retold (Labov &
Waletzky, 1 967). According to Labov and Waletzky ( 1 967), a well-developed narrative will generally include orientatio n (e.g., setting and characters), complicatio n or conflict, evaluation (e.g., describing the overall point o f the story for the listener), resolutio n, and coda (e.g., transitioning to the present day, phrases such as "that' s all").
Children' s narratives can be elicited in many ways. A child may be asked to retell a story found in a wordless picture book or one told by the examiner. Spo ntaneous narratives can also be elicited by asking a child to describe an event he or she has personally experienced (McCabe &
B I LI N G U A L N A R RATIVE ASS ESSM ENT
Bliss, 2005). Narratives can also be analyzed in different manners (e.g., SALT analysis, high point analysis) and compared to developmental criteria in order to determine a child' s narrative proficiency.
2
Typical narrative development must be understood in order to determine a child' s level of proficiency or to identify the presence of a delay. Children begin to demonstrate the skills to provide consistent narratives around the age o f 4-years (Miller, Andriacchi, & Nockerts, 20 1 1 ).
Upon reaching preschool age, typically developing children begin to utilize the past tense, include true or fictional characters, and understand the difference between types of narrative structures (e.g., personal, expository) (Shapiro & Hudson, 1 99 1). A child may begin attempting
to provide more complicated narratives, including such elements as an established plot and an initial reaction, by the age o f 6-years ( Shapiro & Hudson, 1 99 1 ).
In order to accurately assess a bilingual child's narrative abilities, the examiner must be aware o f cultural and linguistic di fferences between the child' s primary language and the assessment language. There are many di fferences between narrative production in Spanish and English. For example, narrative instruction for children in Spanish speaking cultures emphasizes the importance of continuous conversational flow, while English American instruction focuses on the grammatical elements of storytelling (e.g., conflict, resolution) (McCabe & Bliss, 2005).
There are also grammatical differences between languages, such as different verb forms, that are used to describe past events.
In the school setting, the Response to Intervention (RTI) approach is used to identify children who may be in need of early intervention for academic achievement (Fuchs & Fuchs,
3 B I LI N G U A L N A R RATIVE ASS ESS M E NT
intensifies the intervention; i f a child does not respond to the intervention, he may be recommended for more direct special services (Fuchs & Fuchs, 2006).
In order to accurately determine an oral language difference, delay, or disorder in bilingual children, oral narratives must be e fficiently evaluated. The Kindergarten Language Benchmark Assessment (KLBA) is a recently developed screening tool, which includes a narrative story retell subtest. The current study investigated the e ffectiveness o f the KLBA in identifying at risk narrative language s kills in bilingual kindergarten children whose primary language was Spanish, and compared those b ilingual children' s narrative performances to a comparison group o f monolingual English -speaking kindergarten children. In addition, the study examined how results of the KLBA compared to a more in-depth high point analysis o f personal narrative.
B I LI N G UAL N A R RATIVE ASS ESS M ENT
Response to Intervention
Chapter 2 Review of Literature
4
The purpose of Response to Intervention (RTI) is to effectively identi fy at-risk children in schools and respond to observed de ficits with evidence-based intervention (Fuchs & Fuchs,
2006). Within the first month of the school year, a student' s academic performance is observed and evaluated. I f a child is observed to be below a specific level predetermined by the school, he or she is identified as at-risk. Once identified, the child will be observed during general
education over a short period of time, which is known as first tier instruction. At the end of this period, the child will again be assessed for improvement. If no improvement has been made, the child will receive more rigorous education in what is re ferred to as a second tier. Second tier instruction may be inside or outside of the classroom. A fter a determined amount of time, the child is again evaluated. If the child again demonstrates no improvement, he or she will be placed in a third tier, in which instruction is more intensive and may include a smaller group or one-on-one intervention outside o f the classroom. Thus, as a child progresses through the tiers, the intervention becomes more rigorous in its frequency, the sessions become longer, and the number o f students in a group decreases, giving children in the higher tiers more individual attention. Each higher tier also includes more specialized and qualified pro fessionals providing the intervention (Fuchs & Fuchs, 2006).
RTI is a dynamic assessment process. Through continuous observations and tracking o f progress, the pro fessional decides whether adjustments need to b e made in the instruction or intervention. This assessment also provides potential diagnostic information. RTI focuses intensely on reading deficits, particularly in early developing reading skills. This is because R TI
B I L I N G U A L N A R RATIVE ASS ESS M E N T
was developed to be compatible with No Child Left Behind (2002), which focuses on the importance o f reading and effectively distinguishing students who need more focused intervention (Fuchs & Fuchs, 2006).
Emergent Literacy and Narrative
5
Oral language includes vocabulary and narrative skills, which are necessary precursors for reading (Uccelli & Paez, 2007). These skills, known as emergent literacy, start to form in
early childhood and progress as a child observes and takes part in events related to literacy in a decontextualized social setting (Pence, 2 007). There are five areas o f emergent literacy, which, i f fully developed, can lead to advanced reading abilities. These five areas include vocabulary knowledge, narrative abilities, phonological awareness, print knowledge, and literacy motivation (Pence, 2007).
Emergent literacy is imperative in both monolingual and bilingual communicative development, as the precursor to future reading skills. The basis for emergent literacy is
decontextualized language, i.e., speaking about objects or events that are not immediately present or occurring in the moment. Narratives are, because they are decontextualized, an effective tool in the assessment o f a child' s emergent literacy and reading skills (Curenton & Lucas, 2007). Narratives
As defined by Labov and Waletzky ( 1 967), an oral narrative is at least two independent clauses that are related or separated by a temporal juncture. Informally, a narrative can be described as the manner in which a past event is recounted by linking an arrangement o f spoken clauses to the chronology o f the actual event. Although not always identical in structure,
narratives generally share a similar overarching form which includes orientation, complication, evaluation, resolution, and coda. Orientation includes elements such as the setting and characters,
6 B I L I N G U A L N A R RATI V E ASS ESS M E NT
which help the liste ner to understand the co ntext o f the story to come. The complication is the co nflict that presents an obstacle for the characters to overcome. Evaluation provides the listener with relevance; the speaker uses informatio n that emphasizes the overall poi nt o f the story that may be inferred by the listener. An evaluative statement does this by providing an internal or emotional respo nse from the character o f the story. The resolution typically comes after the evaluatio n, i n which the conflict is resolved. The coda may be added after the resolutio n, in which the speaker transitions to the present day by conti nuing the story i nto the current time (Labov & Waletzky, 1 967).
Accordi ng to Shapiro and Hudson ( 1 99 1 ), a narrative at its most basic level includes a beginni ng and e nd, setti ng, character depiction, dialogue, and actio n. There are also several elements that are i ntegral to narrative telling : goals, internal responses, obstacles, repairs,
cohesio n, and coherence. Goals are the end desires of the characters and are o ften grouped within internal responses, which are the characters' reactions to the co nflict o f the story. Obstacles are any conflicts that arise duri ng the story. Repairs refer to the resolutio n o f the narrative (Shapiro
& Hudson, 1 99 1 ) . Coherence and cohesion are also important elements o f a narrative, which are
shaped by the knowledge and skill level of the speaker (Shapiro & Hudson, 1 99 1 ). Coherence
refers to the ability to convey every element of the narrative in a structured manner so that these elements are significantly related to each other. Cohesion re fers to the linguistic re ference devices used to co nnect the elements of the story, such as i nterclausal co nnectives (e.g.,
conjunctions). These devices o f cohesion are a n integral component in the narrative structure, as they serve as part of the temporal juncture connecti ng the two independent clauses.
B I L I N G U AL N A R RATIVE ASS ESS M E NT
Eliciting and Assessing Narratives
There are different methods for eliciting a narrative sample. The first method is asking the child to retell a story he or she has been told by the examiner. Another method is using a wordless picture book. The child is shown a series o f pictures in a wordless book, and he or she is then asked to tell a story according to those pictures. A third method is a spo ntaneous
narrative, in which the child is asked to recall a personal experience. McCabe and Bliss (2005) argue that the spo ntaneous narrative is the truest represe ntatio n of a child' s narrative and language abilities because the clinician has not provided any phrases or words for the child to use. The child must generate the narrative from his or her own experience and abilities.
7
Systematic Analysis o f Language Transcripts (SALT) (Miller, Andriacchi, & Nockerts,
20 1 1 ) is a so ftware program used as a tool i n the evaluation o f elicited language samples. SALT i ncludes a number o f normative databases that can be used as a comparison. According to SALT a nalysis procedures, narrative elicitatio n must be in one o f two forms: both the examiner and the child know the co ntent o f the sample, or the child knows the content while the examiner does not (Miller et al. , 2 0 1 1 ). When the clinician chooses the topic o f the sample, he or she is able to follow specific vocabulary and content in more detail. When the child is completely responsible for generating the content, he or she may be motivated to use more specific vocabulary and exte nsive utterances. Miller et al. (20 1 1 ) believe that an event narrative, which is description o f a life experience, may be an effective type o f narrative for analysis.
The materials used to elicit the narrative may influence event narratives. O ne study found that problem-based booklets (i.e., a series o f related pictures formi ng stories) facilitated
production o f a more coherent narrative than a booklet without a problem. Shapiro and Hudson ( 1 99 1 ) conducted a study in which two types of event booklets were used to elicit narrative
8 B I LI N G UAL N A R RATIVE ASS ESS M E N T
samples from preschool and first grade children. When given the event-based booklet, the children did not generally include obstacles or repairs. Also, more dialogue and actions were used by children who viewed the event-based pictures and by children who were able to preview the pictures beforehand. This indicated that the children concentrated more on the action o f the story and described the characters in fuller detail with an event-based story or a previewed story, as compared to a problem-based story. Although not all of the children in the study were able to provide a conflict and resolution element, they did create an interesting and entertaining narrative through descriptions, many of which were taken from real-life events experienced or observed by the children. In assessing the episodic components (e.g., goals) of the narrative samples, the first graders provided more goals than the preschoolers, but both age groups were able to explain internal responses o f the characters in the problem-based story. This demonstrated that younger children were capable o f providing an effective narrative, even though the structural elements were not yet developed.
In addition to the SALT databases, high point analysis may be used to assess personal narrative samples. High point analysis utilizes the categories laid out by Labov and Waletzki ( 1 967), in addition to those defined by Shapiro and Hudson ( 1 99 1 ). Expanding on this work, Peterson and McCabe ( 1 983) explained that clauses within the narrative sample are separated into one o f the six categories: abstract, complicating action, orientation, evaluation, resolution, and coda. The abstract is a short summary of the narrative as an introduction. A complicating action is any event arising within the story from its outset until reaching its climax, or high point. Orientation is the description of the story' s setting (e.g., place, time, and characters). Evaluation is the explanation for telling the story, or the narrative' s point or significance. Resolution is the event that follows the high point o f the story, resolving the conflict. The coda is the story's
9 B I L I N G U A L N A R RATIVE ASS ESS M E NT
conclusion, o ften connecting the story's key points together. A skilled speaker may bring the listener from the past and back to the present in the coda. Independent clauses (i.e., a clause consisting o f a subject and a verb) may fall under any of the six categories, while dependent clauses (i.e., a clause consisting of either a subj ect or a verb) will always be orientation or evaluation. There are two manners in which the narrative may be organized: the sequential order in which the event happened, and the sequence o f events as retold by the speaker. It is acceptable for the sequence o f the story to deviate from the actual chronology o f events in order to bring suspense to the story.
There are seven patterns of organization described in high point analysis: classic, ending at-the-high-point, leap-frogging, chronological, impoverished, disoriented, and miscellaneous patterns. In a classic narrative pattern, hi gh points are identified when the action o f the narrative is suspended in order to place emphasis on the climax o f the story (Labov & Waletzky, 1 967).
However, high points can be found in many sections of a narrative, depending on the organization the speaker has chosen to u se in the storytelling. Another manner o f narrative organization described by Peterson and McCabe ( 1 983) is the ending-at-the-high-point. In this pattern, the high point is reached and emphasized, as in the classic pattern, but the story immediately ends. These two patterns are considered to be complex in nature; however, other patterns are categorized as simpler (i.e., the leap-frogging, the chronological, the impoverished, the disoriented, and the miscellaneous patterns).
The leap-frogging pattern occurs when a child skips from section to section in the story, leaving out important parts, making it difficult for the listener to understand what originally happened. In the chronological pattern, a simple description of chronological events is given, using only a simple a-b format (i.e., simply stating the events as they successively happened).
10 B I LI N G U A L N A R RATIVE ASS ESS M E N T
The impoverished pattern may include the two consecutive clauses as required to formulate a narrative, but contain so little description that a subj ect matter or arrangeme nt cannot be
identified. The speaker will then co ntinue to repeat these two ideas several times. The disoriented pattern occurs whe n the narrative structure is befuddled and confused to the point where the listener simply cannot comprehend the story. Any narrative sample that could potentially fit i nto one or more o f these categories would be described as miscellaneous. Narratives that are
completely fiction are also included in the miscellaneous category (Peterson & McCabe, 1 983). Typical Narrative Development
According to Miller, Andriacchi, and Nockerts (20 1 1 ), children accrue skills that are needed to begin engaging in conversation through the age o f 4 years. Although narratives begin to take shape between the ages of 3 and 4 years, children's narratives are not typically co nsistent
until the age o f 4 (i.e., preschool). After this, a variety of narrative types begin to appear, manifesting i n different forms, such as personal narratives, retelling of stories, and expository
narratives. Preschoolers can start to use other structures (e.g., expositions) along with narratives, use the past tense, non-real or fictional characters, and traditional language used in the telling o f fairy-tales ( Shapiro & Hudson, 1 99 1 ). Although not generally successful, children begin to make
an effort to establish plot in narratives by the age o f 6 ; they also start to describe a relevant setting, develop goals for characters, and provide i nitial action (Shapiro & Hudson, 1 99 1 ) .
Within high point analysis, the two-event structure should occur between the ages of 3 a nd 4-years, the leapfrog at 4, endi ng-at-high-point at 5 years, and classic at 6 . The chronological structure may occur at all ages. The impoverished, disorie nted, and miscellaneous structures do not occur at any particular age (McCabe & Rollins, 1 994; Peterson & McCabe, 1 983).
1 1 B I LI N G U AL N A R RATIVE ASS ESSM E N T
Shapiro and Hudson ( 1 99 1 ) created a four-level organizatio nal system for describing the narratives elicited. The first level was co mposed o f a simple story with action, orientatio n, or both. The seco nd level consisted o f the utilization o f o ne episodic component, and the third level contai ned two. Level four consisted of the elements described i n the first three levels. The results o f the study i ndicated that the first graders were able to better organize the story than the
preschoolers, as well as include more episodic components. However, the preschool group had the skill to use some o f these components whe n the pictures in the event booklet were organized more sequentially. The study demo nstrated that preschoolers had the ability to use the conflict and resolutio n structure in their narratives when given sequential pi ctures. In additio n, they were able to provide inner thoughts and feeli ngs of the characters, and provided the listener with the context o f the story by offeri ng a setti ng and character description.
Narrative Language in Bilingual Development
Proficiency in both the native language (L l ) and the second language (L2) may be difficult for the bilingual child, who is simultaneously attempting to learn two different
languages. A bilingual child may also face challenges i f he or she is not proficient in the primary language when the seco nd language is bei ng developed. I n addition, li nguistic or cultural
differences might prove to be an obstacle for learning the second language. Cultural and
linguistic differences must be identified before proper diagnoses o f oral language deficits can be made. Grammatical differences can change the manner in which the same story is told i n two different languages. A child' s narrative may reflect his or her individual culture, rather than what is expected o f a typical story told in American English (McCabe & Bliss, 2005). For example,
12 B I LI N G U A L N A R RATIV E ASS ESSM E N T
their children, rather than specific narrative components, such as an initiating event or a resolving event (McCabe & Bliss, 2005).
Decontextualized oral language (i.e., language used in everyday situations) is also an important factor in bilingual development. When primary language skills in deco ntextualized e nvironments are poor as a result of lack of vocabulary exposure in the primary language, children may not be able to translate from LI to L2 in oral or writte n language. It is far easier for a child to comprehe nd a new name in L2 for an already existi ng concept in L 1 , rather than understand an entirely unfamiliar concept (Cummins, 1 979). In addition, narratives may vary depending o n elicitation procedures used (Gutierrez-Clellan, 2002).
McCabe and Bliss (2005) advocate that personal narratives, rather than narratives elicited by picture books, should be used in assessment o f a bilingual child. As with English speakers, a picture book may mask a child's true discourse level; the child could exhibit advanced discourse skills during assessment by simply highlighting what is in the picture. A personal narrative comes from the child himself, and will be a more accurate depiction of the child's language and discourse abilities (McCabe & Bliss, 2005). Therefore, it is important to consider a bilingual
child' s length and type o f exposure to L I a nd L2, cultural variatio ns, and linguistic differences.
Exposure to Ll and L2. Hammer et al. (20 1 2) found that the length a nd timing of exposure to LI and L2 was a significant co ntributor to bilingual development. In the study, 1 9 1 Lati no families completed the Background and Language Questio nnaire, which provided i nformatio n o n the children' s exposure to both Spanish and E nglish, as well as parental status and demographic characteristics. Subtests i n the Woodcock-Munoz Language Survey- Revised, which i ncluded vocabulary and story recall in both English and Spanish, were admi nistered to the participating children to gauge their vocabulary and story recall abilities i n both Spanish and
13 BIL I N G U AL N A R RATI VE ASSESS M E NT
English. Performance on the subtests depended on many factors (e.g., age o f exposure,
conversational partners) which affected the children' s oral language proficiency in either Spanish or English.
The results of this study found that the longer a child lived in the United States, the higher his or her English vocabulary, substantiating that residing in a primarily English-speaking region helped increase English vocabulary and proficiency. Exposure to only Spanish during infancy and to English once in school also appeared to provide a foundation for a child' s Spanish proficiency. On a larger scale, the study revealed the potential importance and timing o f
exposure to the secondary language i n bilingual children and the level o f impact the exposure may have on the development o f both L l and L2 (Hammer et al., 20 1 2) .
Gutierrez-Clellen (2002) found that sampling a child's oral narrative production may reveal academic preparedness in language, as well as measure progress in expressive language in bilingual children. The study included 33 typically developing 7 and 8-year-old children o f Mexican-American or Puerto Rican descent. The participants were prompted t o produce
spontaneous narrative samples from two different wordless picture books and recall two stories. Twenty-eight of the 33 participants were being educated in both Spanish and English, while the remaining children received instruction only in English. All but one child was born in the United States, and parent education ranged from less than 6 years to more than 1 3 years.
The wordless picture book Frog, Where Are You (Mayer, 1 969) was used to elicit the English narrative, while Frog Goes to Dinner (Mayer, 1 97 4) was used for the Spanish narrative; both samples were used to assess each child' s proficiency in English and Spanish. The students were then asked to recall two stories: The Tiger's Whisker (Stein & Glenn, 1979) in English, and
B I LI N G U A L N A R RATIVE ASSESS M E NT
and inferential questions (e.g., why and what i f), were asked in a story comprehension task. Transcriptions o f the story recalls were completed in the SALT program and segmented into events: setting, initiating event, internal response, attempt, direct consequence, and reaction (Gutierrez-Clellen, 2002). Story comprehension answers were also transcribed.
14
The study found that all participants struggled more with the comprehension and recall of the Spanish stories than with English stories. However, the students providing Spanish samples generally performed age-appropriately. Seven of the students who had previously appeared to be fluent in both languages performed below average in both English and Spanish. The greatest differences across languages were found in the narrative recall task. Most children tended to perform better in English than in Spanish. Most o f the children were able to produce coherent narratives that included important events and consequences. Many children struggled in recalling a story, focusing so much on remembering all o f the elements and the complex grammar that they tended to omit important elements of the story. Common errors included the omission of the important events, effects from the story's conflict, and the addition o f superfluous information.
Differences were not as prominent in the narrative production. When the children were asked to spontaneously provide a narrative sample based on the wordless picture books, important events and consequences were included. Most of the participants exhibited better narrative recalls in English than in Spanish. Thus, Gutierrez-Clellen (2002) found that children who are bilingual may not perform equally on a narrative proficiency task in both languages. Most of the children were able to generate adequate grammar, even at the most limited level of proficiency, in both languages. This demonstrated that the children were able to transfer their knowledge o f one language to successfully use the second. A majority o f the students performed
1 5 B I L I N G U A L N A R RATIVE ASSESS M E NT
better in their seco nd language, English, which could be because English is the greater focus in bili ngual classroom programs.
Linguistic and cultural differences. Accordi ng to Melzi (2000), the goals of discourse between a child and parent are particular to the "cultural beliefs, values, and expectations of a community." Latino mothers place a considerable amount o f emphasis on helping their children become bien educados (Melzi, 2000). Although this term is literally translated as "well
educated" i n E nglish, Lati no mothers place another meaning on the term. Bien educado does not only mean well educated, but it also means well versed i n conversation and social expectations. Therefore, Latino mothers place more importance on effective discourse skills, while European American mothers place more emphasis on the actual organization of a narrative. I n addition, Hammer et al. (20 1 2) found that mothers appeared to be i nfluential in recall tasks. Mothers and teachers who read to their children in English helped to build a basis for the child's developing
narrative skills. However, children tended to believe that fathers and teachers valued speaking English more than mothers. Therefore, the child' s conversational partner played a role in the development o f L 1 and L2.
Melzi (2000) conducted a study with 3 1 mothers and their preschool children born in Central America who spoke predominantly Spanish in the home and their preschool children, and 1 5 native-born European American mothers who spoke o nly English in the home, none o f which had a ny k nown communication disorder. The participants were visited in their homes by a bilingual researcher and mothers were asked to converse with their children, discussing recent events in the children' s lives. A total o f four events were discussed: two about shared
16 B I LI N G U A L N A R RATIVE ASSESS M E NT
asked the mothers to start a conversation about one particular experience at a time, and to refrain from drawing from films or stories.
The narrative samples were analyzed using four different categories: quantitative
narrative measures, types o f prompts, conversational functions o f prompts, and types o f narrative information (Melzi, 2000). The quantitative narrative measures analyzed the mothers'
conversation using number of words and mean length o f utterance (i.e., the average number of words utilized in one utterance). Types of prompts used by the mothers included closed-ended questions (i.e., a question receiving a yes or no response), open-ended questions, (i.e., wh questions), memory prompts (e.g., reminding the child that something happened in the recent past and asking about it), statements (e.g., giving the child specific information), and other, which included elements of the discourse that were non-narrative. Conversational function of prompts included initiations, elaboration, maintenance through repetition, maintenance through
other devices (e.g., using interactional markers) and other functions, or guiding the child through the conversation. Narrative elements included description, event, evaluation, reported speech, and generic (e.g., an utterance that continued discourse, but did not fit into any other category for the mothers' speech).
Melzi (2000) discovered no significant differences in the length of discourse between the mothers o f the two cultures. In discussing shared events, both the Central American and the European American mothers utilized more statements. To continue a narrative, however, the Central American mothers tended to use open-ended questions, while the European American mothers chose to use closed-ended questions when discussing an event only the child had experienced. A cultural difference in the construction of a narrative was demonstrated when the European American mothers emphasized a chronological, single event in their children' s
17 B I LI N G U A L N A R RATIVE ASSESS M E NT
narratives, while the Central American mothers asked their children to transition from event to event to create a whole narrative. This indicated that the European American mothers were more active i n building the narrative, while Central American mothers were less directly involved in building the child' s narrative. Overall, the Central American mothers placed more emphasis on listeni ng to their children' s narratives than constructing them. (Melzi, 2000). In e ffect, the Central American mothers, unlike the European American mothers, played a mi nor part in co nstruction o f the narrative. Melzi (2000) explained that European American mothers
participated as co-narrators, while Central American mothers were active listeners. A factor in the different narrative structures was family size. Latino households were generally larger, including extended family members.
Bilingual impacts on elicitation and assessment. Narrative sampli ng that targets vocabulary i n assessment may provide the speech-language pathologist with clearer evidence o f a monolingual o r bilingual child's emergent literacy skills (Uccelli & Paez, 2009). Obtaining a
narrative sample o f spo ntaneous language can complement and justify the diagnosis suggested by standardized measures (Rojas & Iglesias, 2009).
Uccelli a nd Paez (2007) administered standardized tests to 24 typically developing ki ndergarte ners from Spanish-speaking homes o f low socio-eco nomic status (SES) . Each child was tested i n both Spanish and English at the e nd o f kindergarte n, and again at the end of the first grade. Assessments were admi nistered in both English and Spanish to each participant. Because two di fferent languages were bei ng assessed, o nly the language being tested at the time was spoken to the participants to discourage them from code switching (i.e., shi fting from one language to the other in a single utterance). The study examined narrative skills by eliciti ng a spo ntaneous narrative sample from sets of pictures. While the pictures did convey a broad theme
18 B I L I N G U AL N A R RATIVE ASSESS M E NT
or plot, they were general enough so that the children were encouraged to expand upon the plot with spontaneous details. Each child was asked to look at the pictures, and then describe a story related to the pictures. These narratives were analyzed using a scoring scheme in both Spanish and English. The Picture Vocabulary subtests o f the Woodcock Language Pro_ficiency Battery -Revised (WLPB - R) were also administered in both Spanish and English.
Results o f the study indicated that children with "higher story scores in Spanish tended to have higher story scores in English" (Uccelli & Paez, 2007, p. 23 1 ). All vocabulary scores also
increased from kindergarten to first grade in English. While code switching did occur, it was infrequent and the majority occurred when switching from Spanish to English. The bilingual subjects in the study performed lower than average on all assessments during both the
kindergarten and first grade sessions. In addition, vocabulary and narrative stories in Spanish can impact later English development. Uccelli and Paez (2007) found that "Spanish story structure predicted first-grade English vocabulary and English narrative productivity" (p. 232). Telling stories and participating in Spanish conversations with family and friends can be beneficial to the bilingual child's English language development. Because kindergarten and first grade are a critical time for literacy development, results suggested that problems with reading could occur in later grades (Uccelli & Paez, 2007). The study found a positive, but moderate, relationship
between vocabulary and narrative skills in typically developing bilingual children. This research suggests that, although developing at different rates, vocabulary and narrative skills may a ffect each other' s progress.
This study raises a question regarding whether monolingual measures for narrative sampling were appropriate for bilingual children. Because the method of eliciting the sample
19 B I L I N G U A L N A R RATI V E ASSESS M E NT
may affect the child's narrative itself, Uccelli and Paez (2007) suggest that further research needs to explore the accuracy and efficiency of bilingual narrative assessment.
Rationale
Coherent and meaningful oral narratives are expected at the kindergarten level. Oral narratives reflect a child' s ability to convey ideas and events in an organized manner, and are indicative o f the child' s language proficiency. In addition, narratives are effective tools to predict later literacy skills. The ability to accurately screen and assess bilingual children is integral to their academic success. Diagnosis o f a communication difference, delay, or disorder cannot be effectively determined i f the clinician does not understand a child' s proficiency in both his native language (LI) and secondary language ( L2). A narrative produced by a bilingual child may also be a product o f one's own cultural standards for storytelling, rather than the standards o f the language in which assessment takes place (McCabe & Bliss, 2005). Remaining unaware of these
cultural and linguistic differences could result in an inaccurate diagnosis o f a communication delay or disorder in a bilingual child.
School clinicians must effectively and accurately assess a large number of children within each tier o f the R TI system. It is also important to screen numerous students in a short period of time, so as not to interfere with classroom time. The Kindergarten Language Benchmark
Assessment (KLBA) is a screening tool that may be administered in a short amount o f time as compared to others, and therefore, may be an e fficient assessment for the school setting. Although pilot data has suggested that the KLBA is an effective progress-monitoring tool (Preschem & Konikoff, 20 1 3), the KLBA narrative story retell subtest has yet to be validated in
regard to its effectiveness in identi fying bilingual students with di fficulties in narrative
B I L I N G U AL N A R RATIVE ASSESS M E NT
in determining at risk narrative abilities in English Language Learners (ELLs) whose primary language is Spanish, as well as in monolingual English speakers. In addition, the study seeks to compare the effectiveness of the KLBA scoring system with the high point analysis system for
Spanish ELLs, and to examine the performance o f English monolinguals on the same assessment. The following research questions will be examined:
I . Does the KLBA narrative story retell subtest effectively identify at risk children in narrative ability, when compared to high point analysis for:
a. English-speaking kindergarten children? b. Spanish-speaking kindergarten children?
20
II. What are the relative strengths and weaknesses o f the KLBA narrative story retell subtest?
B I L I N G U A L N A R RATIVE ASSESS M E NT
Research Design
Chapter 3 Methods
This descriptive study included between and withi n-subj ects compariso ns o f
kindergarteners' performance when telling a personal narrative. The between-subjects analysis compared narrative skills of a group of E LLs whose primary language was Spanish, to a comparison group o f English monolinguals. The within-subj ects comparison exami ned two methods o f analyzing participants' narrative skills, the KLBA narrative story retell subtest and high point analysis.
Participants
2 1
All kindergarten students atte nding an elementary school in a Chicago suburb were screened during September 2 0 1 3 and January 20 1 4. In order to be included i n the Spanish group o f the current study, children needed to be ELLs with Spanish as a primary language, while also demo nstrating some level of proficiency in English. Typical cognitive abilities were also
required, assessed using teacher report and previous school records. Participants in the comparison group were randomly selected from the English mo nolingual ki ndergarten populatio n. The English monolinguals also needed to exhibit typical cognitive abilities. Nine children participated in the study, six girls and three boys. Three were placed in the ELL group, and six were placed in the monoli ngual comparison group. The average age o f the participants was 5 years, 6 months. Within the E nglish group, three participants identi fied as White, two as Black/ A frican American, and one as Asian/Pacific Islander. Malayalam was the primary language spoken i n the home o f the Pacific Islander participant, but the child identified as a monoli ngual English speaker. The three Spanish group participants identified as ELL with a
B I LI N G U A L N A R RATIVE ASSESS M E NT
primary language o f Spanish. Five o f the nine participants qualified for free or reduced lunch, includi ng all three o f the Spanish group participants. All students were placed in regular kindergarte n classrooms.
Procedures
Approval was granted by the Institutio nal Review Board at Eastern Illinois U niversity.
22
Before screening commenced, consent was obtained from kindergarteners' parents or legal guardia ns . Consent forms were provided in Spanish for parents who were not sufficiently proficient in E nglish. All participants were assessed using the Kindergarten Language
Benchmark Assessment (KL BA) i n September 20 1 3 . Performance o n the KL BA narrative story retell subtest was exami ned in this study. To elicit the narrative, the clinician used o ne o f two prompts: "I want you to tell me a story; Ca n you think o f a time when . . . . " Themes prompted i ncluded: a time you or someone you know got hurt, being stung by a bee, needing a Band-Aid, or getti ng a new pet. If the child responded that he or she had experienced o ne o f these events, the clinician prompted, "Tell me what happened. " One prompt was permitted during a retell: "Tell me more." When the child finished his or her story, the clinician reviewed a series of "wh" questions about characters, setting (i.e., time and place), co nflict, and resolution (i.e., who, when, where, what, and e nding). One poi nt was awarded when the child' s narrative answered that question, while a question unanswered received a score o f zero. For this study, narratives were co nsidered appropriate i f they scored at least four out o f the five on the subtest. A subsequent a nalysis analyzed usi ng five out o f five as the requirement for an age appropriate narrative.
Each narrative sample was audio recorded, transcribed using the SALT program, and then coded based o n high poi nt analysis. Each utterance within a child' s narrative was coded according to narrative elements (i.e., introduction, orientatio n, complicating action, evaluation,
23 B I LI N G U A L N A R RATIVE ASS ESS M E NT
resolution, and coda). The narratives were then categorized by pattern o f structure (i.e.,
disoriented, impoverished, leapfrog, two-event, chronological, ending-at-the-high-poi nt). Classic structure was not i ncluded because this type of narrative is not typically expected u ntil first grade. An e ndi ng-at-the-high-poi nt narrative pattern was co nsidered age-appropriate for the purposes o f the current study.
Reliability
All samples were transcribed by an undergraduate speech-language-pathology student, and then reviewed by a certified speech-language pathologist to evaluate i nter-rater reliability. The same student and speech-language-pathologist also independently coded each narrative sample, and the n compared codes to resolve any discrepancies for 1 00% o f the samples.
Data Analysis
Scores from the KLBA were summarized for E nglish speakers and Spanish speakers. Scores from the Spanish group were compared to scores from the English group to examine similarities and differences in performance on the KLBA narrative story retell subtest. Qualitative analysis was used to examine items missed o n the KLBA by English and Spanish speakers to determi ne whether items were more frequently missed. In addition, individual performance for each student was examined to compare KLBA scores to results o f high point analysis. I ndicatio ns of delay, difference, or disorder were also examined based o n
24 B I L I N G U A L N A R RATI V E ASSESS M E N T
Chapter IV
Results
The purpose o f the current study was to examine the effectiveness o f the KLBA narrative story retell subtest i n identifying both monolingual English speaking and Spanish speaking ELL children at risk for narrative language difficulties. Subtest scores were compared to high point analysis narrative patterns to exami ne the KLBA ' s effectiveness i n screeni ng narrative ability. A certified speech-language pathologist or graduate stude nt in speech-language pathology collected data duri ng two separate testing sessions in September 20 1 3 and January 20 1 4. Each
participant's narrative was scored usi ng KLBA measures, transcribed, and coded using high poi nt analysis. Results are prese nted by language group, as well as by i ndividual performance on the KLBA and high point analysis.
Kindergarten Language Benchmark Assessment Group Results
Narratives were analyzed with the KLBA by determi ning whether the narrative addressed a series o f 'wh' questio ns (i.e., who, where, when, what, endi ng). The narratives of six out of the nine participants met the criteria o f includi ng narrative elements that addressed four out o f five questions. The narratives o f two out o f the three Spanish participants met the KLBA narrative story retell subtest criteria, while four out o f the six English participants met the criteria. When the criteria were raised to five out o f five questions, none of the participants met the criteria. All participants i n both groups answered the 'what' question. All three Spanish participants '
narratives answered the 'who' question, as did all but one narrative in the English group.
Elements fulfilli ng the questions 'where,' 'when, ' and 'ending' were used by approximately half of the participants overall. Only one Spanish participant and three out of the six English
25 B I LI N G U AL N A R RATIVE ASSESSM E N T
participants fulfilled the 'when' response. Similarly, half o f the English participants' narratives addressed the ' endi ng' response. Two out o f the three Spanish participants fulfilled the ' e ndi ng' response. Table A provides summary results of KLBA data elicited in the fall (i.e., September 20 1 3) .
Table A. Fall Number o f Participants Who Met KLBA Narrative Story Retell Subtest Criteria WHO WHERE WHEN WHAT ENDING Criteria Criteria
Met Met
(4/5) (5/5)
TOTAL 8/9 519 419 919 519 619 019
ENGLISH 516 316 316 616 316 4/6 016
SPANISH 313 213 1/3 313 213 213 013
At the winter testi ng (i.e., January 2 0 1 4), the number of participants who met the criteria o f the KLBA narrative story retell subtest remained co nstant; four out o f six English participants met the criteria by fulfilling at least four out of five responses, and two out o f three Spanish participants fulfilled at least four responses. However, only o ne English-speaking participant met the five out o f five criteria. All participants in both groups a nswered the ' who' question (i.e. an i ncrease o f one English participant from the fall). All English group participants answered the ' what' with their narratives, as did two out o f the three Spanish group participants (as compared to all participants meeting criteria in the fall). Overall use o f the ' e nding' response increased from five to seven out of nine i nc luded a resolution was i ncluded by all participants except one i n the E nglish group and one in the Spanish group. Use o f ' where' and ' when' remai ned co nstant. Table B provides summary results o f wi nter KLBA data.
26 B I L I N G U A L N A R RATIVE ASS ESSM ENT
Table B. Winter Number o f Participants Who Met KLBA Narrative Story Retell Subtest Criteria WHO WHERE WHEN WHAT ENDING Criteria Criteria
Met Met (4/5) (5/5)
TOTAL 919 419 419 819 7/9 619 1 /9
ENGLISH 616 316 316 616 516 416 1 /6
SPANISH 3/3 1 /3 113 213 213 213 013
High Point Analysis Group Results
Narratives were transcribed and coded using high point analysis as a compariso n for the KLBA results. Each utterance o f a narrative sample was coded accordi ng to narrative elements (i.e., i ntroductio n, orientation, complicating actio n, evaluation, resolutio n, and coda). The overall narratives were then categorized by structure pattern (i.e., disoriented, impoverished, leapfrog, two-event, chronological, ending-at-the-highpoint). No ne o f the children produced an age-appropriate narrative pattern (i.e., ending-at-the-highpoint) during the fall or winter testi ng sessio ns . The most sophisticated narrative pattern produced was the chronological pattern, which does not occur at a ny particular age, but includes a simple descriptio n o f successive events. During the fall testing session, four of the nine participants produced a chronological narrative. The mo nolingual E nglish group produced three o f the four chronological narratives. The least sophisticated pattern produced by two participants in the English group in the fall was
impoverished; these narratives did not include enough utterances to establish a high poi nt, or they repeated two events multiple times. O ne Spanish participant produced the least
sophisticated pattern for the group, which was a two-event pattern. This i ndicated that the narrative co nsisted of o nly two different narrative elements (McCabe & Rollins, 1 994; Peterson
27 B I L I N G U A L N A R RATI V E ASSESS M E N T
& McCabe, 1 983). The remaining narrative by a Spanish participant was classified as a leapfrog
pattern. Table C provides summary results o f fall high point analysis narrative pattern data.
Table C. Fall Number of Participants Using Narrative Patterns
Disoriented Impoverished Two Leapfrog Chronological Event
TOTAL 0 2 2 1 4
ENGLISH 0 2 1 0 3
SPANISH 0 0 1 1 1
During the winter testing session, five o f the nine participants produced chronological patterns, the most sophisticated pattern p roduced by any of the participants. O f those five, four were in the monolingual English group. The least sophisticated pattern in the winter session, produced by one Spanish and one English participant, was disoriented. This indicated that the narrative was not coherent enough to establish any meaning or point. The English group generally improved narrative pattern or produced consistent patterns from fall to winter. The Spanish group generally p roduced poorer narrative structures from fall to winter, with one participant decreasing from a two-event to a disoriented pattern, and one decreasing from a leapfrog pattern to a two-event pattern. Table D provides summary results o f winter high point analysis narrative patte rn data.
Table D. Winter Number o f Participants Using Narrative Patte rns
Disoriented Impoverished Two Leapfrog Chronological Event
TOTAL 1 0 2 1 5
ENGLISH 0 0 1 1 4
SPANISH 1 0 1 0 1
The narrative elements o f orientation and complicating action were consistently present in both groups ' stories during the fall testing session. All three Spanish participants used a
28 B I L I N G U A L N A R RATIVE ASSESS M E NT
complicating actio n. None o f the parti cipants used all six elements. A resolution was included in approximately hal f o f the narratives, and used by two out o f the three Spanish participants. Introduction, evaluation, and coda were the least included elements, with o nly one English participant using an evaluation and no participants usi ng a coda. Table E provides summary results o f fall high point analysis narrative eleme nt data.
Table E. Fall Number of Participants Using Narrative Elements
INT ORI COM EVA RES COD
TOTAL 219 619 8/9 1 /9 519 019
ENGLISH 1 16 416 516 1 /6 316 016
SPANISH 1 13 213 313 013 213 013
I n winter testi ng, all participants except o ne i n the Spanish group included a complicating action. All participants except one in the E nglish group provided a resolution to their narratives. Similarly, all participants except for one in the English group and one in the Spanish group i ncluded an orientation statement. Overall use o f orientatio n and resolutio n elements improved from fall to winter testing. All three Spanish participants i ncluded a resolution in their narratives. No introductions were included, and o nly o ne English participant used an evaluative utterance. The use of complicating actions remained co nstant overall. O ne English participant used a coda. Table F provides summary results of winter high point analysis narrative element data.
Table F. Wi nter Number of Participants Using Narrative Elements
INT ORI COM EVA RES COD
TOTAL 019 7/9 8/9 1 19 8/9 1 /9
ENGLISH 016 516 616 1 16 516 1 /6
B I LI N G U A L N A R RATIVE ASSESS M ENT
Individual Comparison Results
29
According to high point analysis measures, none o f the children produced an age
appropriate narrative. However, usi ng four out of five as a passing score, six out o f nine children met the criteria to pass the KLBA Narrative Subtest at each testing time. All children who produced a chronological pattern using high point analysis met the KLBA criteria for an age appropriate narrative. All children who produced disorie nted, impoverished, or leapfrog
narratives failed the KLBA. In the fall testing session, two participants, one English and one Spa nish, produced a two-event narrative, which is expected at 3 to 4 years o f age (McCabe &
Rollins, 1 994; Peterson & McCabe, 1 983), but still met the KLBA criteria for an age appropriate
narrative. In the winter testi ng sessio n, an English participant failed the KLBA Narrative Subtest with a two-event narrative, but a Spanish participant passed the KLBA with the same pattern. This suggests some discrepancy in outcomes of the high point analysis and KLBA measures.
If the KLBA narrative story retell subtest criteria require five out o f five responses fulfilled by a child' s story, then all participants in the Spanish and all but one in the English group failed the KLBA. These results coincide more closely to the results from high point analysis. However, the English participant whose narrative did fulfill all five respo nses produced a chronological narrative, which is still co nsidered to be below age expectations . Table G
3 0 B I L I N G U A L N A R RATIVE ASSESS M E NT
Table G. I ndividual Narrative Patterns and KLBA Scores
Participant ID Group Fall Narrative Fall KLBA Winter Narrative Winter
Pattern Fulfilled Pattern Fulfilled
Responses Respo nses
1 00 1 English Chronological 4 Chronological 4
1 003 E nglish Impoverished 1 Chro nological 4
1 005 E nglish Chronological 4 Chronological 5
1 007 E nglish Two-Event 4 Leapfrog 3
1 008 English Impoverished 3 Two-Event 3
1 0 1 1 E nglish Chronological 4 Chronological 4
1 090 Spanish Two-Event 4 Chronological 4
1 098 Spanish Chro nological 4 Two-Event 4
3 1 B I L I N G U A L N A R RATIVE ASS ESS M ENT
Chapter V
Discussion
The current study examined the effectiveness o f the KLBA narrative story retell subtest measures as a screening tool in ide nti fying both monolingual English and Spanish speaki ng ELL (with Spanish as L 1 ) ki ndergarteners at risk for narrative language difficulties. Participant narratives were also analyzed using the established measure o f high point analysis for companso n.
Kindergarten Language Benchmark Assessment
None o f the participants who produced an impoverished or disoriented narrative pattern met the KLBA criteria. All of these participants received a score o f three or less, which indicated that the Narrative Story Retell Subtest was successful i n identifying narrative language
difficulties that are highly at risk. During both the fall and wi nter testing sessio ns, six out of the nine participants' narratives received a passing score i n the current study (using four out o f five responses as the passing criteria). However, as previously stated, all o f the children produced a narrative pattern below age appropriate expectatio ns when assessed using high poi nt analysis, a nd i ncluded, o n average, three or fewer narrative elements during both testing sessions. This indicates that although the KLBA can identify narratives that are significantly below age
expectations, but does not identi fy all narratives below expectations. This could result in children not being identified as at risk for narrative language difficulties, therefore faili ng to receive necessary services.
When the passing criterion for the KLBA was modified to require five out o f five fulfilled responses, o nly one participant in the English group met the criteria, and no ne o f the
3 2 B I LI N G U A L N A R RATI V E ASS ESS M E NT
Spanish participants met the criteria. This indicates that using a more stringent scoring system increased accuracy of the KLBA in identifying narratives that were not age appropriate.
However, one English participant passed the KLBA with a chronological pattern, suggesting that the KLBA does not consistently identify children at risk for narrative language difficulties.
Most of the children' s narratives established a basic character and conflict.
Approximately half of the narratives established a specific time and/or place in which the story occurred. However, the majority of the narratives did not include aspects beyond the more superficial elements of orientation, conflict, and resolution. This could indicate that, although the KLBA criteria require crucial narrative elements (i.e., character, setting, conflict, resolution), the KLBA measures do not assess statements that introduce the story, evaluate characters, or connect the story to the present. Shapiro and Hudson ( 1 99 1 ) found that preschoolers could include feelings of the characters in a story, although consistency of inclusion was not fully developed. By kindergarten, children should be able to reflect on a personal event and connect an emotional response to it. The KLBA did not account for deficiencies in these more advanced narrative language abilities.
If five out of five responses must be fulfilled in order to meet the criteria for an age appropriate narrative, only one English participant passed the KLBA narrative story retell subtest. This would suggest that the KLBA is effective in identifying at risk narrative language abilities in kindergarteners. However, the English participant who met the KLBA criteria did not produce an age appropriate narrative according to high point analysis; therefore, the KLBA did not identify all children who might be at risk for narrative language difficulties.
33 B I L I N G U A L N A R RATIVE ASSESS M E NT
This is also problematic because of a ceiling effect for scores on the narrative story retell subtest. Several participants received four out of five points during both the fall and winter assessments. This allows little room for demonstration of narrative language growth over time. In addition, if children are meeting KLBA criteria without demonstrating age-appropriate narrative elements, the KLBA scoring does not allow for children to demonstrate growth to an age-appropriate level. For example, if the English participant who scored a five on the KLBA during the winter of 20 1 4 did improve from a chronological pattern, the KLBA would not reflect that improvement. Therefore, the KLBA may not accurately depict growth in children' s
narratives over time.
High Point Analysis
None of the participants produced an age-appropriate narrative pattern (i.e., ending-at the-highpoint) during the fall or winter testing sessions. In addition, no individual participant included all six narrative elements. One participant in the English group included four elements in the fall, aw well as one in the English group in the winter. Generally, participants used three or fewer elements when producing the narrative.
Spencer, Clegg, and Stackhouse (20 1 2) found that low socioeconomic status negatively impacted vocabulary development for children. This could be a contributing factor in the below average narrative productions from participants as a whole. The impact of socioeconomic status could contribute to both the English and Spanish groups performing below age expectations.
In addition, the Spanish group may have had more difficulty forming narratives in English because of structural difference between English and Spanish narratives. Children from Spanish speaking homes are generally taught to tell a story without emphasizing the inclusion of
34 B I LI N G U AL N A R RATI V E ASS ESS M E NT
all narrative elements (Melzi, 2000). For example, two out of the three Spanish group participants only used two different elements in the fall. One Spanish group participant only included a complicating action and a resolution, while the other used a series of orientation and complicating action statements . Similarly, two out of the three Spanish group participants used only two different elements: orientation and resolution, and complicating action and resolution. In contrast, three out of the six English group participants included at least three different elements in the fall, and five English group participants included at least three different elements in the winter.
The linguistic differences in English and Spanish could have negatively impacted the bilingual participants' awareness of English story structure. The amount and time of exposure to English could also have influenced the Spanish participants ' knowledge of English vocabulary, impacting narrative production in English (Hammer et al., 20 1 2). Only one Spanish participant produced a chronological pattern during either the fall or winter session. Narratives produced by the two remaining Spanish participants during either session were leapfrog, two-event, or disoriented. All three of these narrative patterns are described as leaving out critical elements or using a limited number of elements. This coincides with the concept of Spanish speaking children learning to tell a story without emphasizing the inclusion of as many elements as possible (Melzi, 2000). All three Spanish participants, in either fall or winter assessment, told a narrative lacking in narrative elements. It is also possible that the high point analysis over
identifies ELL' s due to cultural differences.
Although participants included more than one element in a narrative, often the elements were not combined in a cohesive or coherent manner. For example, one child during the fall testing session stated, "I cut myself at home because I didn't watch, and I got hurt." This
3 5 B I L I N G U A L N A R RATIVE ASSESS M E N T
narrative is not coherent, as the listener cannot infer a clear idea from it. One child' s narrative included, "I put a bandage on. I was painting." This narrative is neither coherent nor cohesive, as the components are not connected to an overall point, nor are they connected by any interclausal connectives to smoothly transition between components.
Limitations and Future Research
The small sample size in the current study, particularly in the Spanish group, did not allow for a complete representation of the monolingual English and bilingual Spanish-English kindergarten population. A larger sample size would better represent the population and yield results which could be more widely applied. Testing was only completed in English, which may not have allowed the bilingual Spanish-English students to demonstrate their complete language abilities. Testing bilingual students in their native Spanish would provide a more complete representation of the bilingual Spanish-English children' s language and narrative skills. This would better assess if a bilingual student was at risk in both L 1 and L2, or if the difficulty was due to acquisition deficits in L l . Considering the level of English proficiency of bilingual students could also be beneficial to future research regarding the effectiveness of KLBA.
In addition, because the KLBA was designed as a brief screening tool, the elicitation method of elicitation limited use of prompts in order to maintain short test duration. If extensive prompting was allowed, participants might produce more developed narratives.
Requiring five out of five fulfilled responses on the KLBA would more accurately identify children who are at risk for narrative language difficulties. Further modifications should also be considered to improve sensitivity of the KLBA measure. For example, the KLBA should include stricter passing requirements, possibly by including more responses that must be
3 6 B I L I N G U A L N A R RATIVE ASSESS M ENT
fulfilled. Assessing narrative pattern would also be beneficial. Additionally, scoring criteria for the KLBA could be expanded to require more complete, cohesive expression of ideas in order to pass each item in the subtest. This might allow for documentation of performance change and growth in its scoring. Future testing should expand the subj ect pool to represent a larger kindergarten population and varied demographics.
Conclusions
The KLBA is a quick and efficient measure of overall language ability in kindergarteners. The Narrative Subtest of the KLBA assesses storytelling skills in the English language,
measuring the narrative elements of character, time, place, conflict, and resolution. The KLBA consistently identified children who were highly at risk for narrative language difficulties that might need to be addressed through RTI services. However, revision of the scoring system should be considered to increase the sensitivity of the measure. Further investigation involving both the bilingual and monolingual population could better identify all children who are at risk for narrative language difficulties, rather than only those at significant levels of risk.