NEAL, SARAH CARRUTHERS. The Differential Impacts of Incredible Years-Teacher Classroom Management Based on Young Children’s Risk Profiles (Under the direction of Dr. Mary Haskett).
Children’s Risk Profiles
by
Sarah Carruthers Neal
A dissertation submitted to the Graduate Faculty of North Carolina State University
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
Psychology
Raleigh, North Carolina 2019
APPROVED BY:
_______________________________ _______________________________ Dr. Mary Haskett Dr. Kate Norwalk
Committee Chair
DEDICATION
BIOGRAPHY
Sarah Carruthers Neal was born in Durham, North Carolina. She completed her Bachelor’s of Arts in Psychology at the University of North Carolina of Wilmington. Sarah began her graduate career at North Carolina State University in 2013. She will graduate with a Ph.D. in School Psychology in 2019.
TABLE OF CONTENTS
LIST OF TABLES ... v
Incredible Years-Teacher Classroom Management ... 2
The Present Study ... 5
Measuring Patterns of Risk ... 7
Research Questions ... 11
Methods... 11
Participants ... 11
Procedures ... 11
Measures ... 12
Analyses ... 15
Results ... 16
Identifying Patterns of Risk ... 16
Baseline Differences in Latent Classes ... 20
Differences in Response to Intervention Within Each Latent Class ... 21
Discussion ... 22
Limitations and Future Directions ... 25
LIST OF TABLES
Table 1 Fit Statistics ... 17
Table 2 Posterior probabilities of the five-class model ... 18
Table 3 Latent Class Means of the five-class model ... 18
Young children growing up in households characterized by low socioeconomic status (SES) are at an increased risk for a number of maladaptive outcomes (Yoshikawa, Aber, & Beardslee, 2012). Low SES has been found to have both direct and indirect effects on many aspects of child development including physical health, emotional and behavioral adjustment, cognitive abilities, language, and academic achievement (Yoshikawa et al., 2012). A greater prevalence of mental health and behavior problems is found among children and adolescents experiencing low SES compared to higher SES peers (Qi & Kaiser, 2016; Reiss, 2013). Children from low SES families are more likely to demonstrate poor behavior and emotion regulation, which underlies both externalizing and internalizing behavior problems (Cadima et al., 2016). The early occurrence of externalizing and internalizing behaviors can have long lasting impacts on a child’s development including impacts on future social adjustment, quality of peer
relationships, academic success, and conduct problems (Loeber, 1996; Fanti & Henrich, 2010). Given the higher prevalence of externalizing and internalizing behavior problems among young children from low SES backgrounds and the long-term impacts of these problems, early
intervention and prevention efforts targeted at children from low SES households are important to decrease the prevalence of these problems and minimize the impacts on other areas of development.
It has been proposed that early social and emotional competency may serve as a protective factor in preventing the future development of behavior problems and
(IY-TCM) program which targets teacher practices and classroom climate in order to improve students’ emotion knowledge, social problem solving, and behaviors (Webster-Stratton & Reid, 2002). IY-TCM has been found to be an effective program in promoting social and emotional competence and decreasing problem behaviors. There is some evidence of individual differences in response to the intervention, but there is limited knowledge about which characteristics of children will predict a positive treatment response. The goal of this study was to explore the differential effects of the IY-TCM program for children with various patterns of baseline risk factors within the context of Head Start classrooms.
Incredible Years-Teacher Classroom Management
The Incredible Years series is a suite of interventions targeting parents, children, and teachers (Webster-Stratton, 2000; 2011). The IY-TCM training program is one component of the Incredible Years Series, aimed at teachers of children ages 3 to 8 (Webster-Stratton & Reid, 2002). The program targets the teachers’ use of (1) attention, (2) academic, persistence, and social-emotional coaching, (3) praise, encouragement, and incentives, (4) use of proactive classroom management strategies, and (5) strengthening students’ social skills, emotion
regulation, and problem solving (Webster-Stratton, Reinke, Herman, & Newcomer, 2011). The goal of the training is to support teachers in structuring their classroom and using positive behavior management strategies with the purpose of preventing problematic behaviors from occurring as well as decreasing existing disruptive behavior problems.
2007), increases in teacher enthusiasm (Raver et al., 2008), and lower use of harsh or negative practices (Raver et al., 2008). IY-TCM has demonstrated decreases in class wide off-task behaviors (Hutchings, Martin-Forbes, Daley, & Williams, 2013). The changes in teacher
behaviors following IY-TCM have an impact on child outcomes through changing the ways that peers interact with those with conduct problems (Snyder et al., 2011), promoting children’s self-regulation skills (Jones, Bub, & Raver, 2013), and improving the quality of the teacher-child relationship (Jones et al., 2013).
IY-TCM leads to improvements across a range of behaviors among preschool aged children. Webster-Stratton et al., (2001) implemented Incredible Years as a prevention program at Head Start centers by providing IY-TCM training to teachers and providing the Incredible Years parent component as an option to all Head Start parents at the intervention centers. At post-intervention they found decreases in conduct problems at home, lower teacher ratings of hyperactivity and antisocial behaviors, and improvements in social competence in the classroom. In other studies, IY-TCM combined with mental health consultation led to improvements in preschool teacher ratings of children’s internalizing and externalizing behaviors (Raver et al.,
Evidence of Differential Treatment Effects of IY-TCM
There is evidence that IY-TCM is not equally effective for all children (Murray et al., 2018; Reinke et al., 2018; Snyder et al, 2011; Hutchings et al., 2013). Children’s behavior prior to intervention has been found to moderate the effects of the intervention on outcomes (Snyder et al., 2011; Hutchings et al., 2013). Within control classrooms, children demonstrated increases in problem behaviors throughout the school year regardless of initial levels of behavior problems (Snyder et al., 2011; Hutching et al., 2013). However, within the IY-TCM classrooms, children with conduct problems or with elevated behavior risk showed decreases in problem behaviors. Children with lower initial levels of behavior problems in IY-TCM classrooms demonstrated no changes to problem behaviors. Of note, Hutchings et al., (2013) found that children in the high behavior risk group who received IY-TCM demonstrated the same level of negative behaviors towards teachers and off task behaviors as those in low behavior risk group. That finding indicates that IY-TCM decreases problematic behaviors among those with higher behavior risk so that their problematic behaviors are consistent with children at lower behavior risk. Children’s initial social competence has also been found to moderate the effects of IY-TCM on outcomes (Murray, Rabiner, Kuhn, Pan, & Sabet, 2018; Reinke, Herman, & Dong, 2018). Children with low social competence at baseline who received IY-TCM had higher prosocial behaviors (Murray et al., 2018), lower inattention (Murray et al., 2018), and greater social competence (Reinke et al., 2018) compared to similar peers in control classrooms following the intervention. The presence of household risk has been found to moderate the impacts of IY-TCM (Raver et al., 2009). Specifically, children who were considered low-risk (one or fewer risk factors)
children’s initial behavior problems, social competence, and household risk moderates the impacts of TCM on problem behaviors and social competence, providing evidence that IY-TCM is not equally effective for all children.
The Present Study
The present study used data from the Head Start Classroom-based Approaches and Resources for Emotion and Social skill promotion (CARES) study. The Head Start CARES study was a large-scale randomized control trial designed to evaluate the impacts of three
different teacher-delivered SEL programs (including IY-TCM) when implemented in the context of Head Start classrooms. Teachers were randomly assigned to receive training in one of three SEL programs or to continue delivering Head Start as usual. Children were assessed through parent report, teacher report, and direct assessments in the fall of the academic year, the following spring, and during their Kindergarten year. In line with previous research, the Head Start CARES study found positive impacts on teacher behaviors following the implementation of IY-TCM as well as improvements in children’s emotion skills, social skills, and learning
behaviors (Morris et al., 2014). However, there were no significant impacts of IY-TCM on the children’s problem behaviors (Morris et al., 2014).
The results of the Head Start CARES study in regards to behavior outcomes following IY-TCM were inconsistent with anticipated findings based on previous research. On one hand, this may indicate that IY-TCM when delivered as a stand-alone intervention is not effective in decreasing children’s problems behaviors. The studies referenced previously all implemented
discrepancy may indicate that IY-TCM as a stand-alone intervention, without the parent training or consultation components, may not have the strength to modify children’s behavior problems. Alternatively, IY-TCM may demonstrate different types of impacts for various children. As described previously, numerous studies have found evidence that children with different baseline characteristics responded differently to IY-TCM (Murray et al., 2018; Reinke et al., 2018; Snyder et al., 2011; Hutchings et al., 2013). If this is the case, examining the main effects of IY-TCM on all children may result in a dilution of the effects. In fact, Greenberg and
Abenavoli (2017) note that when measuring the impacts of universal interventions, the effects of the intervention are often either not detected or underestimated if one focuses solely on main effects. This is due to the heterogeneity of those receiving the intervention, where some may already be exhibiting the target behavior, but others will not develop the target behavior even if they never received the intervention. In addition, universal interventions can lead to different patterns of effects for various subgroups (Greenberg & Abenavoli, 2017). Treatment effects result in a reduction of the target behavior only for children already demonstrating that target behavior. In the Head Start CARES study, this would be demonstrated through a reduction in externalizing and internalizing behaviors among children who demonstrate elevated levels of these behaviors prior to the intervention. Promotion effects result in an increase in positive outcomes or protective factors, such as increases in emotion knowledge or social skills. Promotion effects may later result in prevention effects, such as preventing a child from developing externalizing and internalizing behavior problems when older.
may vary based on baseline risk factors. By understanding which children are most likely to benefit from IY-TCM, stakeholders can more effectively target the program. In fact, the Advisory Committee on Head Start Research and Evaluation (2012) recommended that future research focus on improving the practices of Head Start by developing enhancements to Head Start to support children’s social-emotional learning as well as “investigating program effects for
policy related subgroups including children of families at different poverty levels, children at highest risk…” (p.38). The purpose of the present study was to determine whether there were systematic differences in the patterns of risk factors among children attending Head Start and to determine if children characterized by these patterns of risk factors respond differently to IY-TCM in the Head Start CARES study. Specifically, this study examined (1) whether there were underlying risk profiles based on home-environment, parent, and child characteristics using Latent Class Analyses (LCA) and (2) whether these risk profiles moderated the impact of IY-TCM on children’s internalizing behaviors, externalizing behaviors, social skills, and emotion knowledge.
Measuring Patterns of Risk
studies identified at least one class characterized by poor parent functioning and/or parent distress. Pratt et al., (2016) and Roy and Raver (2014) also identified a class characterized by low resources/income and single parents, but no other risk factors. Lanza et al., (2010) identified a class characterized by single parents and parent history of problems such as drug use or history of arrests.
The present study used six risk factors reflective of aspects of the child’s household risk, parents’ general functioning, and child’s baseline functioning. These risk factors include parent unemployment, parent education, parent relationship status, parent psychological distress, child’s inhibitory control, and child’s expressive language abilities. These risk factors were selected for inclusion based on evidence that each factor independently predicts future problem behaviors, has been meaningful in risk profiles in past studies, and/or may moderate the impacts of early prevention programs. A brief review of research on each risk factor is provided below. Household Risk Factors
Household risk factors such as parent unemployment, low parent education, and single parenthood are markers of low levels of economic resources. These risk factors, either
individually or when co-occurring in early childhood, have been found to predict future behavior problems, especially externalizing behaviors (Burchinal & Roberts, 2006; Fanti & Henrich, 2010; Reiss, 2013). In general, these household risk factors tend to be indirect predictors of behavior problems and are considered markers of other problems.
Parent employment. Parent unemployment has been found to predict future development of children’s behavior problems (Harland, Reijneveld, Brugman,
Parent education. Maternal education is considered an indicator of social class, is correlated with household income, and is associated with children’s outcomes (Jobe-Shields, Andrews, Parra, & Williams, 2015). In fact, low parent education has been found to have a stronger association than parent unemployment with future development of behavior problems and mental health challenges (Reiss, 2013). Although low levels of education can be an indicator of risk, higher levels of education is a protective factor (Jobe-Shields et al., 2015). Parent
education was used in three studies as a risk factor to identify latent classes among young
children (Cooper & Lanza, 2014; Lanza, Rhoades, Greenberg, & Cox, 2011; Lanza et al., 2010). Parent relationship status. Across a number of the studies that identified latent classes based on risk factors among young children, single-parent status consistently differentiated the latent classes (Jobe-Shields et al., 2015; Lanza et al., 2011, 2010; Pratt et al., 2016; Roy & Raver, 2014). The low-risk profiles were typically characterized by the presence of married parents in a household, along with the absence of other risk factors, whereas single parenthood tended to co-occur with a number of other risk factors (Jobe-Shields et al., 2015). In addition, single parenthood has been found to be a strong independent predictor of internalizing behaviors (Bayer et al., 2012).
Parent-Level Risk Factors
Parent psychological distress. Parent psychological distress, especially the presence of depressive symptoms in mothers, has been identified as a strong risk factor for both externalizing and internalizing behaviors (Fanti & Henrich, 2010; Goodman et al., 2011; Heberle, Krill,
of maternal depression can impact the outcomes of various interventions for children with conduct problems and that programs that include a parent-training component are more effective than those that do not (Beauchaine, Webster-Stratton, & Reid, 2005). Latent class status
characterized by maternal depression has been found to moderate the impacts of Head Start on behavior outcomes (Cooper & Lanza, 2014)
Child-Level Risk Factors
Child inhibitory control. Inhibitory control, which is one component of executive functioning, is the ability to suppress a dominant response in favor of a subdominant response (Rhoades, Greenberg, & Domitrovich, 2009). Inhibitory control plays a role in self-regulation and behavior control, by allowing the child to inhibit an inappropriate response. Low levels of inhibitory control in early childhood predict externalizing behaviors (Eisenberg et al., 2009) and internalizing behaviors (Rhoades et al., 2009). Higher levels of inhibitory control predict future social and emotional competence (Rhoades et al., 2009). Initial executive functioning abilities, including inhibitory control, moderate impacts of a SEL intervention in a Head Start setting, where children with lower executive functioning abilities demonstrated greater improvements in social-emotional competence and aggression control compared to children in the control group (Bierman, Nix, Greenberg, Blair, & Domitrovich, 2008).
Children’s expressive language abilities. The ability to comprehend language and communicate effectively underlies children’s success in navigating social settings. Low language ability correlates with current behavior problems and predicts future behavior problems (Chow & Wehby, 2016). Of relevance to the current study, children in Head Start are at risk for low
Research Questions
Research Question 1: Are there underlying latent classes characterized by the presence of children’s risk factors, among the sample of children from the Head Start CARES study? Research Question 2: Within each latent class, do children in classrooms with teachers trained in IY-TCM demonstrate lower internalizing and externalizing behaviors and higher social skills and emotion knowledge when compared to those in the Head Start as usual classrooms?
Methods Participants
Seven children from each Head Start CARES study classroom were randomly selected for measurement at baseline and three additional children (for a total of 10 children) were assessed at the end of the school year. A total of 1014 children were assessed from the IY-TCM group and 941 children were assessed from the control group. For this study, only participants with complete data at baseline and end of the school year were used in data analyses (n =1,037). All children were 4-years-old. Among the 1,037 children assigned to a latent class, 29% were black (n = 298), 40.5% were Hispanic (n = 420), 12.3% were white (n = 128), 6.1% identified as other (n = 63). 128 children were missing data on race. 48.3% of the children were female. 501 children were in the Head Start as usual group and 536 were in the IY-TCM group.
Procedures
fall prior to the start of the intervention implementation. Post-intervention assessments were conducted during the spring of the same academic year.
Teachers in the IY-TCM program participated in six days of training. IY-TCM training focused on establishing positive classroom management strategies and promoting the teacher-child relationship in order to support teacher-children’s behavior regulation. Following the training,
teachers received weekly coaching in their classrooms to support implementation. On a scale of 1 (low) to 5 (high), the average implementation fidelity score for teachers implementing IY-TCM strategies within their classroom was 3.69.
Measures
Risk factors. Home and parent-level risk factors were measured by items on the
parent/guardian survey completed in the fall. The child-level risk factors were measured through direct assessments also completed in the fall. The measures used were as follows:
Parent employment. Parents reported whether they were currently working for pay. The
responses for parent employment were re-coded into two categories: “currently employed” (responses of “yes” or “yes, currently on leave”) and “currently unemployed” (responses of “no” or “laid off”).
Parent education. Parents reported the highest grade of school they had completed.
Responses were recoded to three categories, which included “less than 12th grade,” “GED or
High School Diploma,” and “higher education degree” (i.e. Associates degree, Bachelors degree, Graduate school, Graduate degree).
Parent relationship status. Parents were asked about their current marital or relationship
Parent distress. Parental distress was measured using the Kessler six-item Psychological
Distress Scale (K6; Kessler et al., 2003). The K6 is a non-specific screener for psychological distress and includes six items that screen for depressed mood, motor agitation, fatigue, worthlessness, guilt, and anxiety. Participants rate the frequency with which they experienced each item over the last 30 days on a scale ranging from of 0 (“None of the time”) to 4 (“All of the time”). A psychological distress composite score is created by summing the ratings of all items, with a possible range of 0-24. The developers recommend using a cut score of 13 or higher to denote serious psychological distress and the potential for mental illness. The K6 has demonstrated validity through moderate to high correlations with the Comprehensive
International Diagnostic Interview-Short Form and the World Health Organization Disability Assessment Schedule (Kessler et al., 2003).
Child inhibitory control. Children’s inhibitory control was measured using the Head to
Toes task (Ponitz et al., 2008). Children are instructed to touch their head when the assessor says to touch their toes, and vice versa. This task primarily assesses inhibitory control, but also measures aspects of attention and working memory. Following multiple practice trials, there are 10 trials with directions provided verbally. Children receive one point for a correct response or for a self-correction and zero points for an incorrect response. The total score is the sum of correct trials and can range from 0 to 10. The Head to Toes task has demonstrated validity and inter-rater reliability (Ponitz et al., 2008). Scores on the Head to Toes task have been found to significantly correlate with teacher report of behavior regulation items on the Child Behavior Rating Scale (r = .15 to .47; Ponitz et al., 2008).
Expressive language abilities. Children’s expressive language abilities were measured
2000). The EOWPVT measures children’s vocabulary. Children are shown a picture and asked to produce a word that best describes the picture. Children whose primary language was Spanish were assessed using a Bilingual EOWPVT, which allows children to respond in either English or Spanish. The EOWPVT demonstrates temporal stability with test-retest correlations ranging from .88 to .97 (Brownell, 2000). The EOWPVT has demonstrated concurrent validity with strong correlations between scores on the EOWPVT and other measures of expressive vocabulary such as the Weschler Intelligence Scale for Children III- Expressive Definitions subtest (r = .86) and the Test of Language Development, Third Edition (r = .81; Brownell, 2000).
Outcome measures. Outcomes were assessed through teacher ratings and direct
assessment at pre-intervention (in the fall before the intervention began) and at post-intervention (in the spring of the same academic year).
Behavior problems. Children’s internalizing and externalizing behaviors were measured
in research to measure the presence of behavior problems in children, psychometric data demonstrating evidence of validity is limited.
Social competence. Children’s social competence was measured using teacher report on
the Social Skills Rating System (SSRS), Social Skills Scale (Gresham & Elliot, 1990). The SSRS Social Skills Scale measures children’s cooperation, assertion with peers, and self-control in social situations (Gresham & Elliot, 1990). Ratings range from 0 (“Never”) to 2 (“Very Often”), for a total possible raw score ranging from 0-60. The SSRS Social Skills scale demonstrates strong reliability. Internal consistency is high for each subscale (alpha = .90 - .94) and test-retest reliability is also high (r = .85). The SSRS Social Skills scale has not been validated for
preschoolers, but does show evidence of validity with elementary school-aged children. The total Social Skills Scale was moderately correlated with teacher ratings on the Social Behavior
Assessment (r = -.68; Gresham & Elliot, 1990).
Emotion knowledge. Children’s emotion knowledge was assessed using the Facial
Emotion Identification task. Children were shown a page with a picture of happy, sad, mad, and scared facial expressions and were asked to point to the picture of a specific emotion (e.g., “point to the picture that shows happy”). A total of 16 items were sequentially presented with the order of the facial expressions varied on each page. There were four items for each emotion. The final score is the proportion of all answers that were correct over the total number of items, for a possible range of 0-1. Interrater reliability for the scoring of this task is high (kappa = .77 to 1.00; Garner, Jones, & Miner, 1994).
Analyses
variables, often referred to as indicators (Collins & Lanza, 2010). In the present study, the indicators are the risk factors. The latent variable identified through LCA is categorical and consists of a set of latent classes. Each latent class represents a subgroup of the sample that is characterized by distinct patterns of indicators. The choice of risk factors included in the LCA was based on theory as well as model fit. The identification of the classes was exploratory and, as such, a number of risk factors were initially included in the LCA. However, risk factors that did not meaningfully differentiate latent classes were removed from the model. Model selection was based on Akaike’s information criterion (AIC; Akaike, 1987), the Bayesian information criterion (BIC; Schwarz, 1978), entropy statistics, and conceptual clarity. AIC and BIC compare the relative fit of models with different numbers of latent classes. Lower values suggest a more optimal balance between fit and parsimony. Entropy values reflect the degree of certainty that each individual case is designated to a particular latent class. Values range from 0 to 1, with values closer to 1 reflecting better fit.
MANCOVAs were run to determine whether there were differences in post-intervention social competence, emotion knowledge, internalizing behaviors, and externalizing behaviors within each latent class for children who received IY-TCM versus Head Start as usual. Baseline performance on the outcome measures was included as a covariate.
Results Identifying Patterns of Risk
in subsequent models. A series of LCA models with an increasing number of classes (i.e., 1- to 6-class models) were estimated and compared to explore the number and structure of latent subgroups at baseline. Fit statistics for each of the tested models are presented in Table 1. Although the six-class model had the best fit statistics, the five-class model was ultimately chosen due to conceptual clarity.
Table 1 Fit statistics.
Class AIC BIC Entropy
1 21099.950 21144.447 --
2 19915.122 19994.227 .938
3 19752.537 19866.251 .900
4 19226.489 19374.812 .940
5 19084.466 19267.397 .911
6 18915.669 19133.209 .917
Table 2
Posterior probabilities of the five-class model.
Latent baseline subgroups (proportion) Baseline characteristic Moderate risk (9%) High functioning (13%) Distressed parents (5%) Impulsive kids (66%) Low risk (5%) Parent employment
Employed .389 .562 .336 .512 .449
Unemployed .611 .438 .664 .488 .551
Parent Education
Less than 12th grade .427 .160 .535 .333 .279 GED or High school
diploma
.489 .675 .385 .575 .615
Higher Education .08 .165 .080 .093 .106
Table 3
Latent Class Means of the five-class model.
Latent baseline subgroups (proportion) Baseline characteristic Moderate
risk (9%) High functioning (13%) Distressed parents (5%) Impulsive kids (66%) Low risk (5%)
Parent Distress 3.72 3.71 13.65 3.05 3.97
Child Expressive Language
82.33 89.78 79.75 83.63 85.05
Child Inhibitory Control
2.51 9.48 .25 .16 5.88
Latent Class 1: moderate risk. A total of 94 children (9.0%) were members of the moderate risk latent class. The parents of children in this class were more likely to be
36 (38%) as Hispanic, 12 (12%) as white, and 6 (6%) as other. About half (n = 46; 49%) of these children were female.
Latent Class 2: high functioning. Class 2, labeled the high functioning class, comprised 13% of the sample (n = 141). Their parents were more likely to be employed and more likely to have a GED/high school diploma. These parents had low levels of distress. The expressive language skills of the children in this class were in the average range. Of note, they had the highest language skills compared to the other latent classes. They also had strong inhibitory control skills. Among children in the high functioning class, 39 (27%) identified as black, 44 (31%) as Hispanic, 28 (19%) as white, and 12 (8%) as other. A little more than half (57%) were female.
Latent Class 3: distressed parents. There were 59 children in the third class, labeled the distressed parents class, which comprised only 5% of the sample. Their parents were more likely to be unemployed and to have less than a 12th grade education. These parents also reported the
highest levels of distress compared to the other latent classes and their mean score on the K6 was above the clinical threshold cut score of 13. The children in this class had the weakest expressive language scores and poor inhibitory control. Among these children, 24 (40%) identified as black, 22 (37%) as Hispanic, 5 (8%) as white, and 6 (10%) as other. The majority of these children (67%) were male.
(27%) children identified as black, 300 (43%) as Hispanic, 73 (10%) as white, and 35 (5%) as other. Among these children, 46% (n= 320) were female
Latent Class 5: low risk. There were 54 children in the low risk class, which was 5% of the sample. Their parents were slightly more likely to be unemployed and more likely to have a GED/high school diploma. Their parents were not distressed. The children in this class had low average expressive language scores. Among these children, 15 (27%) identified as black, 18 (33%) as Hispanic, 10 (18%) as white, and 4 (7%) as other. A total of 34 (62%) of these children were female.
Baseline Differences in Latent Classes
To further explore baseline differences in the five latent classes, ANOVAs were run to examine group differences in emotion identification, externalizing behavior, internalizing behaviors, and social competence. At baseline, there were statistically significant differences in emotion identification between latent classes (F (4,933) = 22.14, p < 001). Post hoc comparisons using Least Significant Difference (LSD) test indicated that the mean score for the high
externalizing behaviors (F (4, 1001) = .111, p = .979) or internalizing behaviors (F (4, 999) = .705, p = .588) between latent classes.
Differences in Response to Intervention Within Each Latent Class
MANCOVA analyses were conducted within each latent class to explore the difference between post-intervention scores on emotion identification, social competence, externalizing behaviors, and internalizing behaviors between children who were in the IY-TCM classrooms and children in the Head Start as usual classrooms. Baseline scores for the outcome variables were included as covariates. There were no significant differences in outcomes between the Head Start as usual and IY-TCM groups within the high functioning, the distressed parents, or the low risk classes.
In the moderate risk class there was a significant difference between the treatment groups on the combined dependent variables after controlling for baseline measures. Children in the IY-TCM classroom demonstrated lower externalizing behaviors (M = 1.15) and internalizing behaviors (M =.79) at the end of the year compared to children in the Head Start as usual
classroom (externalizing behaviors M = 3.28; internalizing behaviors M = 1.65). Children in the IY-TCM classrooms had stronger social competence at the end of the year (M = 50.95) compared to those in the Head Start as usual classrooms (M = 45.08). There were no significant differences in emotion identification.
competence at the end of the year (M = 47.24) compared to children in the Head Start as usual classrooms (M = 44.34).
Table 4
Results of MANCOVAs.
Class F (df) p Eta
High Functioning 1.90 (4, 106) .114 .067
Low Risk .362 (4, 33) .834 .042
Moderate Risk 2.793 (4,70) .033* .138
Impulsive Kids 4.588 (4, 518) .001* .034
Distressed Parents 1.366 (4, 26) .273 .174
Discussion
Universal SEL interventions, such as IY-TCM, have been proposed as one option to aid in preventing the development of behavior problems among young children growing up in low-income households. Although past research on IY-TCM indicates positive effects for children, there is also evidence that it is not equally effective for all children (Snyder et al., 2011; Webster-Stratton et al., 2001). Understanding which children are most likely to respond positively to IY-TCM can inform decision-making for teachers and other stakeholders in early childhood education settings. With the goal of contributing to that understanding, the present study explored whether there were systematic patterns of risk factors present among children attending Head Start and whether children with varying patterns of risk factors responded differently to IY-TCM. Five latent classes were identified based on child, household, and parent risk factors present prior to the intervention. Children in these latent classes demonstrated
The results of the present study indicate that there are different patterns of response to the IY-TCM program when compared to Head Start as usual for children with different risk profiles. While differential treatment effects of IY-TCM have been documented in the literature, most studies have focused on the differential treatment effects for children with either existing behavior problems or at risk for behavior problems (Snyder et al., 2011; Webster-Stratton et al., 2001). This study adds to the present literature by exploring differential treatment effects based on family- and child-level risk factors that may be more readily identifiable at a young age and that may occur prior to the onset of behavior problems.
The risk profiles identified in this study are consistent with those found by past
investigators who conducted LCAs based on risk factors in young children. Many of the other studies also identified at least one low risk group (Lanza et al., 2010; Roy & Raver, 2014; Pratt et al., 2015) as well as a group characterized by low parent functioning or depressive symptoms (Lanza et al., 2010; Pratt et al., 2015; Roy & Raver, 2014). However, this study identified one group of children that was unique from past research. Children in the impulsive kids class, which included more than half of the children in this sample, demonstrated the lowest inhibitory
functioning at baseline who were in the classrooms with the intervention had higher social competencies and lower aggression when compared to children in typical Head Start classrooms. There were no treatment effects for children with high executive functioning skills at baseline. There is also evidence that IY-TCM improves self-regulation skills, which in turn results in improvements in behaviors and social competence (Jones, Bub, & Raver, 2013). Our results combined with these prior studies suggest that IY-TCM leads to improvements in inhibitory control, which in turn results in improvements in social competence with the greatest impacts for children with deficits in inhibitory control prior to the intervention.
The differential treatment effects found in this study fit with a tiered service delivery framework of prevention. Within models such as Multi-tiered Systems of Support (MTSS) or the public health model, prevention services are delivered to target populations with varying levels of risk and the intensity of the intervention varies based on risk level (National Research Council and Institute of Medicine, 2009). As such, universal interventions, such as IY-TCM, are expected to be beneficial to select children receiving it, but not all children (Greenberg et al., 2016). The two moderate-risk classes (moderate risk and impulsive kids) were the only classes that
slight worsening of behaviors. Similarly, the present study found that children in the highest functioning and the low risk classes did not appear to benefit from the intervention.
Among children in the distressed parents class, which included children with poor expressive language, poor inhibitory control, and parents who had high levels of distress, IY-TCM was not more effective than Head Start as usual. Because their parents are highly distressed and because parent distress is associated with poor parenting behaviors as well as greater
problem behaviors in children (Goodman, Rouse, Connell, Broth, Hall, & Heyward, 2011; Trapolini, McMahon, and Ungerer, 2007; Turney, 2012), children of highly distressed parents may benefit from supports to ameliorate parent distress and promote positive parenting
behaviors. Prior research indicates that maternal depression moderates the impacts of Incredible Years on observed child behaviors such that the inclusion of the parent training and/or child training program results in greater impacts than IY-TCM alone for children of depressed mothers (Beauchaine et al., 2005). The parent training component of Incredible Years has been found to result in reductions in parent depression and parent stress and improvements in parenting skills. In addition, improvements in both depressive symptoms and parenting skills mediated the
impacts of the intervention on child externalizing behaviors (Hutchings et al., 2012; Hutchings et al., 2007). Our findings support conclusions of these prior studies, which emphasizes the
importance of parenting interventions in addition to classroom-based interventions for parents of preschool aged children at risk for behavior disorders, particularly those who are experiencing high levels of distress.
Limitations and Future Directions
number of strengths to this study. The use of LCA to identify risk profiles paints a more comprehensive picture of the children compared to focusing on single moderator variables or cumulative risk index. We included risk factors occurring at multiple levels of influence on children’s adjustment, including within the home, the parent, and factors within the child. This study was based on a large sample size of children randomly selected from Head Start centers across the nation and the sample is representative of demographic characteristics of all children within Head Start centers. However, a number of limitations to this study exist that should be considered in interpretation of the findings.
First, in terms of quality of delivery of the IY-TCM intervention, the average fidelity rating of IY-TCM principles in the classroom was only moderate (3.42 out of 5; Mattera, Lloyd, Fishman, & Bangser, 2013). This rating indicates that IY-TCM principles and strategies were implemented occasionally, but inconsistently. Though implementation fidelity was moderate, this likely reflects real-world application of IY-TCM, which contributes to the external validity of the study. However, if implementation fidelity had been higher, there might have been greater impacts for some of the risk groups. Although no studies have explored the impact of fidelity on children’s outcomes for IY-TCM specifically, there is evidence in general that higher
behaviors among those not yet experiencing them) often emerge later in time rather than
immediately following intervention (Greenberg & Abenavoli, 2017). Overall, the children in this study demonstrated low levels of teacher-rated externalizing and internalizing behavior problems at baseline and follow up. It is possible that those children who are at greater risk—in particular those in the moderate risk, impulsive kids, and distressed parents classes—were on a trajectory to develop behavior problems when older. Exploring the developmental trajectories of children with various risk profiles beyond a single year from the receipt of IY-TCM would provide important information regarding whether IY-TCM prevents the development of behavior problems over time among these groups of children.
The sample in the present study included only children who attended Head Start. As such, the findings may not generalize to children outside of Head Start settings. Furthermore, all the children in this sample were from low income families and were at an elevated risk for
maladaptive outcomes compared to children in the general population. In addition, since Head Start is an intervention aimed at promoting factors related to school readiness, these children may have different patterns of risk factors than those from low income families that do not attend high-quality early education programs like Head Start. Replication with other samples will be important to determine whether findings generalize beyond the current sample.
There are several limitations related to measures. We used single measures of most constructs; a multi-method approach would have been optimal. In addition, the use of raw scores for the outcome measures limits the interpretations that can be made about children’s functioning at the end of year. Because scores were not based on norms, we are unable to determine the degree to which the emotional, behavioral, and social adjustment of each class was
significant improvements for children in the moderate risk and impulsive kids classes, their behavior could have remained problematic.
REFERENCES
Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 52, 317-332, doi: 00333123/87/0900-SS01
Baker-Henningham, H., Scott, S., Jones, K., & Walker, S. (2012). Reducing child conduct problems and promoting social skills in a middle-income country: Cluster randomised controlled trial. British Journal of Psychiatry, 201(2), 101–108.
https://doi.org/10.1192/bjp.bp.111.096834
Bayer, J. K., Ukoumunne, O. C., Mathers, M., Wake, M., Abdi, N., & Hiscock, H. (2012). Development of children’s internalising and externalising problems from infancy to five years of age. The Australian and New Zealand Journal of Psychiatry, 46(7), 659–68. https://doi.org/10.1177/0004867412450076
Beauchaine, T. P., Webster-Stratton, C., & Reid, M. J. (2005). Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: a latent growth curve analysis. Journal of Consulting and Clinical Psychology, 73(3), 371– 388. https://doi.org/10.1037/0022-006X.73.3.371
Bierman, K. L., Nix, R. L., Greenberg, M. T., Blair, C., & Domitrovich, C. E. (2008). Executive functions and school readiness intervention: Impact, moderation, and mediation in the Head Start REDI program. Development and Psychopathology, 20(2008), 821–843.
https://doi.org/10.1017/S0954579408000394
Bornstein, M. H., Hahn, C.-S., & Hayness, O. M. (2012). Social Competence, Externalizing, and Internalizing Behavioral Adjustment from Early Childhood through Early Adolescence: Developmental Cascades. Developmental Psychopathology, 22(4), 717–735.
Brownell, R. 2000. Expressive One-Word Picture Vocabulary Test Manual. Novata, CA: Academic Therapy Publications
Broekhuizen, M.L., Mokrova, I.L., Burchinal, M.R., Garrett-Peters, P.T. (2016). Classroom quality at pre-kindergarten and kindergarten and children's social skills and behavior problems. Early Childhood Research Quarterly, 36, 212-222.
http://dx.doi.org/10.1016/j.ecresq.2016.01.005
Burchinal, M., & Roberts, J. (2006). Social risk and protective child, parenting, and child care factors in early elementary school years. Parenting: Science and Practice, 6 (1), 79–113. https://doi.org/10.1207/s15327922par0601_4
Cadima, J., Enrico, M., Ferreira, T., Verschueren, K., Leal, T., & Matos, P. M. (2016). Self-regulation in early childhood: the interplay between family risk, temperament and teacher– child interactions. European Journal of Developmental Psychology, 13(3), 341–360. https://doi.org/10.1080/17405629.2016.1161506
Carlson, J. S., Tiret, H. B., Bender, S. L., & Benson, L. (2011). The Influence of Group Training in the Incredible Years Teacher Classroom Management Program on Preschool Teachers’ Classroom Management Strategies. Journal of Applied School Psychology, 27(2), 134–154. https://doi.org/10.1080/15377903.2011.565277
Chow, J. C., & Wehby, J. H. (2016). Associations Between Language and Problem Behavior: a Systematic Review and Correlational Meta-analysis. Educational Psychology Review, 1-22. https://doi.org/10.1007/s10648-016-9385-z
Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. New York: Wiley.
Moderation to Examine Differential Treatment Effects. Child Development, 85(6), 2317– 2338. https://doi.org/10.1111/cdev.12278
Denham, S. A., Bassett, H., Mincic, M., Kalb, S., Way, E., Wyatt, T., & Segal, Y. (2012). Social-emotional learning profiles of preschoolers’ early school success: A person-centered approach. Learning and Individual Differences, 22(2), 178–189.
https://doi.org/10.1016/j.lindif.2011.05.001
Denham, S. A., Caverly, S., Schmidt, M., Blair, K., DeMulder, E., Caal, S., … Mason, T. (2002). Preschool understanding of emotions: Contributions to classroom anger and aggression. Journal of Child Psychology and Psychiatry and Allied Disciplines, 43(7), 901–916. https://doi.org/10.1111/1469-7610.00139
Durlak, J.A., DuPre, E.P. (2008). Implementatino Matters: a review of the research on the influence of implementatino on program outcomes and the factors affecting
implementation. American Journal of Community Psychology, (41), 327-350. DOI 10.1007/s10464-008-9165-0
Eisenberg, N., Valiente, C., Spinrad, T. L., Cumberland, A., Liew, J., Reiser, M., … Losoya, S. H. (2009). Longitudinal relations of children’s effortful control, impulsivity, and negative emotionality to their externalizing, internalizing, and co-occurring behavior problems. Developmental Psychology, 45(4), 988–1008. https://doi.org/10.1037/a0016213
Fanti, K. a, & Henrich, C. C. (2010). Trajectories of pure and co-occurring internalizing and externalizing problems from age 2 to age 12: findings from the National Institute of Child Health and Human Development Study of Early Child Care. Developmental Psychology, 46(5), 1159–1175. https://doi.org/10.1037/a0020659
incredible years teacher programme. New Zealand Journal of Psychology, 42(2), 51–56. Garner, P. W., Jones, D. C., & Miner, J. L. (1994). Social Competence among Low-Income
Preschoolers : Emotion Socialization Practices and Social Cognitive Correlates Author ( s ): Pamela W . Garner , Diane Carlson Jones and Jennifer L . Miner Published by : Wiley on behalf of the Society for Research in Ch, 65(2), 622–637.
Goodman, S. H., Rouse, M. H., Connell, A. M., Broth, M. R., Hall, C. M., & Heyward, D. (2011). Maternal Depression and Child Psychopathology: A Meta-Analytic Review.
Clinical Child and Family Psychology Review, 14(1), 1–27. https://doi.org/10.1007/s10567-010-0080-1
Greenberg, M.T., Abenavoli, R.A. (2017. Universal interventinos: fully exploring their impacts and potential to produce population-level impacts. Journal of Research on Educational Effectiveness. 10(1), 40-67. DOI: 10.1080/19345747.2016.1246632
Gresham, F., & Elliott, S. N. (1990). Social Skills Rating System Manual. Minneapolis, MN: NCS Pearson, Inc.
Harland, P., Reijneveld, S. A., Brugman, E., Verloove-Vanhorick, S. P., & Verhulst, F. C. (2002). Family factors and life events as risk factors for behavioural and emotional problems in children. European Child & Adolescent Psychiatry, 11, 176–184. https://doi.org/10.1007/s00787-002-0277-z
Heberle, A. E., Krill, S. C., Briggs-Gowan, M. J., & Carter, A. S. (2015). Predicting
Externalizing and Internalizing Behavior in Kindergarten: Examining the Buffering Role of Early Social Support. Journal of Clinical Child and Adolescent Psychology , 44 (4), 640-654. https://doi.org/10.1080/15374416.2014.886254
from developing and researching the Webster-Stratton Incredible Years Teacher Classroom Management Training Programme in North West Wales. Journal of Children’s Services, 2(3), 15–26. https://doi.org/10.1108/17466660200700023
Hutchings, J., Martin-Forbes, P., Daley, D., & Williams, M. E. (2013). A randomized controlled trial of the impact of a teacher classroom management program on the classroom behavior of children with and without behavior problems. Journal of School Psychology, 51(5), 571– 585. https://doi.org/10.1016/j.jsp.2013.08.001
Jobe-Shields, L., Andrews, A. R., Parra, G. R., & Williams, N. A. (2015). Person-Centered Approaches to Understanding Early Family Risk. Journal of Family Theory & Review, 7(4), 432–451. https://doi.org/10.1111/jftr.12118
Jones, S. M., Bub, K. L., & Raver, C. C. (2013). Unpacking the Black Box of the CSRP Intervention: The Mediating Roles of Teacher-child Relationship Quality and Self-regulation. Early Education and Development, 24(7), 1043–1064.
https://doi.org/10.1080/10409289.2013.825188
Kaiser, A.P., Hancock, T.B., Gai, X., Foster, E.M., Hester, P.P., (2000). Parent reported behavioral problems and language delays in boys and girls enrolled in Head Start classrooms. Behavioral Disorders, 26(1), 26-41.
Kessler, R. C., Barker, P. R., Colpe, L. J., Epstein, J. F., Gfroerer, J. C., Hiripi, E., … Zaslavsky, A. M. (2003). Screening for serious mental illness in the general population. Archives of General Psychiatry, 60, 184–189. https://doi.org/10.1001/archpsyc.60.2.184
https://doi.org/10.1016/j.infbeh.2011.02.002
Lanza, S. T., Rhoades, B. L., Nix, R. L., & Greenberg, M. T. (2010). Modeling the interplay of multilevel risk factors for future academic and behavior problems: a person-centered approach. Development and Psychopathology, 22(2), 313–335.
https://doi.org/10.1017/S0954579410000088
Mattera, S., Lloyd, C.M., Fishman, M., Bangser, M., (2013). A First Look at the Head Start CARES Demonstration: Large-Scale Implementation of Programs to Improve Children’s Social-Emotional Competence. OPRE Report. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services
Morris, P., Mattera, S. K., Castells, N., Bangser, M., Bierman, K., & Raver, C. (2014). Impact Findings from the Head Start CARES Demonstration: National Evaluation of Three
Approaches to Improving Preschoolers’ Social and Emotional Competence. OPRE Report. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Murray, D.W., Rabiner, D.L., Kuhn, L., Pan, Y., Sabet, R.F., (2018). Investigating teacher and student effects of the Incredible Years Classroom Management Program in early elementary school. Journal of School Psychology, 67, 119-133.
http://dx.doi.org/10.1016/j.jsp.2017.10.004
Parcel, T. L., & Menaghan, E. G. (1988). Measuring behavior problems in a large cross sectional survey: Reliability and validity for children of the NLS youth. Columbus: Ohio State University, Department of Sociology.
Ponitz, C. E., McClelland, M. M., Jewkes, A. M., Connor, C. M., Farris, C. L., & Morrison, F. J. (2008). Touch your toes! Developing a direct measure of behavioral regulation in early childhood. Early Childhood Research Quarterly, 23(2), 141–158.
https://doi.org/10.1016/j.ecresq.2007.01.004
Pratt, M. E., McClelland, M. M., Swanson, J., & Lipscomb, S. T. (2016). Family risk profiles and school readiness: A person-centered approach. Early Childhood Research Quarterly, 36, 462–474. https://doi.org/10.1016/j.ecresq.2016.01.017
Qi, C. H., & Kaiser, A. P. (2016). Behavior Problems of Preschool Children Review of the Literature, 216(2003).
Raver, C. C., Jones, S. M., Li-Grining, C. P., Metzger, M., Champion, K. M., & Sardin, L. (2008). Improving preschool classroom processes: Preliminary findings from a randomized trial implemented in Head Start settings. Early Childhood Research Quarterly, 23(1), 10– 26. https://doi.org/10.1016/j.ecresq.2007.09.001
Raver, C. C., Jones, S. M., Li-Grining, C., Zhai, F., Metzger, M. W., & Solomon, B. (2009). Targeting children’s behavior problems in preschool classrooms: A cluster-randomized controlled trial. Journal of Consulting and Clinical Psychology, 77(2), 302–316. https://doi.org/10.1037/a0015302
Reiss, F. (2013). Socioeconomic inequalities and mental health problems in children and adolescents: A systematic review. Social Science and Medicine, 90, 24–31.
Reinke, W.M., Herman, K.C., Dong, N., (2018). The Incredible Years Teacher Classroom Management Program: outcomes from a group randomized trial. Society for Prevention Research, https://doi.org/10.1007/s11121-018-0932-3
Rhoades, B. L., Greenberg, M. T., & Domitrovich, C. E. (2009). The contribution of inhibitory control to preschoolers’ social-emotional competence. Journal of Applied Developmental
Psychology, 30(3), 310–320. https://doi.org/10.1016/j.appdev.2008.12.012
Rijlaarsdam, J., Stevens, G. W. J. M., Van Der Ende, J., Hofman, A., Jaddoe, V. W. V,
MacKenbach, J. P., … Tiemeier, H. (2013). Economic disadvantage and young children’s emotional and behavioral problems: Mechanisms of risk. Journal of Abnormal Child Psychology, 41(1), 125–137. https://doi.org/10.1007/s10802-012-9655-2
Roy, A., & Raver, C. C. (2014). Are all risks equal? Early experiences of poverty-related risk and children’s functioning. Journal of Family Psychology, 28(3), 391–400.
Schindler, H.S., Kholoptseva, J., Oh, S.S., Yoshikawa, H., Duncan, G.. Magnunson,K.A., Shonkoff, .P., (2015). Maximizing the potential of early childhood education to prevent externalizing behavior problems: a meta-analysis. Journal of School Psychology, 53, 243-263. http://dx.doi.org/10.1016/j.jsp.2015.04.001
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. Snyder, J., Low, S., Schultz, T., Barner, S., Moreno, D., Garst, M., … Schrepferman, L. (2011).
The impact of brief teacher training on classroom management and child behavior in at-risk preschool settings: Mediators and treatment utility. Journal of Applied Developmental Psychology, 32(6), 336–345. https://doi.org/10.1016/j.appdev.2011.06.001
Child: Care, Health and Development, 33(6), 794–803. https://doi.org/10.1111/j.1365-2214.2007.00739.x
Turney, K. (2012). Pathways of disadvantage: Explaining the relationship between maternal depression and children’s problem behaviors. Social Science Research, 41(6), 1546–1564. https://doi.org/10.1016/j.ssresearch.2012.06.003
United States. Advisory Committee on Head Start Research and Evaluation. (2012). Advisory Committee on Head Start Research and Evaluation: Final report. Washington, DC: U.S. Department of Health and Human Services. Retrieved November 28, 2012, from
http://www.acf.hhs.gov/sites/default/files/opre/eval_final.pdf.
U.S. Department of Health and Human Services, Administration for Children and Families (2010). Head Start Impact Study. Final Report. Washington, DC.
Webster-Stratton, C. (2000). The Incredible Years Training Series. Juvenile Justice Bulletin, 1– 23.
Webster-Stratton, C. (2011). The Incredible Years Parents, Teachers and Children’s Training Series: Program Content, Methods, Research and Dissemination 1980-2011.
https://doi.org/10.5093/in2012a15
Webster-Stratton, C., Reid, J., & Hammond, M. (2004). Treating children with early-onset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33(1), 105–124.
https://doi.org/10.1207/S15374424JCCP3301_11
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2001). Preventing conduct problems, promoting social competence: a parent and teacher training partnership in head start. Journal of Clinical Child Psychology, 30(3), 283–302.
https://doi.org/10.1207/S15374424JCCP3003_2
Webster-Stratton, C., Reinke, W. M., Herman, K. C., & Newcomer, L. L. (2011). The Incredible Years Teacher Classroom Management Training: The Methods and Principles That Support Fidelity of Training Delivery. School Psychology Review, 40(4), 509–529.
Yoshikawa, H., Aber, J. L., & Beardslee, W. R. (2012). The effects of poverty on the mental, emotional, and behavioral health of children and youth: Implications for prevention. American Psychologist, 67(4), 272–284. https://doi.org/10.1037/a0028015
