Keywords: Early disruptive behavior, Preschool, Meta-analysis, Categorical approach, Dimensional approach
1. Introduction
Disruptive behavior problems in preschool children are a significant concern, affecting 2–6% of the general population at clinically significant levels. These issues pose challenges for schools, families, and public health systems and are the most common reason for referrals to child mental health clinics (APA, 2000; Keenan & Wakschlag, 2000). Often beginning as early as 3 or 4 years of age (Campbell, Ewing, Breaux, & Szumowski, 1986; Keenan, Shaw, Delliquadri, Giovannelli, & Walsh, 1998; Loeber, 1990), early disruptive behavior can have a poor prognosis, with many “early starters” continuing to exhibit disruptive behaviors throughout childhood and beyond (Earls, 1980; White, Moffitt, Earls, & Robbins, 1990; Zahn-Waxler, Ianotti, Cummings, & Denham, 1990). For instance, Campbell (1990) found that roughly half of preschoolers with disruptive behavior problems continued to show clinically significant difficulties into school age, with two-thirds meeting the criteria for a disruptive behavior disorder (such as Attention Deficit/Hyperactivity Disorder or ADHD, and Oppositional Defiant Disorder or ODD) by age 9. Similar findings have been reported by Egeland, Kalkoske, Gottesman, and Erickson (1990) and Loeber (1982). In summary, disruptive behavior problems, particularly when they emerge in the preschool years, are often associated with chronic, long-term difficulties.
The assessment of preschool disruptive behavior problems is complex due to the common use of two conceptual approaches—Categorical Diagnosis and dimensional assessment—which do not always yield directly comparable data (Eyberg, Schuhmann, & Rey, 1998; Keenan & Wakschlag, 2000; Waldman, Lilienfeld, & Lahey, 1995). A categorical diagnosis approach uses diagnostic criteria to determine the presence or absence of disruptive or other abnormal behaviors, exemplified by the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV, APA, 2000). In contrast, a dimensional approach places these behaviors on a continuum of frequency and/or severity, as seen in tools like the Child Behavior Checklist (CBCL, Achenbach & Edelbrock, 1983; Lavigne et al., 1996). Both methods have significant advantages and disadvantages and are supported by extensive research traditions, including Paul Meehl’s (2001) taxometric work and Jacob Cohen’s (1983) research on the loss of explanatory power when continuous measures are dichotomized (for a comprehensive review, see Beauchaine, 2003).
1.1. Comparing Categorical and Dimensional Approaches to Early Disruptive Behavior
Research into disruptive behavior in children and adolescents has often included comparisons between categorical diagnosis and dimensional assessment to identify their similarities and differences. Studies indicate that while both approaches produce overlapping data, direct comparisons are often challenging due to variations in measurement tools and analytical methods (Scholte, Van Berckelaer-Onnes, & Van Der Ploeg, 2002; Teagarden & Burns, 1999).
Currently, only two studies have specifically compared categorical diagnosis and dimensional assessment in preschool disruptive behavior problems (Pelletier, Collett, Gimpel, & Crowley, 2006; Sprafkin, Volpe, Gadow, Nolan, & Kelly, 2002). These studies employed different instruments: two categorical diagnosis scales and two dimensional rating scales. Pelletier et al. (2006) used the Disruptive Behavior Disorders Rating Scale (DBDRS), a categorical tool, and the School Situations Questionnaire (SSQ), a dimensional tool. They found significant correlations between data from both approaches in measuring Attention Deficit/Hyperactivity Disorder (ADHD) (r =.77, p < .01) and Oppositional Defiant Disorder (ODD) (r =.69, p < .01). Sprafkin et al. (2002) compared the Early Childhood Inventory-4 (ECI-4), a categorical diagnosis rating scale, and the Child Behavior Checklist (CBCL), a dimensional scale. Their findings also showed high correlations between the scores obtained from these two approaches, for both ADHD (r =.70, p < .001) and ODD (r =.81, p < .001).
Comparison of Categorical and Dimensional Approaches in Early Disruptive Behavior Assessment
1.2. Estimating the Stability of Early Disruptive Behavior
Numerous studies using both categorical diagnosis and dimensional approaches have investigated the stability of early disruptive behavior over time (Campbell, 1994; Campbell & Ewing, 1990; Crawford & Lee, 1991; Gadow, Sprafkin, & Nolan, 2001; Olson & Brodfeld, 1991). However, comparing their findings is challenging due to the variety of assessment instruments used.
In general, both categorical diagnosis and dimensional studies suggest that early disruptive behavior problems tend to be stable. Considering categorical diagnosis evidence, Gadow et al. (2001) found that preschoolers rated by parents and teachers as having disruptive symptoms on the ECI-4 continued to show these symptoms eight months later using the same instrument (r =.46, p < .01). Similarly, Lahey, Pelham, Loney, Lee, and Willcutt (2005) used the Diagnostic Interview Schedule for Children (DISC) and the DBDRS for parent and teacher ratings of preschoolers’ disruptive behavior symptoms at baseline and a one-year follow-up, finding relative stability over time (r =.23, p < .05). Pierce, Ewing, and Campbell (1999) demonstrated long-term stability over a decade, assessing problems at age 3 using the Swanson, Nelson, and Pelham Scale (SNAP) by parents, again at age 9 using SNAP, and at age 13 via semi-structured interview. Results showed that 67% of children with behavior problems at age 3 met criteria for an externalizing disorder at age 9, and over half continued disruptive behavior at age 13.
Dimensional studies also support stability. Crawford and Lee (1991) assessed preschoolers twice with the CBCL over six weeks and found consistent externalizing problem ratings. Campbell (1994) using CBCL parent ratings at ages 3 and 6, also showed that children with elevated externalizing problem ratings at age 3 tended to have similarly elevated ratings three years later.
Comparisons across studies on stability are complex due to several factors. Firstly, definitions of disruptive behavior problems vary. For example, Gadow et al. (2001) defined early disruptiveness using separate categorical diagnosis ratings for ADHD and ODD, Campbell (1994) using categorical diagnosis ratings for ADHD only, and Campbell and Ewing (1990) using categorical diagnosis ratings for ADHD and dimensional ratings of externalizing problems. Secondly, follow-up periods vary significantly, from 6 weeks to 10 years. Lastly, statistical measures of stability differ, including chi-square values (e.g., Egeland et al., 1990; Pierce et al., 1999), correlations (e.g., Crawford & Lee, 1991; Gadow et al., 2001), and percentages of children continuing disruptive behavior at follow-up (e.g., Campbell, 1994; Campbell et al., 1986). Meta-analysis, by converting these varied statistics into effect sizes, can significantly improve the interpretation of results across studies, providing quantitative estimates of stability for both categorical diagnosis and dimensional approaches.
1.3. Comparing Disruptive Behavior in Referred and Non-Referred Preschoolers
Several studies have investigated whether categorical diagnosis and/or dimensional approaches can differentiate between preschoolers referred for clinical services due to disruptive behavior problems and their non-referred peers. Generally, both approaches are effective. Researchers have successfully used various instruments in a categorical diagnosis manner to distinguish referred children, including the SNAP (Campbell, 1994), the ECI-4 (Gadow et al., 2001), the Preschool Behavior Questionnaire (PBQ) (Pierce et al., 1999), and the Kiddie-Disruptive Behavior Disorders Scale (K-DBDS) (Keenan et al., 2007).
Dimensional approaches, primarily using the CBCL, also show that referred preschoolers have significantly higher scores (p < .01, < .001) on externalizing behavior scales compared to non-referred peers (Achenbach, Edelbrock, & Howell, 1987; Campbell & Ewing, 1990; Heller, Baker, Henker, & Hinshaw, 1996). Overall, both categorical diagnosis and dimensional studies lead to similar conclusions when comparing referred and non-referred children, despite variations in defining early disruptiveness and the limitations mentioned regarding stability studies.
1.4. Aims of the Present Study
This study aimed to synthesize the existing literature on preschool disruptive behavior problems through a meta-analytic review of studies using categorical diagnosis and/or dimensional approaches. The review sought to determine, regardless of assessment method, if: (a) disruptiveness can be reliably measured in preschool years; (b) early disruptiveness is stable over time; and (c) children referred for clinical services for disruptive behavior can be differentiated from non-referred peers.
2. Methods
2.1. Literature Search
Studies were identified through: (a) PsycINFO and MEDLINE searches using keywords “disruptive behavior,” “preschool,” “categorical,” and “dimensional”; (b) manual searches of key journals in preschool disruptive behavior research2; (c) letters to 13 researchers in early disruptiveness assessment, requesting unpublished manuscripts and in-press articles (9 responses received); and (d) reference list reviews of relevant studies. The search included studies published from 1986 to 2006, reflecting the primary period of research in preschool disruptive behavior.
2.2. Inclusion Criteria
Studies were included if they met the following criteria:
- Published in English.
- Focused on children aged 2–7 years (preschool age range). Studies with broader age ranges (e.g., 2–19 years) were excluded.
- Sample size of at least 25 to ensure adequate power and exclude single case studies.
- Informants were parents and/or teachers of child participants.
- Included one or more categorical diagnosis and/or dimensional measures of preschool disruptive behavior problems and addressed at least one of the three research questions.
- Compared two or more groups to enable effect size calculation (e.g., referred vs. non-referred children).
- Used unique samples to avoid including the same children in multiple studies (e.g., when a research team published multiple studies on the same sample, only the first published study was used).
The search yielded 26 published or in-press studies. Additional identified studies were excluded for not meeting criteria (e.g., Gillion, Shaw, Beck, Schonberg, & Lukon, 2002; Vondra, Shaw, Swearingen, Cohen, & Owens, 2001) or lacking necessary statistics for analyses (e.g., Lavigne et al., 1998; Lavigne et al., 1998; Shaw, Owens, Giovannelli, & Winslow, 2001).
2.3. Coding Procedures
The senior author coded all 26 studies using a standardized coding sheet for this review. Coding included: (1) research question(s) addressed; (2) approach adopted (categorical diagnosis, dimensional, or both); (3) sample sizes, means, standard deviations, and percentages for each group; (4) available test statistics (r, t, χ2, F values); (5) demographic information (child gender, age, ethnicity; parent age, marital status, income, education, if reported); (6) sample type (e.g., referred, community, school); and (7) study setting (e.g., clinic, school). Reliability was assessed by independent coding of five studies (19%) by an additional coder, showing satisfactory inter-rater reliability (kappa=.79).
2.4. Calculation of Effect Sizes
For each research question, two groups were identified in relevant studies, and effect sizes were calculated to determine group differences. Hedge’s g was used for individual effect sizes, and Cohen’s d for overall cumulative effects (averaging Hedge’s g values). R values are also reported in Table 2.
Table 2. Individual effect sizes
| Article | Measure | r | Effect size (CI) | Size | r | Effect size (CI) | Size | r | Effect size (CI) | Size |
|---|---|---|---|---|---|---|---|---|---|---|
| chenbach, Edelbrock, and Howell (2004) | Child Behavior Checklist | DIM | 0.70 | 1.96 (1.60/2.32) | L | 0.40 | 0.87 (0.57/1.18) | L | ||
| Alink et al. (2006)–Study 1 | Physical Aggression Scale for Early Child | DIM | 0.63 | 1.62 (1.27/1.97) | L | |||||
| Alink et al. (2006)–Study 2 | Physical Aggression Scale for Early Child | DIM | 0.72 | 2.08 (1.71/2.44) | L | |||||
| Campbell (1987) | Child Behavior Checklist | DIM | 0.66 | 1.77 (1.32/2.22) | L | |||||
| Campbell (1994) | Swanson, Pelham, and Nolan Questionnaire | C-AD | 0.53 | 1.25 (0.54/1.97) | L | 0.51 | 1.19 (0.65/1.73) | L | ||
| Campbell (1994) | Swanson, Pelham, and Nolan Questionnaire | C-OD | 0.41 | 0.91 (0.38/1.43) | L | |||||
| Campbell and Ewing (1990) | Child Behavior Checklist | DIM | 0.46 | 1.04 (0.61/1.46) | L | 0.49 | 1.13 (0.44/1.82) | L | ||
| Campbell and Ewing (1990) | Child Behavior Checklist | C-AD | 0.86 | 2.69 (2.14/3.25) | L | 0.59 | 1.46 (0.73/2.18) | L | ||
| Campbell et al. (1986) | Child Behavior Checklist | DIM | 0.58 | 1.41 (0.87/1.95) | L | |||||
| Crawford and Lee (1991) | Child Behavior Checklist | DIM | 0.74 | 2.00 (1.56/2.84) | L | |||||
| Egeland et al. (1990) | Child Behavior Checklist | DIM | 0.33 | 0.70 (0.13/1.27) | M | |||||
| Fagot and Leve (1998) | Child Behavior Checklist | DIM | 0.41 | 0.90 (0.67/1.13) | L | |||||
| Gadow et al. (2001) | Early Childhood Inventory-4 | C-AD | 0.46 | 1.04 (0.79/1.29) | L | 0.51 | 1.20 (0.99/1.41) | L | ||
| Gadow et al. (2001) | Early Childhood Invenory-4 | C-OD | 0.56 | 1.35 (1.09/1.61) | L | 0.39 | 0.86 (0.66/1.06) | L | ||
| Heller et al. (1996) | Child Behavior Checklist | DIM | 0.60 | 1.50 (1.14/1.86) | L | 0.53 | 1.24 (0.51/1.95) | L | ||
| Keenan and Wakschlag (2000) | K-SADS-Epidemiological 5th ver. | C-OD | 0.29 | 0.61 (0.24/0.97) | M | |||||
| Keenan et al. (in press) | Kiddie-Disruptive Behavior Disorder Schedule | C-AD | 0.83 | 2.97 (2.59/3.35) | L | |||||
| Lahey et al. (2005) | Disruptive Behavior Disorders Rating Scale | C-AD | 0.23 | 0.47 (0.21/0/73) | M | |||||
| Marakovitz and Campbell (1998) | Swanson, Pelham, and Nolan Questionnaire | C-AD | 0.26 | 0.54 (0.27/0.81) | M | |||||
| Mesman et al. (2001) | Child Behavior Checklist | DIM | 0.01 | 0.02 (–0.19/0.23) | S | |||||
| Mesman and Koot (2001) | Child Behavior Checklist | DIM | 0.09 | 0.17 (0.02/0.32) | S | |||||
| Olson and Brodfeld (1991) | Conners Teacher Questionnaire | C-OD | 0.80 | 2.67 (2.14/3.19) | L | |||||
| Owens and Shaw (2003) | Child Behavior Checklist | DIM | 0.39 | 0.86 (0.68/1.03) | L | |||||
| Pelletier et al. (2006) | Disruptive Behavior Disorders Rating Scale | AD | 0.77 | 2.41 (2.16/ 2.67) | L | |||||
| Pelletier et al. (2006) | Disruptive Behavior Disorders Rating Scale | OD | 0.69 | 1.91 (1.67/ 2.14) | L | |||||
| Pierce et al. (1999) | Swanson, Pelham, and Nolan Questionnaire | C-AD | 0.32 | 0.67 (0.07/1.28) | M | |||||
| Pierce et al. (1999) | Swanson, Pelham, and Nolan Questionnaire | DB | 0.34 | 0.72 (0.30/1.14) | M | |||||
| Pierce et al. (1999) | Swanson, Pelham, and Nolan Questionnaire | DB | 0.77 | 2.41 (1.54/3.28) | L | |||||
| Querido and Eyberg (2003) | Disruptive Behavior Disorders Rating Scale | DB | 0.45 | 0.99 (0.61/1.38) | L | |||||
| Speltz et al. (1999) | Child Behavior Checklist | DIM | 0.18 | 0.37 (0.08/0.67) | S | |||||
| Sprafkin et al. (2002) | Early Childhood Inventory-4 | AD | 0.70 | 1.96 (1.71/ 2.21) | L | 0.13 | 0.27 (0.02/0.56) | |||
| Sprafkin et al. (2002) | Early Childhood Inventory-4 | OD | 0.81 | 2.76 (2.48/ 3.05) | L | 0.20 | 0.42 (0.13/0.70) | S | ||
| Wakschlag and Keenan (2001) | K-SADS-Epidemiological 5th ver. | C-OD | 0.74 | 2.20 (1.75/2.64) | L |
IM = Dimensional Scale; C-AD = Categorical Scale—ADHD; C-ODD = Categorical Scale—ODD; EXT = externalizing; AD = ADHD; OD = ODD; L = large effect; M = medium effect; S = small effect
The DSTAT program (Johnson, 1989) was used to calculate effect sizes using Hedge’s g:
| g=M1−M2σpooled |
|---|
where
| σpooled=[(σ12+σ22)/2]. |
|---|
Hedge’s g was calculated from test statistics like t, F, or p values for each study, adjusting for sample size, method, and cross-sectional variance. When multiple effect size calculations were possible, all were averaged (Hedges and Olkin, 1985). For categorical diagnosis studies measuring ADHD and ODD, effect sizes were calculated and averaged for each type, allowing analyses for each disruptive behavior type and an overall disruptiveness estimate.
Composite effect sizes (Cohen’s d) were calculated by averaging relevant effect sizes with 95% confidence intervals. These weigh each g by the reciprocal of its variance, giving more weight to larger sample studies (Hedges & Olkin, 1985). Homogeneity of g’s was assessed to determine if a single estimate adequately described studies (Hedges & Olkin, 1985). Heterogeneous effect sizes underwent model testing and outlier diagnosis using Hedges and Olkin’s (1985) post hoc procedures to explain variability and/or restore homogeneity. Outlier studies were identified for each research question.
Effect sizes below 0.20 were considered small, around 0.50 medium, and above 0.80 large (Cohen, 1988). Rosenthal’s fail-safe N was calculated to assess publication bias (Wolf, 1986; Wortman, 1994), indicating the number of null-result studies needed to make significant effect sizes non-significant (p > .05) (Rosenthal, 1979).
3. Results
The 26 studies included 4,536 preschoolers aged 2.00–6.10 years (M = 3.86)3, predominantly boys (67%) and Caucasian (68%). These demographics are estimates due to incomplete reporting in 11 studies. Parental demographic data were largely unreported. See Table 1 for descriptive sample information. Studies addressing multiple research questions contributed data to each relevant question.
Table 1. Descriptive statistics of reviewed articles
| Article | N | Child gender | Child age | Child ethnicity 1 | Parent age | Marital status 2 | Income level | Parent education | Clinical/ community | Setting |
|---|---|---|---|---|---|---|---|---|---|---|
| Achenbach et al. (1987) | 87 | – | – | – | – | – | – | – | – | – |
| Alink et al. (2006) | 744 | 390 (52%) boys 354 (48%) girls | M = 2.98 (0.09) Range= 2.75– 3.33 | “Mostly” EA | – | – | – | – | Community | – |
| Campbell et al. (1986) | 68 | 41 (60%) boys 27 (40%) girls | M = 2.92 Range= 2.00– 3.00 | – | – | – | “Range in social class” | – | Parent referred | – |
| Campbell and Ewing (1990) | 54 | 36 (67%) boys 18 (33%) girls | – | – | – | 46 (85%) M 8 (15%) S | – | – | Parent referred | Urban |
| Campbell (1987) | 68 | 41 (60%) boys 27 (40%) girls | M = 2.92 | – | – | – | – | – | Community | – |
| Campbell (1994) | 112 | 112 (100%) boys | M = 3.83 Range= 2.42– 4.83 | 110 (98%) EA | – | – | “Working and middle class” | – | School | – |
| Crawford and Lee (1991) | 30 | 14 (47%) boys 16 (53%) girls | Range= 2.00– 3.00 | 25 (83%) EA 2 (7%) H 3 (8%) Other | – | – | – | – | School | Rural |
| Egeland et al. (1990) | 96 | 52 (54%) boys 44 (46%) girls | Range= 4.50– 5.00 | – | – | – | M = $14,500 | – | School | – |
| Fagot and Leve (1998) | 156 | 82 (53%) boys 74 (47%) girls | – | 148 (95%) EA 3 (2%) AA 2 (1%) H 3 (2%) Other | – | 137 (88%) M | M = $15,000 | – | Community | – |
| Gadow et al. (2001) | 755 | 443 (59%) boys 312 (41%) girls | M = 4.25 (0.75) | 504 (67%) EA 70 (9%) AA 49 (6%) H 7 (1%) Other | – | – | – | – | Outpatient clinic | Urban |
| Heller et al. (1996) | 77 | 37 (48%) boys 40 (52%) girls | M = 4.60 (0.77) Range= 24– 49 | 53 (69%) EA | M = 37 | 12 (16%) S | 67 (87%) Middle class or above | – | Community | – |
| Keenan and Wakschlag (2000) | 79 | 61 (77%) boys 18 (23%) girls | M = 4.00(0.75) Range= 3.00– 4.11 | 10 (13%) EA 63 (80%) AA 1 (1%) H 5 (5%) Other | – | – | Low income (82% on welfare) | – | Inpatient clinic | Urban |
| Keenan et al. (in press) | 100 | 56 (56%) boys 44 (44%) girls | Range= 3.00– 5.00 | 82 (82%) AA 6 (6%) H | – | – | – | – | Outpatient clinic | Urban |
| Lahey et al. (2005) | 125 | 98 (78%) boys 20 (22%) girls | Range= 3.80– 7.00 | 74 (56%) EA 36 (29%) AA 8 (6%) Other | – | – | – | – | Outpatient clinic | Urban |
| Mesman, Bongers, and Koot (2001) | 397 | 204 (51%) boys 193 (49%) girls | M = 5.31 (0.64) | – | – | 345 (87%) M 52 (13%) S | – | – | Community | Urban |
| Mesman and Koot (2001) | 420 | 215 (51%) boys 205 (49%) girls | M = 2.60 (0.80) | – | – | – | – | – | Community | – |
| Marakovitz and Campbell (1998) | 112 | 112 (100%) boys | M = 3.83 | – | – | – | – | – | School | – |
| Olson and Brodfeld (1991) | 53 | 53 (100%) boys | M = 4.60 Range= 4.00– 5.50 | 51 (96%) EA | – | – | Low income | – | School | – |
| Owens and Shaw (2003) | 275 | 275 (100%) boys | – | 160 (58%) EA 110 (40%) AA 5 (2%) Other | – | 179 (65%) M 184 (67%) poverty level | 109 (76%) Unemployed; 160 (58%) no HS | – | Urban | |
| Pelletier et al. (2006) | 200 | 99 (50%) boys101 (50%) girls | M = 4.60 (0.50) | – | – | – | – | – | School | |
| Pierce et al. (1999) | 59 | – | M = 2.92 | – | – | – | – | – | Community | – |
| Querido and Eyberg (2003) | 74 | 40 (54%) boys 34 (46%) girls | M = 5.01 (1.37) Range= 3.20– 6.10 | 55 (74%) EA 9 (12%) AA 6 (9%) H 4 (5%) Other | – | 62 (84%) M | Range= $21,000– $30,000 | M = 14.69 (2.02) | School | Rural |
| Speltz et al. (1999) | 92 | 92 (100%) boys | M = 4.73 (0.52) | 76 (79%) EA 3 (7%) AA 13 (14%) Other | – | 59 (64%) M | – | – | Outpatient clinic | Urban |
| Sprafkin et al. (2002) | 224 | 172 (77%) boys 52 (23%) girls | M = 4.55 (0.77) Range= 3.00– 6.00 | 195 (87%) EA 13 (6%) AA 13 (6%) H 2 (1%) Other | – | – | – | – | Outpatient clinic | Rural |
| Wakschlag and Keenan (2001) | 79 | 59 (75%) boys 20 (25%) girls | M = 4.00 (0.78) Range= 3.00– 4.11 | 6 (8%) EA 63 (80%) AA 6 (8%) H 3 (4%) Other | – | – | 59 (88%) on welfare | – | Inpatient clinic | Urban |
1EA = European American, AA = African American, H = Hispanic, Other = Asian, Native American, or Biracial;
2M = married, S = single. – = Information not reported.
3.1. Comparing Categorical and Dimensional Approaches to Early Disruptive Behavior
Two studies compared categorical diagnosis and dimensional approaches for measuring early disruptiveness (Table 2). Effect sizes were calculated using means, standard deviations, Pearson r correlations, and/or t statistics, with larger effect sizes indicating greater similarity between approaches. High effect sizes suggest that categorical diagnosis and dimensional measures provide comparable information on early disruptive behavior.
Individual effect sizes for both studies were large (Table 2), as was the weighted mean effect size (d = 2.21, N = 388, 95% confidence interval [CI]= 2.08 to 2.34). While one outlier was identified, its exclusion did not significantly change the effect size. Separate calculations for ADHD and ODD also yielded comparable effect sizes (d = 2.17, N = 388, CI= 1.99 to 2.35 and d= 2.25, N = 388, CI= 2.07 to 2.43, respectively). Rosenthal’s fail-safe N indicated that 237 non-significant studies would be needed to render the weighted mean effect size non-significant.
Overall, individual and composite effect sizes demonstrate a highly significant relationship between categorical diagnosis and dimensional measures of early disruptiveness. Practically, preschoolers identified as disruptive by categorical diagnosis measures have a 99% likelihood of similar identification by dimensional measures, confirming that early disruptive behavior problems can be effectively identified using either approach.
3.2. Estimating the Stability of Early Disruptive Behavior
Twenty-one studies reported data on the stability of early disruptive behavior using categorical diagnosis or dimensional approaches. One study provided statistics for both approaches separately. Three studies were excluded for using categorical diagnosis at baseline and dimensional at follow-up, preventing categorization within a single approach. The remaining studies collected categorical diagnosis or dimensional ratings at baseline and follow-up, ranging from 6 months to 5 years. Effect sizes were based on means, standard deviations, and Pearson r correlations, with larger effect sizes indicating greater stability.
For the six studies using a categorical diagnosis approach, individual effect sizes ranged from medium to large (Table 2). The weighted mean effect size was large (d = 1.15, N = 395, CI=.99 to 1.30). Three outliers were identified, but their exclusion did not substantially alter the mean effect size (d = 1.12, N = 180, CI=.89 to 1.34). Separate effect sizes for ADHD (five studies) and ODD (two studies) were also large (ADHD: d =.78, N = 294, CI=.61 to .95; ODD: d = 1.60, N = 193, CI= 1.37 to 1.83). Rosenthal’s fail-safe N indicated that 72 non-significant studies would be needed to make the weighted mean effect size non-significant.
Fifteen studies used dimensional measures, primarily the CBCL, to assess stability. Eleven showed large effect sizes, one medium, and three small (Table 2). The weighted mean effect size for these fifteen studies was large (d = 0.84, N = 1329, CI= 0.76 to 0.92). Homogeneity tests identified nine outliers, including the four studies with small and medium effect sizes. Removing these nine studies increased the mean effect size (d = 1.52, N = 304, CI= 1.34 to 1.70). Rosenthal’s fail-safe N indicated that 107 null-result studies would be needed to make the weighted mean effect size non-significant.
Overall, both individual and composite effect sizes show that early disruptiveness tends to be stable, whether measured categorically or dimensionally. Practically, preschoolers with disruptive behavior problems at baseline are 86% more likely to show similar problems at follow-up when assessed categorically, and 96% more likely when assessed dimensionally, compared to non-disruptive preschoolers at baseline.
3.3. Comparing Disruptive Behavior in Referred and Non-Referred Preschoolers
Fourteen studies compared disruptive behavior in referred and non-referred preschoolers, 11 using a categorical diagnosis approach and three dimensionally. Effect sizes were calculated using means, standard deviations, F values, and χ2 statistics, with significant values indicating differences between referred and non-referred groups. Larger effect sizes represent greater differentiation between these groups.
Individual effect sizes in the 11 categorical diagnosis studies ranged from small to large (Table 2). As three studies measured ADHD and ODD separately, average effect sizes were used for composite effect size analyses. The weighted mean effect size for categorical diagnosis studies was large (d = 1.04, N = 1685, CI = .94 to 1.15). Five outliers were identified, but the effect size remained large after their exclusion (N = 892, d = .94, CI = .79 to 1.08). Rosenthal’s fail-safe N indicated that 127 studies would be needed to reduce the effect size to non-significance. Thus, categorical diagnosis approaches effectively distinguish preschoolers referred for disruptive behavior from non-referred peers, with significant group differences found 83% of the time.
Separate effect sizes were calculated for six studies measuring ADHD and nine measuring ODD (Table 2). ADHD studies showed a large effect size (d = 1.07, N = 1173, CI=.94 to 1.20). Four outliers were identified, but the effect remained large after exclusion (N = 104, d = 1.27, CI=.83 to 1.70). ODD studies (N = 1343) initially showed a medium effect size (d =.70, CI=.58 to .81), which slightly increased after excluding three outliers (N = 1021, d =.75, CI=.62 to .88). Rosenthal’s fail-safe N results showed that 94 null studies for ADHD and 43 for ODD would be needed to make these effects non-significant. In summary, categorical diagnosis approaches yielded medium to large differences between referred and non-referred children when ADHD and ODD were measured separately. Specifically, children referred for ADHD differed from non-referred children 90% of the time, and 77% for ODD referrals.
The three dimensional studies distinguishing referred from non-referred preschoolers all showed large individual effect sizes (Table 2). The composite weighted mean effect size was also large (N = 272, d =.95, CI=.69 to 1.21). Rosenthal’s fail-safe N indicated that 16 studies would be needed to reduce the result to non-significance. The large difference between referred and non-referred groups indicated that dimensional approaches differentiated these groups 83% of the time.
4. Discussion
This meta-analysis compared categorical diagnosis and dimensional approaches in assessing preschool disruptive behavior problems to determine: (a) the adequacy of measuring disruptiveness in preschool years; (b) the stability of early disruptiveness; and (c) the ability to differentiate referred disruptive preschoolers from non-referred peers. Reviewing 26 studies from 1986 to 2006, results showed that: categorical diagnosis and dimensional approaches provide comparable data for early disruptiveness assessment; both yield comparable estimates of stability; and both differentiate between referred and non-referred preschoolers. Each finding is discussed in detail below.
The first objective was to compare the effectiveness of categorical diagnosis and dimensional approaches in measuring preschool disruptive behavior problems. Results indicated close correspondence between data from both approaches at individual and aggregate levels. Although only two studies addressed this directly, their individual and weighted mean effect sizes were large, despite methodological differences. The similarity between categorical diagnosis and dimensional approaches in portraying early disruptiveness aligns with prior research suggesting these approaches are complementary and valuable in young children’s assessment (Arend, Lavigne, Rosenbaum, Binns, & Christoffel, 1996), in both research and clinical settings.
The second research question focused on comparing categorical diagnosis and dimensional measurements of disruptive behavior stability in preschool years. Results showed stability across both approaches, often over extended periods, at both individual and composite levels. Individual effects ranged from small to large, with 80% of studies showing large effects. Homogeneity tests identified outliers, but their removal did not alter the large mean effect sizes for both categorical diagnosis and dimensional approaches, which were similar. This support for early disruptive behavior stability is consistent with extensive research showing that “early starters” on an antisocial trajectory face cumulative challenges into adolescence and beyond (Dodge, Coie, & Lynam, 2006). This highlights the importance of prevention and early intervention programs for parents of preschoolers to redirect this trajectory early (Dumas, Nissley-Tsiopinis, & Moreland, 2006; Kazdin, 2005; Keenan & Wakschlag, 2000).
The third objective was to assess if categorical diagnosis and dimensional approaches differentiate between preschoolers referred for disruptive behavior and those not referred. Individual effects varied from small to large, with 73% of studies showing large effects. Outlier studies with small or medium effects did not change the large mean effect sizes upon removal. Thus, both approaches effectively distinguished referred disruptive preschoolers from non-referred peers. This is consistent with studies showing that referred children often differ from peers from a young age due to alarming oppositional and defiant conduct that hinders their development (Dumas, 1996).
Collectively, this meta-analysis confirms that disruptive behavior problems, whether measured categorically or dimensionally, can appear as early as age 3 and are distinguishable from typical preschool opposition and defiance. Beyond public health implications for prevention and early intervention, these findings suggest that longitudinal studies of behavioral and emotional problems in children should start earlier than many current projects that begin data collection in school-age years (e.g., Braswell, August, Bloomquist, & Realmuto, 1997; Masse & Tremblay, 1999).
4.1. Limitations and Suggestions for Future Research
Several limitations advise caution in over-interpreting these findings. First, this study was primarily methodological, comparing categorical diagnosis and dimensional approaches, rather than focusing on core early disruptiveness behaviors or developmental trajectories (Dodge et al., 2006).
Second, while both approaches show complementary data, they rely on limited assessment tools. Categorical diagnosis measures include SNAP (Campbell, 1994), ECI-4 (Gadow et al., 2001), PBQ (Pierce et al., 1999), and K-DBDS (Keenan et al., in press). Despite different names, they largely reflect DSM-IV criteria for disruptive behavior (ADHD and ODD). Dimensional measures are even more narrowly defined, with CBCL dominating, limiting the empirical definition of “categorical diagnosis” and “dimensional.”
Third, few studies addressed the research questions, especially the first comparison of approaches, indicating sparse research specifically on preschool disruptiveness. Studies including preschoolers, school-age children, and adolescents in the same analyses were excluded due to potentially non-preschool-specific findings. While findings are consistent, they are tentative due to the limited number of studies. Replications and studies breaking down findings by age group (preschoolers, school-age children, adolescents) are needed.
Fourth, sample diversity was limited, particularly in child gender and ethnicity. Estimated samples were mainly boys (67%) and Caucasian (68%). Future studies need more diverse samples to assess categorical diagnosis and dimensional approach adequacy across genders and ethnicities. None of the 26 studies broke down results by gender or ethnicity, and demographic data reporting was often incomplete, especially for parental demographics.
Finally, several studies provided limited statistics for meta-analysis, often omitting basic statistics. More comprehensive statistical data in future studies will enhance confidence in meta-analytic results.
These limitations highlight the lack of an “ideal” study to distinguish preschool disruptive behavior from normative behavior and outline developmental trajectories. Such a study would address the research questions, include referred and non-referred groups, use multiple categorical diagnosis and dimensional measures at baseline and follow-up, include diverse samples, report detailed demographics, and provide comprehensive statistics. Until such studies are available, reviews like this can only point to knowledge gaps and summarize current understandings.
4.2. Conclusion
This meta-analysis provides evidence that categorical diagnosis and dimensional approaches in early disruptiveness measurement: (1) yield comparable data; (2) demonstrate early disruptiveness stability; and (3) differentiate referred disruptive preschoolers from non-referred peers. Despite literature limitations, this suggests disruptive behavior problems can be identified as early as age 3, and both approaches can be used interchangeably or simultaneously for this purpose.
Acknowledgements
This study was supported by grant R21 HD40079 from the National Institute of Child Health and Human Development and by grant R49/CCR 522339 from the Centers for Disease Control and Prevention to the second author.
Footnotes
2Development and Psychopathology, Journal of Abnormal Child Psychology, Journal of Child Psychology and Psychiatry, Journal of Clinical Child Psychology, Journal of the American Academy of Child and Adolescent Psychiatry.
3Reported statistics are based only on the studies which provided demographic information on the children.
References*
[R1] Achenbach, T. M., & Edelbrock, C. S. (1983). Manual for the child behavior checklist and revised child behavior profile. Queen City Printers.
[R2] Achenbach, T. M., Edelbrock, C., & Howell, C. T. (1987). Direct observation of maladaptive behavior: I. Factorially derived syndromes. Journal of Abnormal Child Psychology, 15(3), 327–346.
[R3] Alink, L. R. A., Mesman, J., Van Zeijl, J., Van Ijzendoorn, M. H., Bakermans-Kranenburg, M. J., & Koot, H. M. (2006). The association between harsh discipline and developmental problems in early childhood: testing different models of discipline and different approaches to problem behavior. Journal of Child Psychology and Psychiatry, 47(3‐4), 295–305.
[R4] American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.).
[R5] Arend, R., Lavigne, J. V., Rosenbaum, D., Binns, H. J., & Christoffel, K. K. (1996). Dimensional and categorical approaches to the identification of emotional and behavioral problems in preschool children. Journal of Emotional and Behavioral Disorders, 4(1), 2–13.
[R6] Beauchaine, T. P. (2003). Taxometrics and developmental psychopathology. Development and Psychopathology, 15(3), 499–527.
[R7] Braswell, L., August, G., Bloomquist, M. L., & Realmuto, G. M. (1997). School-based secondary prevention for children with disruptive behavior: integrating family and peer interventions. Journal of Abnormal Child Psychology, 25(4), 303–317.
[R8] Campbell, S. B. (1987). Parent-referred problem three-year-olds: Follow-up at school entry. Journal of Child Psychology and Psychiatry, 28(6), 801–815.
[R9] Campbell, S. B. (1990). Longitudinal studies of active and aggressive children: Implications for prevention and intervention. Journal of Consulting and Clinical Psychology, 58(6), 711–718.
[R10] Campbell, S. B. (1994). Hard-to-manage preschool boys: Externalizing behavior, social competence, and family context during kindergarten and first grade. Journal of Abnormal Child Psychology, 22(6), 647–669.
[R11] Campbell, S. B., & Ewing, L. J. (1990). Hard-to-manage preschoolers: Adjustment at age nine years and predictors of continuing symptoms. Journal of Child Psychology and Psychiatry, 31(6), 871–889.
[R12] Campbell, S. B., Ewing, L. J., Breaux, A. M., & Szumowski, E. K. (1986). Parent-referred problem preschoolers: Mother-child interaction during play at ages 2 and 3. Journal of Abnormal Child Psychology, 14(2), 193–205.
[R13] Cohen, J. (1983). The cost of dichotomization. Applied Psychological Measurement, 7(3), 249–253.
[R14] Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
[R15] Crawford, J. W., & Lee, C. B. (1991). Emotionally disturbed preschoolers: Parent and teacher agreement using the child behavior checklist. Early Childhood Research Quarterly, 6(3), 431–437.
[R16] Dodge, K. A., Coie, J. D., & Lynam, D. R. (2006). Aggression and antisocial behavior in youth. In N. Eisenberg, W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology: Social, emotional, and personality development (6th ed., Vol. 3, pp. 719–788). John Wiley & Sons Inc.
[R17] Dumas, J. E. (1996). Antisocial behavior: A developmental perspective. In M. Lewis (Ed.), Child and adolescent psychiatry: A comprehensive textbook (2nd ed., pp. 559–575). Williams & Wilkins.
[R18] Dumas, J. E., Nissley-Tsiopinis, J., & Moreland, A. D. (2006). Home-based parent training with families of young children at risk for conduct disorder: Follow-up into middle childhood. Journal of Consulting and Clinical Psychology, 74(3), 474–487.
[R19] Earls, F. (1980). Prevalence of behavior problems in 3-year-old children. Archives of General Psychiatry, 37(10), 1153–1157.
[R20] Egeland, B., Kalkoske, M., Gottesman, D., & Erickson, M. F. (1990). Preschool behavior problems: Stability and factors accounting for change. Journal of Child Psychology and Psychiatry, 31(6), 891–909.
[R21] Eyberg, S. M., Schuhmann, E. M., & Rey, J. M. (1998). Disruptive behavior disorders in children. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child psychopathology (3rd ed., pp. 491–529). Plenum Press.
[R22] Fagot, B. I., & Leve, C. S. (1998). Behavior problems in socially rejected children: relations with cost and benefit pay-offs in boys’ and girls’ friendships. Behavior Therapy, 29(4), 663–678.
[R23] Gadow, K. D., Sprafkin, J., & Nolan, E. E. (2001). Preschoolers with ADHD symptoms: categorical or dimensional classification?. Journal of Child Psychology and Psychiatry, 42(6), 757–769.
[R24] Gillion, M. E., Shaw, D. S., Beck, J. E., Schonberg, M. A., & Lukon, J. L. (2002). Trajectories of disruptive behavior from toddlerhood to early school age for children with differing entry points. Journal of Abnormal Child Psychology, 30(6), 599–615.
[R25] Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Academic Press.
[R26] Heller, T. L., Baker, B. L., Henker, B., & Hinshaw, S. P. (1996). Externalizing behavior and cognitive functioning in preadolescent boys with ADHD: stability and prediction over 2 years. Journal of Abnormal Child Psychology, 24(2), 139–164.
[R27] Johnson, B. T. (1989). DSTAT: Software for the meta-analytic review of research literatures. Lawrence Erlbaum Associates.
[R28] Kazdin, A. E. (2005). Parent management training: Treatment for oppositional, aggressive, and antisocial behavior in children and adolescents. Oxford University Press.
[R29] Keenan, K., Shaw, D. S., Delliquadri, E., Giovannelli, J., & Walsh, B. (1998). Evidence for the continuity of early problem behaviors: application of a developmental psychopathology framework. Journal of Abnormal Child Psychology, 26(6), 441–456.
[R30] Keenan, K., & Wakschlag, L. S. (2000). More than just “bad boys”: the clinical significance of early onset conduct problems in boys and girls. In R. Cohen & L. S. Wakschlag (Eds.), Developmental approaches to psychopathology (pp. 202–229). Lawrence Erlbaum Associates Publishers.
[R31] Keenan, K., Wakschlag, L. S., Applegate, B., Hill, C., Danis, B., & Sherman, D. (2007). The Kiddie-Disruptive Behavior Disorders Scale (K-DBDS): convergent validity in clinic-referred preschoolers. Journal of Clinical Child & Adolescent Psychology, 36(3), 407–417.
[R32] Lahey, B. B., Pelham, W. E., Loney, J., Lee, S. S., & Willcutt, E. (2005). Instability of DSM-IV syndromes of ADHD and ODD among children with ADHD at 3-year follow-up. Journal of the American Academy of Child & Adolescent Psychiatry, 44(4), 344–350.
[R33] Lavigne, J. V., Cicchetti, D., Binns, H. J., Christoffel, K. K., Hayford, J. R., Starzyk, K. A., & Arend, R. (1998). Oppositional defiant disorder with onset in preschool years: longitudinal stability and pathways to other disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 37(10), 1077–1085.
[R34] Lavigne, J. V., Cicchetti, D., Binns, H. J., Christoffel, K. K., Hayford, J. R., Starzyk, K. A., & Arend, R. (1998). Oppositional defiant disorder and attention-deficit hyperactivity disorder in preschool children: prevalence and comorbidity. Journal of the American Academy of Child and Adolescent Psychiatry, 37(12), 1265–1271.
[R35] Lavigne, J. V., LeBailly, S. A., Hopkins, J., Gouze, K. R., Binns, H. J., & Lavin, M. A. (1996). The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. Journal of Clinical Child Psychology, 25(4), 465–476.
[R36] Loeber, R. (1982). The stability of antisocial and delinquent child behavior: A review. Child Development, 53(6), 1431–1446.
[R37] Loeber, R. (1990). Development and psychopathology. Development and Psychopathology, 2(4), 327–334.
[R38] Marakovitz, S., & Campbell, S. B. (1998). Predicting behavior problems in school-age boys with early-onset externalizing and internalizing problems. Journal of Abnormal Child Psychology, 26(2), 115–126.
[R39] Masse, L. C., & Tremblay, R. E. (1999). Behavior of boys in kindergarten predicts substance use and criminality during adolescence. Archives of General Psychiatry, 56(1), 62–68.
[R40] Meehl, P. E. (2001). аксometric methods. Lawrence Erlbaum Associates Publishers.
[R41] Mesman, J., & Koot, H. M. (2001). Child behavior checklists for ages 11/2–5. Manual for the ASEBA preschool forms & profiles. Research Center for Child and Adolescent Psychopathology.
[R42] Mesman, J., Bongers, I. L., & Koot, H. M. (2001). Preschoolers’ relationships with parents and teachers: security of attachment and patterns of problem behavior. Journal of School Psychology, 39(3), 205–223.
[R43] Olson, S. L., & Brodfeld, K. J. (1991). Developmental course of conduct problems in preschoolers: Gender differences and maternal determinants. Journal of Abnormal Child Psychology, 19(2), 139–156.
[R44] Owens, E. B., & Shaw, D. S. (2003). Poverty and early child externalizing behavior problems. Applied Developmental Psychology, 24(4), 431–448.
[R45] Pelletier, L. A., Collett, B. R., Gimpel, G. A., & Crowley, S. L. (2006). Convergent validity of categorical and dimensional measures of preschool disruptive behavior. Early Childhood Research Quarterly, 21(1), 116–129.
[R46] Pierce, E. W., Ewing, L. J., & Campbell, S. B. (1999). Diagnostic status and symptomatic behavior of hard-to-manage preschoolers at ages 3 and 9. Journal of Clinical Child Psychology, 28(1), 44–57.
[R47] Querido, J. G., & Eyberg, S. M. (2003). Concurrent validity of the disruptive behavior diagnostic scale: parent form with preschool children. Child & Family Behavior Therapy, 25(2), 1–16.
[R48] Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.
[R49] Scholte, E. M., Van Berckelaer-Onnes, I. A., & Van Der Ploeg, J. D. (2002). Rating scales for child psychopathology: categorical versus dimensional classification. Journal of Child Psychology and Psychiatry, 43(6), 709–720.
[R50] Shaw, D. S., Owens, E. B., Giovannelli, J., & Winslow, E. B. (2001). Infant and toddler pathways to chronic externalizing problems. Development and Psychopathology, 13(4), 801–828.
[R51] Speltz, M. L., McClellan, J., DeKlyen, M., & Jones, K. (1999). Preschool boys with oppositional defiant disorder and conduct disorder: comparison of clinic and community samples. Journal of Abnormal Child Psychology, 27(6), 449–464.
[R52] Sprafkin, J., Volpe, R. J., Gadow, K. D., Nolan, E. E., & Kelly, K. (2002). Convergent validity of categorical and dimensional assessment of ADHD and oppositional defiant disorder in clinic-referred preschoolers. Journal of Psychoeducational Assessment, 20(3), 263–277.
[R53] Teagarden, J. R., & Burns, W. J. (1999). Dimensional versus categorical assessment of child psychopathology. Clinical Psychology Review, 19(6), 679–708.
[R54] Vondra, J. I., Shaw, D. S., Swearingen, L., Cohen, M., & Owens, E. B. (2001). Toddler antecedents of externalizing behavior problems at school entry among children in poverty. Journal of Abnormal Child Psychology, 29(6), 455–472.
[R55] Wakschlag, L. S., & Keenan, K. (2001). Clinical significance versus statistical significance in studies of disruptive behavior in children: what difference does it make?. Journal of Abnormal Child Psychology, 29(4), 313–322.
[R56] Waldman, I. D., Lilienfeld, S. O., & Lahey, B. B. (1995). Toward taxonomic clarification of child and adolescent conduct disorder: categorical and dimensional approaches. Journal of Abnormal Child Psychology, 23(4), 451–470.
[R57] White, J. L., Moffitt, T. E., Earls, F., & Robbins, L. N. (1990). How old do you have to be before it pays to be bad? In L. N. Robbins & M. Rutter (Eds.), Straight and devious pathways from childhood to adulthood (pp. 249–272). Cambridge University Press.
[R58] Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. Sage Publications.
[R59] Wortman, P. M. (1994). Judging research quality. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 97–110). Russell Sage Foundation.
[R60] Zahn-Waxler, C., Ianotti, R. J., Cummings, E. M., & Denham, S. (1990). Antecedents of problem behaviors in children at risk. Development and Psychopathology, 2(3), 271–291.
