Measuring and understanding patterns of change in intervention studies with children: Implications for evidence-based practice

Abstract

Purpose: Comparisons across studies of the effects of intervention are problematic. Such analyses raise both methodological and statistical challenges. A single data set was examined to investigate whether different established approaches to measuring change in children with specific language impairments alter the conclusions that can be drawn regarding the efficacy of an intervention. Methods: Measures of cognitive and language skills were collected at baseline and at six months following an intervention. Reliable and valid psychometric measures were used. Data from the intervention study were used to explore the patterns of results obtained using four different measures of change: change of diagnostic category, differential improvement across assessment measures, item specific changes and predictors of individual change. Results: Associations between different tests purporting to measure similar constructs were modest. The measures identified different children as impaired both at baseline and follow-up. No effect of intervention was evident when a categorical analysis of impairment was used. Both treatment and comparison children changed significantly across time on the majority of measures, providing evidence of development, but specific effects of the intensive intervention were evident using ANCOVAs. Item analysis indicated that one of the standardized language tests adopted in the evaluation was insensitive to change over a six month period. Change in individual children’s performance was predicted by language level on entry to the project. Conclusion: The implications of the results are discussed in terms of the range of analytic approaches available to intervention researchers and the need to consider combinations of methods when analysing outcome data.