Cluttering is a rather poorly defined fluency disorder, thought to be characterised by an
accelerated and/or irregular speech rate with a large amount of disfluencies. As there is not
yet a universally agreed upon definition of cluttering, clinicians have faced challenges
identifying reliable and efficient assessment tools for the diagnosis of cluttered speech. A
lack of diagnostics has also impacted upon the ability to develop treatment strategies and to
monitor their success. A recently developed software programme called the Cluttering
Severity Instrument (CSI; Bakker and Myers 2011) is anticipated to be the first acceptable,
reliable and valid diagnostic tool for identifying cluttered speech. The CSI requires clinicians
to provide perceptual ratings for eight speech characteristics, and to complete a task
estimating the percentage of the sample that is cluttered. The CSI programme uses these two
factors to calculate an overall CSI score.
The principle aim of the present study is to investigate whether the CSI programme can be
regarded as a reliable and valid method of diagnosing cluttered speech. To achieve this, interrater
and intra-rater reliabilities were determined. A second aim was to establish whether the
CSI provides raters with the means to discriminate between impaired and unimpaired speech.
A final aim is to identify whether raters' familiarity with speech analysis impacted upon rater
Twelve raters provided perceptual ratings of eight speech characteristics and indicated the
duration of the speech sample that they perceived to be cluttered. Raters recorded their
judgements using the CSI programme and an overall cluttering percentage was calculated.
Raters completed the experiment for five separate speech samples.
Overall, inter-rater reliability was poor. The combined ratings of speakers and subtests found
a high percentage of correlations, but not many with a strong coefficient value. Additionally,
many of these correlations were determined to have significantly different means. Separate
analysis of subtests indicated that although scores correlated, they often differed in range,
with some raters giving consistently lower scores than others. These correlation analyses
indicate that the CSI does not have strong inter-rater reliability. Intra-rater reliability was
examined by re-testing four participants. Of these four, three had a strong correlation between
their score sets, which provided some evidence of intra-rater reliability. When speakers were
analysed separately, results indicated that raters were successfully discriminating between
impaired and unimpaired speech. Correlation analysis of speech and language therapists
(SLT) and laypersons groups did not determine a familiarity of speech analysis to have an
influencing factor on perceptual judgements.
Based on the results of this study, we cannot conclude that the CSI programme is a reliable or
valid diagnostic tool. Ratings provided by the SLT group were not found to be more reliable
or less variable than the laypersons group. This indicated that a familiarity with speech
analyses did not impact upon perceptual judgements of speech. Although the present study
did not find significant evidence to support the CSI programme as a reliable and valid
assessment tool, it did indicate that it provides raters with the means to successfully
discriminate between impaired and unimpaired speech.||