Transcript The DesPho-APaDy Project: Developing an Acoustic

http://despho-apady.univ-avignon.fr/
The DesPho-APaDy Project:
Developing an Acoustic-phonetic Characterization
of Dysarthric Speech in French
C. Fougeron*, L. Crevier-Buchman*, C. Fredouille°, A. Ghio^, C. Meunier^,
C. Chevrie-Muller, J.-F. Bonastre°, A. Colazo-Simon, C. Delooze, D. Duez^,
C. Gendrot*, T. Legou^, N. Lévêque*, C. Pillot-Loiseau*, S. Pinto^,
G. Pouchoulin°, D. Robert^, J. Vaissiere*, F. Viallet^, C. Vincent*
* Lab. de Phonétique et Phonologie, UMR 7018 CNRS-Paris3/Sorbonne Nouvelle, Paris
° Université d'Avignon, CERI/LIA, Avignon
^ Lab. Parole et Langage, UMR 6057 CNRS Aix-Marseille Univ., Aix-en-Provence
What is Dysarthria?
• Neurologically-based speech disturbances resulting
from damage to the central and/or peripheral nervous
system that impair the transmission of neural messages
to the muscles involved in speech production
– Deficit in the motor execution at any level of speech production:
• respiratory
• phonatory
• supralaryngeal (velopharyngeal and orofacial)
– Requires the consideration of multiple speech parameters
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
2
Rationale and Objectives of the Project
- To constitute a searchable speech database of French
dysarthric patients:
1. for the preservation of existing databases
2. for the need of the acoustic description
- To provide a systematic, quantified acoustic
description of the speech patterns of French
dysarthric speakers
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
3
The Multiple-Field Query Database:
Why?
• Storage and organization of digitized dysarthric speech
corpora (acoustic and physiological data) and associated
patients’ clinical information
• Accessibility (for selected and protected use) through
multiple-field queries (M/F, age, on/off stimulation,…)
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
4
The Multiple-Field Query Database:
Which data?
The CCM (Claude ChevrieMuller) Corpus recorded over the
last 30 years,
~1000 hours of disordered speech,
5000 patients – adults and
children
* 860 patients classified according
to their neurological diagnosis
* 60 control speakers
Data recorded
Sound, EGG (ElectroGlottoGraph)
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
5
The Multiple-Field Query Database:
Which data?
• The ANH (Aix-NeurologyHospital)
Corpus recorded for the past 15
years:
* 990 patients,
* 160 control speakers
Data recorded
Sound, aerodynamic.
2010-05-20
EVA2
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
6
The Multiple-Field Query Database:
How?
• Getting the audio files:
– Digitization of the analog 1/4” audio tapes
• speed change during each recording session
• “real time” auditory control of the recording
• Problems encountered: poor quality recordings, some tapes
in bad condition
– Transfer of DAT tapes
• To date, data from 180 patients have been digitized (analog
tapes), 94 have been captured from DAT
– Segmentation per patient and per speech task (idem
for the control population)
– Data were renamed for anonymous storage
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
7
The Multiple-Field Query Database:
How?
• Designing the database:
– Choice of a relational model to structure the data
– Definition of a set of objects, their attributes and the
relation between them, related to the clinical
environment
– Refinement by confronting the relational data model
with empirical and “real” clinical data
• Developing the query interface:
– Basic sociolinguistic information
– Clinical information
– Recording session information
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
8
Dysarthric Speech in French:
Dimensions to be considered when dealing with dysarthria
• Various parameters can be affected…
• phonation
• articulation
• timing/rhythm
• prosody
• fluence…
• Need of appropriate and sufficiently large quantities of
speech data, homogenous in quality, and sufficiently
documented by clinical information on the patient
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
9
Dysarthric Speech in French
Methods
• Methods and analysis procedures based on both
phonetics and speech engineering approaches
– Manual analysis (by human experts) of the acoustic
properties of the productions
– Automatic acoustic analysis of speech signals
– Continuous back and forth between these two
techniques should gain from the potential of both
approaches
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
10
Dysarthric Speech in French:
Selection of Patients (Acoustic Study)
• Focus on neurophysiologic alterations of 3
neurologic systems:
– Pyramidal system: ALS dysarthria (30 patients)
– Cerebellar system: ataxic dysarthria (30 patients)
– Extrapyramidal system: Parkinsonian dysarthria (30
patients from the ANH corpus)
• Selection based on:
– The clinical information, (the certainty of the
diagnosis, the ongoing treatment, patient
demographics)
– The severity of the dysarthria (work on dysarthria with
relatively intelligible speech)
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
11
Dysarthric Speech in French:
Pre-Processing of the Audio Files
• In order to be able to perform both the
manual and automatic acoustic analyses,
Need of:
– Orthographic transcriptions
– Automatic text-constrained phonetic
alignment
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
12
Pre-Processing of the Audio Files:
Orthographic Transcriptions
Automatic Alignment
1. Orthographic transcriptions
<s> [ext] *vu* *puve* *i* *ale* [ext]*alOR* Mr
Seguin n'avait jamais eu de bonheur avec ses
Each speech production which differs from the original text
was annotated using the SAMPA phonetic alphabet
2. Automatic text-constrained phonetic alignment
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
13
Dysarthric Speech in French:
Automatic Phonetic Alignment
• Evaluation of the automatic phonetic alignment
comparing the production of phonemic labels and the
boundaries annotated by two phoneticians
2010-05-20
Male
Female
control
X
X
Moderate dysarthria
X
X
Severe dysarthria
X
X
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
14
Dysarthric Speech in French:
Automatic Phonetic Alignment
Results:
When comparing segment time boundaries
produced manually and automatically, a shift
above 20ms is observed:
17% of control speakers,
24% of moderate dysarthric patients
56% of severe dysarthric patients
But strong agreement (manual/automatic) for
spectral measurements
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
15
Conclusions/Perspectives
Encouraging results for clinical research on speech
disorders and speech science.
Observing type and range of variations due to motor control
deficit can help for a comprehensive model of speech
variation for normal speakers.
A better understanding of the variations that permit to
consider a dysarthric speech as deviant could provide
insights to the boundary between normal and
pathological speech.
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
16
Acknowledgments
Funded by the ANR BLAN08-0125 of the
French National Research Agency
We deeply thank P. Clément, A. Nurenberg
and O. Panseri who are also collaborating
to this project
For further information, please visit:
http://despho-apady.univ-avignon.fr
2010-05-20
Fougeron et al.: The DesPho-APaDy Project, LREC 2010
17