#### Transcript robust_sid - Faculty of Information Technology

Robust Speaker Recognition JHU Summer School 2008 Lukas Burget Brno University of Technology Intersession variability The largest challenge to practical use of speaker detection systems is channel/session variability • • Variability refers to changes in channel effects between training and successive detection attempts Channel/session variability encompasses several factors – The microphones • Carbon-button, electret, hands-free, array, etc – The acoustic environment • Office, car, airport, etc. – The transmission channel • Landline, cellular, VoIP, etc. – The differences in speaker voice • Aging, mood, spoken language, etc. • Anything which affects the spectrum can cause problems – Speaker and channel effects are bound together in spectrum and hence features used in speaker verifiers NIST SRE2008 - Interview speech Different microphone in training and test about 3% EER The same microphone in training and test < 1% EER Channel/Session Compensation Channel/session compensation occurs at several levels in a speaker detection system Signal domain Feature domain Model domain Score domain Target model Front-end processing Adapt S LR score normalization Background model • Noise removal • Tone removal • Cepstral mean subtraction • RASTA filtering •Feature Mapping • Mean & variance •Eigenchannel normalization adaptation in • Feature warping feature domain • Speaker Model Synthesis • Eigenchannel compensation •Joint Factor Analysis • Nuisance Attribute Projection • Z-norm • T-norm • ZT-norm L Signal domain Feature domain Model domain Score domain Target model Front-end processing Adapt S LR score normalization Background model • Noise removal • Tone removal • Cepstral mean subtraction • RASTA filtering •Feature Mapping • Mean & variance •Eigenchannel normalization adaptation in • Feature warping feature domain • Speaker Model Synthesis • Eigenchannel compensation •Joint Factor Analysis • Nuisance Attribute Projection • Z-norm • T-norm • ZT-norm L Adaptive Noise Suppression Basic idea of spectral subtraction (or Wiener filter): Y(n) = X(n) - N(n) •Y(n) – enhanced speech •X(n) – spectrum of nth frame of noisy speech •N(n) – estimate of stationary additive noise spectrum Reformulate as filtration: Y(n) = H(n)X(n) where H(n) = (X(n) – N(n)) / X(n) It is necessary to • to smooth H(n) in time • make sure magnitude spectrum is not negative • … Adaptive Noise Suppression • • • Goal: Suppress wideband noise and preserve the speech Approach: Maintain transient and dynamic speech components, such as energy bursts in consonants, that are important “information-carriers” Suppression algorithm has two primary components – – Detection of speech or background in each frame Suppression component uses an adaptive Wiener filter requiring: • Underlying speech signal spectrum, obtained by smoothing the enhanced output • Background spectrum • Signal change measure, given by a spectral derivative, for controlling smoothing constants Degraded Speech Suppression Enhanced Speech Suppression Filter Spectral Derivative Background |Spectrum| Speech |Spectrum| Short-Time Spectral Magnitude Detection Time Constant Smooth Delay Adaptive Noise Suppression • C3 example from ICSI • Processed with LLEnhance toolkit for wideband noise reduction SNR = 15 dB SNR = 25 dB Signal domain Feature domain Model domain Score domain Target model Front-end processing Adapt S LR score normalization Background model • Noise removal • Tone removal • Cepstral mean subtraction • RASTA filtering •Feature Mapping • Mean & variance •Eigenchannel normalization adaptation in • Feature warping feature domain • Speaker Model Synthesis • Eigenchannel compensation •Joint Factor Analysis • Nuisance Attribute Projection • Z-norm • T-norm • ZT-norm L Cepstral Mean Subtraction •MFCC feature extraction scheme •Consider the same speech signal recorded over different microphone attenuating certain frequencies twice •Scaling in magnitude spectrum domain corresponds to constant shift of the log filter bank outputs Fourier Transform Magnitude x 0.5 Log() - 0.3 frames Cosine transform Cepstral Mean Subtraction •Assuming the frequency characteristics of the two microphones do not change over time, the whole temporal trajectories of the affected log filter bank outputs differs by the constant. •The shift disappears after subtracting mean computed over the segment. 0.0 •Usually only speech frames are considered for the mean estimation •Since Cosine transform is linear operation the same trick can be applied directly in cepstral domain NIST SRE 2005 all trials Miss probability [%] 2048 Gauss., 13 MFCC + delatas, CMS False alarm probability [%] RASTA filtering •Filtering log filter bank output (or equivalently cepstral) temporal trajectories by band pass filter •Remove fast changes (> 25Hz) likely not caused by speaker with limited ability to quickly change vocal tract configuration 0 Magnitude [dB] •Remove slow changes to compensate for the channel effect (≈CMS over 0.5 sec. sliding window) Frequency characteristic 10 -10 -20 -30 -40 0.01 original 0.1 1 Frequency [Hz] 10 100 Impulse response 0.0 frames RASTA filtered 0.0 -100 0 100 200 Time [s] 300 400 NIST SRE 2005 all trials 2048 Gauss., 13 MFCC + delatas, CMS Miss probability [%] with RASTA False alarm probability [%] Mean and Variance Normalization •While convolutive noise causes the constant shift of cepstral coeff. temporal trajectories, noise additive in spectral domain fills valleys in the trajectories •In addition to subtracting mean, trajectory can be normalized to unity variance (i.e. dividing by standard deviation) to compensate for his effect original Speech with additive noise frames after CMN/CVN Clean speech Feature Warping •Warping each cepstral coefficients in 3 second sliding window into Gaussian distribution •Combines advantages of the previous techniques (CMN/CVN, RASTA) •Resulting coefficients are (locally) Gaussianized more suitable for GMM models 0.0 0.5 1.0 0.0 Inverse Gaussian cumulative density function NIST SRE 2005 all trials 2048 Gauss., 13 MFCC + delatas, CMS with RASTA Miss probability [%] with Feature Warping False alarm probability [%] NIST SRE 2005 all trials 2048 Gauss., 13 MFCC + delatas, CMS with RASTA Miss probability [%] with Feature Warping + triple deltas + HLDA False alarm probability [%] Example of 2D GMM HLDA Heteroscedastic Linear Discriminant Analysis provides a linear transformation that de-correlates classes. HLDA HLDA allows for dimensionality reduction while preserving the discriminability between classes (HLDA without dim. Reduction is also called MLLT) Useful dimension Nuisance dimension Signal domain Feature domain Model domain Score domain Target model Front-end processing Adapt S LR score normalization Background model • Noise removal • Tone removal • Cepstral mean subtraction • RASTA filtering •Feature Mapping • Mean & variance •Eigenchannel normalization adaptation in • Feature warping feature domain • Speaker Model Synthesis • Eigenchannel compensation •Joint Factor Analysis • Nuisance Attribute Projection • Z-norm • T-norm • ZT-norm L Speaker Model Synthesis • It is generally difficult to get enrollment speech from all microphone types to be used • The SMS approach addresses this by synthetically generating speaker models as if they came from different microphones (Teunen, ICSLP 2000) – A mapping of model parameters between different microphone types is applied synthesis cellular synthesis electret carbon button Speaker Model Synthesis Learning mapping of model parameters between different microphone types: •Start with channel-independent root model •Create channel models by adapting root with channel specific data •Learn mean shift between channel models Speaker Model Synthesis Training speaker model: Training data Test data •Adapt channel model which scores highest on training data to get target model •Synthesize new target channel model by applying the shift Ti CD1CD 2 ( i ) i ( iCD 2 iCD1 ) •GMM weights and variances can be also adapted and used to improve the mapping of model parameters between different microphone types Ti CD1CD 2 (i ) i (iCD 2 / iCD1 ) Ti CD1CD 2 ( i ) i ( iCD 2 / iCD1 ) Signal domain Feature domain Model domain Score domain Target model Front-end processing Adapt S LR score normalization Background model • Noise removal • Tone removal • Cepstral mean subtraction • RASTA filtering •Feature Mapping • Mean & variance •Eigenchannel normalization adaptation in • Feature warping feature domain • Speaker Model Synthesis • Eigenchannel compensation •Joint Factor Analysis • Nuisance Attribute Projection • Z-norm • T-norm • ZT-norm L Feature mapping • Aim: Apply transform to map channel-dependent feature space into a channel-independent feature space • Approach: – Train a channel-independent model using pooling of data from all types of channels – Train channel-dependent models using MAP adaptation – For utterance, find top scoring CD model (channel detection) – Map each feature vector in utterance into CI space CD 1 … CD 2 CI CD N yt M iCDCI ( xt ) D.A. Reynolds, “Channel Robust Speaker Verification via Feature Mapping,” ICASSP 2003 Feature mapping •As for SMS, sreate channel models by adapting root with channel specific data •Learn mean shifts between each channel models and channel-independent root model Feature mapping • For each (training or test) speech segment, determine maximum likelihood channel model • For each frame of the segment, record top-1 Gaussian per frame CD i arg max CD p j j ( xt ) 1 j M • For each frame apply mapping to map x with CD pdf to y with CI pdf CI i yt ( xt iCD ) CD iCI i • Target model is adapted from CI model using mapped features • Mapped features and CI models are used in test NIST SRE 2005 all trials 2048 Gauss., 13 MFCC + delatas, CMS with RASTA Miss probability [%] with Feature Warping + triple deltas + HLDA + Feature mapping (14 classes) False alarm probability [%] Session variability in mean supervector space •GMM mean supervector – column vector created by concatenating mean vectors of all GMM components. •For the case of variances shared by all speaker models, supervector M fully defines speaker model •Speaker Model Synthesis can be rewritten as: MCD2 = MCD1 + kCD1CD2, where kCD1CD2 is the cross-channel shift •Drawbacks of SMS (and Feature Mapping) •Channel dependent models must be created for each channel •Different factors causing intersession variability may combine (e.g. channel and language) compensation must be trained for each such combination •The factors are not discrete (i.e. effects on the intersession variability may be more or less strong) •There is evidence that there is limited number of directions in the supervector space strongly affected by intersession variability. Different directions possibly corresponds to different factors. Session variability in mean supervector space Example: single Gaussian model with 2D features Target speaker model UBM 32 Session compensation in supervector space Target speaker model Test data UBM For recognition, move both models along the high inter-session variability direction(s) to fit well the test data (e.g. in ML sense) 33 6D example of supervector space Identifying high intersession variability directions • Take multiple speech segments from many training speakers recorded under different channel conditions. For each segment derive supervector by MAP adapting UBM. • From each supervector, subtract mean computed over supervectors of corresponding speaker. • Find direction's with largest intersession variability using PCA (eigen vectors of the average with-in speaker covariance matrix). supervectors of speaker 1 speaker 2 speaker 3 Eigenchannel adaptation • Speaker model obtained in usual way by MAP adapting UBM • For test, adapt speaker model and UBM by moving supervectors in the direction(s) of eigenchannel(s) to well fit the test data find factors x maximizing likelihood of test data for log p( x | M Ux) t Target speaker model M UBM t • The score is LLR computed using the adapted speaker model and UBM N. Brummer, SDV NIST SRE’04 System description, 2004. Test data NIST SRE 2005 all trials 2048 Gauss., 13 MFCC + delatas, CMS with RASTA Miss probability [%] with Feature Warping + triple deltas + HLDA + Eigenchannels adaptation + Feature mapping (14 classes) Nuisance Attribute Projection U • NAP is an intersession compenzation technique proposed for SVMs • Project out the eigenchannel directions from supervectors before using the supervectors for training SVMs or test 38 Constructing models in supervector space • Speaker Model Synthesis: MCD2 = MCD1 + kCD1CD2 – constant supervector shift for recognized training and test channel • Eigenchannel adaptation: Mtest = Mtrain + Ux – the shift is given by linear combination of eigenchannel basis U with factors x tuned for test data • Eigenvoice adaptation – Consider also supervector subspace V with high speaker variability and use it to obtain speaker model – M = MUBM + Vy – speaker model given by linear combination of UBM supervec. and eigenvoice bases – speaker factors y tuned to match enrollment data – Can be combined with channel subspace: M = MUBM + Vy + Ux • both x and y estimated on enrollment data • only x updated for test data to adapt speaker model to test channel condition Joint Factor analysis • • M = MUBM + Vy + Dz + Ux Probabilistic model – Gaussian priors assumed for factors y, z, x – Hyperparameters MUBM, V, D, U can be trained using EM algorithm – D - diagonal matrix describing remaining speaker variability not covered by eigenvoices u2 u1 d11 v2 d22 d33 v1 NIST SRE 2005 all trials 2048 Gauss., 13 MFCC + delatas, CMS with RASTA with Feature Warping + triple deltas + HLDA + Eigenchannels adaptation Joint Factor Analysis (extrapolated result) False alarm probability [%] + Feature mapping (14 classes) Signal domain Feature domain Model domain Score domain Target model Front-end processing Adapt S LR score normalization Background model • Noise removal • Tone removal • Cepstral mean subtraction • RASTA filtering •Feature Mapping • Mean & variance •Eigenchannel normalization adaptation in • Feature warping feature domain • Speaker Model Synthesis • Eigenchannel compensation •Joint Factor Analysis • Nuisance Attribute Projection • Z-norm • T-norm • ZT-norm L Z-norm • Target model LR scores have different biases and scales for test data – Unusual channel or poor quality speech in training segments lower scores from target model – Little training data target model close to UBM all LLR scores close to 0 • Znorm attempts to remove these bias and scale differences from the LR scores – Estimate mean and standard deviation of non-target, same-sex utterances from data similar to test data Tgt1 scores – During testing normalize LR score – Align each model’s non-target Tgt2 scores scores to N(0,1) ZTgt ( x) LTgt ( x) Tgt Tgt pooled LR scores znorm scores T-norm • Similar idea to Z-norm , but compensating for differences in test data • Estimates bias and scale parameters for score normalization using fixed “cohort” set of speaker models – Normalizes target score relative to a non-target model ensemble – Similar to standard cohort normalization except for standard deviation scaling Target model Cohort Cohort model Cohort model model Tnorm score Ttgt (u ) coh , coh ) • Used cohorts of same gender as target • Can be used in conjunction with Znorm – ZTnorm or TZnorm depending on order Introduced in 1999 by Ensigma (DSP Journal January 2000) L tgt (u ) coh coh Effect of ZT-norm NIST SRE2006 Miss probability [%] telephone trials Eigenchannel adaptation Joint Factor Analysis no normalization ZT-norm False alarm probability [%] Score fusion NISR SRE 2006 all trials Linear logistic regression fusion of scores from: •GMM with eigenchannel adaptation •SVM based on GMM supervectors •SVM based on MLLR transformation (transformation adapting speaker indipendent LVCSR system to speaker) LLR trained using many target and non-target trials from development set Conclusions