Transcript Slide 1
Establishing the psychometric properties of measures of suicidal ideation and attempts Jill Harkavy-Friedman, PhD Reasons to Assess Suicidal ideation and attempts in Clinical Trials Adverse Event Did the event occur during the trial Focus of treatment Did the behavior change during the trial Define change relative to baseline levels There is a need for different assessment tools Dilemma: How to standardize across studies Have well defined universal criterion that represents goal of assessment Final universal criterion must have established psychometric properties and clinical utility Study Measures Permit classification with universal criterion Measures must have established psychometric properties An independent rater can make categorization to universal criterion using results of the specific measure How to chose an instrument Multiple factors that lead to choice of measure Who: Patients, controls What: Ideation, attempts, suicides Where: Office, home, phone/computer When: Baseline, Weekly, Discharge, Follow-Up Why: Adverse event; Reduction of behavior How: Interview (clinical or lay), questionnaire Feasibility: Cost, time, personnel Choosing the variable of interest? Based on goal of study & review of literature Impact/Importance of variable Thoughts of death, Suicidal ideation, Suicide attempt, Suicide Absolute value or change score Did suicidal ideation or attempt occur during trial? Did suicidal ideation or number of attempts change during trial? Emergent suicidal ideation and attempts behavior in non-suicidal sample In high risk samples requires pre-post assessment using prospective design to demonstrate change Multiple measures vs. single measure Suicidal ideation and attempts in context of risk factors vs. suicidal behavior alone Depression, substance use, stress, psychosis Psychometrics Evaluate the psychometric properties Reliability Validity Sensitivity Specificity Variability Ceiling and Floor effects RELIABILITY=REPRODUCABILITY Inter-rater Reliability If 2 or more of us collect and/or see the same information will we make the same rating. Kappa: Categorical (attempt/no attempt) Intra-class correlation: Numerical (Total score) Test-Retest: Over time Correlate time 1 and time 2 If we measure today and measure tomorrow will we get the same answer. Parallel Forms: Across Measurements Correlate forms Avoid learning effects due to repeated assessments Internal Consistency: Within a test Are the items of a measure assessing a single content or construct? Split-Half Spearman Brown Cronbach’s Alpha What does the number mean? Range: 0-1.0 reliability2= percent of variance accounted for by measure (the rest is random error or due to other factors) 0-.30 - weak .31-.69 - moderate .70-1.0 - strong (accounts for 50% of variance or more) Validity Face Validity: Does it look like it measures what it is supposed to ? Content Validity: Is the content representative? Criterion Validity: Predictive, Concurrent Construct Validity: Accrual of meaning through convergent and discriminant validity Content Validity: Is the content representative? What forms of suicidal behavior are of interest Ideation Frequency Planfulness Persistence Intent Preparatory Behaviors Aborted Attempt Interrupted Attempt Attempt Self-Injurious Behavior Criterion Validity: Predictive, Concurrent Predictive Validity Does the measure predict future criterion Requires prospective design Sensitivity and Specificity for future suicidal behavior Concurrent Validity Does the measure relate to variables it ought to relate to at the time of assessment 90+% of people who suicide have a diagnosable psychiatric condition- are you sure behavior in study is due to the medication being tested Depression, Substance Use, Aggression, Impulsiveness, Psychosis Construct Validity: Accrual of meaning through convergent and discriminant validity Validity is accrued Convergent Validity Does the measure related to what it ought to Depression, substance use, stress, health, psychosis Discriminant Validity Is the measure independent of variables not related to suicidal behavior (e.g. positive life events) RELIABILITY IS THE UPPER LIMIT OF VALIDITY Can you find an effect? Sensitivity and Specificity Variability Ceiling and Floor effects Sensitivity and Specificity sensitivity = probability of a positive test among patients with disease specificity = probability of a negative test among patients without disease Example study establishing psychometric properties Goal: Establish the psychometric properties of a self-report measure of suicidal ideation and attempts Sample: Psychiatric clinic sample without controlling for diagnosis Measures: Universal Criterion: measure requiring trained interviewer to determine no suicidal behavior/suicidal ideation/suicide attempt in past week Self-report questionnaire of suicidal ideation and attempts over past week SxList: assesses psychopathology associated with behavior SubUse: assesses substance use LifeList: assesses positive and negative stress Procedures: Patient completes all self-report measures Patient is interviewed by clinical interviewer who categorizes suicidal behavior Patient returns 2-3 days later and completes suicidal behavior questionnaire Initial Data Analysis: Frequency distribution of items, total score and Universal Criterion from suicidal ideation and attempts questionnaire Internal consistency of suicidal ideation and attempts questionnaire Correlate suicidal ideation and attempts questionnaire from time 1 and time 2 Compare categorization by questionnaire vs. clinical rater Examine relationships of questionnaire and interviewer ratings with additional measures using multi-method multi-trait matrix Examine sensitivity and specificity of questionnaire relative to interviewer categorization Considerations for assessing suicidal behavior in clinical trials Suicidal ideation and attempts are multiply determined, intermittent and often recurrent Measures in clinical trials can include categorization into universal criterion that can be used across studies Goal of the clinical trial plays an integral role in determining: Should suicidal ideation and attempts be measured? How will suicidal ideation and attempts be measured? When should suicidal ideation and attempts be measured? How will Adverse Events be measured? When suicidal ideation and attempts are assessed should associated risk factors also be assessed (at least at baseline)? Can evaluate possible causal link to Adverse Event? Validity is accrued across studies