Transcript Chapter 3: Choosing the Study Subjects

Designing Clinical
Research
Anita S. Kablinger, M.D.
Associate Professor
Departments of Psychiatry
and Psychopharmacology
Chapter 2: Conceiving the
Research Question
Research question:
•Addresses uncertainty
•Resolved by measurements
Successful Research
Questions Require:
scholarship, experience, mentor
• Mastering the Literature
-Published literature
-Attend meetings
-Relationships with professionals
• Knowledge of New Ideas and Techniques
• Imagination
FINER Research Questions
• Feasible
Number of subjects, technical expertise, affordable in time and
money, manageable
• Interesting to the investigator
• Novel
Confirms/ refutes/ extends previous findings, provides new findings
• Ethical
No unacceptable physical risks or invasion of privacy
• Relevant
Scientific knowledge, clinical and health policy, future research
Early Stage Outline
• Researcher clarifies plan
• Researcher discovers specific problems
-Is research question finer ?
• Allows colleagues’ reactions
Chapter 3: Choosing the
Study Subjects
• Study sample must be affordable in
time and money
• Study sample must represent the
target population
Inclusion Criteria
• Includes main characteristics of target
population
• What factors are important to the research
question?
• How should these factors be defined?
• Include demographic, clinical, geographic,
and temporal characteristics
Exclusion Criteria
• Characteristics that may interfere with the
data or randomization of the study
• Excessive exclusion may degeneralize the
study
Clinical vs. Community Populations
• Clinical are often already hospitalized so
study subjects are inexpensive, available,
and easy to recruit
BUT
• Their condition may be severe or
extraordinary, so the data may be distorted
and not representative of the population
Convenience Samples
•
•
•
•
Subjects meet entry criteria
Subjects are accessible
Samples often selected consecutively
Minimizes voluntarism
– Volunteers are often healthier than the
general population, so data may be distorted
Probability Samples
• Guarantee that every member in the
population has a specific chance of
selection for the study
• Types:
-Simple random
-Stratified random
-Cluster
-Systematic
2 Goals in Recruiting Study Subjects
• Representative
Response rate affects validity of study
Non-response subjects are often different
than those that respond
– Disease may be the cause of non-response
• Size
Monitor recruitment progress
Note when/ why potential subjects are lost
Chapter 7
Designing an Observational
Study: Cohort Studies
• Cohort studies take place over time
• Descriptive vs. Analytic
• Prospective vs. Retrospective
Prospective Cohort Studies
• Strengths
– Measures predictor before outcome occurs
(Less bias, more accurate)
• Weaknesses
– Expensive and inefficient for studying rare
outcomes
– Conditions present with symptoms before
diagnosis
Retrospective Cohort Studies
• Strengths
– Measures predictor before outcome occurs
(Less bias, more accurate)
– Less costly and time-consuming than
perspective studies
• Weaknesses
– Investigator has limited control over sampling
design or data collection
Nested Case-Control Study
• Used for predictor variables that are
expensive to measure and can be
assessed at end of study
• Subjects from completed cohort study
-Cases: Part of cohort that developed outcome
-Controls: Part of cohort without outcome
*Matching is optional but an unmatched design is preferable
Nested Case-Cohort Studies
• Random sample selected from original
cohort regardless of outcomes
• Advantages:
-Same control group for different studies
-Information on risk factor prevalence
Nested Studies
• Strengths
– Costly measurements are available
– Variables are collected before outcome
– Reduces bias from fatal cases and use of
various populations
• Weaknesses
– Observed association from confounded
variable
– Silent preclinical disease
Multiple Cohort Studies
• 2 different subject samples based on level
of predictor variable
• May compare cohort study outcomes with
census data
• Strengths
– Feasible approach for studying rare exposure or
potential hazards
• Weaknesses
– Cohorts may differ
– Data may be imprecise, incomplete, nonexistent
**Following the Entire Cohort
• Exclude those planning to move/ difficult to
reach
• Obtain information for difficult follow up
-Physician, close friends, SSN, Medicare #
• Periodic contact and repeated follow-up
efforts
Chapter 8: Designing an Observational
Study: Cross-sectional and
Case-control Studies
• Cross-sectional
• Case-control
• Bias
Cross Sectional Study
•
•
•
•
Simultaneous measurements
No follow up
Cause and effect inferred
Predictor and outcome designated
Case Control Study
• One sample from cases
(With outcome)
• One sample from controls
(No outcome)
• Compares levels of predictor in cases vs.
controls
Prevalence vs. Incidence
• Cross sectional studies provide
prevalence (at given time)
– Relative prevalence: Outcome prevalence by
level of predictor
• Cohort studies provide incidence
– Over period of time
Cross Sectional Studies
• Strengths
-Fast and inexpensive
-No loss from follow-up
• Weaknesses
-Difficult to determine cause and effect
Serial Survey
• Series of cross sectional studies
• Inferences can be drawn but a single
group is not followed over time
• May be used in cohort study to prevent
learning effect on data
Case-Control Studies
• Used to compare risk factor prevalence
– How often do predictor variables lead to disease?
• Cases (with disease) vs. controls (without)
– Specified number of each, so incidence or
prevalence of disease not determined
• Retrospective so may contain bias
Case-Control Studies
• Rare diseases
• Diseases with long latency
• Retrospective so can look at various
predictor variables to determine cause of
new outbreak
Case-Control Study Bias
• Control and case separate sampling
-Sampling bias
• Retrospective predictor variable
measurements
-Differential measurement bias
Sampling Bias
Not all affected are available for study
-Undiagnosed, misdiagnosed, dead
SOLUTIONS:
1. Hospital- or clinic-based controls
2. Matching
3. Population-based sample
4. Two+ control groups
Differential Measurement Bias
Imperfect recall
SOLUTIONS:
1. Data recorded before outcome
2. Blinding
Chapter 10
Designing an Experiment:
Clinical Trials I
•
•
•
•
Treatment applied
Effect on outcome observed
Causality demonstrated
Mature research questions
Selecting Participants
• Entry criteria:
Rate of outcome
Treatment effectiveness
Recruitable
Follow up
Generalizability
Compliance
• Sample size and recruitment planning
Measuring Baseline Variables
•
•
•
•
•
Fewer variables may be more efficient
Spend time and money wisely
Tracking information
Participant description
Measure risk factor and subgroup defining
variables
• Material banks
• Initial outcome variable
Randomizing
random assignment to interventions
• Treatment assignment must be random
• Blocked
• Stratified blocked
– Guarantees even distribution of strong predictor in
small sample
Applying Intervention
• Blinding
-Co-intervention effects
-Biased outcome assessment
• Choosing the intervention
-Effectiveness, safety, blindness, generalizability, combinations
• Choosing the control
-The control treatment should mimic the active treatment
-Ethical co-intervention
-Equivalence trials
Chapter 11
Designing an Experiment:
Clinical Trials II
•
•
•
•
•
•
Maximize follow up and adherence
Measuring outcome
Analyzing results
Monitoring clinical trials
Vulnerable populations
Research misconduct
Maximize Follow up and Adherence
•
•
•
•
Tolerable drug
Daily dosage
Pill counts and drug screening
Easy, convenient, and interesting studies
(“participant friendly”)
Clinical Trials
• Clinical relevance should be balanced with
feasibility and cost
• Measurable variables:
Outcome (Clinical)
Risk of outcome (Surrogate)
Measuring Outcome
• Outcomes should be accurate and precise
• Continuous variables preferred over
dichotomous
• Include outcome measures to detect
adverse effects
(See FDA website “Good Clinical Practices”)
Analyzing Results
• Dichotomous: chi squared
• Continuous: t test
• Intention to treat
– Cross-overs covered, prevents bias, may
underestimate results (results are
conservative)
– Preferred over per protocol
• Per protocol analyzes only evaluable
– May differ from those that drop
Subgroup Analysis
• Comparing randomized subjects of trials
• Randomization measurements should be
determined before treatment
• Post randomization factors should not be
considered
• Subgroup size is problematic
– May be too small to detect differences
• Different findings among subgroups
Monitoring Clinical Trials
• Prevent harmful intervention
– Risks are greater than benefits
• Provide beneficial intervention
• Discontinue intervention when research question
becomes unanswerable
• Before study begins, create guidelines and
procedures for monitoring
• Monitor recruitment, adherence, randomization,
blinding, follow up, outcome, adverse effects,
potential confounders
• Who will monitor and how often
• Statistical monitoring methods
Committee Monitoring
• Intervention and continuation decisions
should balance ethical responsibility with
advancement of medical knowledge
• Committee should be composed of
physicians, participant advocates,
biostatisticians, and clinical trial experts
with no connection to study
Options to Randomized
Blinded Trials
• Factorial Design
• Randomization of Matched Pairs
• Group or Cluster Randomization
Factorial Design
•
•
•
•
Answers two separate research questions
Efficient and cost effective
Interaction b/t treatments and outcomes
Useful for studying two unrelated
questions
Randomization of Matched Pairs
• Subject pairs with matching factors
(ex. age, sex)
• Contrast treatment and control in two parts of
the same individual at same time
(ex. one eye is treated, other serves as control)
Group or Cluster Randomization
• Assigning by group (ex. family, sports
team)
• Answers questions about public health
programs
• More feasible and cost effective than
individual assignments
• Complicated analysis
Within Group Designs
• May be used in time-series studies
• Time series studies: Participants are their
own control for evaluating treatment
effects
– Disadvantage: Lack of concurrent control
group
– Efficacy is optimistic b/c learning effects,
regression to the mean, secular trends
– Repeatedly starting and stopping therapy to
establish treatment effects
Cross-over Design
• Each person is their own control so requires
fewer participants
• Due to carry over effects:
Study duration doubled
Complex analysis and interpretation
– Carry over effects: Influence of intervention outcome
after treatment has stopped
– Use washout period to diminish carry over effect
• Use when limited number of subjects and
unproblematic carry over effects
FDA Approval
• “Good Clinical Practices” on FDA website
• Preclinical involves animals
• Phase I: Unblended and uncontrolled with small
number of human volunteers
• Phase II: Small random blind trials of diff doses
• Phase III: Large trials to test hypothesis
• Phase IV: Large studies after drug approval
Study Implementation
Anita S. Kablinger MD
Associate Professor
Departments of Psychiatry and
Pharmacology
Outline of Presentation
• Addressing Ethical Issues
• Designing Questionnaires and Data
Collection Instruments
• Data Management
• Implementing the Study
• Community and International Studies
• Writing and Funding a Research Proposal
Chapter 14: Addressing
Ethical Issues
•
•
•
•
•
•
•
Ethical Principles
IRB
Informed Consent
Scientific Misconduct
Conflict of Interest
Authorship
Confidentiality
Ethical Principles
• Respect for persons- Informed consent,
confidentiality
• Beneficence- Sound research design,
risks are acceptable in comparison to
benefits
• Justice- Vulnerable populations should
not be targeted, equitable access to the
benefits of research
Federal Regulations Ensure
Protection of Subjects (OHRP)
Institutional Review Board Approval
• Minimal risks
• Reasonable risks when compared to
possible benefits
• Equitable participant selection
• Informed consent
• Confidentiality
IRB
• Lack research expertise
– Research design, scientific merit, protocol adherence
• Investigator must uphold ethical research
• IRB reviews include:
– Exemptions: surveys or observations, existing
records, normal educational practices
– Expedited: minimal risk, minor changes in
approved research
Informed Consent
• Nature of research project
• Study procedures
• Risks, benefits, and alternatives
Including medical, psychosocial, economic realms
•
•
•
•
Confidentiality
Voluntary participation assurance
Comprehensible consent forms
Subjects who lack decision-making capacity
Vulnerable Populations
• Avoid vulnerable populations unless
problem under study is especially
prevalent within that specific population
• Vulnerable populations should not be used
unless necessary
– These populations involve additional informed
consent requirements
– Children, prisoners, pregnant women, fetuses,
and embryos, people with impaired decisionmaking capability
Research Misconduct
• Includes fabrication, falsification,
plagiarism
• May lead to incorrect results
• Undermines public confidence and support
• Investigators should have access to all
data, statistical analysis, and publishing
rights
– Sponsor should not veto or censor publication
• Conflicts of interest
Conflicts of Interest
• Blinded studies and peer review prevent
bias in conflicts of interest
• Any conflicting interests should be
disclosed and situations leading to
conflicts of interest should be avoided
• Dual roles for clinician-investigator and
financial conflict of interests may impair
objectivity of study and undermine public
trust
Dual Roles
• Physicians should always uphold patient
welfare
• Data and safety monitoring boards that
make study termination decisions should
not include researchers
• Randomized blinded studies without
controls should have equitable protocols
for both groups in the study
• Principle of Nonmaleficence: Effective
therapies should be provided to controls
Authorship
• Criteria for authorship to avoid questionable
contribution and responsibility
– Substantial contributions
– Drafting or revising the article
– Give final approval of article
• Honorary and ghost authorships
Chapter 15: Designing
Questionnaires and Data Collection
Instruments
• Questionnaires
• Interviews
Questionnaire Format
• Questionnaires should include purpose of
study and how data will be used
• Instructions on answering should be
provided
– Include instructions throughout where format changes
• Wording: Clear, simple, neutralVisual
analog scales
• Branching scales
Questionnaire Questions
• Questions should be mutually exclusive
• Avoid assumptions in questions
• Questions should cover one concept
(avoid “and,” “or”)
• Include “other” or “none of the above” if
questions are not exhaustive
• Group questions from same subject
groups together and include heading
More Questions
• Use series of questions with scores to measure
abstract concepts
• In questions with lists, use “yes” or “no” so each
option requires a response
• Open-ended questions: Better for determining
concept understanding but qualitative analysis is
time consuming and subjective
• Close ended questions: More standardized,
quicker and easier to analyze but may influence
participants’ answers
Questionnaires:
Additional Information
• Emotionally neutral questions at beginning,
sensitive ones in middle, personal at end
– Use introductions for questions about undesirable
behavior
• Set the time frame and quantify responses
– Determine if average or extremes in behavior is more
important for study
– Recent brief time frames increase recall but
responses may not be typical
– Longer time frames make recall difficult and bias
toward recent behavior may result
Questionnaire Editing
• Check for internal consistency to
guarantee acceptable summing or
averaging of scores
1) Draft made from focus group interviews
2) Critical review of draft
3) Pretest new questions or questionnaires
Developing Instruments
for Study
• List variables to be collected and measured in study
• Existing measures for each variable
• Best to not modify unless necessary
– Changes prevent comparison to previous studies with same
instrument
• Make draft
• Revise to simplify and clarify
– Check for length and ambiguity
– Shorten to exclude any unnecessary variables
(Think ahead to data analysis and presentation)
• Pretest and validate
– Face validity, gold standard, predictive validity
Questionnaires vs. Interviews
• Questionnaires:
– Efficient for simple questions, less expensive and time
consuming for research staff, standardized
– Less likely to be completed by people of limited literacy or lower
education so results may be biased
• Interviews:
– Complicated questions, more costly and time-consuming, less
uniform, answers may be influenced by interviewer or
participant/ interviewer relationship
– Should be as standardized as possible
• Wording, nonverbal signals, tone of voice, probing
• Both are subject to imperfect memory
• Both are affected by tendency to give socially acceptable
answers
Chapter 16: Steps in Data
Management
• Define each variable
• Set up the study database
• Test data management procedures before
the study begins
• Enter the data-identify and correct errors
• Back up the dataset regularly
• Create a dataset for analysis
• Archive and store database and results
Chapter 17:
Implementing the Study
• Pretesting (pilot studies)
• Less stimulating than design or analysis
• Either can lead to a change in protocol
once study begins
• Quality control measures: train the team,
certify the team, regular meetings,
performance review
Chapter 18: Community Studies
• Research outside the academic setting
designed to meet the needs of the
community
– Answers questions of local or regional
importance
– May be more generalizable
– Can help local economy and self-sufficiency
Chapter 19: Writing and
Funding a Research Proposal
• Protocol: Detailed written plan of study
• Proposal: Written document to receive funding
– Contains the protocol, the budget and other
administrative and supporting documentation
Proposal Writing Guidelines
• Decide where proposal will be sent
• Follow specific guidelines, requirements, process of
each agency
• Find model proposal specific to agency of interest
• Written criticisms of successful and unsuccessful
proposals submitted to agency
• Contact scientific administrators to review proposal draft
• Organize team, assign leader
– Organization and delegation
– Authority/ accountability chart including all members of research
team
• Timetable and periodic meetings
– Research plan and timetable
– Work from outline
• Review, pretest, and revise repeatedly
Proposal Elements
• Title, abstract, table of contents
• Budget with budget justification, biosketches of
investigators, resources
– Rebudgeting usually acceptable, increased funding may not be
• Specific aims, significance, preliminary studies, and
previous work of investigators
– What has been accomplished, what are the problems, what needs to be
done in the field under study
• Methods, statistical section, timetable
– Special attention to methods because they will be scrutinized
• Ethical issues, consultants, references and
appendices
Scientific Methods
•
•
•
•
Give special attention to this section
Serves as manual for future studies
Should contain table of contents
Should contain general overview
– Diagram may be helpful
• Includes subjects and measurements
• Includes pretest plans, data management,
quality control
Methods Sections
• Overview of design (time frame and nature of
control)
• Study subjects (selection criteria, recruitment
plans)
• Measurements (intervention, outcome variables)
• Pretest plans
• Statistical issues
• Quality control
• Timetable and organizational chart
More Scientific Methods
Statistical Section
• Analysis plans
• Null hypothesis, statistical test, sample
size, power estimate
• Involve statistician in writing or editing this
section
Human Subjects Section
•
•
•
•
•
Address ethical issues
Include children, women, minorities involved in study
Risk and benefit presentation
Obtaining informed consent
Use and value of each consultant, with letter of consent
and biosketch
• Arrangements with collaborating institutes with letters of
agreement addressed to investigator
• References
• Appendices as needed for technical and supporting
material
Funded Research
• Government (NIH)
– Funding is general and determined by peer review
– Summary statement of criticisms and comments
– Resubmission should begin with quotes from summary
statement with corresponding responses and proposal changes
• Private nonprofit (Foundations)
– Funding is specific and determined by executive process
• Private for profit
(Drug companies, Med equipment suppliers)
– Establish right to publish
• Intramural resources
– Local research funds from the universities for their own
researchers
Acknowledgments
• Hulley, Stephen, et. al. Designing Clinical
Research, 2ND ed. Lippincott Williams &
Wilkins. Philadelphia. 2001