Transcript Module 3: Outcome Measures & Statistical Analyses

ClinicalTrials.gov:
A Primer on Results Reporting
Christina McCarthy, MS
Senior Compliance and Regulatory Affairs Specialist
Boston Children’s Hospital
[email protected]
January 14, 2015
1
Outline
• Introduction
– Registration Requirement Re-cap
– Results Reporting Requirement
– Preparing for Results Reporting
• The Four Results Modules
–
–
–
–
Participant Flow
Baseline Characteristics
Outcome Measures
Adverse Events
• ClinicalTrials.gov QA Review Process
A copy of this presentation is available on the BCH intranet:
Research > Translational Research > Regulatory Resources
2
Registration Requirement Re-cap
FDAAA – a law
ICMJE - voluntary
What to
register
“Applicable Clinical Trials”
• Controlled, interventional trials:
drugs, biologics, devices
• “Phase 2 – 4” (not Phase 1; not
feasibility)
• US FDA jurisdiction (e.g IND/IDE
or US site)
• Any human research project that
prospectively assigns human subjects to
intervention or comparison groups, with
or without concurrent controls, to study
the cause-and-effect relationship
between a medical intervention and a
health outcome.
When to
Register
• At trial initiation: not later than 21 days • Prior to enrollment of 1st subject
of enrollment of 1st subject
• Update at least every 12 months
Results
reporting
• “Applicable Clinical Trials” using FDA
approved or cleared (for any use)
products.
If noncompliant
• Public notice; NIH funds withheld; FDA • Cannot publish in select journals
• NO
sanctions, civil monetary penalties (up
to $10, 000/day)
3
3
FDAAA Results Reporting Requirement
Required for:
• FDAAA “Applicable Clinical Trials” that study drugs,
biologics, and devices that are approved, licensed, or
cleared by FDA (for any use)
When:
• Within 12 months of Primary Completion Date (the final
data collection for the primary endpoint)
• If product not approved by Primary Completion Date but is
approved later, then results due 30 days after approval
• Delays are possible, primarily for manufacturer or under
limited special circumstances
o Pending publication is NOT considered a good cause for
delay
http://clinicaltrials.gov/ct2/manage-recs/fdaaa
4
Brief interruption to mention
changes are coming…
Approved or
Unapproved
5
An Overview of Results Reporting
• The process of submitting results information to
ClinicalTrials.gov is conceptually similar to preparing a
manuscript for publication in a journal.
• An individual familiar with the study design and data analysis
(such as the clinical investigator or study statistician) will need
to be involved in order to accurately summarize the results
information in the tabular format required by law and to
ensure that the results are consistent with the
ClinicalTrials.gov review criteria.
6
An Overview of Results Reporting
• Scientific information is submitted as 4
separate modules:
1.
2.
3.
4.
Participant Flow
Baseline Characteristics
Outcome Measures
Statistical Analyses and Adverse Events
7
How are Results Reported?
• Tables are constructed by data providers
– “Stand alone” tables - must be meaningful to
people who are not already familiar with the
study.
– No narrative conclusions
– Columns are study arms
– Rows are measures
– Type of measure determines specific design of
cells
8
Quick Reference:
Steps for Submitting Results
Follow these steps for submitting results:
1.
2.
3.
4.
Learn about the requirements for submitting results.
Login to the Protocol Registration and Results System (PRS).
Update the Protocol Section and release (submit) the record.
Enter the required (and optional) results data elements.
– For basic help with using PRS, review the Quick Start Guide found in the Help section of the
PRS main menu.
– See the Basic Results Data Element Definitions for descriptions of each required data item.
– See the results data preparation checklists and simple results templates for each module,
listed in the Scientific Information section above, for the data you will need and a view of
the data elements in a tabular format.
– See the Helpful Hints (PDF) for tips on entering results data, including three examples of
common study models (parallel design, crossover design, and diagnostic accuracy studies)
and measure types.
5. Preview, inspect, and release (submit) the record.
9
Preparing for Results Reporting
• Accurate registration entry is essential for
results reporting
– Be sure your registration is as accurate and
complete before entering results.
– Registration inaccuracies will be replicated in the
results reporting module as some fields auto
populate.
10
Preparing for Results Reporting
• TIP: Populating the “Detailed Description” field in your
Registration record may assist in the reader’s
understanding of the study and will make your Quality
Assurance review more effective. Remember, the
ultimate goal is clarity for the public.
• TIP: If time allows, release your registration one final
time to the ClinicalTrials.gov QA review team to ensure
you have addressed any errors before inputting results.
11
Preparing for Results Reporting:
Recommended Reading
How to Submit Your Results
Includes preparation checklists and templates, also links to definitions
http://clinicaltrials.gov/ct2/manage-recs/how-report
Simple Results Templates and Examples
http://prsinfo.clinicaltrials.gov/results_table_layout/ResultSimpleForms.html
Basic Results Data Elements Definitions
http://prsinfo.clinicaltrials.gov/results_definitions.html
PRS User Guide
Located on Main Menu in database
10 minute webinars for each results module
http://clinicaltrials.gov/ct2/manage-recs/present
Helpful Hints (with common study designs examples)
http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf
12
Results Overview
• This is your home base for inputting results. You can access the 4 Results
modules from here.
• Click on “edit” next to Participant Flow to edit that module.
13
Edit Mode – Simple but important!
At the bottom of each page in edit mode:
Save: Saves your work and brings you back to the “Results
Overview page.”
Validate: Activates display of Errors, Warnings and Notes, but
does not save entered data.
Cancel: Does not save changes and brings you back to the
“Results Overview page.”
14
Module 1: Participant Flow
15
FDAAA: Participant Flow Definition
“A table…, including the number of patients
who dropped out of the clinical trial and
the number of patients excluded from the
analysis, if any.”
FDAAA[Sec. 282(j)(3)(c)(i)], http://www.gpo.gov/fdsys/pkg/PLAW110publ85/pdf/PLAW-110publ85.pdf#page=82
16
Participant Flow Purpose
• Tabular presentation of progress of research participants
through each stage of trial
– Arms are “copied” from Protocol Section, but may be
modified
– Table may consist of a single Period or multiple Periods,
to represent different stages of the trial
– Each Period must include 2 Milestones: Number
STARTED and Number COMPLETED
• The module should account for all enrolled participants, and
make it clear which participants were analyzed.
See ClinicalTrials.gov Participant Flow Module Presentation, Rebecca J. Williams, PharmD, MPH, at http://prsinfo.clinicaltrials.gov/webinars/module5/index .html
17
Participant Flow Data Elements
• Recruitment Details
• Milestones
• Pre-assignment Details
‐ Milestone Title
• Arm/Group*
‐ STARTED Data*
‐ Title
‐ COMPLETED Data*
‐ Description
‐ Reason Not Completed
• Period Title(s)*
 Type (e.g., Death)
‐ “Overall Study”
 Data
(default) if single period
*Required by ClinicalTrials.gov
For definitions of these elements, see the Basic Results Data Elements Definitions:
http://prsinfo.clinicaltrials.gov/results_definitions.html
18
19
Get Organized Using the Simple Form
20
Participant Flow Structure
• Tabular:
• Arms/Groups:
• Period(s):
• Milestones:
Example: If your
study only has one
period than you
would provide the
title of “Overall
Study” to the
period and only
one Participant
Flow table would
be needed for
your record.
Table showing progress of participants through each stage
pre-populated from Protocol Section (but can be modified)
represent different stages of study (single or multiple)
specific events/time points when # of subjects reported.
STARTED and COMPLETED are required data.
Example: Basic Participant Flow table for parallel design study
For examples of other study designs, see ClinicalTrials.gov’s Helpful Hints: Basic
Results at http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf
21
Participant Flow Structure (Multiple Periods)
EXAMPLE: If your study has 3
different periods, you would
create 3 tables and provide a
descriptive title for each.
STARTED & COMPLETED are
required milestones. You can add
others milestones to show
specific events when the # of
participants was reported.
You will track the # of participants
that do not complete the period
and provide a reason why.
Example adapted from ClinicalTrials.gov‘s Basic Results Helpful Hints, p.13 at
http://prsinfo.clinicaltrials.gov/ResultsExamples.pdf
22
Participant Flow – What is wrong
with this?
Period: First Intervention
STARTED
Arm A: Drug
Bupropion, then
placebo
92
Arm B: Placebo,
then Drug
Bupropion
95
COMPLETED
88
95
NOT COMPLETED 4
Period: Second Intervention
Arm A: Placebo,
then Drug
Bupropion
STARTED
88
COMPLETED
88
NOT COMPLETED 0
0
Arm B: Drug
Bupropion, then
placebo
93
93
0
For the 1st period, there is
no explanation of why some
subjects did not complete
the period.
In the 2nd period, the # of
subjects that started does
not match the # that
finished the 1st period. This
may be a data error. Or if
there was a washout period
between the two periods,
you may want to add that
as a separate period and
explain what happened to
the subjects that dropped
out.
23
Participant Flow – Quick Tips
• Number of participants “started” should match
“Enrollment, Actual” in protocol section
– Pre-Assignment Details can be used to explain any
discrepancies in Enrollment, Actual and total number of
participants Started.
• Specific Periods to reflect study design and to
account for # of participants starting and completing
each Period
• Use of Milestones to convey key events
– For example, number who received intervention
• Reasons for non-completion
See http://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf
24
Module 2: Baseline Characteristics
Baseline Characteristics
• “A table of demographic and baseline data for
the entire trial population and for each arm or
comparison group.” Note that only baseline
measures for Age and Gender are required; all
other baseline measures are optional.
http://prsinfo.clinicaltrials.gov/results_definitions.html
explaining FDAAA Sec. 282(j)(3)(c)(i)
Description of Baseline Characteristics
Module
• Table of demographic and baseline data for the
entire trial population and for each arm or
comparison group
• Accommodates different data types:
– Continuous: measure of central tendency (e.g.,
mean) and measure of dispersion (e.g., standard
deviation)
– Categorical: for each category – (1) a count or (2)
measure of central tendency and measure of
dispersion
28
29
Get organized using the Templates
If it’s in Table 1, it probably belongs in
Baseline Characteristics
These will
become
Default
measures
These will
become
User
Specified
Baseline
Measures
ClinicalTrials.gov Format
33
ClinicalTrials.gov Format
User-Specified Baseline Characteristics
• Do include all meaningful elements that make your study
understandable/ useful to others, both demographic and
clinical measures, such as baseline values of outcome
measures or prior and concurrent treatment
characteristics.
• Make sure units and scales are labeled, and understandable
– (e.g. what the range of a scale is and what it means. If it’s
a well-known scale, within the discipline, refer to it by
name and give a citation, if necessary.)
• Use Table 1, if there is a published article or draft.
• If there is no article from which to work, consult with PI.
Baseline doesn’t always mean only at
the beginning of the study
More than one period or time point can be considered
“baseline” in studies which follow more complicated protocols:
• Crossover
• Dose escalation
• Factorial
• Multiple Period
That is, not every one of these designs would require posting
multiple baselines, but there may be times when the responsible
party feels that to have a clear understanding of the findings of
the study, it helps to show how baseline characteristics differed
between one portion of the study and another.
Example of Multiple Baselines shown
Reporting Groups
Description
Sorafenib
(Nexavar,
BAY43-9006)
Inpatient dose escalation of sorafenib from 400 mg orally twice
daily (bid) for the first cycle, 600 mg bid for the second cycle and
800 mg bid until disease progression, unacceptable toxicity or
withdrawal of consent. Each cycle is 28 days. Dose reductions due
to toxicities were allowed.
Cycle 1 (28 Days)
Sorafenib (Nexavar, BAY43-9006)
STARTED
83
COMPLETED
52
Not Completed
31
Reporting Groups
Description
Sorafenib
(Nexavar,
BAY43-9006
Inpatient dose escalation of sorafenib from 400 mg
orally twice daily (bid) for the first cycle, 600 mg bid
for the second cycle and 800 mg bid until disease
progression, unacceptable toxicity or withdrawal of
consent. Each cycle is 28 days. Dose reductions due
to toxicities were allowed.
Baseline Measures
Sorafenib (Nexavar, BAY43-9006
Cycle 2 (28 Days)
Number of Participants
83
Sorafenib (Nexavar, BAY43-9006)
STARTED
52
COMPLETED
44
Not Completed
8
Cycle 3 (28 Days)
Sorafenib (Nexavar, BAY43-9006)
STARTED
44
COMPLETED
33
Not Completed
11
Age Continuous
[units: years]
Median (Full Range)
Cycle 1 (n=83)
61 (33 to 80)
Cycle 2 (n=52)
63 (33 to 72)
Cycle 3 (n=44)
55 (33 to 72)
From supplemental slide provided at Train-the-Trainer
workshops; courtesy of ClinicalTrials.gov.
Important Quality Checks
Review record to ensure:
• Categories do not overlap and all categories are
presented
• Range and direction of scores in baseline
characteristics description are explained
• Terms are understandable to the general public. Use
footnotes if necessary to describe a score.
MODULE 3:
OUTCOME MEASURES & STATISTICAL ANALYSES
39
FDAAA*: Outcome Measures
“…a table of values for each of the primary and
secondary outcome measures for each arm of
the clinical trial…including the results of
scientifically appropriate tests of the statistical
significance of such outcome measures.”
[Sec. 282(j)(3)(C)(ii)]
*Food and Drug Administration Amendments Act of 2007
40
Outcome Measures Purpose
The Outcome Measures module displays the
results and associated statistical analyses for
each prespecified primary and secondary
outcome measure. Other outcome measures
may also be included.
See ClinicalTrials.gov Outcome Measures and Statistical Analyses Module Presentation, Deborah
A. Zarin, M.D. at http://prsinfo.clinicaltrials.gov/webinars/module7/index.html
Outcome Measures – Data Elements
• To set up table
– Arm/Group title* and Description
– Number of Participants Analyzed*
• To Describe specific Outcome Measures
– Outcome Measure Title*
– Outcome Measure Description
– Unit of Measure*
– Outcome Measure Time Frame*
– Measure Type (e.g., mean, median)*
– Measure of Dispersion/Precision (e.g., Standard
Deviation)*
• Data*
*Required by ClinicalTrials.gov
http://prsinfo.clinicaltrails.gov/results_definitions.html
42
Outcome Measures Data Elements
• Outcome Measure Type*
– Options




Primary
copied from protocol section
Secondary
Other Pre-specified
Post-hoc
• Outcome Measure Reporting Status*
• Outcome Measure Safety Issue? (Y/N)
• Use the Outcome Measure Data Preparation Checklist found at:
http://prsinfo.clinicaltrials.gov/data-prep-checklist-om-sa.pdf
*Required by ClinicalTrials.gov
http://prsinfo.clinicaltrails.gov/results_definitions.html
43
Get Organized Using the Simple Form
Use one simple form for each Outcome Measure you are entering into the database. 44
Outcome Measures – Journal Article Format
Pappas PG, Chetchotisakd P, Larsen RA et al. Clin Infect Dis. 2009
Journal articles have similar information to ClinicalTrials.gov, just presented
in a different format!
© 2009 Infectious Diseases
Society of America
45
Outcome Measures – ClinicalTrials.gov Format, Public View
NCT00145249
46
ClinicalTrials.gov Format (cont.), Public View
NCT00145249
47
Outcome Measure
Outcome measure information:
Please be specific as possible.
•
•
•
•
Title: include the name of the
specific measure. Avoid using verbs,
that is, do not put “To determine…”
Time Frame: must have a time point
at which the outcome is assessed for
the specific metric used (hours, days,
weeks, years) Hint: specify which
study day it is measured - do not use
“until the end of study or death”.
Description: describes what will be
measured, not why it is measured. If
the outcome measure is a
questionnaire or scale, provide the
range and what low or high scores
mean.
Safety Issue: Is this outcome
measure assessing a safety issue?
48
Example of an Outcome Measure
What is wrong with this?
Dysphagia Questionnaire
At the end of the study
Number of Patients with Complete Response to Dysphagia
Baseline to 6 weeks
49
Data Elements – Statistical Analyses*
*The law requires scientifically appropriate tests of statistical significance.
Basic Results Data Elements Definitions: http://prsinfo.clinicaltrials.gov/results_definitions.html
50
Go to Outcome Overview to Add Statistical Analysis
51
Add Outcome Statistical Analysis
52
Add Outcome Statistical Analysis
Choose the statistical test from the drop-down box.
53
Completed Outcome Measure and Statistical Analysis
54
Outcome Measures and Statistical
Analyses – Quick Tips
• Results, including study or outcome conclusions, are not
provided in narrative form.
• The Title and Description are comprehensible and descriptive
(e.g., Arm A, B, C is not informative).
• For measures obtained using a scale:
– Name of scale is provided in “Measure Title.”
– Range and direction of scores (0 = worst; 10 = best) are indicated in
“Measure Description.”
– “Unit of Measure” is “units on a scale” if no other unit.
See http://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf
55
Module 4: Adverse Events
56
Adverse Events Reporting
Adverse Event (AE)
“Unfavorable changes in health, including abnormal
laboratory findings, that occur in trial participants during
the clinical trial or within a specified period following the
trial. Two types of adverse event data are to be reported:
"Serious" and "Other (Not Including Serious)" adverse
events.”
http://prsinfo.clinicaltrials.gov/results_definitions.html#AdverseEve
ntsDefinition
57
Adverse Events Reporting
Serious Adverse Events (SAEs)
– A table of all anticipated and unanticipated serious adverse events
grouped by organ system, with number and frequency of such events
in each arm of the trial
Other Adverse Events (AEs)
– A table of anticipated and unanticipated adverse events that exceed
5% within any arm of the trial, grouped by organ system, with
number and frequency of such events in each arm of the trial.
58
Description of the Adverse Events
Reporting Module
The Adverse Events Module is a summary of results for SAEs and AEs
that are collected during the study
• Data should be tracked and reported in accordance with the
procedures for data collection as defined in the protocol
• Adverse Events are reported as summary data at the end of the study
• SAEs and AEs are presented in separate tabular format
• Not “real time” adverse event reporting while the study is ongoing
• Use the Adverse Events Reporting Additional Description to provide
information on analysis population, methods of event data collection
and other details
http://prsinfo.clinicaltrials.gov/results_definitions.html
59
Adverse Event Reporting
• Adverse Events are entered by 2 modes
– Manual entry directly into the PRS browser based interface
– Download and Upload link
• Use the predefined Adverse Events Simple form for
preparation
– Simple form (template) is helpful in organizing results
entry
– Data is segmented and compiled by
•
•
•
•
Frequency of Event
Total Number of events
Number of events (deaths, myocardial infarction, fever, etc.)
Number events per participant
60
61
Getting Organized
Using the Serious Adverse Events Template
62
Getting Organized
Using the Other Adverse Events Template
63
Adverse Events
• Frequency Threshold for Reporting Other (Not Including Serious) Adverse
Event
– The frequency of Other (Not Including Serious) Adverse Events that, when
exceeded within any arm or comparison group, are reported in the results
database for all arms or comparison groups. The number must be less than
or equal to the allowed maximum (5%), and must not include any symbols
(e.g., >= , %). Expressed as a percentage.
For example, a threshold of 5 percent indicates that all Other (Not Including
Serious) Adverse Events with a frequency greater than 5 percent within at
least one arm or comparison group are reported.
• Total Number Affected by any Other (Not Including Serious) Adverse Event
above the Frequency Threshold
– Overall number of participants affected by one or more Other (Not Including
Serious) Adverse Events above the specified Frequency Threshold (e.g., 5%).
64
Serious Adverse Events
(Public site view)
65
Other Adverse Events
(Public site view)
66
Important Quality Checks
• Total Number of Participants at Risk = Total Number of participants
Started
– Refer to the Participant Flow
– Adverse Events Reporting Additional Description can be used to explain the
Analysis Population (any discrepancies in total number of participants Started
Participant Flow)
• The number of Events is not required when entering Adverse Events,
however a Note will occur for each results entry field that is left blank.
– You are not required to address Notes in order to submit your record
• Be consistent when entering event names for AEs. Use either all lower
case or upper case letters.
• Corrections for text fields in the AE module must be corrected using
“Modify” under event.
• When using the Download or Upload Adverse Events link, the upload will
be rejected if the columns in your file do not match the order and content
in the PRS.
67
ADDRESSING QA COMMENTS
68
ClinicalTrials.gov QA Results Data Flow
Data Provider Inputs Results.
RP “approves” & “releases” data
to ClinicalTrials.gov.
ClinicalTrials.gov conducts QA
review of data within 30 days.
APPROVES:
ClinicalTrials.gov posts data
on public website.
RESETS to “in-progress”:
ClinicalTrials.gov provides
QA comments to Data
Provider.
69
Accessing QA Comments
Records with QA Comments may
also be accessed using the “QA
Review Comments” link on the
Main Page
Comments are accessed from the
Main Menu –
Problems: Link
70
ClinicalTrials.gov PRS
Common QA Comments
General
 Expand acronyms and abbreviations
 Review for spelling errors
 Proper formatting
Content
 Brief title is in lay language and includes condition and
interventions evaluated in this study
 Review for internal consistency
 Check overall recruiting status and compare it to the study start
date, the primary completion date, and the study completion
date listed
71
ClinicalTrials.gov PRS
Common QA Comments
Outcome Measures
 Enter what is being measured, not why it is being measured
‐
General terms as “safety,” “tolerability,” or “feasibility” are not sufficient
 Time frame, must be specific
‐
‐
‐
“At Follow up” or “end of study” is not specific
Most outcome measures have one time point
Each unique combination outcome measure and time frame should be entered separately
 Time frame for change of outcome measure
‐
‐
Time frames indicate time period over which change occurred– generally two points should
be entered
Time-to-event should include a time over which the event will be assessed
 When entering data into ClinicalTrials.gov enter arms and interventions
‐
‐
‐
Specify each study arm first
Specify each intervention
Assign each intervention to one or more study arms
72
REMINDER: New User Access
• Provide the following information to your PRS Administrator,
o
o
o
o
Irine Breytburg ([email protected]):
BCH Username
BCH Employee Number
Full name (e.g., John J Smith, MD)
Email address
• PRS Administrator sends profile request to ClinicalTrials.gov
• ClinicalTrials.gov emails Investigator/staff notifying of account & provides
temporary password (within 2 days)
• Log into the ClinicalTrials.gov Protocol Registration System:
https://register.clinicaltrials.gov/
73
This slide set was made possible by a collaboration of CTSA
organizations (Mayo Clinic, Partners, University of Michigan
Medical School, Duke University) and the National Library
of Medicine.
The Clinical and Translational Science Awards Program
(CTSA) is part of the Roadmap Initiative, Re-Engineering
the Clinical Research Enterprise and is funded by the
National Center for Advancing Translational Sciences
(NCATS), National Institutes of Health (NIH).
74
74
BACK-UP SLIDES
75
FDAAA – Registration Requirements for
Drugs/Biologics
• Is it a Clinical Investigation?
– Defined as “any experiment in which a drug is administered or dispensed to
one or more human subjects
• Is the clinical investigation controlled?
– Is it designed to permit a comparison of a test intervention with a control to
provide a quantitative assessment of the drug effect?
– Concurrent & Non-concurrent controls
• Is the clinical investigation other than a phase 1 clinical trial?
– Per FDAAA, Phase 1 includes initial introduction of an investigational drug into
humans, metabolism, and pharmacologic actions of a drug, mechanism of
action, and early evidence of effectiveness
76
FDAAA – Registration Requirements for
Devices
• Is it a prospective study of health outcomes?
– FDAAA defines ‘health outcome’ where primary purpose is to evaluate a
defined clinical outcome directly related to human health
• Does the study ‘compare an intervention with a device against
a control in human subjects’?
– ‘Intervention defined broadly to include various techniques using the device
such as (among other things): device regimens and procedures, use of
prophylactic, diagnostic, or therapeutic agents’
• Is the clinical study other than:
– a small clinical trial to determine the feasibility of a device
– a clinical trial to test prototype devices (primary outcome measure related to
feasibility, not to health outcomes)
77
ICMJE – Registration Requirements
Required for Prospective studies that:
• Assign subjects to an intervention with or without concurrent
comparison or control groups
• Study the cause/effect relationship between medical intervention
and a health outcome.
ICMJE scope is much broader than the scope of FDAAA:
• Interventions include procedures, behavioral treatments,
dietary interventions
• Health outcomes include any biomedical or health-related
measure obtained in participants, including pharmacokinetic
measures and adverse events
Source: http://www.icmje.org/publishing_10register.html
78