Transcript observational study

Experiments and Observational Studies
Thought Questions
1. In studies to determine the relationship between two conditions (activities, traits, etc.),
one of them is often defined as the explanatory (independent) variable and the other as
the outcome or response (dependent) variable.
In an experiment to determine whether the drug Memox improves cognition of patients
with moderate to severe Alzheimer’s disease, whether or not the patient received Memox
is one variable, and cognitive score is the other.
Which is the explanatory variable and which is the response variable?
2. Suppose you are interested in determining if drinking a glass of red wine each day helps
prevent heartburn. You recruit 40 adults age 50 and older to participate in an experiment. You
want half of them to drink a glass of red wine each day and the other half to not do so. You ask
them which they would prefer, and 20 say they would like to drink the red wine and the other
20 say they would not. You ask each of them to record how many cases of heartburn they have
in the next six months.
At the end of that time period, you compare the results reported from the two groups.
Give three reasons why this is not a good experiment.
Experiments and Observational Studies
Thought Questions
3. When experimenters want to compare two treatments, such as an old and a new drug, they
use randomization to assign the participants to the two conditions.
If you had 50 people participate in such a study, how would you go about randomizing them?
Why do you think randomization is necessary?
Why shouldn’t the experimenter decide which people should get which treatment?
4. In an experiment, researchers assign “treatments” to participants, whereas in an
observational study, they simply observe what the participants do naturally.
Give an example of a situation where an experiment would not be feasible for ethical reasons.
Experiments and Observational Studies
Randomized Experiment versus Observational Studies
• In a randomized experiment we create differences in the explanatory variable and examine
results (response variable).
•
In an observational study we observe differences in the explanatory variable and then notice
whether these are related to differences in the response variable.
Confounding Variables
A confounding variable is related to the explanatory variable, and affects the response variable.
Example: Study of the relationship between smoking during pregnancy and child’s subsequent
IQ a few years after birth.
Explanatory variable?
Response variable?
Possible confounding variables?
• The effect of confounding variables on the response variable cannot be separated from the
effect of the explanatory variable
Experiments and Observational Studies
Designing a Good Experiment
•
•
Randomization
Randomizing the Order of the Treatments
Example: Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp. 595-600)
•Nicotine and control "look-alike" patches were randomly assigned to participants.
•Baseline values were compared for the two groups on numerous characteristics
•Because they examined comparability on 28 characteristics, a few showed substantial
differences in the two groups, as would be expected just by chance.
•For instance, 85% of those receiving the nicotine patches were employed, while only 73.3% of
those receiving the control patches were employed.
Experiments and Observational Studies
Control Groups, Placebos and the Placebo Effect
Control Groups: Handled identically to the treatment group(s) in all respects, except that they
don’t receive the actual treatment. Handled identically to the treatment group(s) in all respects,
except that they don’t receive the actual treatment.
Special type of control group: Placebos. Why do we use them?
What is the Placebo Effect?
Experiments and Observational Studies
Effect of colour of drugs: systematic review of perceived effect of drugs and of their
effectiveness – BMJ 1996; 313 : 1624 (Published 21 December 1996)
Objectives: To assess the impact of the colour of a drug's formulation on its perceived effect and its
effectiveness and to examine whether antidepressant drugs available in the Netherlands are
different in colour from hypnotic, sedative, and anxiolytic drugs.
Design: Systematic review of 12 published studies. Six studies examined the perceived action of
different coloured drugs and six the influence of the colour of a drug on its effectiveness.
•The colours of samples of 49 drugs affecting the central nervous system were assessed using a
colour atlas.
Main outcome measures: Perceived stimulant action versus perceived depressant action of colour
of drugs.
•The trials that assessed the effect of drugs in different colours were done in patients with different
diseases and had different outcome measures.
Experiments and Observational Studies
Effect of colour of drugs: systematic review of perceived effect of drugs and of their
effectiveness – BMJ 1996; 313 : 1624 (Published 21 December 1996)
Results: The studies on perceived action of coloured drugs showed that red, yellow, and orange are
associated with a stimulant effect, while blue and green are related to a tranquillising effect.
•The trials that assessed the impact of the colour of drugs on their effectiveness showed
inconsistent differences between colours.
•The quality of the methods of these trials was variable. Hypnotic, sedative, and anxiolytic drugs
were more likely than antidepressants to be green, blue, or purple.
Conclusion: Colours affect the perceived action of a drug and seem to influence the effectiveness of
a drug.
•Moreover, a relation exists between the colouring of drugs that affect the central nervous system
and the indications for which they are used.
•Research contributing to a better understanding of the effect of the colour of drugs is warranted.
Experiments and Observational Studies
Study: Sham device v inert pill: randomised controlled trial of two placebo treatments
Objective To investigate whether a sham device (a validated sham acupuncture needle) has a
greater placebo effect than an inert pill in patients with persistent arm pain.
Design A single blind randomised controlled trial created from the two week placebo run-in
periods for two nested trials that compared acupuncture and amitriptyline with their respective
placebo controls.
Comparison of participants who remained on placebo continued beyond the run-in period to the
end of the study.
Setting: Academic medical centre. Participants 270 adults with arm pain due to repetitive use
that had lasted at least three months despite treatment and who scored ≥ 3 on a 10 point pain
scale.
Interventions Acupuncture with sham device twice a week for six weeks or placebo pill once a
day for eight weeks.
Main outcome measures: Arm pain measured on a 10 point pain scale.
Secondary outcomes were symptoms measured by the Levine symptom severity scale, function
measured by Pransky’s upper extremity function scale, and grip strength.
Experiments and Observational Studies
Study: Sham device v inert pill: randomised controlled trial of two placebo treatments
Results: Pain decreased during the two week placebo run-in period in both the sham device and
placebo pill groups, but changes were not different between the groups ( − 0.14, 95%
confidence interval − 0.52 to 0.25, P = 0.49)
Longitudinal regression analyses that followed participants throughout the treatment period
showed significantly greater downward slopes per week on the 10 point arm pain scale in the
sham device group than in the placebo pill and on the symptom severity scale.
Differences were not significant, however, on the function scale or for grip strength. Reported
adverse effects were different in the two groups.
Conclusions: The sham device had greater effects than the placebo pill on self reported pain
and severity of symptoms over the entire course of treatment but not during the two week
placebo run in.
Experiments and Observational Studies
Blinding: Single and Double-Blind experiments.
Example: Case Study 5.1, measuring the impact of nicotine patches on smoking, could be doubleblind experiment.
Why is this a good idea?
Matched Pairs, Blocks, and Repeated Measures
Experiments using pairing or blocking are those in which the same or similar experimental units
are used for all treatments. These are sometimes called repeated measures experiments.
Mozart Example: used the same participants to listen to Mozart and to the other conditions
(listening to relaxation tape, silence).
Why is this a good idea?
For other studies, like the nicotine study, it isn't possible to use the same people.
Experiments and Observational Studies
Difficulties and Disasters in Experiments
Example: Quitting Smoking with Nicotine Patches
•
•
•
•
•
240 smokers recruited (all met entry criteria).
Randomly assigned to either nicotine patch or placebo patch for 8 weeks.
Double-blinded.
After 8 weeks: 46% of nicotine group quit, only 20% of placebo group quit.
After 1 year: 27.5% of nicotine group quit, only 14.2% of placebo group quit.
Confounding variables
Example: Quitting Smoking with Nicotine Patches
• Nicotine patch more effective when no other smokers in home.
• If first 120 volunteers assigned to placebo and last 120 to nicotine patch, and if those with no
other smokers in home more eager to volunteer => what would be the problem with the results?
Another Example: In Case Study 1.1 Lee Salk did not randomize the infants to either hear the
heartbeat sound or not. He used the same nursery on subsequent days, with different groups of
babies.
What problems could occur?
Experiments and Observational Studies
Interacting Variables
Effect of explanatory variable on response variable may vary over levels of other variables.
The solution?
Example: Quitting Smoking with Nicotine Patches
•The effectiveness of nicotine patches varied depending on whether or not there were other
smokers at home, but the effectiveness of the control patches did not.
•The presence of other smokers interacted with the treatment in producing the results.
Percent quitting
Nicotine
Placebo (Control Patches)
Smoker at home
31%
20%
No smoker at home
58%
20%
Interacting Variables
Experiments and Observational Studies
Example: Quitting Smoking with Nicotine Patches
Note: The researchers who conducted this study looked at a variety of potential interacting
variables but found only “other smokers at home” to be significant.
Some of the additional possibilities they considered were age, weight and whether or not the
participant suffered from depression.
Experiments and Observational Studies
Hawthorne Effects
Indicate that people respond differently when they know they are part of an experiment.
1920’s Experiment by Hawthorne Works of the Western Electric Company

What changes in working conditions improve productivity of workers?
– More lighting?
– Less lighting?
– Other changes?
Ecological Validity and Generalizability
Variables measured in labs or artificial setting, results do not accurately reflect impact in real
world.
The problem:
lack of generalizability due to:
unnatural settings
sample that is not representative of population
Experiments and Observational Studies
Designing a Good Observational Study
In an observational study, researchers don’t assign choices; they simply observe them.
Manipulation occurs naturally, not imposed. Can’t assume the explanatory variable is the only
one responsible for any observed differences in the response variable.
Case-Control Studies attempts to include an appropriate control group.
‘Cases’ who have particular attribute or condition are compared with ‘controls’ who do not.
Example: Baldness and Heart Attacks
•
•
•
Observational study: compared 665 men admitted to hospital with 1st heart attack to 772
men (same age group) admitted to same hospitals for other reasons.
Percent with pattern baldness higher for heart attack group (42%) compared to no heart
attack (34%).
Included adjustments for age and other heart attack risk factors.
What are the advantages of a case-control study?
What do you think of the controls used in this study?
Example: In determining whether owning a pet bird is related to incidence of lung cancer,
researchers would identify lung cancer patients as the cases and then find people with similar
backgrounds who do not have lung cancer as the controls. They would then compare the
proportions of cases and controls who owned a pet bird.
Experiments and Observational Studies
Designing a Good Observational Study
Retrospective or Prospective Studies
Prospective: participants followed into future, and events recorded.
Retrospective: participants are asked to recall past events.
Which is better and why?
Experiments and Observational Studies
Difficulties and Disasters in Observational Studies
Confounding Variables and the Implications of Causation
Problem: No way to establish causation with an observational study.
Example: Smoking During Pregnancy
• IQs lower for children of women who smoked.
• Difference as high as 9 points before accounting for confounding variables (diet and education);
reduced to 4 points after accounting for those factors.
• Can’t conclude smoking caused lower IQs in children.
Extending the Results Inappropriately
The problem: Because many observational studies use convenience or volunteer samples, it
may not be valid to extend the results to other groups.
Example : Baldness and Heart Attack
• Observational study only used men who were hospitalized.
• Should consider whether results should be extended to all men.
Experiments and Observational Studies
Using the Past as a Source of Data
The problem: In some observational studies participants are required to remember the past.
Partial solution: Use official records if they exist, rather than relying on memory. If possible,
use prospective study.
Example: Pearson, Ross and Dawes (1992, pgs. 75-76) reported on a study in which chronic
headache sufferers were asked to keep a diary of the intensity of their pain for a week.
• At the end of the week, they were asked to rate their current level of pain, then to recall the
maximum and minimum pain they had felt during the week.
•Those who were currently suffering from a high level of pain overestimated their prior levels
of pain, while those with current low levels underestimated their prior pain.
Experiments and Observational Studies
Go easy on the coffee, you could start seeing things – Daily Mail, January 14th, 2009
Drinking cup after cup of coffee dramatically increases the risk of hallucinating, research shows.
Healthy young men and women who had more than seven cups of instant coffee a day were three
times more likely to hear or see things that were not there than those who limited their intake to
less than a cup.
•Confirming the link could lead to new treatments for those who suffer severe hallucinations,
including schizophrenics, some victims of child abuse and the recently bereaved.
•The Durham University researchers asked 219 students to document their caffeine intake,
working on the principle that a cup of instant coffee contains 45mg of caffeine.
•The volunteers were also asked how often they suffered hallucinations.
•The high caffeine users were three times as likely to have had problems as those who rarely drank
coffee.
•Large amounts of caffeine also made people more likely to think they could sense the presence of
ghosts, the journal Personality and Individual Differences reports.
Experiments and Observational Studies
Years Later, No Magic Bullet Against Alzheimer's - NY Times, August 29th, 2010
•The scene was a kind of science court. On trial was the question ''Can anything -- running on a
treadmill, eating more spinach, learning Arabic -- prevent Alzheimer's disease or delay its
progression?''
•To try to answer that question, the National Institutes of Health sponsored the court, appointing a
jury of 15 medical scientists with no vested interests in Alzheimer's research.
•For a day and a half last spring, researchers presented their cases, describing studies and explaining
what they had hoped to show.
•The studies included research on nearly everything proposed to prevent the disease: exercise,
mental stimulation, healthy diet, social engagement, nutritional supplements, anti-inflammatory
drugs or those that lower cholesterol or blood pressure, even the idea that people who marry or
stay trim might be saved from dementia. And they included research on traits that might hasten
Alzheimer's onset, like not having much of an education or being a loner. …..
•''I was surprised and, at the same time, very sad'' about the lack of evidence, said Dr. Martha L.
Daviglus, the panel chairwoman and a professor of preventive medicine and medicine at the
Feinberg School of Medicine at Northwestern University.
•''This is something that could happen to any of us, and yet we are at such a primitive state of
research.'' ….
Experiments and Observational Studies
Years Later, No Magic Bullet Against Alzheimer's - NY Times, August 29th, 2010
•The problem, the group wrote, was that ''the quality of the evidence was typically low.''
•Most studies observed people who happened to use or not use a possible preventive measure and
then determined whether they got Alzheimer's or not.
•Such studies, known as observational ones, are not the gold standard, like those in which people are
randomly assigned to take a pill or do something like exercise, or not. Observational studies are useful
in generating hypotheses but are not proof. Still, if several well-done studies of this type come to the
same conclusion, they can be valuable evidence.
•In the case of Alzheimer's prevention, though, the studies tended to have problems, Dr. Williams
said.
•Often it was not clear precisely what subjects were doing. They might have been using a drug or a
supplement at the start of the study but the dose was not specified, nor was it clear whether subjects
were taking the same doses, or for how long.
•Some studies of drugs to lower blood pressure used self-reports as opposed to, for example,
pharmacy data. A 12-year study asked participants about their use of cholesterol-lowering statins at
the start of the study but never did again.
•A nine-year statin study used pharmacy records but included as users those who took the drugs at
any time during the study period.
Experiments and Observational Studies
Text questions pg 102