CONTACT: BECKY SOGLIN
2130 Medical Laboratories
Iowa City IA 52242
(319) 335-6660; fax (319) 335-8034
Release: June 22, 2000
Observational studies can be as valid as randomized
IOWA CITY, Iowa -- Physicians have never considered
observational studies to be as valid as randomized controlled trials, leading
many of them to disregard observational findings. However, observational studies
published in reputable medical journals often give similar results to randomized
controlled trials, according to an analysis by University of Iowa researchers.
The investigation has implications for how medical
treatments are evaluated for effectiveness. The findings were published in
today's issue of the New England Journal of Medicine.
In randomized controlled trials, clinicians randomly
assign patients to receive one of two or more medical treatments and subsequently
analyze the treatment results. In observational studies, patients are treated
as usual by their physician, who decides the treatment course. Researchers
become involved post-treatment in analyzing the treatment data.
"We found that for several treatment areas the two
types of studies gave similar results," said Arthur J. Hartz, M.D., Ph.D.,
UI professor of family medicine and co-author of the study. "Our findings
suggest that the results from observational studies should not necessarily
be dismissed. We should take a closer look at how to integrate observational
studies with other types of research."
He added, "There is a tremendous amount of computerized
data that has been ignored because of the stigma attached to observational
studies. These data sets may provide an untapped resource for the evaluation
of medical treatments."
Hartz, who co-authored the study with Kjell Benson,
a UI medical student who recently completed his second year, said they had
accepted the conventional wisdom that these observational studies give different
results than controlled trials and that they set out to investigate what types
of observational studies were most likely to be valid. However, as they read
research articles, they found that the observational results were surprisingly
similar to the results from randomized controlled trials, and they began to
pursue that similarity.
"Everyone assumed the question was settled," Benson
said. "While randomized controlled trials will always be the gold standard,
certain observational studies have utility, being cheaper and easier to conduct,
and may be equally reliable."
Using medical databases, including Medline and Cochrane,
the researchers found and analyzed observational studies and randomized controlled
trials published between 1966 and 1998 that evaluated the same treatments.
The researchers analyzed studies of physician-implemented treatments that
assessed the difference between two treatments or between one treatment and
no treatment. They eliminated studies in which patients had a say in treatment
received, such as studies about vitamin use or routine screening for disease.
The investigation resulted in a collection of 136
articles that covered 19 different treatment areas such as hormone replacement
therapy for osteoporosis, calcium-channel blocker therapy for coronary artery
disease and fertility treatments.
In only two of the 19 treatment areas was the difference
between the observational studies and the randomized controlled trials statistically
significant. In a few additional treatment areas, there were qualitative differences
between the observational studies and the randomized controlled trials, but
the patient sample sizes were not adequate to prove a difference. However,
in the majority of treatment comparisons, the results from the two types of
studies were similar.
"We did not cover the universe of medical studies,"
Hartz said. "There are many medical treatment areas for which no one has tried
to do an observational study. In addition, the majority of the observational
studies we assessed were in the higher-quality journals."
The advantage of randomized controlled trials, Hartz
said, is that it increases the likelihood that patients who have one treatment
are on average similar to patients who have the other treatment. However,
the method can be expensive and take considerable time to obtain results.
In observational studies, patients treated with one
method are compared to patients treated with another method for the same condition.
The study is not considered experimental because the person conducting the
research does so only after treatment has been provided by a physician.
"The advantage of observational studies is that the
data may already be available or relatively easy to collect," Hartz said.
"The disadvantage is that differences in the patient outcomes for the two
treatments may be due to differences in the patients. For example, a surgical
intervention may be primarily reserved for the sickest patients."
Hartz said that observational trials are not appropriate
for the definitive evaluation of treatments that have only modest benefits.
Benson noted that, where appropriate, an observational study could be used,
then followed by a randomized control trial.
Benson's involvement in the study was made possible
through the UI College of Medicine Medical Student Research Program and the
Family Medicine Research Program, which are
supported by grants from the federal Health Services and
Resources Administration and the National Heart, Lung and Blood Institute.
University of Iowa Health Care describes the partnership
between the UI College of Medicine and the UI Hospitals and Clinics and the
patient care, medical education and research programs and services they provide.