studies have have multiple errors.
This study1 concerned cardiac patients undergoing outpatient cardiac
catheterization. In this one
study, the authors miscounted their trial's primary endpoint,
included patients that did not belong in the trial, and
failed to appreciate the limitations of the small sample size in regards to
reliably proving that no difference exists.
Finally, the authors failed to be
alarmed at the implications of their own miscounted
An example that would parallel this study would be a trial to evaluate whether drawing blood with needles that have never been sterilized is safe. Assume that 2 out of 100 patients develop an infection and die with the use of the needles that have not been sterilized. Assume 0 out of 100 patients die using the fully sterilized needles. This difference would not be statistically significant. However, it would be a Type II statistical error to state that this study demonstrates that unsterilized needles are safe because statistical significance has not been reached in this small sample size.
A Type II statistical error can easily occur when the numbers being studied are too small to reliably determine that no significant difference exists between the groups being studied. A significant difference, at times may exist, but not reach statistical significance. Larger numbers of patients are often required to prove that no difference exists, compared to the smaller numbers that at times demonstrate that a difference is present. A Type II statistical error occurs if there is truly a difference present, but the study is incorrectly said to demonstrate that no difference exists between the groups.
The details of this flawed study:
1988 study1 was an attempt to assess whether outpatient cardiac
catheterization was a safe approach.
This study reported that 3 of 192 outpatients (1 out of 64) in the study experienced a heart attack with elective outpatient diagnostic cardiac catheterization. (This is in contrast to the usual rate of 1 out of 1000 complication rate for elective diagnostic heart catheterization.) One patient out of 189 patients with inpatient cardiac catheterization had a heart attack.
Implausibly, the authors concluded that these patient results demonstrated that outpatient heart catheterization was safe because this difference was not statistically significant.
a cardiac catheterization lab actually experienced a rate of 3 heart attacks out
of 192 previously in stable patients undergoing diagnostic heart
catheterization, an investigation of that laboratory is in order rather than a
commendation for procedural technique. Though outpatient catheterization is
safe, the data in this study trends in the opposite direction. Further
Subsequent letters to the editor revealed the unusual high frequency of heart attacks was due to the author's miscounting of data points and other trial protocol problems.
The authors answered (details4): They had mistakenly counted patients having a heart attack twice rather than once. This occurred even though having a heart attack as a complication of the procedure was the most important primary trial data endpoint. Furthermore, contrary to their description of the patients undergoing low risk outpatient diagnostic cardiac catheterization, they accidentally included at least one patient in the outpatient group that had a much higher risk procedure, an elective balloon angioplasty who went on to have a heart attack. These errors explained the high frequency of heart attacks that occurred in the outpatient group.
In this one study, the authors
not only misinterpreted their data and ignored adverse trends, they also
miscounted their primary data endpoints and unintentionally broke trial protocol
Block P, Ockene I, Goldberg R, et al. A Prospective Randomized Trial of
Outpatient versus Inpatient Cardiac Catheterization; N Engl J Med 1988: 219:
Letter. Outpatient versus inpatient catheterization. Roehm E. NEJM 1989;
4. Letter. Outpatient versus inpatient catheterization; Block P. NEJM 1989; 320:938-939
copyright 2005-2013 ImprovingMedicalStatistics.com