home   author    contact   web site dedication



Specific guide to this web site for:

 1.  Medical School
      in Statistics

 2.  Medical Students

 3.  Science media writers

 4.  High School & College
     Statistic Teachers


1. Harvard led MI study

2. JACC study 

   (J. of Amer. Coll.

3. NEJM cath study

4. Amer. J. of Cardio.
    review of literature


Oat bran study

Pregnancy & Alcohol

Are Geminis really
9. Columbia 'Miracle' Study  

Additional Topics:


Limitations of Meta-Analyses

Large Randomized Clinical Trials

Tale of Two Large

Advocate meta-analyses

Network meta-analyses




Web Site Entry For High School Teachers And College Statistics Professors

Your feedback is welcome:


 The focus of this web site is how the use of medical statistics and the interpretation of studies in the medical literature can be improved.  The approach employed  is to evaluate individual studies where the statistical analyses have been poorly performed or have conclusions that are poorly formulated.  

A downloadable file of materials from this site is available as noted at the top of this page.  (It would be very useful in addition to obtain the original complete study that is being critiqued. However, most of these are available only through a medical library.) 


This site contains material that can be used in a high school or college introductory statistics course as a source of case studies from the medical literature

Download of selected topics and case study information:  Download (90kb)

The cases most accessible on this web site for high school and college statistic courses are likely to be the following:

1. An example of a completely invalid statistical analysis in the medical literature. In this study in a major medical journal, despite a study involving high powered diagnostic testing involving  P.E.T. scans, the statistical analysis is completely off base.   (i.e. One cannot take two groups and subdivide one of those groups on the basis of high and low values of a certain characteristic, and then say that the two initial groups are different just because the subdivided group with low values of the characteristic differs from the group that was never subdivided).
A statistical analysis in a major medical journal JACC (Journal of American College of Cardiology) which was so bad, that it may have led to a subsequent improvement in the review process of JACC.

2. There is a New England Journal of Medicine article that shows failure in all regards – interpreting data, execution of the trial, incorrectly counting data, and potentially being at risk for a type II statistical error. This may also be of value.  A study in the New England Journal of Medicine where the conclusions run exactly the opposite of what the data suggests. The authors after publication then subsequently determine they had unintentionally miscounted their primary data endpoints. In this one study, the authors  miscounted their trial's primary endpoints, included patients that did not belong in the trial, then incorrectly analyzed the data with a potential for a Type II statistical error, and finally failed to appreciate the implications of their miscounted data  (a grand slam for a poorly conducted and interpreted trial).

3. The underpowered study, how a type II error can occur.
Death of the oat bran fad.  (Murdered by a poorly conceived study.) 
The subsequent meta-analysis published by one of the authors of the initial suboptimal oat bran study is a classic example of how the conclusions of a meta-analysis can be biased.
 case study of biased meta-analysis conclusions

4. There is also an article that is not a clinical study but rather Peter Sleight’s (a leading medical investigator in England) comments about the hazards of subgroup analysis in medical trials. This is a insightful  article and a wonderful presentation of this subject.  The Hazards of Subgroup Analysis  (Are Geminis really different?)

 5. There is also an area that talks of how the medical literature self corrects over time. This may or may not be of interest.  The Reparative Power of Science:  Revisit these controversies
    "10 years later"

6.  Not for your students, but of potential interest to you, is a whole section on the limitations of a meta-analysis and the potential strengths and weaknesses of large randomized clinical trials.  A meta-analysis is often assumed to be a definitive word on a clinical topic by the media as well as some in the field of medicine.  This is certainly not always the case.  The information in this area helps in understanding one of the reasons for variability in the medical literature.

Limitations of Meta-Analyses
    A meta-analysis adds similar smaller trials together in an attempt to better assess the effects of a treatment.  Often a meta-analysis is thought of as the final word concerning a medical topic.  A topic that receives relatively little attention today regards potential weaknesses of meta-analyses. 
The Very Large Randomized Clinical Trial (Strengths and  Limitations)
(This type of trial is increasingly and appropriately used to make recommendations on how to treat patients.  Information on recognizing a well conducted large trial is presented.)

A Tale of Two Large Trials  
    A recent stellar large clinical trial is compared to a large clinical trial with poorly formulated conclusions. Both these trials effect current treatment guidelines for patients.
Beware of Meta-analyses bearing False Gifts  
    A meta-analysis written by a strong proponent of one side of a controversy is particularly problematic.

Download of selected case studies of inappropriately interpreted clinical trials  (90kb)



copyright 2005-2013   ImprovingMedicalStatistics.com