Miscellaneous: Study Evaluation

Study Evaluation

Oxford Criteria Level of Evidence

  • Level 1: Randomized Control Trials
    • 1a: Systematic Review of Multiple RCTs
    • 1b: Individual RCT
  • Level 2: Cohort Studies
    • 2a: Systematic Review of Multiple Cohort Studies
    • 2b: Individual Cohort Study
  • Level 3: Case-Control Studies
    • 3a: Systematic Review of Multiple Case-Control Studies
    • 3b: Individual Case-Control Study
  • Level 4: Case Series
  • Level 5: Expert Opinion

Study Bias

Bias from Observers

  • Selection Bias – Selection of Specific Patients Results in a Nonrandom Sample Group Not Representative of the Population
    • Avoid by Randomizing
  • Detection Bias – Observations in Treatment Group are More Diligently Pursued than in the Control Group
    • Avoid by Blinding
  • Interviewer/Observer Bias – An Observer Makes Subjective Decisions About Outcomes
    • Avoid by Blinding Observer or Making Outcome Measures Objective
  • Confirmation Bias – When Researchers Look for Patterns that Confirm Their Previously Held Beliefs During Interpretation of Data

Bias from Patients

  • Observation Bias/Hawthorne Effect – The Act of Closer Observation on Treatment Group Causes Improved Outcomes to the Control Group
  • Recall Bias – Patients Asked to Answer Questions About the Past and Rely on Memories Which May Not Be Accurate
  • Response Bias – When Patients Enroll Themselves in a Trial Causing a Non-Representative Sample
    • Avoid by Random Sampling the Population

Bias from Testing

  • Lead-Time Bias – When Comparing Two Diagnostic Tests, One Test Allows Earlier Diagnosis but No Change in Outcome Giving a Falsely Perceived Prolonged Survival
  • Length-Time Bias – When a Screening Test Allows Detection of More Indolent Disease that Would Naturally Have a Longer “Pre-Symptomatic” Time Than More Aggressive Disease that Presents Clinically Giving a Falsely Perceived Prolonged Survival

Bias from Non-Reporting

  • Publication Bias – Positive Data is More Likely to Be Published than Negative Data Giving Falsely Positive Evaluations by Meta-Analysis
  • Time-Lag Bias – Research Findings Published More Rapidly or More Delayed Depending on the Nature & Direction of the Results
  • Language Bias – Publication in a Particular Language Depending on the Nature & Direction of the Results
  • Citation Bias – Certain Data is More Likely to be Cited by Other Papers Depending on the Nature & Direction of the Results
  • Location Bias – Publication of Data in Journals with Different Ease of Access & Levels of Indexing in Standard Databases Depending on the Nature & Direction of the Results