Observational and Quasi-Experimental Study Designs

From resources available at the University of Michgian, the Integrated “5S” Levels of Organization of Evidence Pyramid depicts the relationship between the Evidence Hierarchy (the small, inset pyramid) and the “5S” model. The Integrated Pyramid also includes foundational resources that do not have transparent evidence-based methodologies.

{width=100%}

Why not just do randomized controlled trials of everything?: May not be ethical; May not be practical; May cost too much; May be unrealistic to get that many patients; May be difficult to convince patients to participate if well established (but untested) therapy; You often need pilot data, acquired cheaply.

Overview of study designs: From the Centre for Evidence-Based Medicine develops, promotes and disseminates better evidence for healthcare [(CEBM)](The Centre for Evidence-Based Medicine develops, promotes and disseminates better evidence for healthcare.), and Jeremy Howick’s paper:

{width=100%}

Limitations of observational studies: This paper (False alarms and pseudo-epidemics: the limitations of observational epidemiology) from David A. Grimes may not right but interesting.

{width=100%}

Threats to validity in experimental studies:

1. History: We sometimes use “historical” control groups, comparing what happened before and after an intervention occurred. But other things change over time in addition to your intervention.

2. Maturation: People change over time, they grow, learn, mature, get smarter, get demented, etc.

3. Natural history of disease ($\approx$ Regression to the mean): Most of the time, when do you see the doctor? When you are sickest. What happens when you are sickest? You tend to get better.

4. Test-retest: Especially relevant when doing something that can improve with practice or repetition. Example: A patient solves a puzzle, then is given repeated doses of a drug, and is asked to solve the puzzle after each dose.

5. Measurement variation: Example: Doctors at different hospitals have difficulty agreeing on which patients have had a stroke.

6. Hawthorne effect: In clinical trials, refers to an improvement that occurs just because the person is being observed and paid attention to.

All of these threats introduce potential bias and reduce our ability to get at the truth. Key ways to address them include randomization, masking, allocation concealment. But you can’t always do an RCT.

Quasi-experimental study designs: These are experimental studies that evaluate an intervention but do not use randomization.

Ecologic Studies: In ecologic studies, we do not to measure exposure and outcome for individuals.

Post-test only design. Pre-test/post-test design. Non-equivalent control groups/post-test only design. Non-equivalent control groups pre-test/post-test design. Randomized controlled trial. Randomized placebo controlled trial. Time series. Time series with non-equivalent control group.