Meta-analysis of treatment studies
Meta-analyses are systematic attempts to answer a focused clinical question based on the best available evidence.
Why do we do meta-analyses? 1). Comparative effectiveness. 2). Effectiveness in subgroups. 3). Understanding the literature/research base. 4). Identifying gaps in the literature. 5). Guiding practice decisions. 6). Guiding policy decisions
Key elements in a good systematic review or meta-analysis, PRISMA:
-
A focused clinical question: PICO structure
-
An exhaustive search for studies: PubMed, EMBASE, Cochrane Controlled Trials Register, clinicaltrials has some manufacturer sponsored studies
-
Clear inclusion and exclusion criteria
-
Evaluation of study quality: Cochrane Risk of Bias Tool
-
Careful abstraction of data
-
Evaluation for homogeneity: Important to assess homogeneity to determine whether you can trust the results of meta-analysis. Are studies similar enough to combine? Homogeneity: General interpretation: studies found generally similar results; Conceptually: studies were sampled from a single hypothetical population, and any variance is due to sampling error. Thus, all variance is within study rather than between study; Implications: OK to combine results using fixed or random effects model. Heterogeneity: General interpretation: studies found different results; Conceptually: studies were sampled from different populations; The variance seen may be due to sampling error but is also due in part to difference between those populations (between study variation); Implications: Either report results qualitatively, stratify results, or use random effects model (if not too heterogeneous). Sources of heterogeneity: 1. Within study variance. Random error, sampling error. We are sampling from some imaginary large population, and different studies vary due to random error. 2. Between study variance. There are systematic differences between studies due to differences in populations included (adults vs kids), design (blinded or not), intervention, dose, attrition, etc. Cochran’s Q and the $I^2$ statistic. Interpreting the $I^2$, 0 – 40%: might not be important, 30% - 60%: may represent moderate heterogeneity, 50% - 90%: may represent substantial heterogeneity, 75% - 100%: considerable heterogeneity.
-
Clear presentation of results based on best evidence: Fixed effects model, only take into account within study variation. Random effects model account for WITHIN study and BETWEEN study variability. Publication bias: File drawer effect, Funnel plot. Presenting the data: 1). Figure showing how studies were included/excluded. 2). Table 1 with study design data. 3). Table 2 with study quality data (Stoplight table). 4). Table 3 with outcome data (benefits, harms). 5). Relevant forest plots. 6). Funnel plot.
Emerging Methodologies