![]() |
So, you’ve come up against an intervention for which you’re a bit shaky on the evidence supporting it. What do you do? Why, jump on to PubMed or Medline of course and do a search.
You’re in luck, there’s an RCT. Ooh, and there’s another one. And lookie here, some thoughtful soul has performed a metanalysis. You can just read this and go with the recommendations. Sorted! But then an annoying little voice in your head says “It might not be that simple. What about heterogeneity?” |
When you want to see if an intervention improves an outcome, you do an RCT. If you can’t get the numbers for adequate power, or you just want to collate all the existing evidence, you collect all the trials of that intervention and perform a metanalysis. However, for the result of the metanalysis to lead to meaningful conclusions, the old adage of not comparing apples with oranges comes in to play.
[su_tabs]
[su_tab title=”The questions”]
What is trial heterogeneity?
How can you identify it?
What can you do about it?
[/su_tab]
[su_tab title=”The explanation”]
If the trials being analysed examine the same intervention, for similar populations and using similar methods, then it is a fairly homogenous collection of trials whose collective analysis leads to a bigger body of meaningful information.
This rarely happens.
The opposite of homogeneity is heterogeneity (Doesn’t polysyllabism just make you feel better about the world?), which is a variation in data that occurs more than would be expected just by chance. The wider the heterogeneity (scatter, in common parlance), the harder it is to be certain about the outcomes.
A metanalysis may be plagued by statistical heterogeneity or clinical heterogeneity.
Statistical heterogeneity occurs when the statistical elements in each trial vary between the trials. For instance, in a pooled group of trials, some might be RCTs, some might be case-controlled trials and some may be case series. Or, some of the trials might be underpowered, or be conducted as per-protocol studies rather than intention to treat.
Clinical heterogeneity occurs when clinical elements differ widely between trials. The metanalysis in question might include only multi-centre, randomised, double-blind, placebo-controlled, intention-to-treat trials; but, if the trials differ in their definition of the intervention, or its application, or clinical thresholds for initiation, or the characteristics of the trial populations differ across all the studies being analysed, then again it will potentially be difficult to draw useful conclusions.
Trial heterogeneity can be spotted by applying a statistical test called the I2 test. The higher the value (usually a % value) the greater the heterogeneity. If there is no or negligible heterogeneity, trial data is analysed by fixed effects modelling (i.e. assumes any variation seen is due only to chance). If there is a moderate degree of heterogeneity, then a random effects model is applied to the data analysis. However, if the degree of heterogeneity is large, then a summative analysis may be inappropriate and the group of trials in question may have to be analysed individually to draw a meaningful conclusion.
For a good example of the influence of heterogeneity on metanalysis, check out how the guys at TheNNT use its flaws to pick apart a metanalysis of thrombolysis for acute stroke that included the recent IST-3 paper here.
The metanalysis may be considered the pinnacle of Evidence Base Practice, but be aware, it has its pitfalls.
[/su_tab]
[su_tab title=”The references”]
What is a meta-analysis? – http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/meta-an.pdf
TheNNT: Thrombolysis for stroke
[/su_tab]
[/su_tabs]