The use of intention-to-treat analysis has come up at a couple of recent journal clubs, bandied about as if to suggest that the outcomes and conclusions were unassailable. However, ITT analysis may not be quite the bullet proof cladding as it iseems to be made out. So I’ve trotted off to prove to myself that I really did understand the concept and it turns out I almost did. So, courtesy of the Medical Journal of Australia, the Cochrane Collaboration open learning materail resource and Wikipedia (the occasional pinch of salt may be required), here’s a quick summary of intention to treat analysis:
First, ITT analysis is a statistical tool; not a clinical one. It makes the numbers look good and attempts to minimise analytical bias, but that is not the same as telling you that the intervention works perfectly.
Intention to treat analysis is a means of interrogating the data without introducing bias from patients that drop out from the trial or cross over to another arm of the study. It analyses the data from all patients in a given study arm as if they had received the intervention and completed the trial, thus attempting to compensate for patients who might have dropped out of that arm due to death, exclusion due to protocol breach or even recovery. This is different to per-protocol analysis, which analyses only those patients who received the intervention and completed the trial without violating the treatment protocol. Per-protocol analysis can make an intervention look better than it really is because it disregards those who do not complete the trial, including those who may have died or become too unwell to continue, possibly due to the intervention itself.
So ITT analysis may give a truer insight into an intervention; but what if a large proportion of the participants never actually receive the intervention? For instance in one of our recent journal club reviews of early versus late tracheostomy to prevent VAP, nearly a quarter of one of the groups never received the assigned intervention. Does ITT make up for this deficit?
Not really. It is acknowledged that ITT analysis “gives information on the potential outcome of a treatment policy, rather than on the potential effects of a specific treatment”*. So the conclusions drawn need to be interpreted with care. If a patient recovers from a streptococcal pneumonia but never actually had the penicillin administered, is it reasonable to conclude that he got better due to the intention to give it?
There is another way around this, other than per-protocol analysis. You could perform an efficacy-subset analysis, or treatment-received analysis, which analyses only those patients who completed the study and actually got the intervention against those who did not. However, efficacy-subset analysis disregards which group patients were initially randomised to (e.g. patients who were initially assigned to placebo, but subsequently crossed over to the treatment arm for whatever reason) and so has the likelihood of introducing bias (which undoes all of the effort of proper randomisation, which ITT is supposed to preserve) and producing an exagerated false positive rate with an effect that increases with trial size (not good for those well-powered, multi-centre, RCTs).
Finally, to be able to validly apply ITT analysis, a complete data set is required. In other words, if there is any missing data (e.g. patients lost to follow up or subsequently withdraw consent and therefore outcome is unknown) ITT analysis loses its mojo proportionate to the amount of missing data. ITT compensates well for patients who do not complete the study with a known outcome such as death or survival, but works less well if the outcome of the drop-out is not known; i.e. they may have survived or they may have died. Authors will often conduct analyses for each possible outcome to get a range; e.g. all drop-outs assumed to have died or survived or 50:50 outcome.
So, while a trial’s merits are often trumpeted during a review, particularly where it is thrown in as evidence to support a point made in a topic review, and one of those merits is often that there was Intention-To-Treat analysis, be careful to look at the data to check whether the study is compensating for a handful of dropouts, or simply using ITT to gloss over the fact that a large propportion of the sample population never even had the intervention or that the data set is in fact incomplete. This is not to say that ITT analysis in an evil construct used be scheming statisics-savy researchers, rather, it is an important component of research reporting, that must be interpreted with due caution.
For a more peer-reviewed source of information on ITT analysis, check out this eMJA article (MJA 2003; 179 (8): 438-440): Inclusion of patients in clinical trial analysis: the intention-to-treat principle
The Cocrane Collaboration open learning material on ITT analysis is here: http://www.cochrane-net.org/openlearning/HTML/mod14-4.htm