I had an interesting study week on the diploma course I'm doing. One of the things we did during the week was to pick a couple of papers to pieces and look at how they discussed the results.
There are several ways the results of a study can be discussed. Obviously you have the raw numbers, but they are difficult to compare between studies. The three main ways of reporting the benefits of a treatment are relative risk reduction (RRR), absolute risk reduction (ARR) and numbers needed to treat (NNT). Relative risk reductions sound impressive, and are loved by the press and drug companies but they don't actually tell you very much. So you see headlines such as '
aspirin cuts risk of dying by 25%' or 'schoolkids use of drugs doubles'. RRR tells you, as the name suggests, the relative difference between the experiemental arm and the control arm, but, to know if this matters or not you need to know the background incidence of the event. Advertising departments of drug companies and the media often either hide away the background incidence or don't tell you it at all.
Absolute risk reductions are often small, just a few percent, and don't sound particularly impressive, but they tell you infinitely more than RRRs. Again, as the name suggests the ARR tells you the absolute difference between the control and treatment arms. However it can be difficult to translate how an ARR benefits patients. This leads on to the number needed to treat. Now the NNT does what it says on the tin. It tells you how many patients need to be treated for one to benefit and NNTs are very useful in comparing treatments.
Now for some examples. What if I told you there was an easy way to double your chances of winning the lottery? It's easy, just buy two tickets instead of one. But because your chances (or risk, or probability -they all mean the same thing) are vanishingly small to start with, doubling your chances still means they are vanishingly small: they are now just 1 in 15 million compared to the 1 in 30 million before. So the ARR in this case would be absolutely tiny. But now think about a school raffle where one ticket has a 1 in 100 chance of winning. Again there is an easy way to double your chances of winning by buying two tickets instead of one, but in this case your chances have increased significantly from 1 in 100 to 2 in 100, an absolute increase of 1%. Now the NNT is easy to calculate: it's 100/ARR. So if the ARR was 2%, you would need to treat 50 patients for one to benefit.
Sounds straightforward enough? (Or maybe not, depending how well I've explained it.) Well consider the following scenario: your hospital is considering four different cardiac rehabilitation programmes. You must choose which one you will implement. Reliable evidence shows that, during a three year period:
- Programme A reduced the rate of deaths by 20%
- Programme B produced an absolute reduction in deaths of 3%
- Programme C increased patient survival from 84% to 87%
- Programme D meant that 31 people had to enter it to prevent one death.
Which one would you choose?
This was actually the basis of a study performed by
Fahey et al The figures above actually relate to the same data, just presented in different ways. Unsurprsingly they found that the programme that reported the RRR was the one chosen most often for funding.
The moral of the story is this: if you want to sell a drug report the RRR. If you want to know whether the drug is actually worth the money then look at the ARR and NNT (or speak to your prescribing advisor or community pharmacist if they are good)
Ben Goldacre probably explains this kind of thing far better than I do, and there is plenty of information on his website and in his
Guardian columns.