RANDOMISED TRIALS-MORE PLEASE

RANDOMISED TRIALS

According to Goldacre – should be used in education to test policy

Comment

According to Ben Goldacre, a doctor, academic, researcher and journalist, who was very active at the last  Sunday Times/Wellington Education Festival,politicians are ignorant about trials, and they’re weird about evidence.

Goldacre is an engaging charismatic speaker. He is also irreverent and provocative. So good value. He has  spent a lot of time arguing that government should be more evidence based, and that wherever possible, we should do randomised trials to find out which policy intervention works best. We often have no idea whether the things we do in government actually work or not, and achieve their stated goals. It doesn’t need to be this way he says. Quite.

Randomised trials are our best way to find out if something works: by randomly assigning participants to one intervention or another, and measuring the outcome we’re interested in, we exclude all alternative explanations for any difference between the two groups. If you don’t know which of two reasonable interventions is best, and you want to find out, a trial will tell you. Goldacre co-wrote a Cabinet Office paper that   explains why randomised trials of policy are so powerful. He says ‘we explain exactly how to do them; and we explain how to identify a meaningful policy question that can be explored cheaply in a good quality trial.  In international development work, resources are tight, and people know that good intentions aren’t enough: in fact, good intentions can sometimes do harm. We need to know what works. In two new books published last year – “More Than Good Intentions” and “Poor Economics” – four academics describe amazing work testing interventions around the world with proper randomised trials. This is something we’ve bizarrely failed to do at home.’  Is business training useful he asks? There’s a randomised trial on it in Peru. What about business mentors? In Mexico, they ran a randomised trial. Now think about all the different initiatives in the UK to support small businesses, or to help people find work. Do they work? No idea.  Microfinance schemes help small producers buy in bulk to make larger profits, and they change lives. But are group-liability loans better, because people default less, so the project is more sustainable? Or do anxieties about shared responsibility restrict recruitment? Some academics ran a trial.

Do free uniforms improve school attendance, especially in pupils who don’t own one at all?   Someone ran a trial. Contingent payments improve attendance: but what’s the best time to pay, and how?  They also ran a trial which gave a group of adults’ money to attend FE courses, and another group no money to attend the courses. Guess what. The group which had no incentive to attend beat the group that did have a financial incentive to attend.  This is just the kind of counterintuitive result that randomised tests can show up (the hard bit is explaining why) What about streaming in Kenyan schools, with high and low ability classes? Do all kids do better? Someone ran a trial.  The Cabinet Office paper shows, says Goldacre, that ‘policy people need to have a little humility, and accept that they don’t necessarily know if their great new idea really will achieve its stated objectives. We do these using examples of policies which should have been great in principle, but turned out to be actively harmful when they were finally tested. Finally, we address – and demolish – the spurious objections that people often raise against doing trials of policy (like: “surely it’s unfair to withhold a new intervention from half the people in your trial?”). Trials are used widely in medicine, in business, in international development, and even in web design. The barriers to using them in UK policy are more cultural than practical, and this document will hopefully be a small part of a bigger battle to get better evidence into government.’

At Wellington he offered one compelling example of how a randomised trial would have saved thousands of lives.

Steroids reduce swelling-so it followed,  in brain injuries, when the brain swells you  were given steroids,  over a 30 year period.   that was the default  position. But the reality is that steroids actually have no effect on patients with brain injuries and  probably has  a negative effect  possibly killing   many patients- – something that could have been found out years before and saved thousands of lives. A study , coordinated at the London School of Hygiene and Tropical Medicine, involved 10,008 adults with severe head injuries who were randomly allocated either a steroid drip or a fake drip for 48 hours after being admitted to the emergency room.  Within two weeks, 21 percent of the patients given steroids had died, compared with 18 percent of those given the fake drug. The results were the same regardless of how quickly the treatment was administered and regardless of the type or severity of the head injury. Goldacre probably has a point.

 See Cabinet Office paper below-

Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials

www.cabinetoffice.gov.uk/resource-library/test-learn-adapt-developing-public-policy-randomised-controlled-trials

About these ads