Evidence-based medicine
Clinical Unspeak, China, ‘language rapists’, & ‘Melanie’ on Obama
March 3, 2008 70 comments
While Oliver Kamm gaily holds a lit match to whatever shreds remained of his credibility (see update here), a quick round-up of other Unspeak news follows.
• “Evidence-based medicine” is a term for what you might fondly have hoped doctors would always have done: adjust their practice according to the most rigorous possible statistical analyses of what works and what doesn’t. Agree as we do that it is a good thing, we must also acknowledge that its name is a cute bit of Unspeak, assuming as already beyond argument its own definition of “evidence”, and relegating its opponents to the status of witch-doctors. This, after all, was nicely admitted by an evangelist for the paradigm’s clinical power, Druin Birch, during a discussion in the TLS letters pages:
Evidence-based medicine is such an unfairly named movement that there can be no sensible argument against it. J. K. Aronson is right (Letters, February 15) to see it as a front for those who believe some types of evidence are worth more than others, and correct that I accept the reality of this hierarchy.
• In a public debate, someone once asked me why my book did not devote itself to denouncing Chinese Unspeak, rather than carping mostly about the UK and US governments. I replied, reasonably enough I thought, that political English in our time was still worthy of some interrogation, and unfortunately I did not understand Chinese (beyond a very specialized martial-arts vocabulary). Happily, other people do. Chris O’Brien wrote a fascinating article for Forbes on how he spent two years as a “language polisher” for the official news agency:
“The three closenesses” and “The four steadfasts” are just a couple of the catchphrases championed by President Hu Jintao that are guaranteed to send English spell-check programs whirring to life. “Eight honors and eight disgraces” is another party favorite. The government views such buzzwords as essential tools in maintaining its influence over the morality of the people.
From O’Brien I also learned of the China Media Project, which offers among other things a useful list of definitions for current phrases of Chinese government Unspeak. The stipulation that the media should adhere to the principle of “correct guidance of public opinion”, for example, means that they should report whatever the Party tells them. Failure to do so would be a violation of “propaganda discipline”. And “civilized creation [and management] of the Web” means, um, censoring it.
• For some comic relief, readers may peruse an extraordinary “essay” entitled “Feminism and the English Language“, by a man called David Gelernter, for the “American Enterprise Institute”. The delicious irony of this blubbering rant, crammed as it is with violent misogyny and lies, is that Gelernter professes to be trying to defend “our ability to write and read good, clear English”, associating himself through his title with George Orwell, while at the same time perpetrating sentences like the following:
So feminist authorities went back to the drawing board. Unsatisfied with having rammed their 80-ton 16-wheeler into the nimble sports-car of English style, they proceeded to shoot the legs out from under grammar–which collapsed in a heap after agreement between subject and pronoun was declared to be optional.
For good measure, he calls his imaginary army of feminist language-police “language rapists”. Ho ho ho.
• Lastly, the operator of the sockpuppet known as “Melanie Phillips” has been really pushing the envelope recently, reporting that:
Barack Obama failed unequivocally to repudiate the support expressed for his candidacy by the black power, Islamist, racist antisemite Louis Farrakhan.
Shall we refresh our memories on how Obama failed unequivocally to repudiate Farrakhan’s support? Uh, he said he “would reject and denounce it”. (Update: See Alex Higgins’s magnificent satire in comments on that whole line of questioning.) It is only natural that the fervid, spasming bolus that is the microbrain of “Melanie” finds this somehow equivocal, because “she” believes that Barack Obama is the Trojan horse for a shadowy group of anti-Semitic Islamists who want to take over America and nuke Israel. Whatever he actually says cannot in principle dislodge this paranoiac pellet from her bolus. But then, if “Melanie”‘s writing were in any way evidence-based, the satire wouldn’t be so entertaining, would it, readers?
Home Secretary Jacqui Smith is trying to build a ‘consensus’ on 42 day detention without charge – unfortunately pretty much everyone other than Government Ministers is against it.
To Jacqui Smith, a consensus is when everyone agrees with her.
I don’t get your beef with the term “evidence-based medicine”. Do you think it should be called, “good evidence-based medicine”, “scientific evidence-based medicine” or even “remotely plausible-based medicine” instead to differentiate itself from alternative or CAM medicine?
Medicine should be evidence-based, so the ‘evidence-based’ bit is redundant (and Unspeak).
“Evidence-based medicine” is not so called so as to distinguish itself from “alternative medicine” (or rather anti-medicine, the subject of a forthcoming post if I can find the damn book again), but in contradistinction to other conceptions of medicine (ie actual medicine) which have, or so it is argued, not got straight in their minds what sort of thing should count as proper evidence as to the efficacy or otherwise of certain practices. (Aronson, the guy to whose letter Druin Birch is responding is himself seemingly a clinician or clinical researcher.)
Hi Steven, I’ve been following the recent US presidential candidates’ debate and was uncomfortable with some of the questions put to Obama.
I hope you don’t mind if I take the liberty of posting a portion of the transcript from the recent debate in Texas, which no doubt Melanie Phillips is alluding to:
_________________________________________________
Moderator: Senator Obama, I’d like to ask you now about a difficult subject for you. Earlier on this week, you received an endorsement from the controversial figure Louis Farrakhan, who said, “I’d rather that half-devil was in the White House than that white bitch.”
Now, Mr Farrakhan is on record as making a number of anti-Semitic statements. Can you reassure American Jews that you reject those statements?
Senator Obama: Well, I have denounced his disgraceful remarks on many occasions, and I have a great deal of support…
Moderator: Ah-ha, you denounce Farrakhan, but do you reject his endorsement?
Obama: Well I can’t stop anyone hating me less than other candidates…
Moderator: But do you reject him, Senator?
Obama: Yes, I reject him.
Moderator: OK, so you reject him, but is that repudiation? Do you repudiate him?
Obama: Doesn’t that count as a repudiation?
Moderator: You repudiate, but do you eschew him? Perhaps I should remind the audience that in 1963, Louis Farrakhan said he wished “to stick forks in the eyes of rabbis” and that he once tried to distract people on Yom Kippur by coughing loudly for the whole day?
Obama: Yes, I am aware of what he said, as are we all, and I have on many occasions…
Moderator: So do you eschew him?
Moderator 2: If you saw Farrakhan, in front of you, right now, would you spit in his face? Would you spit right in his face, Senator?
Moderator: Would you spit on his grave even after he is dead?
Obama: I don’t see that that’s necessary.
Moderator: Can you assure American Jews that you don’t personally plan to have Israel demolished, sold and converted into a giant mosque dedicated to the perpetrators of 9/11?
Obama: As I was saying before, I have a lot of support from the Jewish community for my campaign. I regard Israel’s security as central to our Middle Eastern policy and…
Moderator: We have to return to this issue, I’m afraid, Senator. Isn’t it true your pastor’s daughter’s cousin’s boyfriend’s friend published an article in a student magazine praising Louis Farrakhan for his excellent bow-tie? And in fact, awarding the Nation of Islam his magazine’s annual prize for Achievements in Bow-Ties? Do you denounce and reject that award?
Obama: I have always condemned anti-Semitism in quite clear language and I have said that sometimes I disagree with my pastor’s daughter’s cousin’s boyfriend’s friend’s magazine on a number of issues.
Moderator 2: Have you repudiated the comments of Pastor Scaryblackdude Nutjob, who greeted Farrakhan in Georgia last week and praised him with words I imagine went like, “Yo, my main man, Farrakhan, I dig your Jew-hating jive, brother!”
Obama: Are you sure that quote’s even accurate?
Moderator 2: You know what I mean. I can’t do the lingo.
Obama: Well I have no actual connection to this particular individual but obviously I condemn, repudiate, reject, denounce and eschew…
Moderator: You know, this may sound kind of funny, but… The other week this er… African-American guy was trying to break into my car…
Moderator 2: Do you reject and repudiate this attempt by one of your own to steal my colleague’s car?
Moderator: More to the point, can you assure me and other moderate Americans that it was not in fact you, who was trying to break into my car? I mean, you look pretty similar… I couldn’t say for sure you were the guy…
Obama: I was in Wisconsin that week. It was on TV.
Moderator: Good point. Can I ask you about the powder issue? I mean you used to… you know, right?
Obama: Well, as I explained in my memoir, I grew up as a troubled teenager, unsure of my identity, without a father, like many of my generation, and occasionally…
Moderator: No, what I mean is… can you get me some?
Obama: I beg your pardon?
Moderator 2: I mean we got some friends over this weekend, a little new to the neighbourhood, and we were looking to party, and we thought you might be able to get us a little something to put us in the mood?
Obama: I can’t believe you asking me this.
Moderator: Senator, let me rephrase that – have you got some blow? You know we’re good for it.
Moderator 2: I mean, if you don’t have any on you right now, maybe you know the right guy to go to around here?
Police Officer: Hey, hey, hold it right there!
(Moderators hide)
Police Officer: Hey you, can you come to the side a moment, sir?
Obama: What? Are you crazy? This is live on television – it’s the candidates’ debate! I wasn’t about to… I’m not a… Is this a sting?
Police Officer: Sir, can you with me to the side, sir?
Police Officer 2: You heard him! Just what in the hell do you think you’re doing?
Obama: I’m participating in the CNN Democratic presidential candidates’ debate!
Police Officer 2: Yeah, and what would you be doing in that?
Obama: I’m running for President! I’m the junior Senator for Illinois!
Police Officer 2: Smart guy, huh? Book ‘em, Dan. Maybe we’ll take this boy down the station and straighten him out.
Police Officer: Didn’t we pull this guy over earlier today?
(Senator Obama arrested, taken off set. Senator Clinton waits for her question)
Moderator: Did you catch what Obama told that officer?
Moderator 2: I think he said, “What the fuck you at, fool?” Or at least that’s what I’m telling the Washington Post. I don’t always know what they’re saying, you know? Good questions, by the way – still the best in the business!
Moderator: Thanks!
[faints clean away in admiration]
It’s contributions like that one that prevent my confessedly haphazard and slothful posting schedule from petering out completely. Thank you!
*claps*
“evidence-based medicine” ought to call itself something like “econometrics-based medicine”. Basically it’s about de-emphasising case studies and similar in favour of large-sample statistical studies (and in particular, metastudies). It is a bit of an academic land grab by a subset of medical statisticians. It’s healthy stuff in small doses, but in its current expansionary state, you get two issues:
1) it’s not so good for rarer conditions and treatments, and much better for drugs than surgical or preventative medicine.
2) there is an ever-present danger that practically unimportant effects get picked up as “statistically significant” as the sample size increases.
It seems to be potentially pretty excellent for preventative/safety measures too – cf this EBM guy Don Berwick who appears to have saved lot of lives in the US simply by insisting to reluctant doctors that they wash their hands more than they were doing a decade ago.
Round of applause for Alex – fantastic!
As it has been explained to me, there are also some pretty big ethical issues with the insistence of EBM on RCT. So, for example, suppose we’ve been treating child cancer patients with treatment X for decades and it seems – in the experience of clinicians – to be reasonably effective. The EBM people are going to say that there’s no actual evidence, what we need are RCTs. But doing an RCT means giving a largish bunch of kids a placebo (and denying them treatement X).
Yes, there’s certainly a tension there, though it might be a bit of a straw man to imply that EBM folk brusquely demand RCTs in such a situation without paying attention to ethical concerns.
But RCTs are not necessarily done against placebo. At least as I understood from a book I reviewed a while ago (The Body Hunters), RCTs on treatments for life-threatening conditions in the west already test the candidate not against placebo but against the best current treatment, for precisely the ethical reasons you outline. (Of course there are then ethical concerns that the experimental group might be getting treatment that is inferior to that offered the control group.)
David Gelernter’s “blubbery rant”: what a wonderfully apt below-the-belt description, judging from what we can see of him. Does this qualify as gut-speak?
Ouch, that wasn’t even intentional.
There’s an extremely good more-in-sorrow-than-in-anger demolition of Gelernter’s screed by Geoffrey Pullum at Language Log.
Thanks for the tip. I’ve just read Pullum’s post. Awfully good and indeed sad. Were you also struck by Gelernter being a computer scientist? Surely, those “guys” have done more raping and pillaging of our language than any birds-of-a-feather feminists. :-)
The “evidence-based medicine” debate has festered in medical circles for years, and one gets tired of BMJ jokes about advocates of EBM doing studies on parachute efficacy. And as has been noticed, this is because there really are so many straw men peopling this debate. Please find me a single advocate of EBM who says anything close to: “Take the kids off the lifesaving cancer drugs until we can prove it works in a placebo-controlled trial in which we will give half of the dying children sugar pills.”
“Evidence based medicine” practitioners or advocates are like nerds with their broken spectacles and pocket protectors – everybody loves to beat up these know-it-all goody goodies, while kowtowing to the badass renegade who does it his own way. “I don’t follow no goddam rules.”
Steve, you question the ethics of putting people into a branch of the trial that may not include so-called gold standard treatment: the ethics of EBM, and of clinical research in general, are immensely complex, and will invariably include some assessment of how you will be giving some people worse treatment – unless you are conducting a noninferiority study, you have hypothesised that one arm will do better than another. Blinding is one way of ensuring that there is some measure of fairness in this ethical quandry (this is not the only purpose for blinding, of course).
And there are some truly striking examples of how practitioners of EBM have challenged such common sensical, obviously helpful practices as hormone replacement therapy for post-menopausal women because it will save their lives . . . Or not, as the case may be. So what is “evidence” unspeaking here? “Common sense”? Isn’t it doing what it purports to do? And are there not lengthy debates about what constitutes “evidence”, and arguments about the hierarchies of evidence that go from the paragon of double-blind, adequately powered RCTs to the nadir of case studies published in the ignominious letters section.
So, “EBM” is unspeak because it says that other practitioners are not using “evidence”? That may be cutting it a bit fine. Rather, it is about subjecting your knowledge to something that can be visible to everybody in a public space. There is something out there called “evidence” that you, or I, or my cats, can access: there is no magic, there is no mystery. It isn’t entirely packaged in the gauze of personal experience. Or rather, there is mystery, but it is available to all, not to the select few; and the magic is all the more magical for occurring out in the open.
The demand that there is “evidence” is not necessarily a bad thing. This means that my claims need to be subject to some approximation of good science (accepting, as I do, that the paradigms of science for molecular physics cannot map perfectly onto the messier world of clinical medical research). So, if I believe that vitamins are good for people who smoke, because vitamins are full of antioxidants that will bind to the dangerous free radicals in tobacco smoke and because vitamins in general are healthy things, then I should run studies to produce some sort of evidence to justify this claim, because what if the opposite is true? What if vitamins actually appear to increase rates of cancer in smokers? Well, that’s preposterous, and obviously not true, because vitamins are good for you.
Except it looks like both vitamin A and E might be implicated in increasing rates of lung cancer in smokers.
By the way, dsquared, I think you’re a quarter right. EBM does lead to problems with “statistical significance”, but then surely you should concede that EBM advocates have worked extraordinarily hard to improve statistical and epidemiological literacy amongst clinicians (for example, numerous medical journals run or have run series explaining how to make sense of EBM and how to “read the papers”). I think you are not right when it comes to EBM and rare diseases and treatments, simply because the collection of data requires the aggregation of affected populations, and this has brought together people with rare conditions, their families, interested investigators, etc..
Thank you, sw, that is fascinating.
A couple of random observations I am pulling out of a chapter on EBM from another book I reviewed a while ago (Super Crunchers):
EBM dudes don’t think that statistics are everything: one of the movement’s founders, Gordon Guyatt, has said that statistical evidence “is never enough”.
Re: sw’s “renegade who does it his way”, Don Berwick (the hand-washing evangelist, among other things: see here) is quoted as saying: “The more I have studied it, the more I believe that less discretion for doctors would improve patient safety. Doctors will hate me for saying that.” No doubt! The book’s author then makes a provocative comparison between doctors and airline pilots.
Is “evidence-based medicine” nonetheless Unspeak? Sure, for the reasons Birch cited. You might think it’s a relatively virtuous kind of Unspeak, but it’s surely a polemically argumentative label nonetheless.
Not at all; I love what the world of computers has given to our language. But then I am maybe more l33t than many of my fellow literary hax0rz.
^Indeed. When are you going to write about games again? All this wordy crap is really hard to follow. :P
Trigger Happy (along with Redeye) was like the highlight of my month.
Hmmm, no, I don’t agree that “evidence-based medicine” has necessarily improved statistical literacy among doctors. It’s brought in a more systematic learning of a toolkit that has a number of quite serious flaws, statistically speaking (it seems to me based on a cursory examination that medical statistics is getting a really bad case of the econometrician’s “curse of the 5% significance level”, and there is a worrying assumption that big-ass metastudies will iron out any structural modelling problems when they won’t).
Also, it *is* biased toward drug solutions and against long-term or structural public health solutions, for the same “manage what you measure” reasons that management consultancy has an intrinsic bias toward short-term cost-cutting solutions.
But my real problem with the evidence-based medicine community and their appropriation of the term “evidence” to mean their kind of statistical study is that they’re trying to treat the average patient when no such entity exists. It makes a bit of sense in social and economic contexts to reify a statistical aggregate as a “representative agent”, but not to try and treat his gout. There’s an implicit assumption of homogeneity among human beings that is very unsupported by the evidence indeed; conversations with young doctors who have just finished their EBM courses suggests very strongly to me that the people who are pushing this movement in universities really don’t understand how serious a missing variable is as a departure from the underlying assumptions of a regression model.
[The demand that there is “evidence” is not necessarily a bad thing. This means that my claims need to be subject to some approximation of good science (accepting, as I do, that the paradigms of science for molecular physics cannot map perfectly onto the messier world of clinical medical research). So, if I believe that vitamins are good for people who smoke, because vitamins are full of antioxidants that will bind to the dangerous free radicals in tobacco smoke and because vitamins in general are healthy things, then I should run studies to produce some sort of evidence to justify this claim, because what if the opposite is true? What if vitamins actually appear to increase rates of cancer in smokers? Well, that’s preposterous, and obviously not true, because vitamins are good for you.
Except it looks like both vitamin A and E might be implicated in increasing rates of lung cancer in smokers. ]
whoa hang on; this kind of thing is *exactly* what we should expect more of in a world of evidence-based medicine, because nobody is going to do this kind of interaction study for every single pair of therapies, and the assumption of independence is woven into the weft of the statistical models used. I also question your example of HRT above; it is only very recently that there’s been any questioning of the value of this in the literature and the early evidence-based studies were very positive indeed.
But some of the most visible, empirically robust successes of EBM are precisely the simple structural rules of care put in place by the campaigns of Berwick and others. These kinds of things:
If there is a “bias” here, it isn’t one towards drug solutions, so far as I can see.
Well, there’s a balance to be struck, isn’t there? If you refuse ever to assume that persons with similar symptoms will respond similarly to similar treatment, and just say “Hey! Everybody’s unique!”, then a lot of people will probably die while you are trying to map their wonderfully unique biochemistry and build an entire predictive molecular model of each patient.
Aw, shucks. Thanks! I do think that, for a few years there, Edge was actually the best magazine in the world — and not just because of my contributions. ;)
As for returning to the subject in future, I do not rule it out. But who wants to print it? The general prejudice against the form has not dissipated as much as I once fondly hoped it would.
hmmmm … I would question whether all of those success stories can actually be claimed 100% for “Evidence-Based Medicine”. There wasn’t ever any movement of doctors who thought that it was OK to make decisions about washing your hands on a case-by-case basis based on individual judgement. I think we need to draw a distinction between normal process improvements which don’t require any particular philosophical commitment to a view about how medical science progresses, versus things that specifically arise out of science as it’s practised in journals (note as well that there’s a big equivocation here between “scientifically grounded” and “based on statistical evidence” which is itself a bit of Unspeak).
In any field in which Taylorisation is being carried out (which means basically every field) there’s always a combination of genuine gains from process simplification and improvement, combined with substantial over-reach. There are also always evangelists of Taylorisation who tend to assume that they’ve got a silver bullet which solves all the problems of the field and to attribute the worst possible motives to anyone who disagrees with them. Like you say, a balance has to be struck.
Also note that a genuinely evidence-based approach to the whole field would be aware that the purpose of Taylorisation is to reduce the human capital requirement. If you standardise the procedures that qualified doctors carry out, then as night follows day, in a few years’ time that procedure will be carried out by people who aren’t doctors.
I’m in general in favour of this because of the success that Taylorisation has enjoyed in so many other industries, but a lot of doctors aren’t and for pretty obvious (non-self-serving) reasons. At the very least, a lot of work needs to be done on ensuring that the initial diagnosis is correct, because if it isn’t, nobody further down the track is going to realise this because they’re following the process. And it means that a lot of these studies (which were carried out on the basis of the processes being carried out with a big human capital input) will need to be revisited in ten years’ time to see if they work as well when they’re being carried out by less qualified technicians. A lot of the evidence-based community seem to really skate over these issues.
From Super Crunchers, about the first great evangelist of hand-washing based on statistical evidence, Ignaz Semmelweis, in 1840s Austria:
The author says that, even though it is now understood why hand-washing is a good idea, Berwick still met similar resistance in the mid-2000s when he insisted that even more hand-washing would reduce infections in patients with central-line catheters, and that many physicians still don’t wash their hands enough.
But Berwick’s suggestion of hand-washing in that particular context arose exactly from an EBM approach, ie reviewing hospital mortality studies with a particular focus, something that no one else had bothered to do. Until then, if they even thought about it, people had just assumed that they were washing their hands enough already.
As it happens, EBM people are very hot on things like the Isabel database, precisely in the hope of reducing diagnostic error.
Good point. The implicit relationship or contradistinction between the two is quite interesting.
That’s a very interesting point — but the extent to which it is cause for worry, or not, just depends on what the procedures in question are. You don’t actually need qualified doctors to elevate the heads of people on ventilators (to take one EBM-recommended standard procedure). A lot of proposed standardization of procedure involves such primum non nocere stuff which would in fact waste doctors’ time if no one else were allowed to do it (and obviously a lot of it is already done by nurses etc). I don’t know if any EBM people are seriously suggesting that complicated medical procedures should be performed by people without medical training. That might be another straw man.
If you google the term “nurse practitioners”, you’ll see that this absolutely isn’t a straw man (and if you google the term “nurse quacktitioners” you’ll see what calibre of idiot dominates the EBM-Luddite community, admittedly).
A lot of diagnosis happens during routine procedures, which IMO is the legitimate worry associated with Taylorisation here. If you can nip into the clinic and get your verucca burned off by a nurse working to a protocol, for example, then that’s convenient for you and saves a lot of wasted effort for the doctor. But if we installed that as the standard procedure, then we would need to have some other method of picking up all the veruccas which are actually skin cancer. Every medical procedure is potentially a complicated one, and the skill of deciding which ones are actually complicated and which ones aren’t is something that appears to me to be pretty resistant to a standardised approach.
[But Berwick’s suggestion of hand-washing in that particular context arose exactly from an EBM approach, ie reviewing hospital mortality studies with a particular focus, something that no one else had bothered to do. Until then, if they even thought about it, people had just assumed that they were washing their hands enough already.]
I think this is a Freakonomics/Moneyball style Sunday School triumph story. There have been all sorts of people for the last hundred years trying to enforce hand-washing policies; this one took off and that’s a genuine victory for which he should be given the credit, but I don’t agree that he was a lone voice in the wilderness.
So that I can be sure this isn’t a straw man: which EBM advocates are actually suggesting that complicated medical procedures/diagnoses be performed by people without medical training? (The “nurse practitioners” to whom you have introduced me appear to have extra medical training compared to nurses and so can presumably be trusted with more tasks, though not including open-heart surgery or the like.)
Surely it all depends on the procedure, and surely people are not just blithely refusing to think about this. Obviously there are some procedures, like elevating the head of a bed, scrubbing with chlorhexidine and perhaps more, that do not really have the potential suddenly to become complex and require the skills of an M.D., and can in fact safely be delegated to others.
Anyway, there’s an interesting short article by a Yale MD on precisely this topic here (which also mentions nurse practitioners).
Re: Berwick again:
He was in fact the first person to demand that hospitals implement a standardized protocol of extra hand-washing (and other specific hygiene procedures) specifically in cases of patients with centre-line catheters, so as to reduce the incidence of infection. (He actually got the info from a statistical study showing that where they were implemented, as they had been here and there on an ad hoc basis, they cut the risk of infection from central-line catheters substantially.) It’s pretty hard to argue that this isn’t “really” EBM.
Zounds! I’ve been left behind! I just want to jump in with a few thoughts here. There is a polemical approach that takes a subject’s worst characteristics or, worse, the abuse of otherwise sound characteristics and makes these fallacies or corruptions exemplary of that subject. To wit, the use of statistics. Foolish, manipulative, deceptive, or thick-witted abuse of statistics – like imagining that the p-value is certain proof of veracity – does not discount the subtle, imaginative and ultimately very useful modelling possible with statistics. Has the statistician George Box not already been quoted on unspeak.net? “All models are wrong. Some are useful.” This humility about the nature of statistics and his optimism about its use are not unwarranted. If a bunch of mooks could potentially misuse biostatistical information, in spite of the extensive didactics about EBM that medics are inundated with and despite the frequent articles about EBM in journals (dsquared, I put the burden on you to show that I’m wrong about this), it is hardly the fault of “EBM”. I don’t understand your arguments in 21, dsquared. But my main argument about EBM has been left behind: it is about being willing to subject claims to empirical testing. Now, nobody is saying that “empirical” testing is 100% super-duper objective, or perfect, or the final say in the matter; rather, it is the expectation that all medical claims are subject to validation, and that “my own experience” is not good enough (though clearly never entirely rejected: indeed, “clinical significance” tends to trump “statistical significance”, and the latter is expected to justify itself in the face of claims made for the former).
The notion that there is some sort of homogenization in studies is not worth discussing as evidence against EBM: one central concern of EBM is making sense of this very process of subject selection, and the biases that arise. “Generalisation” is a core subject of scrutiny. Plus, it throws the ethical baby out with the bathwater: the relationship of individuals to the multitude is a key feature of public health, and EBM is but one part of that dialogue.
I’m really unconvinced by – although intrigued by – the argument that EBM is responsible for people thinking they can do things that under others circumstances they would not be attempting to do. Does publishing studies in (relatively) freely available journals like JAMA or BMJ make your nextdoor neighbour think that she can perform colorectal surgery? I think that the rise of nurse practitioners or “physician assistants” speaks to a different set of arguments – not so much about “evidence”, but about health care economics (why pay an MD to do what someone can do on a boosted nurse’s salary?). I do agree that there is an argument to be made about what sorts of knowledge are put to work in medicine, about patterns of specialisation, and, something you refer to that is of central importance, the nature of referral: the practice of medicine involves an expertise in referral (the GP who sends one hypothyroid patient to an endocrinologist, but treats another in her own practice). I’m not taking a position here on NPs or APs – I’m just saying that I don’t think that the rise in NPs or APs can be explained by the popularity of EBM.
Is EBM Taylorization? That’s an interesting problem. Efficiency and breaking down practices into essential components so that they can be studied are both components of EBM, but I fail to see how EBM voids a job of responsibility: it simply isn’t that authoritative. Indeed, as far as I know, most of the evidence suggests that people take whatever is offered to them as evidence for a particular practice and then just go and do what they want to do anyway.
with respect to straw men, here’s Richard Smith, who surely can’t be considered straw, claiming that there is no evidence “from randomised controlled trials” that the US approach of screening for liver cancer leads to better outcomes. I don’t think that the idea of having “randomised controlled trials” of an entire medical infrastructure including screening tests even makes any sense; I spend my life arguing this point with econometricians (it’s basically an endogeneity problem; a model in which the variable of interest cannot be separated from a load of other factors) and am distressed to see it popping up in other fields.
which EBM advocates are actually suggesting that complicated medical procedures/diagnoses be performed by people without medical training?
Well, depends whether you’d call childbirth a complicated medical procedure or not, for example. Statistically, there is no difference at all between childbirth outcomes between home births, midwife-led practices and hospital births, but in individual cases it can clearly make all the difference.
Or take the example of the dispensation of pain medication; just handing out the pills and calculating the doses isn’t a complicated medical procedure, but diagnostic interpretation of patient reports of changes in the pain they’re experiencing is a really difficult matter that is very difficult, even for doctors; this is potentially a very complicated piece of medicine indeed.
I also think there’s a lot of ambiguity in the phrase “medical training”. Nurse practitioners have extra training in the procedures they’re qualified to carry out, but it isn’t medical training; they aren’t doing anything like a medical degree.
Thanks for the link. He’s striking rather a self-questioning attitude, though, isn’t he? Not like your average EBM straw-man who doesn’t care about individuals. After all, he does actually write:
It seems like he worries about just those problems you bring up, rather than being ignorant of their existence. You go on to say:
I might misunderstand the protocol, in which case I hope you or sw will correct me, but isn’t it just a case of doing some retrospective analysis of outcomes of screened vs nonscreened, after trying to control for other factors? (Not trivial, I know, but maybe not useless.)
Mmm, so which EBM advocate not made out of straw, exactly, is saying to pregnant women: “Hey! It doesn’t matter where you have your baby! Get your local locksmith to deliver if you like! The statistics say it’s all good!”?
isn’t it just a case of doing some retrospective analysis of outcomes of screened vs nonscreened, after trying to control for other factors?
but a) the controls aren’t possible – we don’t even know what we ought to be controlling for b) trying to control ex post in a retrospective analysis is a very big difference from the normal procedure of setting up a control group ex ante and c) even if one were to get over these two massive stumbling blocks, one still wouldn’t have a randomised trial; people are not assigned to “screened versus nonscreened” in the US healthcare system in a manner which is uncorrelated with other factors affecting their health outcomes. I am guessing that this was just a bit of loose speaking on his part (I don’t believe that there have been any RCTs carried out on the effectiveness of screening because I think it would be pretty impossible to do so) and what he really means is no more than a complicated and authoritative-looking way of saying that the USA doesn’t have a longer life expectancy or better liver-cancer outcomes than the UK. But the general question of bolstering one’s arguments with spurious references to “randomised controlled trials” is basically the whole topic here isn’t it?
It seems like he worries about just those problems you bring up, rather than being ignorant of their existence
But he has quarantined his worries about them; he’s put them into a logical place where (for rationalists) they are a proper object of “worries” but aren’t allowed to affect policy or practice. So in practical terms, he might as well have ignored them. This is the hallmark of the Taylorising over-reacher; he uses the valid arguments of scientific management in contexts where the actual scientific support is weak or non-existent (which is pretty much the definition of Unspeak).
I’m afraid that I’m really getting lost in this argument, dsquared.
Just type “randomized controlled trial screening” and then the disease of your choice into google and you will find RCTs of screening procedures (I tried “lung cancer” and “cardiovascular disease” for two reasons: both are relatively common, often lethal conditions where there is the hope and the possibility, respectively, of addressing morbidity and mortality with interventions when detected early, all of which are necessary components of a “screening” test). “Screening” by the way is entirely based on concepts of population, rather than the individual, and so I don’t understand how it cannot be subject to population-based studies.
I’m also getting lost with the whole birthing thing.
There is no difference between these location of births with respect to what? Neonatal morbidity and mortality? Maternal survival? Not true. Identifying complications of pregnancy and getting obstetrical care is an essential feature of reducing maternal mortality (it is, for example, one of five elements of the Minimum Initial Service Package for reproductive health in refugee crises – midwifery and TBAs, or Traditional Birth Attendants, do not lower morbidity and mortality, whereas 24/7 access to emergency OB/GYN services does). So, I’m not quite sure what you’re saying here.
You attack “spurious references to randomised controlled trials”. I agree, “spurious references to randmoised controlled trials” are bad things.
Ah, right, thanks for the hint, sw.
Here, for example, is the report of a randomized-controlled trial of screening for liver cancer; and this paper mentions another one.
Hmmm maybe I’m wrong then. Those are both papers referring to the screening of carriers of hepatitis virus though, so with respect to a general screening program as carried out in the USA they wouldn’t be randomised (and I think I’m right about Richard Smith as there doesn’t appear to be any studies of people who aren’t hepatitis carriers in the USA). In general, the larger the population you’re proposing to screen for a given disease, the more serious the statistical issues (also methodological isses here and here (including a blunt statement that they’re not suitable for cancer screening, which might be a bridge too far) and I’m not at all convinced that these papers are giving remotely reliable results; the quoted statistical significance levels are of less and less practical relevance. Looking around, the main controversy appears to be in the context of breast cancer screening – there are a lot of RCTs which have been carried out on this, but there are also a lot of sceptics of the validity of these RCTs and I think I’m with the sceptics.
There is no difference between these location of births with respect to what? Neonatal morbidity and mortality? Maternal survival? Not true.
True, at least for births identified as low-risk beforehand, and I don’t see why (given the actual evidence), anyone who believes in evidence-based medicine would not say that it doesn’t make a difference where a healthy woman with no pre-existing pregnancy complications gives birth.
Hold on, how can somebody be going on and on about the uselessness of EBM then say “true” and link to a prospective cohort study to back up his or her contention that something is “true”? Is that not an example of EBM?
As for your subject matter – you are simply wrong insofar as there is a difference between home births, midwife births and OB/GYN births, which is partly responsible (although by no means entirely responsible) for the single largest disparity in health outcomes indicators across the world: maternal mortality in childbirth.
There is a difference, and your caveat (“identified as low-risk beforehand”) is a population measure, again a feature of “EBM”. It is becoming increasingly clear that you are the biggest proponent of EBM here!
By the way, in your cited study:
how can somebody be going on and on about the uselessness of EBM
Oh FFS. Read back this thread then look me in the eye and tell me that’s even remotely fair. I’m really not going to carry on a discussion on that basis.
Well, to be fair, you’ve hardly addressed a single point I’ve made, so there’s not much of a discussion to begin with; but I do apologise for accusing you of “going on and on”, which is not fair.
As for re-reading your posts, I’ll admit to having a hard time constructing out of your posts a case that you consider EBM even remotely useful, so I don’t think that’s terribly unfair of me. That having been said, and I’ve pointed this out at several junctures along the way, I’m not clear that I’ve been following this thread, which may be why you’ve not felt it necessary to address any of my other points. And perhaps your attacks – and they certainly do sound like attacks, apologies if I misread the tone – on the statistics and pedagogy of EBM and the “spurious” citations of EBM and the fetishisation of “randomised controlled studies” and “Taylorisation” and purported problems in screening, etc., are directed at something other than EBM. I’m just not sure what.
That all having been said, you remain factually wrong about maternal mortality in posts 34 and 36, whether related to EBM or not, and I don’t think that your representations of EBM overall, as I understand you to have characterised it in these posts, is terribly fair.
I think this whole thread would blow Cass Sunstein’s tiny mind.
“It’s healthy stuff in small doses”
“In any field in which Taylorisation is being carried out (which means basically every field) there’s always a combination of genuine gains from process simplification and improvement, combined with substantial over-reach”
“I’m in general in favour of this because of the success that Taylorisation has enjoyed in so many other industries”
give over man. And as far as I can see (from the evidence), I am not wrong on childbirth. The BMJ and NICE both agree that there is no measurable difference in birth outcomes for planned home birth, the article I cited (which is by no means atypical) expressing this conclusion as:
“Planned home birth for low risk women in North America using certified professional midwives was associated with lower rates of medical intervention but similar intrapartum and neonatal mortality to that of low risk hospital births in the United States”
There isn’t, statistically, a difference in outcomes which appears in the data. Now the answer to this might be that in fact childbirth can be Taylorised and ought to be a job for midwives, or maybe there is something up with the data (a lot of these studies really do have a dose of 5% signifiance cult), or maybe this is an example of the difference between statistical and practical significance, but as far as I can see, the only response consist with rigourous application of evidence based medicine is the first (which is the actual view of NICE). Maybe NICE are right, but lots and lots of doctors disagree with them and not obviously for self-serving reasons; I think it makes sense to look at the EBM controversy in cases where it is seriously at variance with “medical common sense” as well as in things like handwashing and raising patients heads.
Dsquared, you originally claimed that “Statistically, there is no difference at all between childbirth outcomes between home births, midwife-led practices and hospital births”. What the study you cite appears actually to say, if I am reading it correctly, is that there is no statistical difference in outcomes between i) home births “identified as low-risk beforehand” and conducted with a midwife and where mother/baby can be quickly driven to A&E if necessary; and ii) hospital births. Those are two quite different claims, aren’t they?
They’re not very different claims though, are they? The set of unplanned home births plus planned home births from high-risk pregnancies is pretty small.
The differences appear quite crucial to me.
As to what you report NICE’s view to be, I’m a little confused, because its guidelines (pdf) actually say:
(This being Britain, access to emergency services is assumed.) This makes for an interesting comparison with your study result. I suppose it could be that the worse outcomes of home births with complications leading to transferral to hospital are balanced out by the better outcomes when home births go well. But it is of note that, if this is the case, those dastardly EMBers at NICE are not just taking the overall result and saying there is no difference without alerting the public to negative possibilities.
They also say at the outset that their advice — that home birth is generally okay — specifically does not cover cases of women:
So the implicit figure of an EBM advocate saying “Go ahead and have your baby wherever the hell you like, it makes no difference!” is really a straw man.
Just get a room guys.
I think there’s an intransitive noun here; your implicit figure is my straw man.
We can certainly trade quotes here can’t we? Look at the preceding sentence:
So as far as I can see, NICE do in fact give the advice that the evidence suggests (because it does) that for the relevant class of births (which is nearly all of them), home births and midwife-led units have outcomes which are at least as good. If we look carefully at the language here (back on topic!) you can see that the positive sentence I quoted is referring to a definite, evidence-based benefit while the negative sentence following it is talking unspecifically and in the conditional. NICE doesn’t do this out of sheer cackling evil or red-faced ideological obsession btw; it does this because it’s right, and the evidence does actually show this.
As far as anyone can tell, this is because (again, according to the evidence), doctors in the UK (and more so in the USA) tend to perform Caesarians too often; this is an absolute paradigm case of evidence-based medicine, because it directly brings into conflict the individual judgement of the doctors in each individual case with the evidence that they do in fact make systematic errors on average. (There is a clear parallel here with a lot of the things that the index funds crowd say on the basis of “evidence-based finance”).
These are the real hard cases of evidence-based medicine, because we know that:
a) in any individual case of a decision to perform a Caesarean, it is more or less impossible to gainsay the opinion of the obstetrician attending.
b) however, on average, there is good evidence (by which I mean genuine, scientific evidence of the sort that should be believed; it does rather get my goat to be portrayed as someone with no appreciation for statistical methodology when I have made my living out of it for ten years) to believe that he is likely to be wrong.
This is a genuine, non-straw distinction between the average patient and the individual patient and demonstrates, as far as I can see, that there is a genuine scientific issue here and it is not just a matter of doctors’ egos and Luddism. I think that these kinds of issue are actually very common in medicine and I don’t think that science is well served by not taking them seriously.
Sure: but however long we trade quotes it will remain true that your claim about what the statistics showed in #31, and your claim about what the BMJ and NICE views were at #42, were not actually correct. You might blithely dismiss all their crucial caveats about how their results and judgments apply only to specific classes of cases and contexts as unimportant because those that don’t fit are only a “pretty small” number, but isn’t that kind of attitude exactly what you’ve been accusing your EBM straw-men of doing all along?
(Was there any evidence, btw, for your further claim that NICE thinks “childbirth can be Taylorised and ought to be a job for midwives”? I don’t see that view expressed in their guidelines; perhaps it is somewhere else.)
Anyway, now we are on the same more nuanced page of what various people are recommending, the Caesarean case is indeed interesting, and no doubt worthy of discussion by people who know more about obstetrics than I do. Has someone in this thread actually been seeking to dismiss such issues as cases of “doctor’s egos and Luddism”? Or is there a serious EBM advocate on the issue who has said such a thing? Exactly who are you arguing with?
Has someone in this thread actually been seeking to dismiss such issues as cases of “doctor’s egos and Luddism”? Or is there a serious EBM advocate on the issue who has said such a thing? Exactly who are you arguing with?
I’m arguing with Druin Birch, linked above:
and SW, in this thread:
among others. I realise that neither of those quotes actually used the phrase “doctors’ egos and Luddism” but I think that the paraphrase is clearly fair.
On the issues of post #31 I’ll clarify: I still don’t think that these tests of screening regimes make sense, and they make less sense the broader-based the screening and the longer the onset of the condition screened for. The simple fact that a lot of them have been done in the literature doesn’t necessarily mean that they make sense; I think a lot of the published econometrics literature doesn’t make sense. And in the one instance where these trials have actually been scrutinised (breast cancer screening), they have not come through that scrutiny well. I was surprised that this literature existed though, I admit that.
On #42 I don’t agree that I am “blithely dismissing” anything. I said that the BMJ and NICE found no measurable difference in outcomes between home and hospital delivery for planned births; in as much as I was wrong on this, it’s because NICE found small evidence that home births had lower incidence of forceps and Caesareans.
Was there any evidence, btw, for your further claim that NICE thinks “childbirth can be Taylorised and ought to be a job for midwives”? I don’t see that view expressed in their guidelines; perhaps it is somewhere else
The expansion of midwife-led units under the NHS National Service Framework (the objective is to offer the option to every woman in the UK) was what I was thinking of. Guideline 55 also seems pretty clear to me; it provides an algorithm for births taking place with no involvement at all from a medical doctor; I’m not sure what else might be needed to establish that the process could be Taylorised. And I repeat again, they are almost certainly right on this question.
The point here is that evidence based medicine here has fairly solidly established a point which is very controversial indeed with doctors. Lots and lots of doctors don’t accept the concept of a “low risk birth” (which is in fact a concept established by evidence-based medicine in the first place) and believe that all births are potentially risky. Whatever one believes about this – personally I think that the evidence on home births is actually very good and the doctors are wrong – it clearly isn’t a question which can be settled by the kind of evidence which is the subject of EBM, and it can’t be solved by Unspeaking the objections away either.
I suppose Druin Birch can look after himself, but that really isn’t a fair characterization of sw’s position at all, if you look again eg at his comment #30.
Your claims at #42 were not correct: they were much more general than the claims actually made by the BMJ and NICE, and you simply swept the crucial caveats and limitations to the findings, that I pointed out in #43 and #45, under the carpet. The point at which you actually blithely dismissed them, just like some EBM strawman who doesn’t care about individual outcomes, came at #44 (“pretty small”) and then again at #47 (“nearly all of them”).
So, if I understand your latest correctly, you are saying that evidence-based medicine itself cannot adjudicate a dispute between evidence-based medicine and individual doctors? That seems true, though I don’t see that it means any specific such dispute can’t in principle be adjudicated, or even that anyone on this thread or elsewhere is really trying to cover such a dispute up where it exists. I still agree, though, that “evidence-based medicine” is Unspeak!
The expansion of midwife-led units under the NHS National Service Framework (the objective is to offer the option to every woman in the UK) was what I was thinking of. Guideline 55 also seems pretty clear to me; it provides an algorithm for births taking place with no involvement at all from a medical doctor; I’m not sure what else might be needed to establish that the process could be Taylorised.
In a word no. I think the most you could claim is that midwives have been helped in their long fight against doctors by a move towards taylorisation in the NHS, but you’d have to provide some evidence for this (I don’t know of any).
You’re also ignoring other factors:
1) Midwives are not deskilled doctors. They have their own skills, and are good at aspects of childbirth that doctors (largely) are pretty hopeless at.
2) Part of the problem with childbirth was that doctors had (for the most part) over-medicalised it. Why is debatable (my guess is a combination of a poor assessment of actual risks of complication, combined with the way that they are trained to see things through a medical paradigm), but it led to a more uncomfortable, painful and (possibly) dangerous procedure for the majority of births.
3) A lot of women wanted home births.
Lots and lots of doctors don’t accept the concept of a “low risk birth” (which is in fact a concept established by evidence-based medicine in the first place) and believe that all births are potentially risky.
This is an interesting thing which comes up in medicine a lot. Its about risk, and the problem is that in reducing one risk you often increase other risks. You can’t eliminate all risks, but unless you look dispassionately at the evidence there is (a very human) tendency to focus on the visible risks. Doctors are right, in that some “low-risk” births will have medical complications. On the other hand, midwife led births are less likely to have such complications. But of course, they don’t see that.
Are you talking about the bracketed caveat below?
I think that this is still saying that statistical evidence trumps doctors’ anecdotal experience and case studies. The concession is made that it has to be high-quality statistical evidence (which is a genuine step back from the frontiers of evidence-based medicine; there were plenty of people talking smack about breast cancer screening on the basis of statistical studies that were very weak indeed), and there’s proper respect for the distinction between statistical and practical significance but it still very much seems to me that SW is committed to the view that doctors who disagree with (for example and because it’s a clear cut case) the evidence on home births must be doing so for reasons that are illegitimate – I can’t see what “all medical claims are subject to validation” means here otherwise.
just like some EBM strawman who doesn’t care about individual outcomes
by the way you’re wrong about this. The EBM strawmen do care about individual outcomes, but believe that individual outcomes will be better if general rules are followed. In other words they believe that doctors can’t systematically beat the averages. This is a specific claim that a class of general rules based on averages over a sample space are applicable to particular new cases. EBM advocates become less realistic and more straw-like the greater the space of such rules they’re willing to back.
The special cases in which everyone agrees that hospital births are indicated aren’t relevant to point I was making (they are also statistically uncommon but that’s a different point). Which point was, in response to your request for an example of a complicated medical procedure which EBM suggests can be carried out by people who don’t have medical training, that an example of such a procedure was vaginal delivery of single births to healthy mothers. It’s not a perfect example because there seems to be a controversy over whether it’s a complicated medical procedure (which was the subject of the recent detour). But I don’t see how its usefulness as an example is compromised by the existence of other kinds of births which are acknowledged by everyone to be too complicated for midwives to handle – which means that ignoring those other categories of birth isn’t blithe, it’s sensible.
2) Part of the problem with childbirth was that doctors had (for the most part) over-medicalised it. Why is debatable (my guess is a combination of a poor assessment of actual risks of complication, combined with the way that they are trained to see things through a medical paradigm), but it led to a more uncomfortable, painful and (possibly) dangerous procedure for the majority of births.
not trying to make yet more enemies here, Cian, but would you agree that “a poor assessment of actual risks of complication” might be described as “Luddism” and “the way that they are trained to see things through a medical paradigm” might be described as “doctors’ egos”?
I think it’s more complicated than that. There is a genuine, philosophical question of standards of evidence and of what statistical reasoning can and can’t achieve here, which I would know more about if I read some of those philosophy of science books I keep buying. There’s also something I’m going to write a proper post about, which is that there’s a clear role for Goodhart’s Law to take effect here; now that NICE has been established and large amounts of government money (and professional advancement etc) get spent in a manner directed by EBM, then everyone is going to make sure that their personal clinical preferences can be well-supported by scientific evidence. With predictable and baleful consequences for the general standard of science produced (a genuine problem; something similar in economics more or less did for the Lawson-era monetary policy framework).
By the way, I hope everyone will realise that I’m making a joke rather than a serious point when I suggest that the tendency of doctors to over-medicalise pregnancy might be put on a plane with the ideological bias toward the theatrical of Shakespeare scholars?
I am really at a loss as to how you manage to read sw’s observation that “indeed, ‘clinical significance’ tends to trump ‘statistical significance'” as saying “that statistical evidence trumps doctors’ anecdotal experience and case studies”. Can you run that one by me again? sw seems to me to have shown himself throughout ready to sort of have the metaevidential discussion about the virtues and demerits of EBM and other forms of “validation” that you have been complaining no one is willing to have.
As to whether I am wrong about what your EBM strawmen say: I concede that I might have been. (After all, they’re your strawmen, so you are the expert.) Still, you have said that EBMers care about/try to treat the “average” patient and it is individual doctors who concern themselves about the “individual patient”. Apologies if I read too much into this. I think it is actually an interesting philosophical issue, even if the two camps are not so starkly opposed in reality; but then it must have been an issue since the dawn of medicine — how do you treat the person in front of you with regard to what you know (or think you know) about how what generally causes such cases, and what generally resolves them? — and so doesn’t seem to me exclusively a problem with EBM. (Nor indeed exclusively with medicine as a science.)
Re the question about complex medical procedures being done by people with no medical training, and whether home birth counts as that. I would say not, but you are right that a lot of it is about issues of interpretation of language (hey, we’re on topic again!). I would argue: surely, just to the extent that delivery is considered a “medical procedure”, the training that midwives receive by definition constitutes “medical training”?
We should also keep in mind that the studies and guidelines etc we have discussed do not for a moment countenance delivery by midwives alone without the mothers being able to get to A&E if necessary, as it was indeed considered necessary in a full eighth of cases in the study you cited. So they are actually not recommending in any sense that delivery of babies be left entirely to people without medical training, even if one wants to insist that midwives’ training isn’t really “medical”.
No doubt everyone had it easier when our aquatic ape ancestors just happily plopped out babies underwater.
I don’t read the bracketed caveat that way at all (if I have misread SW then of course I’m wrong but I don’t think I have). I think it reads straightforwardly as a comment about the distinction between practical and statistical significance, which is a distinction within the statistical analysis.
That distinction would be (and these are examples I have not checked so treat them as hypothetical) – if we found out that dentists were more prone to alcoholism than oncologists, then it’s quite possible that this could turn out to be a statistically valid fact (in the sense of passing all relevant tests of significance), but it’s obviously of zero practical or clinical signifiance. On the other hand, the first studies on taking aspirin as a prophylactic for heart disease didn’t really show a statistically significant effect, but it was immediately recognised as massively clinically significant, because if you can deliver even a small improvement in such a widespread cause of death that’s huge.
These are just sensible principles of data interpretation – they are actually not very well followed in practice anywhere in statistics (including in medicine – SW cites loads of strictures against these bad practices in the EBM literature, but as in econometrics, this is more likely to mean that bad practice is rife). But they’re about getting the right interpretation of the statistical evidence, not about any limitations of the statistics correctly interpreted.
There’s a qualitative difference here between that and, say, a psychiatrist sitting down with a patient with mild depression and making the judgement “cognitive behavioural therapy is never going to work for this guy, he hasn’t got the personality for it, I’m going to prescribe him a little bit of prozac”. (And this really isn’t a strawman example; a clinician doing this would be going against the NICE guideline which recommends that antidepressants aren’t prescribed for mild depression because the risk/benefit ratio is too low. There are also plenty of people out on the net who think that CBT should always be tried first in cases of more serious depression; admittedly the majority of these are CBT enthusiasts who have marshalled the evidence-base in their favour, but as I keep saying, this is not a problem that EBM can just wash its hands of). I really don’t think that the caveat about “clinical significance” could be stretched far enough to include something like that if the rest of the comment is to preserve any sense at all.
I guess we’re going to just disagree on the midwives then. I think there’s a real, qualitative difference between the way in which a medical doctor makes the decision about the progress of a birth (and therefore about the escalation of the care) and the way in which a midwife does. One’s working off a great big base of medical knowledge and the other is working off a checklist (both are of course also working off a whole load of practical knowledge but I think that the distinction in the training is really clear).
Mmm, but if there weren’t loads of strictures in the EBM literature, you could say “People haven’t even thought about these issues!” So you win either way, it seems.
But a) it is still doctors who make the decision as to whether a home-birth is appropriate in the particular case; and b) it is still doctors to whom care is transferred if things become difficult (also, the midwives appear to be operating under a sensible precautionary principle w.r.t. the escalation of care, given that in your study 12.1% of home-birthers were transferred to A&E, but the transfer was considered “urgent” in 3.4% of the cases). So as I say, whether you consider midwives “medical” staff or not, I see zero evidence that anyone is actually suggesting they should handle all the intrapartum care in all cases.
Going back to comment number 1 about consenus. You might remember that the UK gvernment claimed that it was trying to reach a consensus in the UN Security Council just before the invasion of Iraq, going on to say “but unfortunately it wasn’t possible”. Well no it wasn’t possible because a small minority of members (including the UK) wanted to invade anyway while more than two-thirds wanted inspections to continue (and quite right too). For Jacqui (and the UK Government in general) a consensus is when everyone agrees with them even when there is no good reason why they should.
I’m sure this explains why dsquared and I are unable to reach a consensus.
But a) it is still doctors who make the decision as to whether a home-birth is appropriate in the particular case
Not sure this is true – surely it’s the patient rather than the doctor who makes the decision, based on advice given, and it’s precisely the content of that advice that is the subject of the controversy.
I see zero evidence that anyone is actually suggesting they should handle all the intrapartum care in all cases.
But that’s a claim which would be much stronger than the one which I actually need in order to make the point I want to make (I am a bit bored with the phrase “straw man”). It is, provably from the service framework, an NHS priority to provide the option of home births and midwife-led units. Lots of doctors don’t think that this should be the case, based on their view of the risks and benefits of home births. The evidence appears to be that these doctors are wrong, as long as you accept that the results of statistical analyses like the one I cited above are “the evidence” in the relevant sense. Whether or not one accepts that proposition is the matter of controversy.
I mean, there are procedures at Tescos for referring things up the management structure, but that doesn’t mean that Sir Terry Leahy is stacking the shelves.
dsquared, going back to #55, you write:
If I did a study on whether feathers or bricks were heavier, using a randomised, single-blind trial matching feathers and bricks for length, it is quite possible that I would provide evidence in support of the hypothesis that bricks are heavier that feathers. A bullshit study is a bullshit study. Any study begins with a question. “Are bricks heavier than feathers when matched for length?” and “are dentists more prone to alcoholism than oncologists?” seem to me to be useless questions; perhaps there are circumstances in which those very questions are not useless, but then the study would have to designed to take into account those circumstances. “Does aspirin prevent morbidity and mortality in cardiovascular disease?” and “how do CBT and medications compare in mild depression?” may be considered useful questions, and so studies might be designed that would provide evidence in support of an answer to these questions. If the question has no clinical significance, then all the evidence and statistics in the world can’t do much to make the answer worthwhile. This is one of the key features of “EBM” – what sort of question does a study try to answer and can I find evidence to support or guide my answer to a question (such as whether I should start diuretics or ace inhibitors in my patient with diabetes and hypertension, or whether I should recommend that this woman with a history of preeclampsia should deliver at home or in the hospital . . . one asks a clinically meaningful question and then can do a literature review to see what studies have been performed and what the assessments of these studies are and whether they are suitable to the person sitting in front of you; those studies don’t hold a pistol to your forehead demanding that you do one thing or the other).
I think this is where you’re going when you then write:
I don’t know whether they’re well practiced or not, but I pretty much commented on that already in a previous post. The “right” interpretation of statistical evidence may include an assessment of clinical significance, depending on what sort of answers the statistics are modelling. And that’s all they’re doing: modelling an answer to a question. If the question is one you want asked, or one you are asking, then you can look at the model and try to make sense of it and its relationship to reality.
As for midwives, I can’t bear not to chime in – with the very same point I made earlier, so apologies for redundancy. Hospital and home deliveries are both safe, but if you want to reduce maternal mortality, there has to be access to 24/7 OB/GYN services – and I don’t know anybody who doesn’t agree with that, so surely we have some broad consensus there?
Yeahbut, it is actually the claim you made, when you said that NICE and the BMJ think “childbirth can be Taylorized and ought to be a job for midwives”. I’m happy to agree that you didn’t need to make such a strong claim!
“Ought to be a job for midwives” is possibly too strong a gloss on what NICE do think and say, which is that in the situation which most pregnant women find themselves in, midwife-led births[1] have the same risks of harm to mother or baby ex ante and better risks of having a Caesarian or forceps delivery. But I’m not giving an inch on “Taylorised”; that’s exactly what the NICE guideline does. The fact that some steps on the algorithm involve passing the case on to a hospital unit doesn’t change that – it’s what you’d expect to see in any Taylorised process. And the purpose of that algorithm is to allow women the choice of a home birth rather than having to have a doctor-led process (which a) lots of them simply want and b) is clearly cheaper than having the same degree of individual attention provided by a qualified MD).
[1] (in the context of the general availability of hospitals etc, although I know we absolutely hate strawmen here, so I’m sure nobody would want to claim that I ever actually said “by which I mean births with absolutely no involvement from doctors at all who can be taken out of the process and it makes no difference if you give birth in a ditch attended by one of Macbeth’s witches”)
SW:
but this is my point; they more or less do hold a pistol to your head if they have been codified into NICE guidelines and you are a doctor working for the NHS. A British psychiatrist or GP who regularly prescribed Prozac for mild depression (which is not in and of itself a crazy thing to do; it’s very common to do so in the USA) would be prescribing against guideline and sooner or later would get into some sort of trouble (most likely with his local Health Authority, which is audited against targets for compliance with NICE guidelines). This is “government science”; if you don’t view this role of codification and enforcement as being part of the meaning of evidence-based-medicine (and I guess the final paragraph of #30 suggests you don’t) then fair enough, but it certainly seems to me that Richard Smith and Druin Birch do.
(and just to add that because it’s “government science”, it’s very vulnerable to Goodhart’s Law as I mention earlier – once you start using a scientific measure as a guide to policy, you immediately begin to degrade its informational value)
I simply can’t be unsympathetic to your concern that such tools as “EBM” or, one might add, the Diagnostic and Statistical Manual of Mental Disorders (DSM) or the International Classification of Diseases (ICD) might be misunderstood, misused, or frankly abused in the design of health policy, by lawmakers, by health care bureaucrats, etc. At the same time, however, the potential to misread a novel, or a song (as in Reagan’s appropriation of Springsteen’s Born in the USA), or a medical technology is not necessarily the fault of that book, song, or practice. Similarly, the failure to appreciate the subtleties of an approach – as in EBM – should be laid at the feet of this who are clumsily handling it, rather than in the approach itself. So, yes, I don’t think that the misapplication of EBM or misunderstandings of it are an inherent fault of EBM. That all having been said, I can’t quite agree with you: I would hope that governments’ health care policies do take into account scientific evidence, including one of EBM’s great areas of concern, epidemiology. One would hate for the NHS to set aside as many beds for patients with Hunter’s Disease as for patients with cardiovascular disease.
sorry guys – reply forthcoming on Wednesday – travelling around …
OK, a day late and a dollar short …
No, I do think that the problems of over-reach and institutional misuse are intrinsic to the evidence-based system. Which is not to say that these are killer objections or that taken as a whole one should reject evidence-based medicine, but taking it as a whole is what you have to do.
By which I mean, it is intrinsically part of the point of evidence-based medicine that it’s a centralised system in which there is a single right and wrong way of doing things – it’s what a Foucauldian (or the more irritating kind of net.libertarian) would call a “totalitarian” system. I’d argue that the motivation for this is Taylorisation and standardisation – as I understand it you and Steven would see it more as the outcome of the scientific method, delivering us a current best estimate of the underlying objective facts (and these two aren’t so different since Taylorisation basically is the scientific method applied to management) – but I think we’re all agreed that the end result is that over an increasing part of the field, there’s a settled single way of doing things.
When you create a centralised system like this, there are known, specific institutional weaknesses that are intrinsic to it – and this is as far as I can see an empirical fact, as the relevant results of public choice economics and sociology are facts of social science which are not much worse empirically established than a lot of things in medicine. Specifically, any regulatory system is vulnerable to “regulatory capture”, anything which has the power to allocate significant amounts of government money is going to become the object of a lot of rent-seeking behaviour, and any information source which is relied on for purposes of public policy will tend to have its usefulness degraded over time, as the subjects of that public policy have an incentive to take actions which degrade it (that’s Goodhart’s Law).
You can build in institutional safeguards against these problems (and they are indeed often overstated a lot – only a small proportion of regulatory agencies are so badly compromised by regulatory capture as to be worse than no regulation at all), but they do exist and they exist as an intrinsic part of systems of the kind of which evidence-based medicine is an example. There are, of course, plenty of problems and pathologies of ad hoc medicine (or whatever the opposite of EBM is), but they’re not the same ones.
So I think this ends up as a dispute in the philosophy and sociology of science. I don’t agree with you that the blame can be taken off EBM and placed onto the people who misuse it on the “Born in the USA” principle. I think that given the sociological facts (which are proper facts) about EBM as a social and political institution (and it’s a pretty controversial view that something like EBM even is a social or political institution, but I think it is), it’s inevitable that exactly these sorts of over-reachers and misusers will come into being, because otherwise someone would be leaving money (or its equivalent) on the table.
and just responding to something I missed at the time but which I think is a peripheral skirmish:
If the question has no clinical significance, then all the evidence and statistics in the world can’t do much to make the answer worthwhile
in that case, for the relative propensity to heart attacks of different kinds of medical man, substitute a lot of the results that the oncology literature repeatedly comes up with – it’s a truism of the profession that “more or less anything can be shown to cause cancer in lab rats”. There is a massive academic industry in the generation of findings that everyday compounds are potentially carcinogenic, all of them (or all of the published ones anyway) passing all relevant tests of statistical significance, but the vast majority generally regarded (correctly) as being of no clinical significance.
Zounds! Now I see why nobody responds to my blosts. When they’re this long, it’s hard to know where to start!
In science, as in EBM, nothing is ever “settled”; it is a fundamental principle of scientific inquiry that everything is ultimately up for grabs. Now, such an assertion ought not be reduced to sniffy “So, you’re saying everything’s relative?” or a pouty “So, you’re saying there’s no such thing as objectivity?” Rather, it is an ideal position, almost Platonic in its total purity; science wants to find new ways of looking at things, more closely, from different angles, and does so by asking questions and figuring out how to answer those questions (sometimes, things don’t work out this way; sometimes technologies, designed to answer one set of questions, accrue data and create patterns, and then the question is, “Why did this pattern emerge?”) Science, I always argue, isn’t about answers, it’s about questions. So, the authoritarian hegemony of EBM just isn’t convincing. If a physician says, “I have to prescribe Fosamax because of the evidence base”, then that physician is misusing fundamental principles of EBM, which include asking questions: “Does the literature and the evidence suggest that this patient in front of me would benefit from this medication?” I could find numerous citations that support my approach to EBM, but none that officially sanction its misuses. Now, you do raise an interesting question about responsibility: how responsible is anybody or any system for its abuses and misuses? To some extent, the burden is on the critic to show responsibility in a specific way, rather than just drawing on generalizations. And I don’t think you have done this (I’m afraid I’m not convinced by your hypotheticals . . . )
Unfortunately, I simply don’t know where EBM is “centralised”, nor do I know where or how it is exerting its Taylorising effects. I think that your critique is fascinating, but it smacks to me – perhaps inaccurately – of the concern of doctors who insist that EBM will make of them cyborgs who simply input “evidence” and output “prescriptions”; that is a caricature of what EBM intends and certainly has not become a common practice, as far as I can tell. But what would I know? As I insisted in my defence of Kamm, who has been so thoroughly tortured on this site, I am lawyer.
This Goodhart’s Law stuff is interesting: but what precisely is being “degraded” and how does EBM’s influence on public policy degrade either EBM as an abstract principle or the “evidence base” of medicine itself?
In terms of #68, the question, though, remains clinically significant: what substances are carcinogenic in humans? The answers have sometimes been fascinating; your argument that
doesn’t quite hold; see the site I linked to for a more sober, humble approach, based on science, statistics, and “evidence”.
I don’t know why the link didn’t work; my html usage is crap. Here’s the link:
http://www.cancer.org/docroot/.....nogens.asp