The Debunking Handbook: now freely available for download
Posted on 27 November 2011 by John Cook
The Debunking Handbook, a guide to debunking misinformation, is now freely available to download. Although there is a great deal of psychological research on misinformation, there's no summary of the literature that offers practical guidelines on the most effective ways of reducing the influence of myths. The Debunking Handbook boils the research down into a short, simple summary, intended as a guide for communicators in all areas (not just climate) who encounter misinformation.
The Handbook explores the surprising fact that debunking myths can sometimes reinforce the myth in peoples' minds. Communicators need to be aware of the various backfire effects and how to avoid them, such as:
It also looks at a key element to successful debunking: providing an alternative explanation. The Handbook is designed to be useful to all communicators who have to deal with misinformation (eg - not just climate myths).
The Authors:
John Cook is the Climate Change Communication Fellow for the Global Change Institute at the University of Queensland. He created and runs Skeptical Science and co-authored the book Climate Change Denial: Heads in the Sand with environmental scientist Haydn Washington. In 2011, Skeptical Science won the Australian Museum Eureka Prize for the Advancement of Climate Change Knowledge.
Professor Lewandowsky is an Australian Professorial Fellow and a cognitive scientist at the University of Western Australia. He received a Discovery Outstanding Researcher Award from the Australian Research Council in 2011. His research examines people's memory, decision making, and knowledge structures, with a particular emphasis on how people update information in memory. He has published over 120 scholarly articles, chapters, and books, including numerous papers on how people respond to misinformation. (See www.cogsciwa.com for a complete list of scientific publications.) Professor Lewandowsky is an award-winning teacher and was Associate Editor of the Journal of Experimental Psychology: Learning, Memory, and Cognition from 2006-2008. His research has been funded continuously since 1990 by public agencies in 5 countries, but he has no commercial interests of any kind. He has also contributed numerous opinion pieces to the global media on issues related to climate change "skepticism" and the coverage of science in the media. A complete list of his public essays can be found athttp://www.shapingtomorrowsworld.org/inthemedia.htm, which is a blog run by academics from W.A.'s three major universities.
Update: Translations
The Debunking Handbook has now been translated into the following languages:
Dutch |
French |
|||
German |
Hungarian |
Icelandic |
||
Indonesian |
Italian |
Polish |
||
Spanish |
Swedish |
Update: Open Access Slideshow
In collaboration with Ullrich Ecker from the University of Western Australia, we have put together the “Myth Debunking Slideshow” (Ecker, Cook & Lewandowsky, 2012) - an open-access slideshow summarizing both the basic research on misinformation and recommendations on how to deal with misinformation in practice. The presentation is designed in popular Microsoft PowerPoint software, and is licensed under Creative Commons, meaning it is free to use, adapt and distribute under most conditions.
This post has been cross-posted at Shaping Tomorrow's World.
Obviously, the handbook works well for any debunking in any subject (creation-evolution, HIV-AIDS, vax-antivax, etc).
Global warming: Propaganda creates myth of disagreement.
So, here is my point.
Schmittner et al 2011 find a low sensitivity. In this article , SkS explains to its readers why there are some reasons to take the results with caution : you discuss in details the models, the proxies, the methods, etc.
Lunt 2010 and Pagani 2010 find a high climate sensitivity. In this article , SkS doesn't explain to its readers why there could be some reasons to take the results with caution : you do not discuss in details the models, the proxies, the methods, etc.
In my opinion, this is a potential double standard and without explanation from your part, it hurts the confidence I can place in your impartiality. Comparing these two articles on a very similar subject, it seems you deploy a high skepticism for a low sensitivity study but a low skepticism for a high sensitivity study. It looks like a cognitive bias in favour of the most alarming publications, and anyway an unequal standard of quality for the information you give to your reader.
Could you explain me why I'm wrong?
Thank you.
PS : please, no strawman, I'm not denialist and I fully agree with IPCC WG1 conclusions. But I prefer to know if, beyond the very legitime and useful demonstration of denialists' misinformations, I will find on SkS a fullly impartial and convincing view on current climate science results, or a discretely biased presentation of these results.
There are many threads on climate sensitivity (use Search), but consider this one as a starting point. In it you find a review of many sources, offering estimates of climate sensitivity as high as 8C. However, the graphics shown remain consistent with the 2-4.5C range of estimates. Clearly, those papers arguing for 8C have not made their case.
This continues to be a stunning example of false equivalence. Many posts here attempt to put multiple papers in context and consider multiple lines of evidence. Rather then posit the appearance of cognitive bias, please provide more concrete examples than the sensitivity question.
It is interesting though that sensitivity in warmer times (albeit Earth Systems Sensitivity, also discussed at Serendipity ) is high while at cool times it's low if this holds up.
Rather, what is happening in the main is papers that are counter to what science has established are treated more skeptically. So a paper that looks at high sensitivity (say 4.5 as the paper you referenced suggests) is completely in keeping with the understood science, that the most likely number is ~3C, bounded below at 2C (in other words, tons of evidence that it is NOT lower than that) and not well bounded above (meaning a paper that merely claims that, with the long term impacts baked in, the number is more like 4.5c - is well within the understood science).
So you are seeing the case that extraordinary claims require extraordinary proof, not a case of bias. If a future paper claims 8-10C for the sensitivity figure, look for caveats similar to the Schmittner paper.
Also note the paper itself urges caution, and beyond any concept of bias or proof there is the simple fact that the author would be misrepresenting the paper to not include the caution an a review of the paper.
Everyone involved at Skeptical Science would love to see the accepted science overturned, but there is no credible evidence to support that position.
Science means going where the evidence takes you. I know of no blog site that beats Skeptical Science in that regard.
#6, #7 : OK, so context... SkS treats information in a different manner if this information is susceptible to be deformed by denialist. I keep it, but fort that, it would have been sufficient to say that Schmittner 2011 is in the range of sensitivity of IPCC models (previous point), without trying to suggest there could be some particular uncertainties or flaws in their study (all the more so that it seems the tamino critic is irrelevant, because UVIc is not CCCMa).
#5 Two papers, same subject, different treatment on SkS... Sorry, I do give a correct example (not apple and orange, not cherrypicking), the fact that there are many papers on ECS on SkS is not relevant if I choose to compare the treatment of these two similar papers in particular. Your answer is the less convincing for me, I prefer #6 and #7 as more plausible. A detail concerning the paper you link : the figure from Knutti and Hegerl 2008 in the conclusion is incomplete. In the original paper, the two authors not only gave the different sensitivity distributions (part a of the figure reproduced), but also a "partly subjective classification of the different lines of evidence for some important criteria" (part b of the figure, non reproduced).
I hardly think the degree to which a particular paper can be spun is a worthy criterion for rebuttal. If a paper is on a relevant topic and has some evident flaws, it is questioned. Why do you interpret that as bias?
You are missing the bigger question: If Schmittner's sensitivity is within the 'classical range of sensitivities' - although they claimed to have lowered the uncertainty - why are the pseudo-skeptics running with 'new paper finds global warming overstated'? Isn't that an indication of just how bad things are in the literature of denial: A paper that makes no statistically significant change to our understanding is trumpeted like they've transmuted lead into gold?
We saw this behavior with CERN CLOUD, Murry Salby, Spencer/Braswell, etc. In the vernacular, it's called 'clutching at straws.' Myths do not debunk facts.
I don' think there is a real issue here - you are putting SkS under a microscope that no "skeptical" site could handle - and at that SkS is coming through with flying colors.
So, while we did not undertake a thorough going critique of Paganini or Lunt, neither have we embraces their results uncritically.
In contrast, deniers have not only embraced Schmittner et al uncritically, but they have used it as justification of very low values of climate sensitivity that Schmittner et al exclude.
This difference results in a clear difference in treatment at SkS. The primary purpose of SkS is not to explain climate science. Rather, it is to debunk denier myths. Because deniers have been building a myth around Schmittner et al, it requires detailed treatment to debunk that myth. In contrast, Lunt and Paganini are merely mentioned so that people interested in climate sensitivity will be aware of recent papers. So while the two papers did receive a different treatment, that is because of how they relate to the stated purpose of SkS, not because of any bias.
#11 actually thought: yes but it seems to me there was some particular efforts to go beyond the author reservation (typically, to refer a tamino non peer reviewed critic suggesting the model is incorrect: climate science is not done on blogs! Even if I appreciate and read tamino). I myself suggest in commentaries some critics that went to be wrong, see for example Dr Schmittner's recent precisions about the proxies data (better than Shakun 2010).
#10 muoncounter : of course denialists are trumpeting politically-biased and fanciful 'victories' on 'mainstream conspiracy'. My worry is that the counter-argument of denialists' myths translate in an unbalanced 'over-criticizing' of some works they unduly exploit (Schmittner is not Spencer!).
With regard to your response to muoncounter, I do not think SkS is guilty of unbalanced "over-criticizing" of some works. It you want to see an unbalanced over-criticizing, compare Dana's post on Schmittner et al to that by Joe Romm, who calls Schmittner et al "a new deeply flawed study". That is, of course, a load of nonsense. I am not aware of any flaws in Schmittner et al. They applied a valid technique to the best proxy set of LGM temperatures they could find. To the extent that they are wrong about climate sensitivity, it will be because the best proxy set is neither accurate nor extensive enough, and because due to computational limitations they used only one mid-level climate model. Both limitations are not flaws but simply the constraints of real science, which must work with the data available, and within a budget.
Of course, I don't consider that SkS is in the same category that Joe Romm's website.
Further, your council in this case makes the perfect the enemy of the good. Like it or not, denier misrepresentations are often so fundamentally flawed that you will not find peer reviewed literature discussing the relevant facts, and or debunking argument. Claims by one prominent Australian denier that CO2 actually cools the Earth's surface relative to what it would be with no greenhouse gases in the atmosphere fall under this category. In other instances, flaws in denier papers are picked out and highlighted long before any peer reviewed literature will discuss them, simply because of the slow turn around time on peer reviewed literature. In both cases, clear and compelling discussion of the issues can be found on several blogs. The suggestion that SkS should either discuss the issues without reference to those other discussions (thereby pretending that while other blog discussions are suspect because they are blog discussions) or not discuss them at all because we do not have peer reviewed literature addressing the issue are both unwise. The simple fact is that Tamino, or RC, or Science of Doom will often have clear and accurate analysis on particular issues that are (in many instances) better than we can produce ourselves. In that instance, we serve our readers best by drawing attention to those discussions.
#13: "an unbalanced 'over-criticizing' of some works they unduly exploit (Schmittner is not Spencer!). "
I'm sorry, but I cannot determine what you mean by this phrase.
#15: "Tamino is not a climate scientist. If you don't scotch to this editorial rule, why not consider the McIntyre's"
Another false equivalence. If you are suggesting that 'all climate blogs are created equal,' you are severely mistaken. If you think that tamino = McIntyre, why not tamino = WUWT? Why not use authoritative sources whose work stands up to inspection? Why equate said sources with those whose work fails inspection?
Anyway the blog question was not the principal matter of my #4 point. You correctly explained me the editorial reasons for which Schmittner 2011 deserves a more attentive treatment than others publications about sensitivity. I agree with these explanations. You also show that the canonical 2-4,5 K range is the usueal reference for sensitivity on SkS. Idem. Case closed for me.
The this is an interesting perspective ... if the temp sensitivity is low then we are seeing a high climate change sensitivity to that small change. So it would mean things would be worse than expected. That is the risk aspect that we have to take in to account which deniers and avoiders ignore. A human trait and short coming.
Media Misleads On Flawed Climate Sensitivity Study: Avoiding “Drastic Changes Over Land” Requires Emissions Cuts ASAP
http://thinkprogress.org/romm/2011/11/27/376197/media-flawed-study-climate-sensitivity/
Many have considered McIntyre's arguments, and found then to be wrong in nearly (granted, not quite all) every way. McIntyre's results were deliberately inflated by himself and even more by others, yet when you dig and understand exactly what he did, you don't see an accurate argument. That he still goes on about the hockey stick nearly a decade on, after it's been independently verified many times speaks strongly of his approach to science, and does not argue for his points being either relevant or accurate, or worthy of interest.
So by your logic, we need to have limited trust in your arguments as well?
The differences should be abundantly clear. On this blog (or tamino, neven, science of doom, etc), anything technical is solidly supported. If you show up with a pet theory that makes no sense (and there are plenty of those characters), you will be rebutted with the science that shows you make no sense. On the McIntyres, Goddards, Currys, Watts, etc, there is plenty of opinion, distraction, disinformation and hot air and very little in the way of solid scientific support.
But I suppose you are free to buy your soap from the store of your choice. My observation is that someone like tamino knows a heck of a lot more about what he does than I do -- and I can learn from that.
So, McIntyre has misattributed quotes, taken them out of context, and stitched them together out of chronological order to aid misinterpretation. Further, he ignores the fact that the person discussed was in fact accepted as a contributing author, thus invalidating his thesis. It has long been my firm belief that out of context quotations are simply a form of lying. So while Tamino gives us clear analysis of statistical issues relating to climate science, McIntyre gives use data (in this case quotations) deliberately out presented out of context with intent to deceive. That is why it is inappropriate to use McIntyre as a source, but appropriate to use Tamino.
Indeed, I would go further than that. McIntyre and Watts also read the scientific literature, but that does not make their comments worthwhile. The simple fact is that their reading is not comprehensive and their understanding is limited for a variety of reasons. Therefore, (like me) they are not expert. An expert has been defined as a person who knows all the basic mistakes in a field, and how to avoid them. In a field as complex as climate science, that means to be an expert you require a genuine comprehensive knowledge of the whole field. A detailed and accurate knowledge of a particular area does not prevent you from making basic mistakes in others, as example after example of well educated non-climatologist deniers have demonstrated.
Therefore, a blog comment is trustworthy only to the extent that:
1) The author reads the primary literature;
2) The author focuses on that area of the primary literature which intersects with their particular area of expertise (in Tamino's case statistics);
3) To the extent the author ventures outside their expertise, they primarily summarize and report on the primary literature - and are clear on the rare occasions when they venture beyond that their comments are speculative;
4) The author does not disagree with the scientific consensus in their blog posts.
The last is simply a measure of appropriate humility - a recognition that they also can make mistakes, and as they are not expert, may make fundamental errors. That does not mean such people should not criticize the consensus. On the contrary, it means only that they should have the courage of their convictions and criticize the consensus in the peer reviewed literature, and only after that criticism stands the test of time should they present it in their blogs.
There are a number of excellent blogs that meet all these criteria, and there is nothing wrong with drawing on the particular expertise of those authors when it exceeds that of a particular SkS author in writing an article.
McIntyre has spent much of the last 8-9 years submitting papers and blogging against Mann et al and similar papers (without much justification or traction, and multiple debunkings, mind you) - that should be sufficient grounds regarding McIntyre to take what he writes with a large grain of salt.
Thanks for the excellent and informative paper. I will certainly take the recommendations into account from now on.
If I may add one thought: I think it is very important to realize that things that are obvious to yourself are not necessarily obvious to others. Assessing the amount of knowledge your discussion partner possesses is very important.
I can remember only a couple of years ago, when I didn’t know much about climate change, I had the vague notion: “Yes the earth is probably warming, but it’s not absolutely sure yet and it may not be all bad”. If at that time some maniac foaming at the mouth had shouted at me: “Beware! The end of the world is near!” I would not be inclined to believe him either.
I am absolutely convinced that professional climate deniers – people who deliberately defend a viewpoint they don’t believe themselves - are the biggest criminals on the planet, even worse than terrorists since they do more damage than any terrorist could ever achieve. Their lies will cost millions of lives. They are deliberately squandering the future of our planet in exchange for some money. But when dealing with normal people who don’t believe in climate change, you are just talking to people who have formed a judgement based on what they know/what they read. It is not useful to insult people – I just think back of how I thought just a couple of years ago (although I never was a climate skeptic).
Some questions that weren’t answered by the paper, that I am sometimes struggling with:
- What to do if someone gives you several myths in one go ?
An example: -START OF MYTH-"The IPCC is untrustworthy since they have falsified studies. Yes, the climate changes but it has always changed. The influence of mankind on the amount of CO2 is really minimal. Besides, CO2 is not even the strongest greenhouse gas. If humans are responsible for the earth’s warming, are they also responsible for the warming of the other planets in the solar system ? The sun has a much larger influence on the climate than we thought.”-END OF MYTH-
If I try to answer these myths separately, people lose patience. If I don’t answer certain myths people continue believing that some of it is true.
- How strong should my message be ? The paper recommends not to use dramatic language, otherwise people will feel offended and will reject the statement right away. If however I try to convey the message in a nuanced and non-frightening way, I am also giving the wrong impression, since people might get the impression that the problem is not that urgent or the consequences are not going to be that bad.
- The paper also mentions that people who are less fixed in their minds can be convinced more easily using information. If you argue with someone on a public forum, is it your first aim to convince your discussion partner, or do you have a better chance trying to convince the people who are reading along ? Your discussion partner may be stuck so much in his convictions that he will never change his mind.
(Your results may vary)
One thing that might be worth considering is kindle (or other ebook) versions of this and The Scientific Guide to
Global Warming Skepticism. I find the kindle to be a much more convenient way of reading stuff, but most ebook readers don't do a great job with PDF file.
Might even be worth while to sell them for 99 cents to put some money back into the SkS coffers.
Either way, I might be able to try and create a kindle version, but I would need a non-pdf copy of the guide to take a crack at it. And since it I will soon be spending 6 weeks travelling mostly out out of internet range I might not be able to get it done until I get back.
Just a thought.
cheers
The gap created by this debunking is the question “how can there be a 97% consensus if 31,000 scientists dissent?” This gap if filled by explaining that almost all the scientists in the Petition Project are not climate scientists.
Please replace
"This gap if filled..."
with
"This gap is filled..."
Please follow your own advice :)
http://not-that-sane.blogspot.com/2011/11/how-to-debunk-myth.html
I'll be curious to hear others' opinions on whether we nailed the key points. The post at http://www.climatebites.org/2011/11/30/climate-change-rebuttalmust-read-the-debunking-handbook-from-skeptical-science/
I happen to have degrees in cognitive psychology and computer science; I'm familiar with a common misconception that human memory works like computer memory. It is not, at all. When you write to computer memory, it is like flipping a switch; whatever information was there before is erased and the new information is put in its place. There is no "memory" of what was there before. However, brains are organic by nature; memory works as associations between neurons and these have to be "grown". I've been out of the field for some time, but, to my knowledge, there is no mechanism to erase prior associations. Whatever the strongest association is becomes "the memory", and whatever weak ones may be present become some barely conscious addenda. It takes some repetition and active processing to develop a new connection to the point where it is stronger than the old one. I think that using this as a working model of brain behavior might help with understanding and predicting how it can be difficult to change someone's mind about an issue.
For instance, with the one-myth, many-facts scenario, you have one strong pattern of association, and the multitude of weak associations never receive enough processing resources individually to overcome the strong one. The loudest demon metaphor comes to mind. I suspect that it applies to more that just our perception neurons.
Changing a set of firmly entrenched, interwoven beliefs, can be like trying to create a paradigm shift. Sometimes it takes a traumatic event to cause such a shift. For example, young people often have the sense that nothing bad will happen to them because nothing really bad has ever happened to them before. Often, nothing short of something like getting in a car wreck, breaking a leg, or getting arrested and thrown in jail will convince them otherwise.
I have some curiosity to look up other works by Professor Lewandowsky.
The term "belief perseverance" comes to mind. (references available through Google). One of the more effective ways to overcome this is to get the other person to try to imagine as if some bit of information were true, and then build an logical argument from there. I think it is important to remember that the other person is likely to honestly believe the incorrect information, and refrain from accusations otherwise.
Terms like "climate Armageddon" are entirely unhelpful to the discussion, and rather inaccurate. While the effects of climate change will be very bad for a lot of people, and while if really pushed to an extreme civilization may find itself being refashioned (as has happened, one should note, frequently and at an accelerating pace since the end of the dark ages), "Armageddon" is not an applicable term.
We discuss "climate change," not climate "Armageddon" or "catastrophic climate change" or "catastrophic anthropogenic global warming" or any other unnecessarily dramatic and overwrought superlative.
Note also that this site is not about "selling" a view point. This site is about explaining the science. The truth behind the science is all that one needs to make a rational decision about any course of action (or inaction).
Perhaps SkS can team up with these guys.
Glasgow is clearly a civilized place. We don't often have science lectures in pubs in the US.
Hello, the link to the english version of the debunking handbook does not seem to work for me? cheers
[DB] Curious. Try this one:
http://www.skepticalscience.com/docs/Debunking_Handbook.pdf
Fantastic information but I would also like to request either a Kindle version or at least abandon the two column format. It's pretty hard to read this pdf on a Kindle or tablet.