Fact checking
Journalism |
---|
![]() |
Areas |
Genres |
Social impact |
News media |
Roles |
Fact checking is the act of checking factual assertions in non-fictional text in order to determine the veracity and correctness of the factual statements in the text. This may be done either before (ante hoc) or after (post hoc) the text has been published or otherwise disseminated.[1]
Ante hoc fact-checking (fact checking before dissemination) aims to remove errors and allow text to proceed to dissemination (or to rejection if it fails confirmations or other criteria). Post hoc fact-checking is most often followed by a written report of inaccuracies, sometimes with a visual metric from the checking organization (e.g., Pinocchios from The Washington Post Fact Checker, or TRUTH-O-METER ratings from PolitiFact). Several organizations are devoted to post hoc fact-checking, such as FactCheck.org and PolitiFact.
Research on the impact of fact-checking is relatively recent but the existing research suggests that fact-checking does indeed correct misperceptions among citizens, as well as discouraging politicians from spreading misinformation.
Contents
Post hoc fact-checking[edit]
Consistency across fact-checkers[edit]
One study finds that fact-checkers PolitiFact, FactCheck.org, and Washington Post's Fact Checker overwhelmingly agree on their evaluations of claims.[2][3]
However, a study by Morgan Marietta, David C. Barker and Todd Bowser found "substantial differences in the questions asked and the answers offered." They concluded that this limited the "usefulness of fact-checking for citizens trying to decide which version of disputed realities to believe."[4]
A paper by Chloe Lim, Ph.D. student at Stanford University, finds little overlap in the statements that fact-checkers check. Out of 1065 fact-checks by PolitiFact and 240 fact-checks by The Washington Post's Fact-Checker, there were only 70 statements that both fact-checkers checked. The study found that the fact-checkers gave consistent ratings for 56 out of 70 statements, which means that one out every five times, the two fact-checkers disagree on the accuracy of statements.[5]
Effects[edit]
Studies of post hoc fact checking have made clear that such efforts often result in changes in the behavior, in general, of both the speaker (making them more careful in their pronouncements) and of the listener or reader (making them more discerning with regard to the factual accuracy of content); observations include the propensities of audiences to be completely unswayed by corrections to errors regarding the most divisive subjects, or the tendency to be more greatly persuaded by corrections of negative reporting (e.g., "attack ads"), and to see minds changed only when the individual in error was someone reasonably like-minded to begin with.[6]
Correcting misperceptions[edit]
A 2015 study found evidence a "backfire effect" (correcting false information may make partisan individuals cling more strongly to their views): "Corrective information adapted from the Centers for Disease Control and Prevention (CDC) website significantly reduced belief in the myth that the flu vaccine can give you the flu as well as concerns about its safety. However, the correction also significantly reduced intent to vaccinate among respondents with high levels of concern about vaccine side effects--a response that was not observed among those with low levels of concern."[7] A 2017 study attempted to replicate the findings of the 2015 study but failed to do so.[8]
A 2016 study found little evidence for the "backfire effect": "By and large, citizens heed factual information, even when such information challenges their partisan and ideological commitments."[9] A study of Donald Trump supporters during the 2016 race similarly found little evidence for the backfire effect: "When respondents read a news article about Mr. Trump’s speech that included F.B.I. statistics indicating that crime had “fallen dramatically and consistently over time,” their misperceptions about crime declined compared with those who saw a version of the article that omitted corrective information (though misperceptions persisted among a sizable minority)."[10]
Studies have shown that fact-checking can affect citizens' belief in the accuracy of claims made in political advertisement.[11] A paper by a group of Paris School of Economics and Sciences Po economists found that falsehoods by Marine Le Pen during the 2017 French presidential election campaign (i) successfully persuaded voters, (ii) lost their persuasiveness when fact-checked, and (iii) did not reduce voters' political support for Le Pen when her claims were fact-checked.[12] A 2017 study in the Journal of Politics found that "that individuals consistently update political beliefs in the appropriate direction, even on facts that have clear implications for political party reputations, though they do so cautiously and with some bias... Interestingly, those who identify with one of the political parties are no more biased or cautious than pure independents in their learning, conditional on initial beliefs."[13]
A study by Yale University political scientists Gordon Pennycook and David G. Rand found that Facebook tags of fake articles "did significantly reduce their perceived accuracy relative to a control without tags, but only modestly".[14] A Dartmouth study led by Brendan Nyhan found that Facebook tags had a greater impact than the Yale study found.[15] A "disputed" tag on a false headline reduced the number of respondents who considered the headline accurate from 29% to 19%, whereas a "rated false" tag pushed the number down to 16%.[15] The Yale study found evidence of a backfire effect among Trump supporters younger than 26 years whereby the presence of both untagged and tagged fake articles made the untagged fake articles appear more accurate.[14]
Based on the findings of a 2017 study in the journal Psychological Science, the most effective ways to reduce misinformation through corrections is by:[16]
- limiting detailed descriptions of / or arguments in favor of the misinformation;
- walking through the reasons why a piece of misinformation is false rather than just labelling it false;
- presenting new and credible information which allows readers to update their knowledge of events and understand why they developed an inaccurate understanding in the first place;
- using video, as videos appear to be more effective than text at increasing attention and reducing confusion, making videos more effective at correcting misperception than text.
A forthcoming study in the Journal of Experimental Political Science found "strong evidence that citizens are willing to accept corrections to fake news, regardless of their ideology and the content of the fake stories."[17]
Political discourse[edit]
A 2015 experimental study found that fact-checking can encourage politicians to not spread misinformation. The study found that it might help improve political discourse by increasing the reputational costs or risks of spreading misinformation for political elites. The researchers sent, "a series of letters about the risks to their reputation and electoral security if they were caught making questionable statements. The legislators who were sent these letters were substantially less likely to receive a negative fact-checking rating or to have their accuracy questioned publicly, suggesting that fact-checking can reduce inaccuracy when it poses a salient threat."[18]
Political preferences[edit]
One experimental study found that fact-checking during debates affected viewers' assessment of the candidates' debate performance and "greater willingness to vote for a candidate when the fact-check indicates that the candidate is being honest."[19]
A study of Trump supporters during the 2016 presidential campaign found that while fact checks of false claims made by Trump reduced his supporters' belief in the false claims in question, the corrections did not alter their attitudes towards Trump.[20]
Controversies and criticism[edit]
Political fact-checking is sometimes criticized as being opinion journalism.[21][22] In September 2016, a Rasmussen Reports national telephone and online survey found that "just 29% of all Likely U.S. Voters trust media fact-checking of candidates’ comments. Sixty-two percent (62%) believe instead that news organizations skew the facts to help candidates they support."[23][24]
Organizations and individuals[edit]
This section may rely excessively on sources too closely associated with the subject, potentially preventing the article from being verifiable and neutral. (July 2015) (Learn how and when to remove this template message)
|
The Reporters’ Lab at Duke University maintains a database of fact checking organizations that is managed by Mark Stencel and Bill Adair. The database tracks more than 100 non-partisan organizations around the world. The Lab's inclusion criteria is based on whether the organization
- examines all parties and sides;
- examines discrete claims and reaches conclusions;
- tracks political promises;
- is transparent about sources and methods;
- discloses funding/affiliations;
- and whether its primary mission is news and information.[25]
Africa[edit]
- Africa Check:[26] Africa's first independent fact-checking organisation with offices in Kenya, Nigeria, South Africa, Senegal and the UK checking claims made by public figures and the media in Africa.
India[edit]
- Boom is a fact checking digital journalism website.[27][28][29]
- SMHoaxSlayer is a broad spectrum fact checking website with verifying social media hoaxes and scams circulating in india.[30][27][28][29]
Japan[edit]
- GoHoo: Launched by a nonprofit association Watchdog for Accuracy in News-reporting, Japan (WANJ or 一般社団法人 日本報道検証機構) on November 16, 2014. Crowd-funded approx. 1.6 million yen through Ready For.[31] Awarded Social Business Grand Prize 2012 Summer.[32]
- Japan Center of Education for Journalists (JCEJ): Fosters journalists and fact checkers by referring to a Journalist's Guide to Social Sources published by First Draft News, a project of the Harvard Kennedy School’s Shorenstein Center. JCEJ itself also debunks falsehoods.[33]
Europe[edit]
- BBC Reality Check[34]
- Full Fact:[35] An independent fact checking organisation based in the UK which aims to "promote accuracy in public debate", launched in 2009.
- The FactCheck blog:[36] A fact checking blog run by the Channel 4 News organization in the UK.
- Les Décodeurs:[37] French fact-checking blog run by Le Monde.
- Pagella Politica:[38] an Italian fact-checking website.
- Ferret Fact Service:[39] Scotland's first fact-checker launched in April 2017[40] after a grant from the Google Digital News Initiative.[41]
Latin America[edit]
- Argentina: Chequeado.com[42]
- Central America: Rete al candidato[43]
United States[edit]
- FactCheck.org and FactCheckEd.org: non-partisan,[44] nonprofit sister websites that are self-described "advocates for voters that aims to reduce the level of deception and confusion in U.S. politics," and serving as an educational resource for high school teachers and students, respectively (the latter founded 2005). They are projects of the Annenberg Public Policy Center of the Annenberg School for Communication at the University of Pennsylvania, and are funded primarily by the Annenberg Foundation.
- Fact Checker (The Washington Post):[45] A project of The Washington Post, known for grading politicians on the factual accuracy of their statements with one to four "Pinocchios."[46] Created September 2007 by Post diplomatic writer Michael Dobbs specifically for the 2008 presidential campaign.[47] Ceased operation 4 November 2008,[48] but relaunched with a broader focus in January 2011, led by veteran Post diplomatic correspondent Glenn Kessler.[49]
- PolitiFact.com:[50] A service of the Tampa Bay Times - Created August 2007, uses the "Truth-o-Meter" to rank the amount of truth in public persons' statements. 2009 Pulitzer Prize Winner.
- Snopes.com focuses on, but is not limited to, validating and debunking urban legends and other stories in American popular culture.
- TruthOrFiction.com validates and debunks urban legends, Internet rumors, e-mail forwards, and other stories of unknown or questionable origin.
Ante hoc fact-checking[edit]
Among the benefits of printing only checked copy is that it averts serious, sometimes costly, problems, e.g. lawsuits and discreditation. Fact checkers are primarily useful in catching accidental mistakes; they are not guaranteed safeguards against those who wish to commit journalistic frauds
The possible societal benefit of honing the fundamental skill of fact checking has been noted in a round table discussion by Moshe Benovitz, who observes that "modern students use their wireless worlds to augment skepticism and to reject dogma," but goes on to argue that this has positive implications for values development. He argues:
"We can encourage our students to embrace information and vigorously pursue accuracy and veracity. Fact checking can become a learned skill, and technology can be harnessed in a way that makes it second nature… By finding opportunities to integrate technology into learning, students will automatically sense the beautiful blending of… their cyber… [and non-virtual worlds]. Instead of two spheres coexisting uneasily and warily orbiting one another, there is a valuable experience of synthesis…".[51]
He closes, noting that this constitutes "new opportunities for students to contribute to the discussion like never before, inserting technology positively into academic settings" (rather than it being seen as purely as agent of distraction).[51]
Controversy[edit]
One journalistic controversy is that of admitted and disgraced reporter and plagiarist Stephen Glass, who began his journalism career as a fact-checker. The fact checkers at The New Republic and other weeklies for which he worked never flagged the numerous fictions in Glass's reporting. Michael Kelly, who edited some of Glass's concocted stories, blamed himself, rather than the fact-checkers, saying: "Any fact-checking system is built on trust ... If a reporter is willing to fake notes, it defeats the system. Anyway, the real vetting system is not fact-checking but the editor." [52]
Individuals[edit]
- Sarah Harrison Smith spent some time and also headed the fact checking department for The New York Times. She is the author of the book, The Fact Checker's Bible.
- Jim Fingal worked for several years as a fact-checker at The Believer and McSweeney's and is co-author with John D'Agata of The Lifespan of a Fact which is an inside look of a battle between himself as fact-checker and author D'Agata regarding one of his essays that pushes the limits of "artistic license" that is acceptable of a non-fiction work.
Alumni of the role[edit]
The following is a list of individuals for whom it has been reported, reliably, that they have played such a fact checking role at some point in their careers, often as a stepping point to other journalistic endeavors, or to an independent writing career:
- Susan Choi – American novelist[53]
- Anderson Cooper – Television anchorman.[54]
- Esther Dyson – technologist[citation needed]
- Nancy Franklin – New Yorker staff writer[55]
- William Gaddis – American novelist[56]
- Virginia Heffernan – New York Times television critic[57]
- Roger Hodge – Former editor, Harper's Magazine[58]
- David D. Kirkpatrick – New York Times reporter[59]
- Daniel Menaker – Former editor-in-chief at Random House[60]
- David Rees – cartoonist
- Sean Wilsey – McSweeney's Editor and memoirist[61]
See also[edit]
Further reading[edit]
- Silverman, Craig (23 October 2007). Regret The Error: How Media Mistakes Pollute The Press And Imperil Free Speech. Penguin Canada. ISBN 9780143186991.
- Amazeen, Michelle (2015) "Monkey Cage: Sometimes political fact-checking works. Sometimes it doesn’t. Here’s what can make the difference.," The Washington Post (online), 3 June 2015, see,[62] accessed 27 July 2015.
- Davis, Katy (2012) "Study: Fact-checkers disagree on who lies most," The Center for Media and Public Affairs (CMPA), George Mason University (online, press release), 22 October 2012 see,[63]
- Lewis-Kraus, Gideon (2012) "RIFF: The fact-checker versus the fabulist," The New York Times Magazine (online), 21 February 2012 (print edition, 26 February 2012, p. MM45, title, "I Have Taken Some Liberties"), see,[64]
- Heffernan, Virginia (2010) "The Medium: What 'fact-checking' means online," The New York Times Magazine (online), 20 August 2010 (print edition, 22 August 2010, p. MM14). Accessed 27 July 2015.
- Silverman, Craig (2010) "Top fact checkers and news accuracy experts gather in Germany," Regret the Error (online), 4 September 2010, see,[65] accessed 28 July 2015. Cited by Tobias Reitz & Kersten Alexander Riechers (2011) Quo vadis Qualitätssicherung? Corrigo, Konzeption eines Crowdsourced Media Accountability Services," p. 151, Fachbereich Media, 31 May 2011 (Hochschule Darmstadt, University of Applied Sciences), see,[66] accessed 28 July 2015.
References[edit]
- ^ "Ante hoc – Oxford Reference". doi:10.1093/acref/9780195369380.001.0001/acref-9780195369380-e-221. Archived from the original on 21 February 2015.
- ^ Amazeen, Michelle A. (1 October 2016). "Checking the Fact-Checkers in 2008: Predicting Political Ad Scrutiny and Assessing Consistency". Journal of Political Marketing. 15 (4): 433–464. doi:10.1080/15377857.2014.959691. ISSN 1537-7857.
- ^ Amazeen, Michelle A. (2 January 2015). "Revisiting the Epistemology of Fact-Checking". Critical Review. 27 (1): 1–22. doi:10.1080/08913811.2014.993890. ISSN 0891-3811.
- ^ Marietta, Morgan; Barker, David C.; Bowser, Todd (2015). "Fact-Checking Polarized Politics: Does The Fact-Check Industry Provide Consistent Guidance on Disputed Realities?" (PDF). The Forum. 13: 577. Retrieved 27 September 2016.
- ^ "Checking how fact-checkers check".
- ^ Amazeen, Michelle (2015) "Monkey Cage: Sometimes political fact-checking works. Sometimes it doesn’t. Here’s what can make the difference.," The Washington Post (online), 3 June 2015, see [1], accessed 27 July 2015.
- ^ Nyhan, Brendan; Reifler, Jason (9 January 2015). "Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information". Vaccine. 33 (3): 459–464. doi:10.1016/j.vaccine.2014.11.017. ISSN 1873-2518. PMID 25499651.
- ^ Haglin, Kathryn (1 July 2017). "The limitations of the backfire effect". Research & Politics. 4 (3): 2053168017716547. doi:10.1177/2053168017716547. ISSN 2053-1680.
- ^ Wood, Thomas; Porter, Ethan (5 August 2016). "The Elusive Backfire Effect: Mass Attitudes' Steadfast Factual Adherence". Rochester, NY: Social Science Research Network. SSRN 2819073
.
- ^ Nyhan, Brendan (5 November 2016). "Fact-Checking Can Change Views? We Rate That as Mostly True". The New York Times. ISSN 0362-4331. Retrieved 5 November 2016.
- ^ Fridkin, Kim; Kenney, Patrick J.; Wintersieck, Amanda (2 January 2015). "Liar, Liar, Pants on Fire: How Fact-Checking Influences Citizens' Reactions to Negative Advertising". Political Communication. 32 (1): 127–151. doi:10.1080/10584609.2014.914613. ISSN 1058-4609.
- ^ Rodriguez, Barrera; David, Oscar; Guriev, Sergei M.; Henry, Emeric; Zhuravskaya, Ekaterina (18 July 2017). "Facts, Alternative Facts, and Fact Checking in Times of Post-Truth Politics". Rochester, NY.
- ^ Hill, Seth J. (16 August 2017). "Learning Together Slowly: Bayesian Learning about Political Facts". The Journal of Politics: 000–000. doi:10.1086/692739. ISSN 0022-3816.
- ^ a b Pennycook, Gordon; Rand, David G. (12 September 2017). "Assessing the Effect of "Disputed" Warnings and Source Salience on Perceptions of Fake News Accuracy". Rochester, NY.
- ^ a b Nyhan, Brendan (2017-10-23). "Why the Fact-Checking at Facebook Needs to Be Checked". The New York Times. ISSN 0362-4331. Retrieved 2017-10-23.
- ^ Chokshi, Niraj (18 September 2017). "How to Fight 'Fake News' (Warning: It Isn't Easy)". The New York Times. ISSN 0362-4331. Retrieved 19 September 2017.
- ^ Porter, Ethan; Wood, Thomas; Kirby, David (2017-11-17). "Sex Trafficking, Russian Infiltration, Birth Certificates, and Pedophilia: A Survey Experiment Correcting Fake News". Rochester, NY.
- ^ Nyhan, Brendan; Reifler, Jason (1 July 2015). "The Effect of Fact-Checking on Elites: A Field Experiment on U.S. State Legislators". American Journal of Political Science. 59 (3): 628–40. doi:10.1111/ajps.12162. ISSN 1540-5907.
- ^ Wintersieck, Amanda L. (5 January 2017). "Debating the Truth". American Politics Research. doi:10.1177/1532673x16686555.
- ^ "Taking Corrections Literally But Not Seriously? The Eects of Information on Factual Beliefs and Candidate Favorability" (PDF).
- ^ Riddell, Kelly (26 September 2016). "Eight examples where 'fact-checking' became opinion journalism". Washington Times. Retrieved 27 September 2016.
- ^ Graves, Lucas (2016). Deciding What's True: The Rise of Political Fact-Checking in American Journalism. Columbia University Press. p. 27. Retrieved 27 September 2016.
- ^ Reports, Rasmussen. "Voters Don't Trust Media Fact-Checking - Rasmussen Reports™". Retrieved 17 October 2016.
- ^ Lejeune, Tristan (30 September 2016). "Poll: Voters don't trust media fact-checkers". Retrieved 17 October 2016.
- ^ "How We Identify Fact-Checkers – Duke Reporters' Lab". 22 June 2016. Retrieved 14 September 2016.
- ^ Lyman, Rick (23 July 2013). "Nonpartisan Fact-Checking Comes to South Africa". New York Times.
- ^ a b "Fake News Warriors". hindustantimes.com/. Retrieved 30 August 2017.
- ^ a b "In an era of fake news, check out India's homegrown Hoax Slayer". hindustantimes.com/. 13 January 2017. Retrieved 30 August 2017.
- ^ a b "Boom! This team of four journalists is on a mission to hunt down and destroy fake news in India | FactorDaily". FactorDaily. 28 July 2017. Retrieved 30 August 2017.
- ^ "This 'Hoax Slayer' has busted 1,300 fake forwards on WhatsApp". YourStory.com. 1 April 2017.
- ^ "私たちは、マスメディアの情報の真偽をチェックするニュースメディア、「メディアのメディア」を創ります!". Ready For. Retrieved 2017-11-01.
- ^ "ソーシャルビジネスグランプリ 2012夏 結果発表". Social Entrepreneur School. Retrieved 2017-11-01.
- ^ Kajimoto, Masato (2017-10-30). "A snap election (and global worries over fake news) spur fact-checking collaborations in Japan". Nieman Foundation at Harvard. Retrieved 2017-11-01.
- ^ Jackson, Jasper (12 January 2017). "BBC sets up team to debunk fake news". The Guardian. ISSN 0261-3077. Retrieved 16 September 2017.
- ^ "Full Fact". FullFact.org. 15 February 2012. Retrieved 15 February 2012.
- ^ "The FactCheck Blog". Channel 4. 15 February 2012. Retrieved 15 February 2012.
- ^ "Fact-checking blogs turn up heat on French candidates". France 24. Retrieved 2 August 2013.
- ^ "Italian politics: Pinocchio's heirs". The Economist. 22 February 2013. Retrieved 2 August 2013.
- ^ "Ferret Fact Service". 4 April 2017.
- ^ "Journalism.co.uk interview". 28 April 2017.
- ^ "Google DNI: The Ferret". 19 September 2017.
- ^ "Chequeado.com: Fiel defensor de los hechos". Lanacion.com. Retrieved 2 August 2013.
- ^ "El Financiero lanzó aplicación para retar a los candidatos presidenciales". elfinancierocr.com. Retrieved 17 March 2014.
Rete al candidato is the first political fact checking digital platform in Central America. It is based in Costa Rica and was launched in 2013 by the weekly newspaper El Financiero to monitor the political debate of the 2014 presidential elections in that country. It is supported by the Konrad Adenauer Foundation.
- ^ "About Us". FactCheckED.org. Retrieved 28 July 2015.
- ^ Rucker, Philip. "Fact Checker". Washingtonpost.com. Retrieved 7 June 2009.
- ^ Kessler, Glenn. "About the Fact Checker". Blog.washingtonpost.com. Retrieved 7 June 2009.
- ^ "washingtonpost.com Launches "FactChecker"". Findarticles.com. 2007. Archived from the original on 8 July 2012. Retrieved 7 June 2009.
- ^ Rucker, Philip. "Fact Checker". Voices.washingtonpost.com. Retrieved 7 June 2009.
- ^ Kessler, Glenn (19 July 2012). "Welcome to the new Fact Checker". The Washington Post.
- ^ "St. Petersburg Times Online". Politifact.com. Retrieved 7 June 2009.
- ^ a b Moshe Benovitz et al., 2012, "Education: The Social Media Revolution: What Does It Mean for Our Children?" Jewish Action (online), August 24, 2012, New York, NY, USA:Orthodox Union, see [2], accessed 28 July 2015.
- ^ John Watson (2017-04-02). "What is Fact Checking? – FactCheck Sri Lanka". Factchecksrilanka.com. Retrieved 2017-12-07.
- ^ "An Interview With Susan Choi". Archived from the original on 18 February 2001. Retrieved 18 November 2006.
- ^ "CNN.com – Transcripts". Transcripts.cnn.com. 1 June 2006. Retrieved 18 October 2011.
- ^ "Contributors". Archived from the original on 19 March 2006. Retrieved 17 November 2006.
- ^ "William Gaddis (American author)". Britannica.com. Retrieved 18 October 2011.
- ^ Skurnick, Lizzie. "Content". Mediabistro.com. Retrieved 18 October 2011.
- ^ "Hodge, Roger D". Archived from the original on 8 March 2007. Retrieved 18 November 2006.
- ^ Kirkpatrick, David D. "David Kirkpatrick". The New York Times.
- ^ "Swarthmore College Bulletin". Swarthmore.edu. July 2011. Archived from the original on 27 October 2008. Retrieved 18 October 2011.
- ^ "Sean Wilsey – About Sean Wilsey – Penguin Group". Us.penguingroup.com. Archived from the original on 27 September 2011. Retrieved 18 October 2011.[verification needed]
- ^ Amazeen, Michelle (14 December 2012). "Sometimes political fact-checking works. Sometimes it doesn't. Here's what can make the difference". The Washington Post. Retrieved 28 July 2015.
- ^ "Study: Fact-Checkers Disagree on Who Lies Most". Archived from the original on 9 March 2015. Retrieved 28 August 2015. . Accessed 28 July 2015.
- ^ Lewis-Kraus, Gideon (21 February 2012). "The Fact-Checker Versus the Fabulist". The New York Times. Retrieved 27 July 2015. (Subscription required (help)).
- ^ "Archived copy". Archived from the original on 8 September 2015. Retrieved 28 July 2015.
- ^ "Wayback Machine" (PDF). Web.archive.org. Archived from the original on 6 September 2015. Retrieved 7 December 2017.