Opinion poll

Post Debate Polls: Why You Shouldn t Believe Most Online Polls

• Report rights infringement
• published: 28 Sep 2016
• views: 10

Post Debate Polls: Why You Shouldn t Believe Most Online Polls On Tuesday morning, Donald J. Trump boasted of what he described as widespread approval of his performance in the first presidential debate with Hillary Clinton. Those so-called “final debate polls” consisted of readers on news sites who were asked for their opinion of who had won. While it looked good for Mr. Trump, pollsters and some journalists offered a protest: Informal, unscientific “polls” on news sites produce junk data that does not indicate how the public actually feels, and should not be believed as an indication of — well, much of anything. “Those do a good job of engaging audiences online, and they do a good job of letting you know how other people who have come to the webpage feel about whatever issue,” said Mollyann Brodie, the executive director for public opinion and survey research at the Kaiser Family Foundation. “But they’re not necessarily good at telling you, in general, what people think, because we don’t know who’s come to that website and who’s taken it.” Professional pollsters use scientific statistical methods to sure that their small random samples are demographically appropriate to indicate how larger groups of people think. Online polls do nothing of the sort, and are not random, allowing anyone who finds the poll to vote. They are thus opento manipulation from those who would want to stuff the ballot box. Users on Reddit 4chan directed masses of people to vote for Mr. Trump in the instant-analysis surveys, according to The Daily Dot. Similar efforts were observed on Twitter and other sites. Even when there is no intentional manipulation, the results are largely a reflection of who is likely to come to a particular site and who would be motivated enough to participate. Intuitively, it’s no surprise that readers of sites like Breitbart News and the Drudge Report would see Mr. Trump as the winner, just as Mrs. Clinton would be more likely to find support on liberal sites. It’s not just in politics; take the question of identifying the best undefeated team in the N.F.L. after three weeks. The New England Patriots and the Denver Broncos are likely the most obvious candidates for football fans, but when a news site in Philadelphia asked the question on Sunday after a big win by the Eagles? You can guess what happened. While that’s an extreme case of hometown bias, similar factors are at play on every website. Frank Newport, the editor in chief of Gallup, said most reader response surveys n news sites are done more to benefit the news organization than to educate the public. “In our business, the key is generalizability,” he said, referring to the ability of a sample group to apply to a wider population. “That’s the core of what we do. Typically, it takes a lot of time, and a lot of effort, and a lot of money to do it.” The New York Times conducts scientific polling and generally avoids unscientific reader polls on its website. The New York Times Magazine asks a selection of its readers offbeat questions, but each edition discloses the unscientific methodology. Confusing matters is the rise of online polls that attempt to use scientific methodology. Those are also causing concern in the profession. Andrew Seaman, the ethics committee chairman for the Society of Professional Journalists, said the common use of online opt-in polling warranted a review in newsrooms. In a hypothetical situation, he imagined a local news site that crowned the best pizza shop based on 13 responses in a city of 10,000 people. It might seem benign, but other pizza shop owners could lose business — and the stakes would be higher in more significant cases, like a presidential election. “My advice would be to avoid opt-in polls that provide little value to the public,” Mr. Seaman said in an email. “Otherwise, the news organizations run the risk of being irresponsible and possibly harmful.” Does the public understand what makes a poll educational and what makes one just for fun? Mr. Newport said he couldn’t quite answer. “I would need a scientific poll to tell you that,” he said.