Review of Levitt and Dubner's "Think like a Freak" in Barron's: "Beware of Populist Economics"

This is a copy of my review from Barron's:

The new lessons from the Freakonomics guys are compelling, but deeply flawed. 

Reviewed by John R. Lott Jr. 

The Freakonomics franchise certainly has legs. According to legions of admirers, in their best-selling series that includes Freakonomics, SuperFreakonomics, and now Think Like a Freak, University of Chicago economics professor Steven Levitt and journalist Stephen Dubner have taught us to use economic reasoning to shed light on real-life situations.  In the process, they have also shown that economics can be fun. 

But the fun quotient is ultimately diminished by the fact that their stock in trade is naive economics. Typically, Levitt and Dubner fail to understand that when a problem arises in a market, it generally provides an incentive for those involved to remedy the problem.      Take the sour-lemon story. "A new car that was bought for $20,000," they assured us in Freakonomics, "cannot be resold for more than perhaps $15,000. Why? Because the only person who might logically want to resell a brand-new car is someone who found the car to be a lemon. So if the car isn't a lemon, a potential buyer assumes that it is."  

Stories like these have clearly appealed to those who enjoy clever portrayals of a dysfunctional world. But a little research would have revealed that, contrary to Levitt and Dubner, used cars with only a few thousand miles on them sell for almost the same price as when new. One obvious reason: Since car manufacturers allow warranties to be transferred to new owners, potential buyers know that even if they do buy a lemon, they will not be stuck with it. 

In Think Like a Freak, the authors promise to teach us what "it takes [to be] a truly original thinker." But rather than promoting original or critical thought, their book tries to convince us of one main thing: People are stupid. We are thus confronted with half-baked theories similar to those in their previous books. 

Take the example that Think Like a Freak starts out with: soccer players in the World Cup doing what is best for their own reputations rather than what is best for their team. We are told that, when a player kicks penalty shots, aiming toward the center has a better chance of success, but that fear of shame prevents players from doing so. The potential shame of kicking the ball right into the hands of a goalie standing in the middle of the goal, especially during the World Cup, keeps players from doing what is best for the team. 

The data cited to support this view are a bit rough. We are informed that "only 17% of kicks are aimed" at the center of the goal, even though "75% of penalty kicks at the elite level are successful." But the real problem is that, per their usual habit, the authors assume no one else involved is smart enough to detect this cheating. If, by kicking the ball to the side, players really are failing to score, it defies belief that team owners and coaches would be blind to this abuse and allow it to continue. 

Since the team's gain from winning is far greater than any shame the player risks, incentives can be used to make sure that players do what is best for the team. Stiff financial penalties can be imposed, including the penalty of being fired from the team. Then there are other kinds of possible shame, meted out to these players in front of other team members, for not serving the interests of the team. 

In the ivory-tower world of Levitt and Dubner, however, owners and coaches are completely ignored, since including them would only ruin a clever insight. I contacted actual soccer coaches at three different colleges, and found that they did not agree with the authors' premise that the center shot in a penalty kick is the best strategy. Not surprisingly, then, players seem to obey their coaches' dictates on penalty kicks. . . .

The rest of the review is available here.

________________________________

Space constraints limited what I was able to write.  For those interested, here is part of what I had originally included in the review that I sent in.

Or take their discussion about wines.  People supposedly keep buying the expensive wine even though they really can’t tell which is the premium wine and the cheap stuff.  If you believe Levitt and Dubner, these wine drinkers are making a mistake to pay more for the so-called higher quality wine. Again, possibly people are just self-deluded or dumb, but there are other possibilities.  Aged wines may not taste better, but simply different.  For example, it is costly to store wine for decades.  As wine ages, the tannins in the wine disappear and one can find out how the wine tastes uninhibited by the tannins.  Price differences would thus be due to the different costs of producing different wine, not a result of differences in demand. 

Alternatively, Professor Orley Ashenfelter at Princeton found that he could very accurately predict the price of wine by simply looking at the amount of winter and harvest rainfall and the average summer temperature in the vineyard.  If wine prices are random, how is it that prices can be predicted so accurately based on growing conditions?  

As a primary example of the “truly original” thinking Levitt and Dubner claim to have done, they point again to the assertion discussed in Freakonomics that liberalizing abortion lowers crime rates.  The problem with this bragging is that neither the basic idea that “unwanted children” who are brought up in bad environments that lead to crime was not a new idea nor was their test of it. 

 While these authors take credit for the idea, it is actually an old argument.  The 1972 Rockefeller Commission on Population and the American Future cited research purporting that the children of women denied an abortion didn’t get the attention that others received and “turned out to have been registered more often with psychiatric services, engaged in more antisocial and criminal behavior, and have been more dependent on public assistance.”  Roe v. Wade even discusses the consequences of “unwanted children” not getting the attention that they need. 

The research the commission cited went back even further.  For example, a 1966 study followed the lives of children born to 188 women who were denied abortions from 1939 to 1941 at the only hospital in Gothenburg, Sweden.  They compared the lives of these kids over the next twenty years to the next child of the same sex who was born after them.

Oral argument in Lott v. Levitt

The oral argument heard before the 7th Circuit Appeals Court can be found here.

Levitt's original correction letter.

More on "Dispute Over the Economics of File Sharing Intensifies"

David Glenn has a long discussion about the paper published in the Journal of Political Economy by Oberholzer-Gee and Koleman S. Strumpf on file sharing. Glenn has a useful and well-done discussion of the debate. For example, he notes:

Mr. Strumpf replies that most of Mr. Liebowitz's criticisms are trivial, even if correct. "Almost every point raised in Liebowitz's various pieces," he says, "involves incidental points which are not central to our conclusions."

Mr. Liebowitz, however, says his objections cast doubts over the entire study. Because he has not been able to scrutinize the OpenNap data at the heart of the study, his criticisms are largely aimed at a few non-OpenNap-based auxiliary tests that are presented at the end of the paper. But those tests are so weakly constructed, he says, that they call into question the validity of all the work.

One of the tests has to do with seasonal variations in record sales. "The number of file-sharing users in the United States drops 12 percent over the summer ... because college students are away from their high-speed Internet connections," Mr. Oberholzer-Gee and Mr. Strumpf write. Because of that seasonal dip, we might expect to see higher CD sales during the summer. But summer CD sales, as a proportion of the full year's sales, have not risen during the file-sharing era. Ergo, there doesn't seem to be much relationship between file sharing and CD sales.

That all sounds logical enough. But Mr. Liebowitz points out that one of the argument's premises—"the number of file-sharing users in the United States drops 12 percent over the summer"—is profoundly misleading.

The authors' citation is to a report on monthly file-sharing usage prepared by BigChampagne, a company that measures traffic on peer-to-peer networks. Mr. Liebowitz has a copy of the same report, which covers the period from August 2002 through May 2006. It's true that summer file sharing drops by an average of 12 percent (11.7 percent, to be precise) during the three summers—2003, 2004, 2005—covered in that report. But all of that effect comes from a severe drop in the summer of 2003, during a much-publicized wave of industry lawsuits against file sharers. During the summer of 2004, file sharing was flat, and during the summer of 2005, it actually rose slightly. So the test Mr. Oberholzer-Gee and Mr. Strumpf have set up—which is based on the ratio of summer-to-full-year CD sales—tells us nothing, Mr. Liebowitz says.

"If one my undergraduates did that, I would fail him," says Bruce D. McCullough, a professor of decision sciences at Drexel University who became interested in the dispute because he is a proponent of data transparency in economics publishing. "To take one decline, one flat, and one advance, and to suggest that it always goes down in the summer is just wrong." . . .

As too often seems to be the way of academic debate from people in part of the academy, this has a personal attack on Stan Liebowitz:

In an e-mail message to The Chronicle, Mr. Strumpf defends his work and suggests that Mr. Liebowitz's zeal stems from the fact that an academic center he directs, the Center for the Analysis of Property Rights and Innovation, receives grants from the Recording Industry Association of America and other commercial interests. "One might ask why Professor Liebowitz has remained so engrossed with our study," he writes.

I agree with Craig Newmark that Stan Liebowitz's concern that he couldn't redo their results and that they wouldn't share their data is enough to motivate many academics, including Liebowitz.

Other comments by CaveatBettor, Peter Klein, Josh Wright, and sivacracy.

The Controversy over how the Journal of Political Economy treated papers on Downloading Music

I have covered the controversy here. Stan Liebowitz has a follow up piece on the who debate here. His abstract notes:

Through a stroke of luck, a referee report in the review process at the JPE has been positively identified as the Oberholzer-Gee/Strumpf (O/S) response to my earlier comment. Regardless of the response's provenance, what counts is whether it solidly refuted my comment. This 'sequel' analyzes the O/S response. The O/S response only deals with four of the nine points discussed in my comment, leaving the five remaining critiques unchallenged. The conclusion of my review is that the O/S response fails as a defense of these four points and contains many of the same types of errors that marred their original paper. This sequel also discusses the history of this dispute including O/S' various reasons for not making their data available. Finally, this sequel provides full documentation on the JPE's decision not to publish the comment.

Did the Journal of the Political Economy make a serious mistake in accepting a paper?

John Palmer has an extremely lengthy excerpt from an investigative report on a paper on file sharing that was published by the JPE (for a previous post see here). This discussion involves a case where the editor was apparently told about severe data errors in the piece before publication, told that the authors were unwilling to share their data before publication and have still refused to do so (coming out with multiple conflicting reasons for not doing this), and questions about whether the rejection of a comment pointing out the data errors was done properly. When told that there was improper behavior with the data the editor's response was interpreted by Stan Liebowitz to mean this: "There appears to be no way within the profession to adjudicate such claims. In fact, the profession tends to operate as if all economists are truthful all the time."

Craig Newmark has his comments here. Peter Klein has comments here, who classifies this under the general category of "Academic Journal Fakery." Other posts here (Paul Walker) and here ("provocateurjim") also draw concerns over what happened in this case.

Lott v. Levitt: The Appeal with all the briefs filed

A couple of weeks ago I posted the original brief and Levitt's response in my Appeal in the Levitt case. Here is my final reply brief.

The Levitt correction letter is available here.

Lott v. Levitt: The Appeal

The settlement in the original lower court case resulted in Levitt's "correction letter" and allowed for me to appeal the decision on the first count. Here is a copy of my filing in the appeal and Levitt's response. This article on the Chronicle of Higher Education blog reviews the discussion, though I wish that it had mentioned the "e-mails," not just the "e-mail," and that the libel per quod had been brought up in the lower court, not just now. It would have been nice if the Chronicle reporter, David Glenn, had noted that the settlement with Levitt in the lower court preserved the right to appeal the first count.

The Problematic "famous file-sharing paper"

Craig Newmark has previously covered the problems with Felix Oberholzer-Gee and Koleman Strumpf's widely cited and discussed paper in the Journal of Political Economy entitled "The Effect of File Sharing on Record Sales: An Empirical Analysis." Stan Liebowitz had found numerous errors in the paper, but was not given access to their data. Unfortunately, under Steven Levitt's editorship at the JPE, this general type of claim has arisen previously. Anyway, Craig has a follow up posting on all this. Florenz Plassmann has translated an article from Handelsblatt, a leading German financial publication. This is what Craig posts from the article:

Recently, this paper has sparked a heated discussion. The relevance of the debate extends far beyond the paper in question. It questions the reliability of empirical studies in economics, and may ultimately challenge the way in which the crème de la crème of scientific journals deals with scientific evidence.

The key question is: how can a study that is based on secret data that nobody has double-checked be printed without close examination by one of the most prestigious economics journals? This is especially puzzling because the supplier of the data has a special interest in a certain result. The study of the two economists from Harvard and Kansas is based on proprietary data on music downloads, which the authors received from the file sharing services "MixmasterFlame" and "FlameNap."

. . . .

Liebowitz knew of the filesharing study before it was published because it had been circulated as a working paper. In his letter he told Levitt that, despite repeated requests, the authors did not provide him with an opportunity to check their results. Could he please use his influence as editor of the "JPE" to make such checks possible? Levitt declined to tell Handelsblatt whether he followed up on this request.

It appears that he did not. Even one year after publication, the authors still keep their data to themselves. Oberholzer-Gee told Handelsblatt that they had to sign an agreement not to share the data to get them from the file sharing service. The authors argued that they had to "protect their sources" and declined to provide Handelsblatt with either a copy of the agreement or the name of a reference at the file sharing service who could confirm their version.

Liebowitz pressed Levitt, the editor of the "JPE," to at least correct several mistakes and ambiguities before publishing the paper.

For example, the authors write that about half the reductions in music CD sales are the result of the increase in market share of music discount stores with smaller inventories. Liebowitz argues that this cannot possibly be correct. He calculates that, even under extreme assumptions, the reduction in inventories can at most account for one-sixth of the decrease in sales. "It is unbelievable that a top-journal like the "JPE" would publish such claims without any evidence," Liebowitz complains in his letter, and he points Levitt to an entire series of additional errors or ambiguities.

Levitt forwarded Liebowitz’ letter to the authors, who ignored it—their study was published with only minor changes. Since then, file sharing services can refer to an academic paper in one of the top economics journals to defend themselves against the music industry.

In principle, like many other journals, the "JPE" requires that authors publish not only their results but also disclose the data and the methods that they use to derive them. However, this requirement does not apply to Oberholzer-Gee and Strumpf—their paper was accepted before the requirement became binding. "This has nothing to do with science," criticizes Bruce McCullough, professor of decision sciences at Drexel University in Philadelphia. "Without scrutiny, there can be no science," says the expert on the replicability of empirical results in economics. . . .

Given the huge coverage of this by the American press, it would be useful if someone in the American Press would write about this problem.

Thanks very much to Florenz Plassmann for translating this.

New Op-ed: On Giving Terrorists the Best Advice

Here is a new piece that I have at Fox News:

Should a website post the best ideas for successful terrorist plots? Should we even discuss publicly how to stop terrorist attacks? This week, New York Times blogger Steven Levitt publicly posted terrorist plot suggestions. He claims that “by getting these ideas out in the open, it gives terror fighters a chance to consider and plan for these scenarios before they occur.”

Levitt clearly assumes that terrorists have already figured out the best ideas, but that our side has not. If anything, the reverse is probably true. There are vastly more Americans than terrorists possessing detailed information on American infrastructure, traffic flows, policing practices, etc. So terrorists could easily learn something. To make matters worse, there are also many home-grown mental basket cases who could get ideas on how to obtain worldwide attention. . . . .<.dd>

Steve Levitt's Correction Letter

Click on above letter to enlarge.

Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Levitt at least acknowledged that he sent multiple problematic "emails" over more than one day. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. The letter contradicts many of the false claims made in Levitt's book (he knows that everything in his book about me is false). One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:

"I also was aware at the time of the May 2005 emails to you that in connection with the preparation of conference issues for the JLE that the organizer of each conference issue needs to provide funding to the JLE to cover publication and mailing expenses. I did not mean to suggest that Dr. Lott did anything unlawful or improper in arranging for the payment of the publication expenses for the Conference Issue."

The Chronicle of Higher Education has had a couple of articles on all this, including characterizing Levitt's letter as offering "a doozy of a concession."

See this from the Chronicle.

See also this:

According to the motion, new facts have come to Mr. Lott’s attention since last year that significantly alter the character of his complaint. For one thing, he says, new information has come out about what he calls Mr. Levitt’s malice toward him. The motion alleges that Mr. Levitt has publicly referred to Mr. Lott as “the anti-Christ” and that Mr. Levitt “offered publicly to pay colleagues if they would humiliate” Mr. Lott. (Mr. Levitt did not immediately reply to a request for comment today.)

One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.

The date on this letter does not match when I received it.
Update: See also this follow up filing.

Here is the exchange that Levitt had with economist John McCall on May 24th and 25th, 2005:

Email from John McCall to Steven Levitt

You also state that others have tried to replicate [Lott’s] research and have failed. Please supply me with appropriate citations so that I might check for myself.

Email from Steven Levitt to John McCall

There was a NRC/ natl acad of sciences panel I was part of about research on guns that came out in 2004. That will point you in the right direction. . . .

Email from John McCall to Steven Levitt

Hi Steve,

I went to the website you recommended -- have not gone after the round table proceedings yet -- I also found the following citations -- have not read any of them yet, but it appears they all replicate Lott's research. The Journal of Law and Economics is not chopped liver.

Have you read through any of these?http://johnrlott.tripod.com/postsbyday/RTCResearch.html

Cordially,
John McCall PhD

Email from Steven Levitt to John McCall

John,
It was not a peer refereed edition of the Journal. For $15,000 he was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.
Steve

Email from John McCall to Steven Levitt

Returning to the $15,000 bribe issue of JLE and -- although I have not yet gone (I will find it tomorrow) to the NRC/natl. acad. of sciences panel you recommended -- I am wondering whether those deliberations were published somewhere, and, if so, who paid for that. Could such not have been essentially the same thing? I noticed that this issue of JLE was the results of a conference on crime safety and guns that was sponsored by AEI and the Yale Center for Studies in Law. I understand how your best friend the editor could have been outraged, and I hope he had the principle to resign his position in protest. However, we all eventually realize that an editor is but a small cog in a big wheel.

Email from Steven Levitt to John McCall

John, if you read the paper by Duggan in JPE, and Ayres and Donohue in Stanford Law Review, and the NAS/NRC report (which was not paid for by anyone, it is done by the National Academy of Sciences), you will see the other side of the debate. Steve

Judge Castillo issues decision on Lott v. Levitt

A copy of the judge's decision can be downloaded here. A Copy of Levitt's Correction Letter can be obtained here.

1) The Judge has found that one of the two counts of defamation involving Levitt can proceed.

p. 15: ". . . Levitt's email sounds as if he was "in possession of objectively verifiable facts." In his email Levitt states: 'It was not a peer refereed edition of the Journal. For $15,000 [Lott] was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.' First, it would be unreasonable to interpret Levitt's unqualified statement that the journal edition was not 'peer refereed' as Levitt merely giving his opinion on the 'peers' chosen to review,or referee, the Special issue. Indeed, the editor of the Journal might be able to verify the truth or falsity of whether the Special Issue was reviewd by peers. Furthermore, while Levitt argues that one person's "'peer' in the academic realm may be another person's 'hack'," this distinction is not reasonable when discussing the review process at a top university's academic journal. Second, a reasonable reader would not interpret Levitt's assertion that "For $15,000 [Lott] was able to buy an issue and put in only work that supported him" as simply a statement of Levitt's opinion. Levitt's email appears to state objectively verifiable facts: that Lott paid $15,000 to control the content of the Special Issue. . . . Levitt's motion to dismiss Count II of Lott's Complaint is denied."

2) p. 7: "The applicable standard, however, is not that of the 'world of academic research and scholarship' that Lott describes. Rather, the critical question is how a 'reasonable reader' would interpret the phrase. The reasonable reader in this case is the general population . . . . In everyday language replicating results does not necessarily mean analyzing identical data in identical ways . . . ."

Response: I think that the market for the book was also aimed at academics. The book is apparently marketed to a large number of economics classes and is read by academics.

Levitt's correction letter


Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. Unfortunately, at this point, I am not allowed to really comment on this. One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:

"I also was aware at the time of the May 2005 emails to you that in connection with the preparation of conference issues for the JLE that the organizer of each conference issue needs to provide funding to the JLE to cover publication and mailing expenses. I did not mean to suggest that Dr. Lott did anything unlawful or improper in arranging for the payment of the publication expenses for the Conference Issue."

The Chronicle of Higher Education has had a couple of articles on all this:

See this from the Chronicle.

See also this:

According to the motion, new facts have come to Mr. Lott’s attention since last year that significantly alter the character of his complaint. For one thing, he says, new information has come out about what he calls Mr. Levitt’s malice toward him. The motion alleges that Mr. Levitt has publicly referred to Mr. Lott as “the anti-Christ” and that Mr. Levitt “offered publicly to pay colleagues if they would humiliate” Mr. Lott. (Mr. Levitt did not immediately reply to a request for comment today.)

One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.

The date on this letter does not match when I received it.
Update: See also this follow up filing

New Research that Abortion Increases Violent Crime Gets Attention

From today's Chicago Sun Times:

A high-profile economist is challenging the conclusion in the best-selling book Freakonomics by University of Chicago professor Steven D. Levitt that the legalization of abortion in the early 1970s led to a major drop in murder and other violent crimes a generation later.

John R. Lott Jr., a former U. of C. economist now teaching in New York, says the Supreme Court's 1973 Roe v. Wade decision actually caused violent crime to rise.

Lott and fellow researcher John Whitley plan to publish a paper in October in Economic Inquiry that questions Levitt's research on abortion and crime.

Lott and Levitt already were feuding over Lott's charge that Levitt had defamed him in Freakonomics.

In their new paper, Lott and Whitley say that legalization of abortion prompted a cultural change that increased the number of children born out of wedlock. Those children of unwed mothers caused murders to rise by more than 700 cases in 1998 alone, saddling the public with more than $3.3 billion in "victimization costs," the paper says.

On the other hand, Levitt's research found that Roe v. Wade resulted in a savings of $30 billion a year that crime would have cost the public.


More unwed mothers

His Freakonomics, co-authored by Stephen Dubner and published last year, says legalized abortion led to a large drop in murder and other violent crime in the late 1980s and early '90s, and continues to reduce crime.

The book suggests that if the aborted fetuses had instead been born, they would have become adults more likely to commit crimes because they were unwanted by their mothers.

To illustrate the point, the book says the five states that allowed abortion three years before Roe vs. Wade saw major declines in violent crime between 1988 and 1994 -- earlier than the other states.

But Lott says the Levitt study did not fully consider the increase of children born out of wedlock. His theory is that with the option of abortion, women became more likely to have premarital sex, but then had their babies and raised them as single parents.

Children born out of wedlock have had smaller investments in "human capital" by their parents and are more likely to get into trouble when they grow older, Lott says.

On average, his paper says, about 5 percent of whites were born out of wedlock from 1965 to 1969, rising two decades later to 16 percent. For blacks, the figure rose from about 35 percent to about 62 percent, the paper says.

Before legalized abortion, more than 70 percent of children born out of wedlock ended up in families with a father, but the fraction fell to 44 percent in 1984, according to the paper. . . . .

If you want to read the research, you can find it here. THe newspaper article says that I concede that abortion through its effect on "unwanted" births slightly reduces violent crime, but what I believe that I said is that it is possible. The net effect however is abortion increases violent crime.

Ariel Rubinstein Reviews Freakonomics

Ariel Rubinstein provides a critical review of Freakonomics. I guess that I had wish that Ariel had gotten more into the specific economics examples. I think that there were many mistakes in the book.

I thank him for sending it to me.

Levitt's argument for dismissal

Levitt's response to my lawyer's response to their motion to dismiss can be found here.

I haven't really had much of a chance to look at their response so far. A past post on this can be found here.

Lott v. Levitt Response FIled

Yesterday my lawyer filed a response to Levitt and HaperCollin's motion to dismiss the defamation case. For those interested, the response can be found here.

A past post on this can be found here.

LA Times Letter to Editor: Re "Shooting holes in a lawsuit," Opinion, May 31

Unfortunately, this letter was substantially shortened and somewhat altered the meaning due to space limits on letters. It is also unfortunate that the LA Times did not publish any letters from other academics who had written in.

Accusations on gun law research are unfounded
June 12, 2006

Re "Shooting holes in a lawsuit," Opinion, May 31

Jon Wiener claims that "nobody" tried to replicate my research, which demonstrated that right-to-carry laws reduce crime rates.

This is false. Not only has everyone who tried managed to replicate my findings, but many academics have gone beyond that and shown that right-to-carry laws reduce violent crime under a variety of approaches (see johnrlott.tripod.com/postsbyday/RTCResearch.html).

Wiener fails to note that since my research was published, not a single peer-reviewed publication has found that right-to-carry laws increase crime, and there is research that finds even larger drops than I found.

Wiener's Google search implies that many other scholars agree with his claims of research fraud. Yet Wiener fails to note that only about a sixth of the postings are actually scholars of any type.

Wiener claims that my defamation suit wants to "silence" certain claims of fraud. I and other academics have tried to engage Stephen D. Levitt in discussions on the accuracy of his claims, but he won't respond to us.

Instead, [Levitt] falsely charges that the research with which he disagrees was published only because the University of Chicago Press overrode the journal's editor, and that I bought the press' decision.

JOHN R. LOTT JR.

Here is one of the other letters that I know of from two academics:

In his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But this is not the issue. In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate [Lott’s] results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation. We replicated Lott’s results, as have many other scholars. If Levitt did not know this, then he was lax in his scholarship.
Levitt should know that among scholars “unable to replicate” is interpreted as “something is seriously wrong here.” Moreover, the phrase carries the suggestion of, at best, incompetence and, more likely, dishonesty.
Levitt should express his disagreement with Lott in a way that does not imply incompetence or dishonesty. It is thoroughly reasonable to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.

Nicolaus Tideman
Blacksburg, VA

Florenz Plassmann
Ithaca, NY

See also:

Dear Mr. Goldberg:
At the end of his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But the relevant issue is not “a literal interpretation of a single word.” In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate his results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation of the words.
In our understanding, and in the understanding of colleagues we consulted, what is generally meant in economics by replicating someone’s work is gathering the same data and analyzing them in the way that the original researcher had analyzed them. We took an interest in the issue of guns and crime when Lott’s work was first published. We gathered the data that he had used, analyzed it in the way that he described and got essentially the same results. Thus we replicated Lott’s work.
In some circumstances, what is meant by replicating someone’s work is to gather similar data and analyze it in the same way. But that interpretation would not be relevant here, because (as often happen in economics) the data that Lott used were all of the data that were available.
Should “replicate” be stretched to mean “undertake similar analyses and reach similar results”? Well, we analyzed the guns and crime data in additional ways that we regarded as interesting and reached results that were broadly in agreement with those that Lott had reached.
Is there any reason why our work (“Does the Right to Carry Concealed Handguns deter Countable Crimes? Only a Count Analysis Can Say,” Journal of Law and Economics, October 2001, Volume 44, Number 2, Part 2, pp.771-798), should not count as a replication of Lott’s work? Wiener quotes in apparent endorsement Levitt’s assertion that, “for $15,000 [Lott] was able to buy an issue [of the Journal of Law and Economics] and put in only work that supported him.” This is not what happened. We presented our work at a conference that Lott organized. At Lott’s invitation, we submitted our work for a special issue of the Journal of Law and Economics that Lott arranged. A letter from an editor (Sam Peltzman, Professor of Economics at the University of Chicago) informed us that we would need to address the concerns of a referee who had reviewed our work before a decision would be made as to whether it would be published. A later letter for Peltzman informed us that our revision in response to the referee’s comments was accepted. Thus our work was not “put in” the Journal of Law and Economics by Lott but rather accepted for publication after scholarly review.
The basic issue between Lott and Levitt is not whether the results of either scholar can be replicated, but rather what statistical analyses are appropriate to make inferences about the effect on crime of allowing more citizens to carry guns. It is not unreasonable to require Levitt to find a way to express his analytical disagreement with Lott without implying that others who analyze the data in the way that Lott did do not get the same results. While lawyers are better suited to opine on what legal measures are appropriate to achieve this end, it seems to us not unreasonable, prima facie, to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.

Sincerely,

Florenz Plassmann
Associate Professor of Economics, Binghamton University
Nicolaus Tideman
Professor of Economics, Virginia Tech

Will Donohue and Levitt predict that a massive new crime wave is coming?

Guttmacher Institute, associated with Planned Parenthood, claims that: "The rate of unintended births — unintended pregnancies carried to term — rose by 44 percent among poor women from 1994 to 2001, but declined by 8 percent for wealthier women." If true, I assume that those few who still believe that abortion massively reduces violent crime rates after the errors were discovered in the original research will be predicting that we will soon see a big increase in violent crime rates. If up to 80 percent of the changes in murder rates can be explained by abortion as has been argued, those who believe this relationship must be worried we could be in for big increases in murder. (Of course, the abortion is more likely to slightly increase crime.) People born in 1994 will become teenagers starting next year. Presumably, proponents of this theory will start warning people very soon.

News story on defamation suit: Lott v. Levitt

(Update) From the University of Chicago student newspaper the Maroon:

Freakonomics claim sparks defamation lawsuit
By Kim Velsey
April 21, 2006 in News
John Lott, Jr., a former visiting professor at the University filed a defamation lawsuit on April 10 against economic professor Steven Levitt, co-author of the New York Time bestseller Freakonomics

Lott said the book misrepresents his work on guns and crime, according to court documents. The lawsuit does not name journalist Stephen Dubner, though he co-wrote the book with Levitt.

Freakonomics, which melds Levitt’s economic essays with Dubner’s flowing prose, remains high on the bestseller list. The book’s success, however, may have prompted the legal action, as the lawsuit references the popularity of Freakonomics as a factor contributing to Lott’s damaged reputation.

The lawsuit states that the book “damages Lott’s reputation in the eyes of the academic community in which he works, and in the minds of hundreds of thousands of academics, college students, graduate students, and members of the general public who read Freakonomics.”

The contested material is on pages 133–134 of Freakonomics, in which Levitt writes that researchers have been unable to confirm Lott’s conclusion that right-to-carry gun laws actually reduce crime.

Freakonomics states, “Then there was the troubling allegation that Lott actually invented some of the survey data that supports his more-guns/less-crime theory. Regardless of whether or not the data was faked, Lott’s admittedly intriguing hypothesis doesn’t seem to be true. When other scholars have tried to replicate results, they found that right-to-carry laws simply don’t bring down crime.”

According to the lawsuit, Lott acknowledges that his findings have come under scrutiny in the academic community, but he maintains that he used “different data or methods to analyze the relationship between gun control laws and crime.”

The lawsuit states that scholars who have replicated Lott’s work have achieved the same results. “Every time that an economist or researcher have tried to replicate [Lott’s] results, he or she has confirmed Lott’s conclusion.”

Carl Moody, a professor of economics at the College of William and Mary, said he successfully replicated Lott’s findings and published the results in 2001. Moody said Levitt’s accusation is wrong.

The lawsuit, which also names Levitt’s publisher HarperCollins, states that the publisher acted with malice by failing to verify the statements. It seeks a court order to halt sales of Freakonomics until the statements are retracted or amended and also demands that Levitt and HarperCollins pay unspecified monetary damages.

HarperCollins would not comment on the lawsuit, but a company representative said, “HarperCollins Publishers firmly stands behind Freakonomics and its authors, Steven Levitt and Stephen Dubner.”

The ABC show 20/20 featured Freakonomics in an hour-long special on April 14. However, there was no mention of the lawsuit, and Levitt has yet to comment on it publicly. The book’s website, Freakonomics.com, which has Levitt’s and Dubner’s weblogs, includes a brief mention of the pending litigation.

“While we were away [in London promoting the paperback edition of Freakonomics], the economist John Lott filed a lawsuit claiming that Freakonomics has libeled him,” wrote Dubner on his blog.

Lott’s website made no mention of the lawsuit.

The lawsuit has opened up discussion on the veracity of Levitt’s claims and whether a lawsuit is an appropriate forum for an academic debate.

The litigation has also shed light on what can happen when an academic book attains blockbuster status.

“Most academic debate is so trivial no one would care,” Moody said. “If the book had appeared and no one had bought it, it wouldn’t be an issue. But Levitt is accusing this guy of falsifying his results in front of millions of people.”

Here is an earlier story from the Chicago Tribune:

A scholar known for his work on guns and crime filed a defamation lawsuit Monday against University of Chicago economist Steven Levitt, co-author of the best-seller "Freakonomics."

John Lott Jr. of Virginia, a former U. of C. visiting professor, alleges that Levitt defamed him in the book by claiming that other scholars had tried and failed to confirm Lott's conclusion that allowing people to carry concealed weapons reduces crime. Publishers Weekly ranked "Freakonomics" eighth this week for non-fiction hardcover books.

According to Levitt's book: "When other scholars have tried to replicate [Lott's] results, they found that right-to-carry laws simply don't bring down crime."

But according to Lott's lawsuit: "In fact, every time that an economist or other researcher has replicated Lott's research, he or she has confirmed Lott's conclusion."

By suggesting that Lott's results could not be replicated, Levitt is "alleging that Lott falsified his results," the lawsuit says.

Lott is seeking a court order to block further sales of "Freakonomics" until the offending statements are retracted and changed. He is also seeking unspecified money damages.

Lott acknowledged in the suit that some scholars have disagreed with his conclusions. But he said those researchers used "different data or methods to analyze the relationship between gun-control laws and crime" and made no attempt to "replicate" Lott's work.

The lawsuit alleges that Levitt and his publisher, HarperCollins Publishers Inc., made the statements with reckless disregard for whether they were true and that the book damaged Lott's reputation.

Neither Levitt nor HarperCollins officials could be reached Monday.

According to the lawsuit, Levitt also defamed Lott in an e-mail that Levitt sent to an economist in Texas last May. The e-mail described work that Lott published in an academic journal in 2001. It falsely stated that Lott's work had not been peer-reviewed and that Lott had blocked scholars with opposing views from appearing in the same issue of the journal, the lawsuit said.

Lott's books include "More Guns, Less Crime: Analyzing Crime and Gun Control Laws," published in 1998. Levitt won the John Bates Clark Medal for economists younger than 40 from the American Economic Association in 2003.

The case was assigned to U.S. District Judge Ruben Castillo.

From the Chronicle of Higher Education:

Last week Mr. Lott filed a defamation lawsuit against Steven D. Levitt, a professor of economics at the University of Chicago and a co-author of the best-selling Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (William Morrow, 2005). (A copy of the complaint can be found on the blog Overlawyered.) Mr. Lott charges that in the book and in private e-mail messages, Mr. Levitt spread lies about the quality and integrity of Mr. Lott's work (The Chronicle, April 13). Much will hinge on exactly what Mr. Levitt meant by the words "replicate" and "peer refereed."

UPDATE: For the conclusion of part of the case, please see this link.

Interesting conference on the abortion and crime link

Today we just had an interesting conference on the abortion and crime link. There is a video of the event. We invited all the Americans who had finished work on this topic by the middle of February, so while it might appear that almost everyone who had finished a paper thought that the link between abortion and crime was nonexistent or possibly even positive. There was one paper that claimed to find evidence in favor of Donohue and Levitt, but that when the issues raised by Chris Foote were dealt with the significance of the effect went away. Video is available of the two sessions at the top right of the page.

The Economist Magazine on Steve Levitt and Abortion

Of course, lots of people have always thought Mr Levitt was in the wrong. Even if abortion cuts crime, it is still immoral, they fulminate. But this is largely beside the point: Mr Levitt's research does not take a position on abortion's social virtues, but aims merely to uncover its societal effects. Besides, for someone of Mr Levitt's iconoclasm and ingenuity, technical ineptitude is a much graver charge than moral turpitude. To be politically incorrect is one thing; to be simply incorrect quite another.

But what is most amazing is that despite all the statistical significance being eliminated from their panel data set being eliminated when the results are done Donohue and Levitt's way with the the data they want to use, John Donohue now says what they wanted to do:

It may be asking too much of the numbers to convince everybody. “The debate over abortion and crime will not be resolved within the parameters of our paper,” says Mr Donohue. He thinks the arrest figures are “muddy” and the state population data “sloppy”. Combining the two generates so much noise, it is hard for the statistical tests to hear anything. Ted Joyce, a professor at Baruch College (part of the City University of New York), who has had his own methodological disagreements with Messrs Donohue and Levitt, also thinks the debate is stretching the data too far. He points out that if you add controls for 50 states and 12 years—as Messrs Foote and Goetz do, and as Messrs Donohue and Levitt meant to do—you are, in effect, holding another 600 things constant. This robs the data of most of their variety, and of much of their ability to explain anything.

There was no warning that these were the wrong tests at all in either of their previous papers or in Levitt's book. Levitt now refers to the “collage of evidence," but the panel data was the only test that really amounted to anything worthwhile. The rest of the data was merely cross sectional or time series.