Empirical Legal Studies

Cornell's 2015 Junior Empirical Legal Scholars Workshop

Cornell Law School will host its Junior Empirical Legal Scholars Workshop next weekend in Ithaca, NY. Similar to past workshops, "Junior" is defined to include pre-tenure (or pre-tenure-track) individuals who, through their early empirical scholarship, have demonstrated considerable scholarly promise. At the Workshop, each junior presenter is matched with a field-appropriate "senior" empirical legal scholar discussant. The 2015 Junior scholars reflect a diverse range of academic disciplines and include: Maya Sen (Harvard), Sarath Sanga (Columbia), William Hubbard (Chicago), Jed Stiglitz (Cornell), Jacob Goldin (Princeton), and Avani Sood (UC Berkeley).

Replication Problems in Psychology

No doubt partly prompted by a recent spate of academic fraud, Science magazine published results from a study that set out to replicate 100 published psychology studies. An "international team of experts," however, could "reproduce only 36% of original findings." (News coverage here.) While I'm sure many in the psychology field will seek to explain such findings, an inability to replicate 74% of studies published in "top" psychology journals is, at the very least, jarring.

Update: From today's NYT (here).

Modeling Law School Clinics' Influence On Legal Employment Outcomes

Over at The Faculty Lounge blog, Michael Risch (Villanova) dissects a pair of papers that seek to empirically assess whether "law clinic opportunities are likely to improve a school’s graduates’ employment outcomes."

A paper by Jason Yackee (Wisconsin), Does Experiential Learning Improve JD Employment Outcomes?, finds that "there is not much evidence that law schools that provide greater opportunities for skills training have substantially better employment outcomes than do those law schools that provide fewer opportunities.” A subsequent paper by Robert Kuehn (Wash U), Measuring Legal Education's Employment Outcomes, however, concludes that the "expanded empirical analysis finds it is not possible to draw any reliable conclusion from his models about the likely effects of law clinic courses, or other activities like law journal and interschool skills competitions, on employment outcomes, and surely not any negative suggestion about clinic opportunities or participation. The most realistic conclusion from available data is that nationwide models provide inconclusive results, as they do not achieve statistical significance and yield both positive and inverse relationships depending on the year of graduation, control variables, and outliers. In fact, other evidence shows that law clinic experiences are important to potential employers and do aid some students in securing employment."

Setting aside the overheated rhetoric that frequently accompanies discussions about clinical and "experiential" law school offerings, the pair of papers, taken together, illustrates the potential complexities incident to modeling something as complex as legal employment outcomes.

Empirical Assessment of Law Faculty Scholarly Impact

While some may quibble over how best to measure, a recent paper by Sisk (St. Thomas--Minnesota) et al., Scholarly Impact of Law School Faculties in 2015: Updating the Leiter Score Ranking for the Top Third, updates prior work assessing law faculties' scholarly impact. As Brian Leiter observes, "citations to faculty scholarship is, of course, only one metric of scholarly distinction." That said, among Sisk et al.'s interesting findings includes how the scholarly impact rankings for some ("several") schools diverge from US News rankings.

M-Turk's Important Limitations

Dan Kahan (Yale) has been a leading critic of the increasingly-popular "M-Turk" studies. And despite, as Andy Gelman (Columbia--Statistics) notes, "M-Turk’s combination of low cost and low validity [make] it an attractive option for many researchers," such attractiveness, as Kahan points out (here), should be measured due to critical limitations incident to an array of M-Turk's sampling issues. Whether one agrees or disagrees with Kahan, researchers using M-Turk-generated data certainly need to engage Kahan's critique.

Data on Lawyer Malpractice

While legal scholars don't flinch at studying medical malpractice (for one recent example, click here), a similar focus on attorney malpractice has not yet emerged. To this end, in When the Lawyer Screws Up: A Portrait of Legal Malpractice Claims and Their Resolution, Bert Kritzer (Minnesota) and Neil Vidmar (Duke) present initial data that help outline the basic contours of attorney malpractice. While their analyses are largely descriptive, their initial findings point to "essentially two hemispheres of legal malpractice." An excerpted abstract follows.

"In this paper we provide an empirical portrait of legal malpractice claims and their resolution. We draw on a wide range of data sources including reports published by the American Bar Association, reports from individual insurers, and data sets obtained from insurance regulators in Florida and Missouri….

A central finding of our analysis is that there are essentially two hemispheres of legal malpractice paralleling the two hemispheres of the bar identified by Heinz and Laumann in their study of the Chicago bar. In the personal services sector one finds mostly relatively small stakes cases often involving plaintiffs’ personal injury, real estate, family law, and collections and bankruptcy; much of the insurance coverage is provided by mutual insurers started by and/or affiliated with state bars. In the corporate sector stakes tend to be very large, and cases involve corporate matters, corporate litigation, and high stakes areas such as intellectual property and securities; insurance is provided by either by specialized insurers or through brokers who put assemble a group of insurers to cover a large law firm."