Gun violence research should be peer-reviewed and rely on verifiable facts What happens when it isn’t?
The NRA relies heavily on research conducted by economist John Lott. But, Lott’s research is not peer-reviewed, and his most notable publication has been questioned by researchers from both sides of the gun policy debate. A little over a decade ago, he was disgraced and his career was in tatters. Not only was Lott’s assertion that more guns leads to more safety formally repudiated by a National Research Council panel, but he had also been caught pushing studies with severe statistical errors on numerous occasions. An investigation uncovered that he had almost certainly fabricated an entire survey on defensive gun use. And a blogger revealed that Mary Rosh, an online commentator claiming to be a former student of Lott’s who would frequently post about how amazing he was, was in fact John Lott himself. He was all but excommunicated from academia.
Below is a recent story published by the online magazine Vox that details the academic and ethical concerns with his research.
The bogus claims of the NRA’s favorite social scientist, debunked
The War Against Guns: Arming Yourself Against Gun Control Lies byJohn Lott is well on its way to ascending into the pantheon of pro-gun literature. A parade of conservative heavyweights has lavished praise on the book since its publication this month, among them Ted Cruz, Marco Rubio, Newt Gingrich, Grover Norquist, Ann Coulter, and Sean Hannity. Cruz opined that “the Second Amendment has no better defender than John Lott,” adding that with this book, Lott “has done his country and the cause of gun rights a great service.”
Lott is also enjoying a moment as a cause célèbre on the right: Conservative sites, and the NRA itself, are suggesting that there was some nefarious reason that an interview with Lott was cut from Katie Couric’s documentary on gun violence, Under the Gun. The documentary’s director has replied that Lott’s work had been “largely discredited” and that it made more sense to reserve the time for “responsible gun owners.”
Lott’s 250-page book covers a wide range of territory, and seeks to debunk what he sees as many of the most common arguments by gun control advocates and their allies in the media. This is familiar territory for Lott, whose first major book, More Guns, Less Crime, published in 1998, marked a turning point in the gun debate. For the first time, it allowed the gun lobby to couch its arguments in the language of social science. The central finding in that book — that rates of gun ownership and the existence of “right to carry” laws reduce violent crime — have been the subject of numerous subsequent studies, the most sophisticated of which conclude Lott’s results are specious. (A National Research Council report found that the data did not support the theory.)
After controversies involving his work and his aggressive, sometimespseudonymous defense of it online, Lott left academia to become the founder and president of the Crime Prevention Research Center. Yet that has hardly dented Lott’s reputation among defenders of minimal restrictions on guns.
In our view he remains the single most important quasi-academic figure opposing gun control: There are many people making Second Amendment arguments in favor of the right to bear arms, but he is the originator of two of the NRA’s favorite talking points: that “right to carry” laws reduce crime and that murderers seek out gun-free zones for their “sprees.”
But despite the effusive praise from gun advocates, and respectful treatment by the mainstream media, The War on Guns is riddled with errors and falsehoods that undermine most, if not all, of its major claims.
Lott dismisses the link between guns and suicide
Every single case-control study done in the United States hasfound the presence of a firearm in the home is a strong risk factorfor suicide. (That’s 24 separate studies.) Indeed, a 2014 meta-analysis by Andrew Anglemyer and two colleagues examining 16 studies found that gun availability tripled the overall risk of suicide (not just firearm suicide — suggesting that it wasn’t a question of victims substituting guns for other options). Indeed, if there is one aspect of the gun debate that is decidedly uncontroversial, it is that more guns means more suicides — although, of course, much research remains to be done on the subject.
Lott returns to the subject of suicides several times in his book, each time insisting that gun ownership is unrelated to suicide. He complains that reporters push the suicide-gun link without looking at the data. In particular, he writes, they ignore “the National Research Council’s research showing that suicidal individuals had merely ‘substituted other methods of suicide.’”
This is a remarkable example of quoting out of context. The NRC quotedoes not refer to the dozens of studies examined by the NRC on the topic of suicide and firearms, but only two — specifically, two studies examining the connection between recent gun purchases and suicide. And the statement actually reads: “The most important limitation is that these [two] studies do not indicate whether handgun purchasers would have substituted other methods of suicide if a gun were not available, and do not measure other factors.”
The NRC was merely summarizing, as academics tend to, the limitations in two studies among many others. Lott is using the NRC’s academic modesty and fastidiousness — something his own work is not known for — to undermine the central findings.
The NRC report’s larger conclusion, after reviewing the research? “Overall, the U.S. studies have consistently found that household gun ownership is associated with a higher overall risk of suicide.” The exact opposite of Lott’s claim.
Lott is progenitor of the myth that mass shooters seek out “gun free” zones
A key Lott thesis, hammered home again in this book, is that mass shooters deliberately and overwhelmingly target places where guns have been banned. Lott rests his contention on two foundations: statements made in journals and on social media by three mass killers and his own determination that all but three mass shootings in the US since 1950 have occurred in areas that banned firearms. Both of these assertions wither under scrutiny.
Lott claims, for instance, that James Holmes, who killed 12 people in a movie theater in Aurora, Colorado, in 2012, specifically targeted the Cinemark theater because it had a gun-free zone policy, and that Holmes had ruled out attacking an airport because of its “substantial security.”
Yet the main reason Holmes decided against attacking an airport, according to passages in 36 pages of handwritten notes gathered during the post-shooting investigation, is that he didn’t want his motive to be construed as terrorism. (He had no “cause” except his own brand of violent nihilism.) Nowhere in those notes does he mention civilians carrying firearms, although the notes are remarkably detailed, covering virtually every detail of the crime’s planning. So detailed, in fact, are Holmes’s notes, that we can be confident that he would have mentioned gun-free zones if he cared about them.
Holmes chose the specific theater he did, in his own words, because it was “isolated, proximate, large” — qualities three other theaters in his immediate vicinity lacked. (Holmes actually appeared far more concerned about finding a good parking place than encountering armed resistance.)
Moreover, the entire basis for Lott’s claims about the psychological motivation of Holmes is a highly unscientific search Lott conducted reported on his website — using MapQuest and movies.com. This research revealed that, of the seven theaters showing “The Dark Knight” within 20 minutes of the killer’s house, only one prohibited firearms. (It’s worth mentioning just how arbitrary and suspiciously convenient that 20-minute cutoff is. Had he extended the study to theaters that were 21 minutes away, he’d be forced to include the Arapahoe Crossings 16 complex, which also posted signs forbidding guns.)
In classifying theaters as gun-free zones, Lott makes the suspect assumption that any theater without a visible “No Weapons Allowed” sign must necessarily allow firearms. This ignores the fact that businesses might ban guns as a matter of internal policy but nevertheless be reticent to advertise that policy. Indeed, we know that Lott’s assumption is wrong, because we made follow-up calls to the theaters in Lott’s data set. Esquire Theater, for instance, which is within the 20-minute circle of Holmes’s apartment, disallows weapons as a matter of policy, yet it has no posted signs.
More broadly, are we really to believe that sociopaths like James Holmes are hunched over their computers, searching for theaters within a certain radius of their home, calling up place after place to hash out the nuances of their weapons policies — all without any concern about arousing suspicions on the other end of the line? Lott appears to believe that mass shooters are incredibly attentive to the minutiae of gun policy when there is little evidence to support such a claim.
Lott’s argument also relies on the assumption that “good guys with guns” dutifully adhere to “no weapons” signs, so mass shooters can reliably infer that any place with a “no guns” policy will be free of people with guns. But that’s a highly dubious assumption, given how readily gun advocates will admit they ignore gun-free signs.
Along with his claims about the Aurora shooter, Lott also cites statements in the rambling “manifesto” of Elliot Rodger, who killed six people and wounded 14 others in Isla Vista, California, near the University of California at Santa Barbara campus. But Rodger makes it crystal clear that it was police officers he was worried about encountering, not armed civilians. (“There would be too many cops walking around during an event like Halloween, and cops are the only ones who could hinder my plans.”)
Lott cites as well the social-media account of Justin Bourque, who killed three Canadian policemen in Moncton, New Brunswick. This might be the oddest example of all. Bourque did mock gun-free zones on social media, but when he carried out his attack he deliberately targeted only police officers, all of whom were armed. Pause to let it sink in: Lott holds up as proof of his theory that mass shooters avoid men with guns — a mass shooter who specifically targeted men with guns (as did the killer of the Dallas police this summer).
Lott’s empirical claim that only three mass shootings have occurred where civilians are allowed to carry firearms is also false. He misclassified shootings in Umpqua Community College in Oregon; Hiahleah, Florida; and elsewhere as occurring in gun-free zones. But as Politifact has reported, Umpqua permitted people with concealed-carry licenses to carry arms on campus. And it was widely noted that several students there were armed at the time of the shooting. The college “was never designated as a ‘gun-free zone’ by any signage or policy,” a spokesperson for the college told Politifact. (Using Lott’s accounting methods, the attack in 1966 by the University of Texas “clock tower” shooter, Charles Whitman, took place in a no-guns zone, although a veritable militia of armed Austin residents emerged to try to take him out.)
The best available evidence suggests that shooters do not deliberately choose gun-free zones for their attacks, and instead usually choose locations where they have a deep-seated emotional grievance — or, in some cases, they simply go where the public is gathered and where police are likely to be absent.
There’s little to no evidence that wielding a gun makes you safer
Lott asserts that “What scares criminals most is the thought of their victims having guns. A gun response when confronted by a criminal is the safest, according to the Bureau of Justice Statistics’ National Crime Victimization Survey.” He continues: “Gun control advocates seem to think that waiting for police to arrive is a good enough option for poor Americans.” Unfortunately for Lott, the actual data he cites undermines his claim. A recent study examining NCVS data showed that doing nothing and using a gun in self-defense have almost exactly the same injury rates (11 percent versus 10.9 percent). In fact, calling the police or even just running away both led to less risk of injury after the defensive action (injury rates of 2.2 percent and 2.4 percent respectively) than gun use (4.1 percent).
Lott also opines that “Guns are used defensively some 2 million times each year.” This is an old chestnut that really needs to be retired. The survey he pulls this number from is two decades old and hopelessly flawed.
In 1992, Gary Kleck and Marc Gertz, criminologists at Florida State University, surveyed 5,000 Americans at random, by telephone, asking them if they had used a firearm in self-defense in the past year. They identified 66 cases of successful defensive gun use in the sample, and then extrapolated their findings to the general population, leading to the number Lott cites.
But as David Hemenway of the Harvard School of Public Health hasconclusively demonstrated, this kind of small-sample survey of an inherently rare event — especially when that event is socially desirable (like fending off a criminal) — are so prone to bias as to be useless. Whether the survey is about NRA memberships or magazine subscriptions, such surveys produce estimates that can be many times larger than empirically known rates in the population. (As Hemenway has written, “It is useful to realize that in a random 1,000 people, about 90 say that they personally have seen something they believe was a spacecraft from another planet.”)
One kind of error common to such surveys is “telescoping”: Respondents report any time they’ve had the experience inquired about, regardless of the time period specified. More importantly, gun owners justify their purchases to themselves in terms of self-defense, so they have a strong motivation to say they’ve used their guns that way.
In any case, extrapolating from the Kleck-Gertz survey leads to manifestly absurd results. For there to be more than 2 million defensive gun uses, homeowners would have to defend themselves with a firearm in more than 100 percent of burglaries (to choose one category of crime). And if other findings in the Kleck-Gertz survey were correct, more than 100,000 criminals would be injured by law-abiding gun owners annually. Hospital records reveal no such armies of wounded. (Kleck stands by his findings.)
Instead, the best available empirical evidence from the Gun Violence Archive finds fewer than 1,600 verified defensive gun uses each year. Where are the remaining 99.92 percent of Lott’s claimed gun uses hiding?
Evidence-free assertions about guns and children
Lott states that accidental gun deaths among children “rarely involve children shooting themselves or other children” and “Overwhelmingly, the shooters are adult males with alcohol addictions, suspended or revoked driver’s licenses, and a record of arrests for violent crimes.” The second claim has no citation at all. His citation for the first claim is “a study that [he] did on the years 1995-2001” for an earlier book, The Bias Against Guns. Yet, turning to that book, we find reference only to one unpublished, un-peer-reviewed LexisNexis search of newspapers in 1998 and 1999.
This claim, too, is easily refuted. Data from the National Violent Death Reporting System (compiled for us by the Harvard Injury Control Research Center) finds that of 72 fatal accidental shootings from 2003 to 2006 involving children between the ages of zero to 14, two-thirds of the victims were shot by another child. When you combine self-inflicted injuries by children with shootings by other children, the proportion rises to three-fourths. Lott’s assertion is clearly outlandish.
A Lott of disingenuousness, as well as unskeptical media attention
Unfortunately, Lott’s book fits the pattern in a career distinguished by dishonesty. If you are interested in an honest, fact-based analysis of the gun debate, books such as Private Guns, Public Health by David Hemenway and The Gun Debate by Phillip Cook are excellent places to start. The War on Guns is not worthy of your time.
Nonetheless, Lott will likely continue his book tour on a near-endless list of right-wing radio shows, and unfortunately continue to be cited as an expert in mainstream media outlets such as the Washington Post, the Wall Street Journal, LA Times, Politifact, CBS, CNN, Fox News, and many others. For mainstream outlets, Lott fills a need for “balance.” He’s a gun advocate who wields social science in a way that sounds authoritative, or at least “academic,” to non-experts. But news outlets who cite John Lott as a gun expert are falling for a charade.
Evan DeFilippis is a master’s student at Princeton’s Woodrow Wilson School. He is currently a summer fellow at Harvard’s Government Performance Lab. Devin Hughes is a CFA Charterholder and the founder of Hughes Capital Management, LLC, a registered investment adviser.Both write at ArmedWithReason.com