“Eugene Volokh” writes:
Please, please, let’s take special care to be polite in these exchanges. This is a sensitive subject, but even when we think the other person is dead wrong, it’s better to say this in a subtler way.
OK, I’ll do my best to be polite. I won’t say anything in reply to the ill-mannered Frank Warren other than to note that argument ad hominem is a fallacy, as is arguing from authority (”Kleck is infallible” type arguments) and as are straw man arguments (falsely claiming that I have asserted that Kleck is lying or asserted that guns are pointless.) I think critical analysis of Kleck’s estimate is especially important for those who because of their biases (and we all have our biases) would like to accept it as true.
On the other hand, there is something odd about the assertion that guns are used defensively in over 700,000 burglaries and 400,000 robberies. It’s particularly interesting that the defenders perceived that someone “almost certainly would have died” without the gun use in 15.7% of the cases, or over 300,000 cases total. Surely that’s a remarkable number, one rather inconsistent with our presumably reliable homicide totals. (I realize, though, that it’s possible that people wildly mis-estimated this but were truthful about other, more objective, judgments.)
The estimate of 200,000 woundings of criminals in DGUs is also remarkable. It is 10-20 times higher than Kleck’s earlier estimates of this number derived from multiple independent data sources. It is double the NEISS derived estimate of the number of ALL gun shot wounds (criminal, defensive, accidental and attempted suicides) treated in hospitals.
Kleck also reports that two respondents volunteered that they killed the perp. Kleck’s own earlier estimates of the number of defensive gun killings indicate that it would be most unlikely to have even a single such case in a survey of this size.
Another way to estimate DGUs is to ask criminals how often they were thwarted by armed victims. This is what Wright & Rossi did. 34% of the criminals that they surveyed admitted this. These criminals had an average of 10 prior arrests, so if the 34% who had been thwarted had an average of two thwartings each, it would appear that being arrested is roughly 15 times as likely for these criminals as being stopped by an armed victim. Combine this with about 988,000 arrests per year (this number from Kleck’s “Crime Control” paper) and we get a (very rough) estimate of 70,000 crimes stopped each year with guns. The uncertainties in this calculation are large: it might be off by a factor of two or three, but a factor of thirty is implausible.
To put it another way, if Kleck’s estimate is correct Wright and Rossi’s criminals should have had an average of 20 encounters with armed victims each. And yet most said they had never not once encountered an armed victim. OK, maybe most of these people lied. Or maybe 4% of Kleck’s respondents made stuff up.
Finally, much as one might fault the Crime Victimization Survey, it seems to me to use methods that are not that far different from Kleck’s. True, its nonanomyous nature and failure to ask the right questions makes it singularly inapt at measuring defensive gun use. But I’m not sure it’s equally inapt at measuring, say, robbery or burglary. Am I mistaken on this?
I’m puzzled as to why you say that the NCVS fails to ask the right questions. Let’s focus on the number of DGUs against burglaries, where the Kleck estimate is 25 times that of the NCVS one. In essence, the questions asked by Kleck were: 1. Did you use a gun defensively against someone? and 2. (Only asked if answer to question 1 was yes) Was it against someone attempting a burglary?
While the relevant NCVS questions were in essence: 1. Did someone attempt a burglary? 2. (Only asked if answer to question 1 was yes) Did you use a gun to defend against that person?
That is, the questions are roughly the same, only the order is different. It is certainly possible that asking the questions in a different order will make some small difference to the numbers who answer “yes” to both questions. It most implausible that the difference would be large. The NCVS experimented with different questions in their recent redesign and found that asking several times about the same crimes in different ways tended to jog their respondents memories and deliver more “yes” responses, but the increase was about 20% or so, not factors of 20 or more.
As to the non-anonymous nature of the NCVS: It is true that Kleck’s survey was anonymous, but not in the usual sense of the word, where the identity of the interviewee is unknown. Since there are not usually large numbers of adults of a particular sex living in the average American household, it would be quite easy for someone conducting such a survey to determine the identity of the interviewee. Kleck’s survey is anonymous in the sense that the identity of the INTERVIEWER is unknown. Sure he identified himself as a research assistant working for Gary Kleck, but how does the interviewee know that he is not really a burglar casing the joint, or a police officer looking for innocent gun owners to persecute?
Here’s what Professor Biderman (author of the book “Understanding Crime Incidence Statistics: Why the UCR diverges from the NCS”) had to say on this point during a discussion on the NCVS mailing list:
Albert Biderman wrote:
Another issue Weingarten raises relates to respondent fears of use of (e.g.) her survey responses against her. Respondent confidentiality, by law and by practice, is better protected in Census Bureau-conducted surveys, such as the NCVS, than in any other ones. Fears of uses of information against a respondent who gives it are groundless. Nobody beats the Census Bureau either in the level of cooperation and confidence it achieves from its samples of the public. A few decades of attacks against government in general, and the Census Bureau, in particular, appear to have caused some, but limited, damage to cooperation with federal surveys.
To put things another way: if you want to believe that Kleck got the right answer and the NCVS the wrong one because people distrust the NCVS and lied, then you must believe that 96% of the population are sufficiently paranoid to believe that the Census Bureau lies when it promises confidentiality AND you must believe that NONE of these people are sufficiently paranoid to believe that a government agent would lie to them and claim to be a private researcher conducting research into DGUs. I find explanations that require 96% of the population to be complete lunatics somewhat absurd.
Alternatively, 4% of Kleck’s respondents may have told some tall tales. I personally believe that it the explanation “4% of the population are braggers” to be more plausible than the explanation “96% of the population are paranoid loons”.
To summarize: While Kleck’s survey appears to have been properly carried out, the methodology does not defend against untruthful responses by interviewees and it only requires a very small fraction of untruthful responses to generate his enormous estimate for DGUs.
Kleck’s estimates are inconsistent (usually by factors of ten or more) with:
CDC counts of homicides
UCR counts of homicides
Kleck’s own, earlier, estimate of defensive woundings
Kleck’s own, earlier, estimates of defensive killings
Wright and Rossi’s survey of criminals
NEISS counts of gunshot wounds
NCVS counts of burglaries and violent crimes
UCR counts of burglaries and violent crimes
NCVS counts of DGUs
The most parsimonious explanation for all these discrepancies is that a small fraction of Kleck’s interviewees told some tall tales.
Finally, let me note that none of the forgoing proves that the NCVS estimate is correct, only that the Kleck estimate is probably in error. It is possible that it underestimates DGUs, maybe even by a factor of two. There is just no good reason to believe that the same people who wouldn’t tell the NCVS about a defensive gun use would tell Kleck.