Deltoid

Gary Kleck writes:

my position that estimates of DGUs with a wounding are unstable is correct. The prevalence of DGUs with a wounding in the Kleck-Gertz (K-G) survey was 0.0011 (1.326% of U.S. adults had a DGU of some kind in the previous year, and 8.3% of DGUs involved a wounding – see pp. 184-185 of K-G article; 0.083 x 0.01326 = 0.00110058). Assuming simple random sampling, the 95% confidence interval estimate of the national annual prevalence of DGUs with a wounding would be 0.0011 +/- 1.96((.0011 x .9989)/4,977)) = 0.0011 +/- 0.0009, or 0.000179-0.00202.

This is not the correct way to do the calculation. Kleck has effectively assumed that there were only 0.0011*4977=5.5 cases with woundings in the sample. If there were only that many, then the confidence interval would indeed be as large as Kleck claims. (Well, almost — the formula that Kleck uses doesn’t give a very approximation to the confidence interval when the number of cases is as small as 5.)

However, on pages 173-4 Kleck reports that there were 17 sample cases with woundings, 8.3% of the DGUs. This lets us work out a 95% confidence interval: 8.3% +/- 1.96 x sqrt((.083 x .917)/205) = 8.3% +/- 3.8% or 4.5% - 12.1% of DGUs involve a wounding.

Multiplying these proportions times the U.S. population age 18+ yields 34,438-388,673 DGUs with woundings per year, a considerably wider range than Lambert’s miscalculations implied.

Multiplying the percentages by Kleck’s preferred estimate of 2.5M DGUs yields 110,000 - 300,000 DGUs with woundings per year. Allowing for the sampling error in Kleck’s 2.5M DGU estimate and rounding things off to avoid spurious precision in the confidence interval gives the 100,000-300,000 interval that I gave earlier.

Further, the assumption of simple random sampling tends to understate the magnitude of sampling error for a survey that actually used a more complex sampling design, which means that the estimate of DGUs with a wounding is actually even more unstable (i.e. has an even wider interval estimate) than even these calculations imply.

The confidence intervals given on pages 166-167 assume simple random sampling. Kleck and Gertz do not mention in their paper that because of their sampling design they have understated the sampling error of their DGU estimates.

In sum, my position that the Kleck-Gertz survey sample, however adequate for estimating total DGUs, was too small for estimating specific subtypes of DGUs such as those with a wounding (see K-G rebuttal of David Hemenway, J. Crim. Law & Criminology, Summer 1997,p. 1453), was correct, and Lambert is wrong.

Kleck is mistaken. Just to double check: the relative uncertainity of an estimate of a proportion is roughly proportional to the inverse of the square root of the number of positive cases. This implies that the relative uncertainty of the wounding estimate (based on 17 positive cases) is about twice that of the 2.5M DGU estimate (based on 66 positive cases).

I have no idea where Lambert came up with 100,000-300,000.

I hope I have explained the derivation to everyone’s satisfaction.

However, even if it had been correct, an interval estimate as wide as 100,000-300,000, with the upper limit three times as large as the lower limit, is itself quite imprecise.

It is however precise enough to observe that it is not all consistent with the estimate of 7700-18,500 that appears on page 164 of “Targetting Guns”. It seems likely that most of the people who reported a wounding in Kleck’s survey, did not, in fact, wound a criminal. This could be because they were honestly mistaken, or embellishing a real DGU or making the whole thing up.

While you would expect that some could make an honest mistake about the matter, it seems unlikely that 90% would, especially since that implies that when DG users fire shots at a criminal they only get a hit a mere 5% of the time.

I argued that the estimate of 200,000 DG woundings derived from Kleck’s survey (p163 of TG) was inconsistent the estimate of 7700-18,500 DG woundings on page 164 of TG. Kleck accuses me of sloppy reading for not noting that the p 164 estimate is for medically treated wounds only. However, even if we accept Kleck’s generous estimate that there are as many untreated gunshot wounds as treated ones (chapter 1 of TG), it is quite clear that if we multiply the page 164 estimate by two to allow for this possibility, that it is still not at all close to the estimate from Kleck’s survey.

In any case, we can do better than multiplying it by two: The crucial number used to compute the p164 estimate is the ratio between hospital treated nonfatal shootings and fatal shootings. On p 164 Kleck uses 5.67 for this ratio, in chapter 1 he uses 5.25. Since I’m trying to get an upper bound, I’ll use the higher number, 5.67.

Now we need to allow for woundings that were not treated in a hospital. NCVS data shows that more than 90% of people suffering a gun shot wound in a gun crime received hsopital treatment. Of course, criminals who are shot while attempting a crime will be less likely to seek hospital treatment since it might lead to their capture, so let’s assume that NONE of them get hospital treatment. On page 163 of TG, Kleck estimates that 5-13\% of homicides are legal defensive ones. Since we are trying to get an upper bound, we’ll use the 13% figure.

Putting it all together we get a revised ratio of: 5.67/0.9/(1-0.13) = 7.2. This is the upper bound of the ratio between ALL nonfatal shootings and fatal ones.

Redoing the calculation on p164 using this ratio yields an upper bound of 22,600 legal shootings of criminals. A similar calculation yields a lower bound of 8,400.

These estimates are inconsistent (by a factor of at least nine) with the survey derived estimate of 200,000.

Another way to look at this is to take the 7.2 ratio and apply it to the survey derived estimate of 200,000 DG woundings. This suggests that there were at least 24,000 lawful killings of criminals. This exceeds the total number of gun homicides by a healthy margin, let alone the number of lawful ones.

W A Collier writes:

How the NCVS miscounted DGUs

Undersized sample, poor methodology, bias in the questions, unsound methods and procedures in eliminating bias, and unlike Kleck, they started with the conclusion (there are only a small number of gun defenses) as an objective to be proven (not the scientific method) whereas Kleck started with the question (How many DGUs are there) and let the numbers supply the answer, pro or con.

You need to inform yourself better about the NCVS.

1. The sample size is about 100 times that of Kleck’s survey.
2. The NCVS methodology has been refined over 25 years of use and is the benchmark for all other victimaization surveys.
3. The questions are not biased (I bet you don’t even know what they are.)
4. The NCVS does not use unsound methods to eliminate bias. There is extensive documentation on the methods they use. I suggest you consult it.
5. The NCVS did not start with a conclusion. The NCVS was designed to measure crime and responses to crime. The DGU estimate is just one of the statistics you can get from it. On the other hand, Kleck was famous for his spirited advocacy of the existence of large numbers of DGUs before he conducted his DGU survey.

Ray writes:

Danny’s obvious reading disability has not allowed him to read this when I posted it before. Maybe he can get a friend to read it to him this time:

W A Collier writes:

Ray, one question for you: If all these other folks including Marvin E. Wolfgang (widely acclaimed as a statistician) found no fault serious enough to invalidate the methodology of Kleck, then how do you account for your posting? Are you a better analyst than Wolfgang and Kleck/Gertz - or are you simply making up a some of this, cutting bits and pieces from their contexts, and then changing contexts (as I have seen you do repeatedly - a bad flaw) to make it look as if it is damning enough to discredit Kleck? After all, Wolfgang himself wrote: “What troubles me is the article by Gary Kleck and Marc Gertz. The reason I am troubled is that they have provided an almost clear-cut case of methodologically sound research ..”

Seems pretty clear to me an acclaimed and admittedly anti-gun researcher found no flaws significant enough to discredit the work by Kleck - so where does your opinion and expertise come from (compared to a PhD and life long awarded fellow like Wolfgang)?

Here’s what James Q. Wilson has to say: (from The New Republic, August 25, 1997 v217 n8 p38(4))

Using the data compiled by the National Crime Victimization Survey (NCVS) of 56,000 families, scholars have estimated that there are, at a minimum, between 65,000 and 80,000 defensive gun uses per year. Some estimates based on private polls suggest much higher defensive uses, ranging up to 1.5 or even 2.5 million. The data supplied by private polls are controversial, since so much depends on inferring society-wide effects from the answers of a tiny number of respondents. (If, to take a recent study, only 54 people out of 2,500 surveyed said they used a gun to defend themselves, then each of the 54 represents 68,000 Americans. Reporting errors–lies, exaggerations, poor memory–on the part of just a few people can have huge effects on the total number of defensive gun uses.) So consider instead the much larger and more reliable NCVS, conducted by the Census Bureau, according to which defensive gun uses in America are not trivial: 65,000 to 80,000 uses each year.