Deltoid

Yes, the measure of shooting DGUs is inaccurate because of the small number (16) of sample cases. A 95% confidence interval is 100,000 to 300,000 criminals are shot by armed citizens each year.
However, even allowing for the inaccuracy, the number does not seem possible.

Eugene Volokh writes:

(2) Could it be that this figure is not that far off the mark considering that a number of career criminal-career criminal confrontations may accurately be called “defensive gun use”?
Criminals, after all, sometimes have to defend themselves, too, and they might be quite honest when they say they were acting (in that situation) defensively. Of course, some might be less thrilled by those DGUs than by “honest citizen” DGUs; but the same overestimate may take place in the NCVS — it may be that many of the robberies and assaults counted by the NCVS, for instance, are basically bad guys robbing/assaulting bad guys. Am I missing something?

This does not seem likely. The number is greater than the NEISS estimate of all shootings (criminal, justified, suicidal and accidental). Kleck has used an estimate of a 15% fatality rate from defensive shootings. 200,000 such shootings would therefore result in 30,000 shooting deaths from self-defense shootings, which is much more than the total for all gun homicides, criminal or lawful.

(3) Finally, one thing that does make me doubt that the respondents were lying is that — as Kleck points out — the great majority of the reports did not involve the gun being fired. Why would someone lie in this sort of survey? Presumably, I’d guess, to make themselves seem macho to the interviewer (pointless, of course, but possible).

Some may believe that it is a citizen’s duty to fight back against crime, just as it is to vote. I believe that surveys tend to overstate the number that vote by a few percent.

But if so, why would a lot of the liars invent such humdrum incidents?

This seems to be buttressed by the other details; in most cases, the offender was said to be unarmed; in very many, there was no threat or attack by the offender; in 47% of the cases there was only one offender. Again, not very macho stuff.

It may be that they are describing real DGUs but changing some of the details as in who it happened to and when it happened. There is support for this in that Kleck found that 85% of the time the gun user was the respondent rather than someone else in the household. Since the respondent was randomly chosen from within the household the correct number should have been 51% (the average household size is slightly less than 2). Kleck found this surprising and decided that it was because the respondent was concealing DGUs by other household members. In table 1 it would appear that he multiplies the numbers from the other surveys that were household based by 1.7 to compensate for this under counting.

Alternatively, a respondent making up a DGU, or describing a friend’s DGU as if it happened to someone in the respondent’s household will tend to make him or herself the defensive gun user. It may be that of those that reported a DGU to Kleck, 70% made it up or changed the user to him or herself. That would leave 15% where the respondent was the user and 15% where it was someone else in the household.

There is a similar anomaly with when the incident occured. You would expect 20% of the DGUs to have occured in any one of the five years. But twice as many (40%) occured within the last year.

Alternatively, a respondent making up a DGU, or describing a friend’s DGU as if it happened to someone in the respondent’s household will tend to make him or herself the defensive gun user.

Joel Friedman writes:

While I do not disagree, your estimate of 70% / 15% is just that an estimate. It is also possible to argue that the numbers could be 50%/ 25% or 60 / 25% ? If this is not correct, please enlighten me as to why the 70% / 15% is more correct?

Out of each 100 reported DGUs, 85 involved the respondent, and 15 some other member of the household. The real numbers must be equal. Kleck believes that the true numbers are 85 and 85, that is, the respondents concealed 70 DGUs involving other household members. Alternatively the true numbers could be 15 and 15, that is, the respondents made up (wholly or partially) 70 DGUs involving themselves. This one factor alone makes an enormous difference to your estimate of DGUs: 170 vs 30, a factor of almost six.

It may be that of those that reported a DGU to Kleck, 70% made it up or changed the user to him or herself. That would leave 15% where the respondent was the user and 15% where it was someone else in the household.

There is a similar anomaly with when the incident occured. You would expect 20% of the DGUs to have occured in any one of the five years. But twice as many (40%) occured within the last year.

Seems likely, but doesn’t Kleck himself discuss this phenomena as well as the telescoping phenomena as the reason for this? Are not his explanations suitable to the discussion?

Telescoping that inflated the estimate by 100% could certainly cause such an anomaly. Kleck believes that the overestimate due to telescoping was no more than 21%. Another possible explanation: If you are making up a DGU, or perhaps changing the time of a real DGU to fit into the five year period that Kleck asks about you might randomly answer yes or no to the question about whether it was during the past year. If so, and if all the DGUs reported were made up, then you would see 50% of the DGUs reported as occurring in the past year. The fact that the number was neither 50% nor 20%, but 40% suggests that we are seeing some mixture of false and true reports of DGU.

Professor Volokh commented on cuing and the NCVS of DGUs. Cuing is mentioned quite a lot in the BJS report on the NCVS redesign. Does anyone know what estimate you get for DGUs from the 93 (redesigned) NCVS? Since the redesign turned up more crime, it should be a bit higher than previous estimates. One of the stated objectives of the NCVS is to measure victim’s response to crime and the people who did the redesign seemed well aware of the need for multiple cueing, so I’d be a bit reluctant to assume that messed up somehow and produced an instrument that gave wildly incorrect estimates of the responses to crime.

Discussion about correlation between gun ownership and crime: This arose from my statement that Kleck’s survey implied that 16% of burglaries resulted in a DGU. (Obtained by dividing the number of DGU vs burglary given by his survey by the total number of burglaries given by his survey.) I went on to suggest that this implied that at least 32% of US burglaries would have to have someone at home. Mark Gibson argued that the number might be smaller if burglaries were concentrated in gun owning households. Pim van Meurs argued that this was not true since crime and gun ownership were uncorrelated in general. I have to disagree here. AFTER CONTROLLING FOR OTHER FACTORS crime and gun ownership were uncorrelated. If you are interested in the question of whether general gun ownership causes or prevents crime, or if actual (as opposed to perceived) crime rates cause gun ownership, then this is the appropriate measure. However, we are want to know if 50% of burglaries are of gun-owning households. In this case we are just interested in the univariate correlation. Bordua (in a county level study of Illinois) found that crime was (quite strongly) negatively correlated with gun ownership in his univariate analysis. Consequently, it would seem that less than 50% of burglaries were of gun-owning households and the 32% figure a very conservative underestimate. It doesn’t seem likely that all burglars would seek a confrontation with a gun-toting resident so the true number would be quite a bit more than 32%.

James B. Clark writes:

Here’s your table. You didn’t include the data from 1967-1972, so I have no idea what it looks like. If you pick 1975, 1976, 1977, or 1978 I would bet that a decline in the homicide rates would be detected by the methods in the Loftin study.

Probably. All this means however, is that because the data is noisy, statistics cannot tell us exactly when the drop occured.

Charles Scripter writes:

Curious. If the data is so noisy that one cannot determine exactly when the “drop” occurred, then Pim certainly cannot claim that the “drop” corresponds to the gun ban.

He cannot claim that the drop occured exactly at the time of the ban. He can claim that the drop occured at about the time of the ban.

Even if we believe that all of Kleck’s respondents were truthful, his estimate of the number of DGU’s is plus or minus 600,000 (95% confidence interval). I don’t see you complaining that people cannot claim that the number is 2,549,862 because of this uncertainty.

James B. Clark writes:

If they had used used 1976 as the cutoff instead of 1977, they’d have gotten even more statistically significant results. The problem is, you can’t look at data, notice a trend, then test for the trend.

Which they didn’t do. (Else they would have used 1976 as the cutoff as you note.)

Charles Scripter writes:

If the “statistical significance” test would have indicated 1976, then how did they choose 1977? Ah, I understand… The authors wished to find a correlation with the 1977 gun ban, and massaged the data to match…

You’re confused. I’ll say it again — The problem is, you can’t look at data, notice a trend, then test for the trend. This is real trap in the analysis of quasi-experimental data. With an experiment you can spot a trend and then collect more data to test the significance. This is not possible in a quasi-experiment. You have to start with the hypothesis to be tested before you go looking at the data.