The survey is confidential. The person surveyed cannot get into
trouble for mentioning a defensive use with an illegal weapon.
Nonetheless someone is less likely to report such a use, so the NCVS
will likely underestimate such uses. However, this hardly supports
your claim that it is a “gross underestimate” unless almost all
defensive gun uses are conducted with illegal weapons.
David Veal writes:
Considering the problems occasionally (and well publicized)
encountered by people using armed resistance to crime, it is possible
that people would be hesitant to talk to a government official even
if they were fairly certain they were in the right. Many jurisdictions
(especially those which are both firearms hostile and have a lot of crime)
have policies which effectively mean any self-defense, no matter how
justified, goes to the prosocuter’s rubber stam …. er, I mean
Grand Jury. :-)
And while we may be talking about a confidential survey, and the
wrong level of government, a lot of people may not be capable of
either making that distinction, or interested in doing so.
If someone really does think that the Census Bureau will breach
confidentiality and pass things to the local prosecutor the smart
thing to do is to refuse to participate in the survey (rather than
participate and then lie). After all, if you participate you may
accidently let something slip. The NCVS gets a very high
participation rate, so this suggests that the number who lie in this
way is very small.
It is also possible that Kleck’s high estimate was partly caused by
people telling stories about gun defences that happened to someone
else, or completely making it up. For example, his study implies that
citizens shoot 200,000 criminals each year, which is ten times as high
as the estimate he obtained by other means. This suggests that at the
very least, some exaggeration is going on.
We can reconcile the Kleck estimate and the NCVS one by assuming that
some people falsely claimed to Kleck that they used a gun (false
positive) and that some people falsely claimed to the NCVS that they
had not (false negatives). If this is so, the true estimate lies
somewhere in between 2.5M (Kleck) and 80,000 (NCVS).
To determine where it lies we need to know the relative likelihoods of
false positives and false negatives. If they are equal (that is, the
percentage of non-(gun defenders) who claim to use a gun to Kleck is
the same as the percentage of gun-defenders who claim not to have used
one to the NCVS) we can solve some equations to find the true
estimate: Letting l be the fraction who lie, and d be the true
estimate we have:
d*l = d - 80,000 (The fraction of gun defenders who lie is the
discrepancy between the NCVS estimate and the true figure.)
(250M-d)*l = 2.5M - d (The fraction of non-(gun defenders) who lie is
the discrepancy between the Kleck estimate and the true figure.)
Solving for d:
d = 80,000 /(1 - (2.5M - 80,000)/250M)
= 81,000
Under this assumption, respondents to the NCVS and to Kleck are both
99% truthful.
Readers may be puzzled as to why, if both groups are equally truthful,
the true estimate is not halfway in between the NCVS and Kleck
estimates. The reason is the number of people who did not use a gun
for self defence is much much more then the number who did. A very
small percentage of non-(gun defenders) corresponds to a very large
percentage of gun defenders.
Even if NCVS respondents are 20 TIMES as likely to lie as Kleck’s
respondents (surely a vast overestimate), the true estimate is still
only 99,000.
On pages 136-138 of “Point Blank” Kleck discusses Kennesaw burglaries.
He states that after Kennesaw passed a (purely symbolic) law requiring
a gun in every household, residential burglaries fell by 89%. His
explanation for this decrease is that publicity about the law reminded
criminals of the risks they faced from potential victims’ gun
possession and scared them away from burglaries in Kennesaw.
Kleck goes on to criticize a study that came to a contrary conclusion.
He writes “an ARIMA analysis of monthly burglary data found no
evidence of a statistically significant drop in burglary in Kennesaw
(McDowall et al. 1989). This study, however, was both flawed and
largely irrelevant to the deterrence hypothesis.” Kleck argues that
there were two flaws in the study:
1. Using a data source that lumped residential and non-residential
burglaries together. He considers this a flaw because his theory
predicts an effect only on residential burglaries.
2. Using raw numbers of burglaries instead of rates at a time when the
population of Kennesaw was increasing.
He offers the following table in support of his claim that these two
“errors” are significant:
Total Burglaries or % Change
Raw Number or rate? Just Residential 1981-82 1981-86
Raw Total -35 -41
Rate Total -40 -56
Raw Residential -53 -80
Rate Residential -57 -85
From looking at this table, it appears that these two “errors” made
McDowall et al report an 85% decrease as a mere 41% decrease which
they found not statistically significant.
As happens with a disturbing frequency with Kleck’s writings, when you
check out the source he cites you get a very different picture.
McDowall et al report the following (from UCR data)
Kennesaw Burglaries 1976-1986
76 77 78 79 80 81 82 83 48 85 86
41 21 22 35 35 54 35 35 29 32 70
McDowall et al note that percentage changes based small frequencies
can be misleading. For example, the decrease from 1981 to 1982 was
just 19 burglaries, but seems more when expressed as a 35% reduction
as Kleck does. Note further that if we compare 1979 (35 burglaries)
or 1980 (35 burglaries) with 1982 (35 burglaries) no reduction at all
is seen.
OK, so Kleck used a misleading presentation of the data in the
McDowall paper even though the paper specifically warned about such a
presentation. There are still more problems with Kleck’s table.
Firstly, the last column is mislabelled. The numbers in it correspond
to the % change from 1981-85. The sources that Kleck used to
construct the table gave raw numbers, not % changes. Here they are:
1981 1982 1985
Total Burglaries 54 35 32
Residential Burglaries 55 26 11
Now Kleck must have looked at these numbers to construct his table.
He comments on the difference between the two 1985 numbers, but does
not comment on the 1981 numbers. I find this extraordinary. The 1981
numbers are INCONSISTENT. It is not possible for total burglaries to
be less than residential burglaries. One or both of the sets of
figures must be incorrect. The figures for total burglaries come from
the FBI’s UCR, while those for residential burglaries come from the
mayor of Kennesaw, a strong supporter of the Kennesaw law. The most
likely explanation for the discrepancy is that we have another case of
a politician bending the truth for political advantage. I don’t
understand how Kleck could possibly have missed this.
Anyway, Kleck’s claim that it was an error to use UCR data is false,
since the other data from the Mayor is probably bogus. His claim
that it was an error to use raw numbers is also incorrect, and reveals
a lack of understanding of interrupted time series analysis. The
ARIMA model would be fitted to a steady increase in crime caused by
increasing population and enable such an increase to be controlled
for. The only way a population increase could mask a decrease in the
burglary rate associated with the law would be if the increase
occurred abruptly at the same time as the law (which it didn’t).
Kleck goes on to make two more erroneous criticisms of the McDowall
study. Firstly he argues that his theory predicts a deterrent effect
on occupied residential burglaries only. If these are displaced to
unoccupied and non-residential burglaries then the hypothesized
deterrent effect could occur without changing total burglaries. Kleck
accuses McDowall et al of ignoring his discussion that a major effect
of residential gun ownership may be to displace burglars from occupied
homes. Yet it is Kleck who has ignored a key fact from that
discussion: the occupied burglary rate in the US is quite low: about
14% according to NCS data. This means that Kleck’s own theory
predicts a reduction in residential burglaries of AT MOST 14% (and
that’s only if we assume complete success in deterring occupied
burglaries, no displacement to unoccupied residential burglaries
whatsoever, and that Kleck’s theory that high gun ownership areas
(like Kennesaw) would have lower occupied burglary rates is
incorrect.) The much larger decreases that Kleck claimed supported
his theory are in fact INCONSISTENT with it.
Finally, he argues that if McDowall et al had used a temporary-change
model (instead of a permanent one) and excluded the high 1986 burglary
data they might have found that the impact parameter was negative and
significant, supporting the deterrence thesis. This argument once
again reveals a lack of understanding of the interrupted time series
model. With a permanent-change model, excluding 1986 makes the impact
parameter smaller (since 1986 is one of the years the law is supposed
to affect). With a temporary-change model, excluding 1986 makes the
impact parameter LARGER (since 1986 is not one of the years the law is
supposed to affect). Consequently, if McDowall et al had used a
temporary-change model (instead of a permanent one) and excluded the
high 1986 burglary data they would have found that the impact
parameter was POSITIVE (though probably not significant).
To summarize: Kleck presentation of the Kennesaw data was misleading,
he failed to note obvious inconsistencies in the data, nor did he note
that even the faulty data did not support his hypothesis and his
criticism of the McDowall paper was wrong on each of its four points.