.
That is 100,000 (five orders of magnitude) times smaller than
the effect he actually attributes to carry laws.
You can also get a pdf version of this document or an html version that is all on one page.
However, as the next sections will show, it does not seem that these laws caused there to be significantly more guns, whether you use Lott's meaning for ``more guns'', or some other one.
Lott also presents an analysis (page 114) based on two surveys of gun ownership (conducted in 1988 and 1996) that purports to show that a 1% increase in a state's gun ownership causes a 4.1% decrease in the violent crime rate and a 3.2% decrease in auto theft.
Lott's two polls indicate that gun ownership increased by 50% in just eight years, from 26% to 39%. This is contradicted by everything else we know about gun ownership:
Since 1959, there have been at least 86 different surveys on gun ownership [21]. There doesn't seem to have been in any increase over that period, let alone over 1988-1996. The percentage of the population that declared they were gun owners varied between 25% and 35%, but there was no clear trend. It seems that the changes in the numbers are caused by sampling error, differently worded questions, and changes in the willingness of people to admit to gun ownership. Lott's apparent increase is an artifact of his having looked at just two polls instead of many.
Data on gun sales in the period 1988-1996 shows that the number of guns per person increased by just 10% [21,18]. Any increase in ownership rates is going to be less than 10% since the majority of gun purchases are by people who already own guns. Furthermore, gun sales per capita in the period 1988-1996 were only 5% more than in the previous eight year period. This does not seem consistent with the dramatic increase in gun ownership suggested by Lott.
Lott argues (page 36) that a doubling of spending on private security from 1980 to 1996 suggests that more people have been obtaining guns. However, despite an increase in population, gun sales in 1996 (4.8 million) were less than in 1980 (5.8 million).
For this reason, Gary Kleck (who strongly believes that armed citizens deter crime) does not think it likely that the carry law caused crime to decrease. He writes [21]
Lott and Mustard argued that their results indicated that the laws caused substantial reductions in violence rates by deterring prospective criminals afraid of encountering an armed victim. This conclusion could be challenged, in light of how modest the intervention was. The 1.3% of the population in places like Florida who obtained permits would represent at best only a slight increase in the share of potential crime victims who carry guns in public places. And if those who got permits were merely legitimating what they were already doing before the new laws, it would mean there was no increase at all in carrying or in actual risks to criminals. One can always speculate that criminals' perceptions of risk outran reality, but that is all this is--a speculation. More likely, the declines in crime coinciding with relaxation of carry laws were largely attributable to other factors not controlled in the Lott and Mustard analysis.
However, his claim is wrong since the group with the highest risk of being a crime victim are those with a criminal record (as Lott himself notes on page 8), and those with a criminal record are not eligible for permits.
Hood and Neeley [17] analyzed permit data for Dallas at the zip-code level and found exactly the opposite pattern from that predicted by Lott, that is, those zip codes with the highest violent crime rate before Texas passed its carry law had the smallest number of permits per capita.
A similar pattern occurs with seat-belt wearing. People who wear seat belts are less likely to have accidents [14], not more likely.
Lott's error is in making the false assumption that everyone is equally risk averse. People who are risk averse will both wear seat belts and drive more carefully, or get a concealed weapon permit and avoid dangerous situations.
Furthermore, permit holders will not always carry their weapons and will not always get a chance to use them, implying that permit holders will use guns for defense in a much smaller percentage of crimes than the number of permit holders suggests.
There is empirical data on how often permit holders use their weapons:
Dade county police kept records of all arrest and non-arrest incidents involving permit holders in Dade county over a 5 year period [8]. Lott cites this study to show that gun misuse by permit holders is extremely rare (page 11):
A statewide breakdown on the nature of those crimes is not available, but Dade county records indicate that four crimes involving a permitted handgun took place there between September 1987 and August 1992 and none of those cases resulted in injury.
Lott fails to note that the same study shows that defensive gun use by permit holders is also extremely rare (page 692 of [8]):
The Dade police recorded the following incidents involving the defensive use of licensed carry firearms: two robbery cases in which the permit-holder produced a firearm and the robbers fled, two cases involving permit-holders who unsuccessfully attempted to stop and apprehend robbers (no one was hurt), one robbery victim whose gun was taken away by the robber, a victim who shot an attacking pit bull; two captures of burglars, three scaring off of burglars, one thwarted rape, and a bail bondsman firing two shots at a fleeing bond-jumper who was wanted for armed robbery.
There were only 12 incidents where a criminal encountered an armed permit holder. Compare this with the roughly 100,000 violent crimes in Dade county in that period. Clearly the chance of a violent criminal encountering an armed victim increased by at most 0.012 percentage points. The true figure is considerably less since some permit holders may have carried legally or illegally before the law, only half of the 12 incidents involved defence against a violent crime, and crimes where are a gun is used defensively are more likely to be reported to the police than crimes in general. (NCVS data [38] indicates that 65% of crimes where a gun is used defensively are reported to the police, compared with 43% of crimes in general. Kleck's survey [22] indicates that 64% of defensive gun uses are reported to the police.)
Nor was Dade county unusual in that there were very few carry permits issued--at the end of the five year period there were 22,000 permit holders in Dade, about the same percentage as their was in the rest of Florida.
As far as Lott's assertion that permit holders face a higher risk of being attacked than the general population goes, the Dade study shows that the rate at which permit holders in Dade use their weapons for defence against a violent crime is only 12 per 100,000 permit holders per year. Compare this with a violent crime rate of about 1,000 per 100,000 population in Dade county. Needless to say, 12 is not higher than 1,000.
Since the gun stock can only increase, if by ``more guns'' you mean an increase in the gun stock, whether ``more guns'' are associated with ``less crime'' or ``more crime'' depends on whether crime rates went up or down. Since the US crime rate has been declining in the last few years, but the long term trend has been upwards, you can have ``more guns, more crime'' if you look in the long term, or ``more guns, less crime'' if you look at the short term. This is another reason why it is not useful to have ``more guns'' refer to the size of the gun stock.
However, while this theory is not advanced by Lott, there is a way that carry laws could have caused a reduction in crime without having ``more guns''. The publicity about the new law could have caused criminals to overestimate the risks that they faced from permit holders. If this was the case, you would expect to see an immediate decrease in crime followed by a gradual increase as criminals learned that they were not at any greater risk. Lott does not test for this possibility--his models either have an immediate effect or a gradual one, not a combination. However, Ayres and Donahue [2] test such a combination model and find exactly the opposite of what the publicity theory predicts--an immediate increase followed by a gradual decrease.
Goertzel argues convincingly that
When presented with an econometric model, consumers should insist on evidence that it can predict trends in data other than the data used to create it. Models that fail this test are junk science, no matter how complex the analysis.In the case of Lott's model we are in the fortunate position of being able to test its predictive power. Lott's original data set ended in 1992. Between 1992 and 1996, 14 more jurisdictions (13 states and Philadelphia) adopted carry laws. We can test the predictive power of Lott's model by seeing if it finds less crime in those jurisdictions. Ayres and Donahue [2] have done this test. They found that, using Lott's model, in those jurisdictions carry laws were associated with more crime in all crime categories . Lott's model fails the predictive test.
Ayres and Donahue go on to examine all the states adopting carry laws using data up to 1997 and found that carry laws were associated with crime increases in more states than they were associated with decreases. They rather pointedly observe that
Those who were swayed by the statistical evidence previously offered by Lott and Mustard to believe the more guns, less crime hypothesis should now be more strongly inclined to accept the even stronger statistical evidence suggesting the crime- inducing effect of shall issue laws.
In response to a similar critique by Alschuler [1] Lott argues that these correlations make sense because black females aged 40-49 could be more likely to be crime victims (page 144). It is true that if they were about 24 times more likely to be homicide victims than the general population, then the correlation would make sense. However, the FBI's Uniform Crime Reports [36] shows that black females aged 40-49 made up just 1.3% of murder victims. This is more than the 0.4% of the population that is black female aged 40-49, but clearly not 24 times more. And the association with a 30% decrease in rape makes even less sense. Even if no women in this group were ever rape victims, this would only account for an association with a 1% decrease.
Even more troublesome are the results of the two-stage least squares (2SLS) regressions. The correlations discussed so far were computed assuming that crime rates do not affect arrest rates, which does not seem a reasonable assumption. Table 11 of [31] reports the results of rerunning the regressions using two-stage least squares, which allows arrest rates and crime rates to affect each other. The size of the effect associated with the carry law are spectacularly different from those in table 3 of [31]. For example, the effect on property crime changes from a 3% increase to a 67% decrease and the effect on violent crime changes from 5% decrease to a 72% decrease. The 2SLS regressions are clearly spurious and indicate severe problems with the model.
Dezhbakhsh [9] offers further evidence that Lott's 2SLS model is incorrect:
[Lott] obtains mostly negative numbers for arrests. For example, more than 19,000 of approximately 33,000 county-level auto theft arrests are "negative"; the number of negative arrest rates for aggravated assault and property crimes are, respectively, 9,900 and 13,500. What does a negative arrest rate mean? Obviously, the number of individuals arrested for crimes can only be zero or positive.
Black and Nagin [5] report further evidence that Lott's model is not correct. They applied Heckman-Hotz tests [16] which indicated the presence of systematic factors, not modeled by Lott, which significantly affected the crime rate.
Figures 1-4 of their paper show that the carry laws were not consistently associated with a crime reduction in any crime category: that is, there were some models where the law was associated with an increase for each crime category studied. I should note, however, that if we restrict things to just models that include a trend component, homicide and robbery show consistent reductions. For this reason, Bartley and Cohen argued that Lott's results should not be dismissed as unfounded.
Dezhbakhsh and Rubin [10,11] re-examined the data using a more general model that allowed the carry law to have different effects in each county and to affect other parameters in the model. With this model they found the carry law did not have any clear effect on rape or assault, that it was associated with a reduction in homicide in six out of 33 states, and with an increase in robbery in 13 out of out of 33 states. The evidence here is stronger for an increase than for a decrease.
Plassmann and Tideman [39] point out that Lott's analysis technique assumes that crime rates are normally distributed and that this is not even close to being true for low crime counties. When they made some plausible changes to the specification, the effects on murder vanished. However, when they did their own analysis assuming that the murder rate was Poisson distributed, they found an even stronger effect (a 12% decrease). They also looked at the effects on each state and found a confusing pattern of results, with the effect varying from a statistically significant increase of 6.5% (Virginia) to a statistically significant decrease of 35% (Montana). While we would not expect the laws to have exactly the same effect in every state, it seems hard to see how the effects could be so radically different.
Duggan [12] points out another problem with Lott's analysis:
One problem with these regression estimates is that Lott and Mustard are implicitly assuming that these laws are varying at the county level, when in fact they are varying only at the state level.The reason this is a problem is that you would expect crime rates in counties within the same state to be correlated. This problem does not bias the estimates of the law's effect, but causes the standard errors to be underestimated, so that some results may appear to be statistically significant when they are not.
On page 278, note 3, Lott comments on this problem, but erroneously claims that including dummy variables for all counties solves the problem. This is clearly false. The dummy variables only account for fixed differences between counties and do not address the within-state correlations between counties.
After adjustments to account for this problem, Duggan found that
none of the coefficient estimates on the CCW variable remain statistically significant.
Lott's response to Duggan's paper was to repeat his false claim:
The correlation of the error terms across counties is picked up when one has county fixed effects included in the regression. He does not do the adjustment recognizing that the county fixed effects are already picking up what he wants to adjust for. [25]
Moody [35] noticed the same problem as Duggan:
Merging an aggregate variable with microlevel variables causes ordinary least squares formulas to severely overestimate the t-ratios associated with the aggregate variables. ...I reestimated the model using the original county-level data set but adjusted the standard errors for clustering within states. The results were somewhat different from the original Lott and Mustard findings. ...While shall-issue laws reduce violent crime in general in all models, the effects seem to be concentrated in robbery. Murder and rape are significantly reduced in only one version of the model.
In Lott's response to Moody [29] he still did not admit to making a mistake but rather stated that he ``had already discussed this issue''.
in their current condition, county level UCR crime statistics cannot be used for evaluating the effects of changes in policy.I should note, however, that state level analyses do not have the same problem and these give somewhat similar results to the county level one.
Crime goes up and down, and we do not understand all the reasons for this. Although Lott makes a commendable effort to control for as many factors as he can, there are still significant changes in the crime rate that are not explained by his model. A way to test to see changes in the crime rate were caused by the carry law or one of the other factors is to look for differential effects--we would not expect the carry law to affect all crime rates equally. For example, since juveniles are not eligible for the concealed weapon permits, we would not expect the law to affect the juvenile homicide rate as much as it effects the adult rate. With one somewhat problematic exception, we do not find differential effects.
However, the evidence from the Dade county study [8] mentioned earlier shows that the carry laws did not increase the probability of criminals encountering an armed victim.
Lott reckons that the carry law caused a reduction of 8% in murders, 5% in rapes, 7% in aggravated assaults and 2% in robberies. For Dade county that translates to 1,500 fewer aggravated assaults, 450 fewer robberies, 65 fewer rapes and 30 fewer murders each year, amounting to a total of about 10,000 fewer violent crimes over the five year period. This seems to be far too large an effect to be caused by a mere 12 incidents where a criminal encountered an armed permit holder.
By how much did the carry law increase the cost of crime to criminals? Arrest is 5,000 times more likely than encountering an armed permit holder. If the criminal considers these to be equally bad outcomes, then the carry law increased the cost by no more than 0.02%. It is absurd to expect a 7% decrease in crime from such an insignificant change in the cost.
Anyway, since Lott has estimated the effect of increasing the arrest
rate we can do a much more direct calculation. In chapter 4 he
reports that concealed carry had an effect of 4.8% reduction in
violent crime while a 100 percentage point increase in the arrest rate
reduced violent crime by 0.48%. The 12 incidents out of 100,000
violent crimes is equivalent to increasing the arrest rate by 0.01
percentage points. Consequently, Lott's model predicts that this
would reduce the violent crime rate by
.
That is 100,000 (five orders of magnitude) times smaller than
the effect he actually attributes to carry laws.
To be fair, it looks like Lott has made an error in reporting his results and understated the effect of increasing the arrest rate by a factor of 100. (The numbers seem ridiculously small, and the corresponding numbers in the table giving effects when the data is aggregated by state are about 100 times larger.) Even if we allow for this possible error, the effect is still 1,000 times smaller than the effect he actually attributes to carry laws.
Even if (contrary to what criminals said in Wright and Rossi's study of criminals attitudes to firearms [46]) criminals are not afraid of police guns but are afraid of victim's guns, the change in the cost is still insignificant. If you believe Kleck's survey of defensive gun use [21] there were at least 100,000 DGUs (defensive gun uses) in Dade county over the five year period. If you believe the NCVS [40] there were at least 2,500. Either way, the 12 by permit holders makes no significant difference to the total.
If we decide that the only cost that criminals care about is the chance of getting shot by a victim, then the comparison is even starker. Not one criminal was shot by a permit holder during the entire five year period, while probably around 500 criminals were shot by non-permit holders acting in self-defence [21].
Nor is it plausible that there were large number of DGUs by permit holders that were not reported to the police. While Kleck's survey and the NCVS give wildly different estimates for the number of DGUs they agree that about half of DGUs are reported to the police.
Some have argued that the publicity about the passing of the law caused criminals to mistakenly believe that the risk they faced increase. This is possible, but Lott didn't test this model (since he didn't have a variable for the publicity). Nor is this consistent with the results of his trend analysis, which shows that the decrease was small at first and gradually increased as time passed. The publicity about the law would have been greatest at the time the law was passed and lessened as time passed.
NCVS statistics indicate 36% of robberies and 10% of rape/sexual assaults occurred ``on street other than near home'' [37]. If the carry law caused a 20% decline in crimes in public places, you would expect a 7% decline in robbery and a 2% decline in rapes. The effect of the law might be something other than a 20% decline, but in general we should expect the effect on robbery to be about 3 1/2 times as great as that on rape. Instead, Lott found a 5% decline in rape and a 2% decline in robbery.
In response to a similar criticism made by Webster [45] Lott (page 133) offers two arguments:
First, that the effect on the slope was larger for robbery, that the robbery rate was increasing before the law and decreasing afterwards. However, this is undercut by Table 4.13 which shows the result of adding data for 1993 and 1994. This causes the change in robbery rates associated with the law to go from negative to positive. This suggests that robbery rates went back up after 1992.
The robbery rate was changing in a way that was not explained by Lott's model. Since we don't know what was making robbery increase, we have no way of knowing when the increasing trend would end (certainly it couldn't go on increasing forever). Assuming that robbery would have continued to increase for the whole period of the study without the carry law seems a little unwarranted.
There is also an element here of shifting the goal posts. If you can find a decrease in the rate, report that. If that doesn't work, look at the trends.
Second, that some robberies were not street crimes and the laws could cause an increase in other robberies by making them relatively more attractive. This argument seems to miss the point. The carry law could equally well cause an increase by substitution in non-street rapes. Lott offers no evidence at all that this supposed effect was different for rapes than for robberies.
Criminals respond to the threat of being shot while committing such crimes as robbery by choosing to commit less risky crimes that involve minimal contact with the victim.Unfortunately for this argument, the law was not associated with a significant decrease in robberies. In fact, when data for 1993 and 1994 was included, it was associated with a small (not statistically significant) increase in robberies.
The law was associated with a significant reduction in assaults, but there does not seem to be any reason why criminals might substitute auto theft for assault.
In the second edition of Lott's book table 9.1 shows the results of his latest analysis, using data up to 1996. This table shows that the effect on violent crime (-2.3%) is very similar to the effect on property crime (-1.6substitution effect, but, since it is not plausible that a carry law could cause a greater reduction in auto theft than in murder, there is evidence that both reductions were caused by some other factor. Rather than point this difficulty out to his readers, Lott quietly drops further discussion of the substitution effect.
In response to a similar argument made by Black and Nagin [5], Lott writes (page 143):
The difference that did exist across states can be explained by differences in the rate at which handgun permits were issued.
Lott has data on handgun permits for three states (table 4.7). This shows that the percentage of the population with permits in 1994 was 1% in Florida, 1.4% in Oregon, and 4% in Pennsylvania. However, this is exactly the opposite ordering of the changes in the violent crime rate: -4% in Florida, -3% in Oregon, and -1% in Pennsylvania. If differences in the rate at which handgun permits were issued explain this, it can only be if more handgun permits cause more violent crime.
Black and Nagin [5] also observe that the effects on murder and rape depend on the inclusion of Florida in data set--if Florida is excluded, the effect on murder changes from a 9% decrease to a 1% decrease, while the effect on rape changes from 3% decrease to 1% increase.
Lott offers several objections to this argument:
Firstly, he objects to Black and Nagin conducting their analysis using only counties with populations of more than 100,000. This is a strange objection to make, since Lott states that restricting the sample in this way makes the effects of shall-issue law stronger and more significant. In any case, it makes no difference whether the sample is restricted in this way since the effects on murder and rape also vanish when Florida is excluded from a sample containing all counties.
Secondly, Lott reports that when he reran all the regressions in the book without Florida, in only eight out of one thousand did the result change from significant to not significant. This goes some way towards alleviating the concerns about the influence of Florida on the results, but the fact remains that it did make a large difference to the most important regressions.
Thirdly, Lott suggests that Black and Nagin conducted a search for a specification that weakened the results and that traditional statistical tests of significance are based on the assumption that the most favourable (or unfavourable) one out of a number of tests has been chosen. However, it well known that all that is necessary to do in such a situation, is to adjust the level of significance accordingly, and Black and Nagin did so.
In response to a similar argument made by Alschuler [1], Lott argues (page 148) that concealed handguns could defend against estranged family members. While this is possible, it misses the point of the argument, which is not that carry laws would have no effect on in-family homicides, but that the effect would be less than that on stranger homicides.
Ludwig [33] used juvenile homicide rates to control for unobserved variables that may vary over time and found that, if anything, the carry laws resulted in an increase in adult homicide rates.
Lott offers two arguments in response to this criticism (page 147).
First, that ``criminals may well tend to leave an area where law-abiding adults carry concealed handguns''. This is contradicted by his own results: a substitution into property crime, and no displacement of violent crime to nearby areas without carry laws.
Second, that gun-carrying adults may be able to protect some youngsters. However, even we make the extremely generous estimate that this happens half the time, the effect of the law on the juvenile homicide rate would only be half that of the adult rate. The effect, as seen above was the same. A more realistic estimate, combined with the substitution effect mentioned on the previous, suggests that there should have been no effect, or a small increase in juvenile homicides.
His table 4.14 clearly shows that the violent crime in such counties did not change at all when the carry law was passed. However, because three of the four categories of violent crime showed an increase, and one a decrease, Lott argues that this shows that there was a spillover effect.
To illustrate that the results are not merely due to the ``normal'' ups and downs for crime, we can look again at the diagrams in chapter 4 showing crime patterns before and after the adoption of the non-discretionary laws. The declines not only begin right when the concealed handgun laws pass, but the crime rates end up well below their levels prior to the law. Even if laws to combat crime are passed when crime is rising, why would one believe that they happened to be passed right at the peak of any crime cycle?(You can see an example of one of the diagrams here). This is wrong. Lott's diagrams do not show crime rates at all, but rather plot two quadratic curves that he fitted to the data. It is no surprise that there is a peak when the laws were passed--this is one of the few places where it is possible for the fitted curves to peak. Even if the crime rate started to decline before the laws passed, Lott's diagram could still show a peak coinciding with the law.
I ran some experiments by fitting a similar pair of quadratic curves to a sequence of random numbers. Almost always the curves seemed to show that something had happened at the junction of the two curves, even though nothing had.
If your browser supports Java, the applet below lets you repeat my experiments. Each time you click on the ``Randomize'' button, a curve is fitted to a new set of random numbers. You can also use the left mouse button to move points to new positions to see how the fitted curve changes.
There are two problems here: firstly, he is using a counties' population as a proxy for the number of permits issued without validating that proxy. This is puzzling, since he has county level data for the numbers of permits issued for two states. It would be simple to check to see if relatively more permits were issued in high population counties.
Secondly, high population centres show greater variations in crime rates. The following table shows the standard deviation of the log of the homicide rate for US cities of different sizes over a 20 year period.
| City size | s.d. of log homicide rate |
| One million and over | 0.15 |
| 500k-999k | 0.12 |
| 250k-499k | 0.15 |
| 100k-249k | 0.10 |
| City size | annual change | annual change |
| in homicide rate | in homicide rate | |
| 1976-1991 | 1991-1997 | |
| One million and over | 0.5 | -1.7 |
| 500k-999k | 0.3 | -0.8 |
| 250k-499k | 0.2 | 0.4 |
| 100k-249k | 0.2 | -0.2 |
Lott and Landes also offer another argument why we might expect a greater effect on multiple victim public shootings: since there are more people present in such shootings, the chance of a criminal encountering a permit holder is greater than that for a single-victim crime. However, this argument is flawed, since all the costs of crimes are higher for multiple victim crimes--the criminal is more likely to be caught and convicted, the penalty will be higher, and they are more likely to encounter resistance from non-permit-holders. To put it in economic terms, what is important is not the absolute increase in cost of a crime, but the relative cost. There is no good reason to expect an increase of $2 in the price of something costing $20 to have a greater effect on demand then an increase of $1 in the price of something costing $10.
Furthermore, there is an upper limit to the cost of these crimes. Since the criminal dies in many of these crimes it is hard to see how the cost can increase from this.
For these reasons it is unclear whether you would expect to see a greater deterrent effect on multiple victim public shootings than on other crimes.
As to the question of whether ``more guns'' cause ``less multiple victim public shootings'', there are similar problems to the question as to whether ``more guns'' cause ``less crime''.
Firstly, there weren't significantly more guns.
Secondly, there are doubts about whether there were ``less multiple victim public shootings''. Although there was an 89% reduction when comparing before and after rates in states that introduced shall issue laws, (table 2 of [30]) shows that the average murder and injury rate from multiple victim shootings was 0.042 in states without such laws and 0.029 in states with shall issue laws (just 31% lower). The difference between 89% and 31% is enormous and suggests that something is wrong--the only way that the reduction could really have been 89% is if there was some other mysterious factor operating that would have caused the rate of mass public shootings in shall-issue states to have been much higher than in the other states, were it not for the shall-issue law. Lott and Landes do not identify such a factor.
Also, table 3.2 of the book indicates that the overall murder rate was 9.5 in states without such laws and 5.1 in states with shall issue laws (46% lower). That is, multiple victim shootings were actually relatively more common in states with shall issue laws.
Furthermore, it is absurd to suppose that the carry law could cause a decrease as large 89% in multiple victim public shootings. It is possible that some perpetrators might be deterred, but since many of them die from police weapons or their own weapons it is surely less than 89%. Indeed it is unlikely that as much as 89% of adults are even aware of the carry laws.
Lott and Landes found that that neither the frequency nor the severity of punishment had an effect on public shootings, suggesting that these crimes are not easy to deter. Lott believes that carry laws work to reduce public shootings not by increasing the cost of the crime, but by decreasing the value of the crime to the criminal. Lott writes [27]:
What motivates most of these criminals seems to be the desire for publicity. They want to kill as many people as possible. The possible presence of concealed weapons can limit the carnage, and thus the incentive to begin the attack.However, the average number of deaths per incident in public shootings in states with shall issue laws was 1.7 (my analysis of table 1 of [30]), almost the same as the 1.8 deaths per incident in states without shall issue laws. This very small difference does not seem like anywhere enough to make such shootings worthless to the perpetrators.
Also, out of the hundreds of cases studied, Lott and Landes fail to present a single case where a concealed handgun was used to limit the carnage in a mass public shooting. The closest they come is the description of two cases where a civilian gun was used for defense, one involving a shotgun and one involving a handgun retrieved from a car.
Lott and Landes present an analysis in table 10 that seems to indicate that the shall-issue law reduced the number of deaths per incident by 2.2, a figure much greater than the number of deaths per incident in states without shall-issue laws. This once again suggests that something is wrong with the model.
Lott is undoubtedly sincere in his belief that more guns caused less crime and one cannot be other than impressed by the energy he has devoted to marshaling the evidence in favour of his proposition and dismayed by the tactics of some of his opponents that he describes in chapter 7. However, he is none the less completely wrong--there weren't significantly more guns, there wasn't less crime, and the mechanism for causation just isn't there.
There may be good reasons for a state to introduce ``right-to-carry'' laws but reducing crime is not one of them.
In the list below, ``7.1'' refers to criticism number 1 in chapter 7.
His argument here is not even internally consistent. In his last paragraph he argues that if permit holders face the same risk of being attacked as everyone else, only 0.65% of permit holders need to thwart an aggravated assault to account for the observed drop in the assault rate. But in the previous paragraph he stated that only 0.18of the population are victims of aggravated assault, so if permit holders face the same risk as everyone else, only 0.18% of them will even have a chance to thwart an aggravated assault. This is much less than 0.65%, so if permit holders face the same risk, it is not possible for 0.65% of them to thwart an aggravated assault.
The fact that permit holders are less likely to be crime victims makes this comparison even worse for Lott.
Lott claims that the reductions in crime begin right when the carry laws were passed and that it too much of a coincidence to expect a crime cycle to have peaked exactly when the law was passed. However, as shown elsewhere the fact that the declines in Lott's graphs begin at the time of the law is an artifact of the way the graphs were created--it is not easy for a decline to begin anywhere else in his graphs.
If those states without right-to-carry concealed gun provisions had adopted them in 1992, county- and state-level data indicate that approximately 1,500 murders would have been avoided yearly. Similarly, we predict that rapes would have declined by over 4,000, robbery by over 11,000, and aggravated assaults by over 60,000.
These numbers are based on the constant-effect model, so it does not seem unreasonable for critics to concentrate on that model.
While there do seem to be some problems in classifying when Maine and Virginia passed their laws, showing that the results do not depend on Maine or Virginia is a satisfactory response.
Fortunately for Lott, his second argument is a better one--excluding arrests from the analysis has little effect on the results.
More about the problems with Lott's graphs is here.
Lott argues that concealed carry laws could cause an increase in gun ownership and hence deter crimes in homes and other private places. However, Lott himself noted (page 28) that a Texas poll suggests that 97% of first-time applicants for concealed-weapon permits already owned a handgun, that is, concealed carry laws do not significantly increase gun ownership.
Lott argues that while this objection was plausible when only a few states had implemented the laws, now that there is data from many states, the objection is now longer plausible.
The problem here is that while there is now data from many states, there is not much data from those states that have only recently changed their laws, and the results are dominated by the few states that have had their laws for a long time. When more data becomes
Lott goes on to argue that he can link the laws to the crime reductions because:
Lott does have a good point when he argues that it would be better if critics who argue that some other factor caused the reduction in crime would specify what that factor was, however, as Lott concedes, it remains possible that some unknown factor caused the crime changes. Lott's own analysis that found crime trends before the laws were passed demonstrates that there were factors operating that were not explained in his model.
Lott goes on to argue that the reductions were not caused by ``other factors'' because:
This is a very strange criticism, since Lott and Mustard did exactly the same the thing: Page 35 of [31]
We reran all the regressions in this section first by limiting the sample to those counties over 10,000, 100,000, and then 200,000 people. Consistent with the evidence reported in Table 7, the more the sample was limited to larger population counties the stronger and more statistically significant was the relationship between concealed handgun laws and the previously reported effects on crime.That is, according to Lott and Mustard's original paper, Black and Nagin's analysis was biased towards finding a beneficial effect for the gun laws.
Lott then writes:
Despite ignoring all these observations, it is only when they also remove the data for Florida that they weaken my results for murder and rape.This statement is false. Black and Nagin report that removing Florida makes the effects on murder and rape not statistically significant whether or not the analysis is restricted to large counties.
As Zimring and Hawkins note, this is not correct. If the critics are correct and the models wrong, you cannot draw any conclusion about the effect of the gun laws on crime. It would be possible for the laws to cause an increase, a decrease or to have no effect.
He also complains about ``the reluctance of gun-control advocates to release their data'', mentioning Kellermann's study (data available from the ICPSR, study 6898) and the Police Foundation study on gun ownership and use (data available from the ICPSR, study 6955). While Lott may have had trouble getting this data, it is publicly available on the Internet.
Lott goes on to argue that the reductions were not just the result of crime trends because
However, because the model he fits contains a term that includes the number of years since the carry law was passed, this result is also consistent with a crime trend.
To resolve the situation it would be necessary to fit a model with both individual nonlinear trends and a parameter that allowed the trend to change at the time of the carry law. We could then test to see if the change in the trend was statistically significant.
Unfortunately, while some of the models considered by Lott contain a parameter that measures the change in a trend at the time of the carry law, none of them contain nonlinear trends for individual states (and so would be expected to fail the Heckman-Hotz tests for misspecification). And while the model considered by Black and Nagin contains nonlinear trends for individual states it only allows the carry law to change the crime rate, and not the crime trend.
To summarize, Lott is wrong when he claims that nonlinear trends in each state make no sense, and we don't know whether his results on changes in crime trends associated with the carry laws would still be significant if nonlinear trends in each state were controlled for.
Now that Lott has moved to a different preferred model, the appropriate way to answer their objections would be test to see if the new model satisfies the Heckman-Hotz tests. Lott has not done this.
The explanation of the quote is simple. As the quote from Kleck inside the front cover of ``More Guns, Less Crime'' indicates, Kleck has no problem with the design or methodology of Lott's study. Kleck disagrees with Lott's conclusions because Kleck's own research indicates that a significant number of people carry guns for protection even without carry laws. The number of carry permits issued represents a relatively small increase in the number of people carrying, insufficient to cause the crime decreases that Lott claims the carry laws caused.
In a later paper on safe storage laws [32] Lott and Whitley do not use these exit polls. Instead they use GSS surveys to measure how gun ownership changes as a result of a state passing a safe storage law. Using these polls they find that gun ownership declined by one percentage point per year in the states with the laws and argue that the laws caused increases in crime rates.
Lott does not explain why, after stoutly defending his use of the exit polls to measure changes in gun ownership at the state level he abandoned them for his later paper. One possible explanation is that the exit polls say the opposite thing to the GSS surveys. The exit polls show substantial increases in gun ownership in the states that passed safe storage laws. I computed a regression relating the change in gun ownership as measured by the exit polls to the number of years that a safe storage law had been in place and found that the laws were associated with a 0.06 percentage point per year increase in gun ownership rates. This increase is not statisitically significant, but it is the opposite sign to Lott's result using the GSS surveys.
Since the exit polls give the opposite result from the GSS surveys it is quite possible that if Lott has conducted his analysis in chapter 3 using the GSS surveys he would have gotten the opposite result and found that more guns were associated with more crime.
It is not good practice to choose your data source to get the result you desire.
In the second edition Lott changes ``national surveys'' to ``a national survey that I conducted''. This is a fabrication. There is no evidence that Lott ever conducted such a survey. Lott provides some details about what he claims to have done in a letter to The Criminologist [28] and a phone conversation with James Lindgren [23].
Unfortunately for Lott, he first made the claim about 98% brandishing on Feb 6, 1997 [24] before the survey he alleges it came from was completed. Even if he did conduct a survey (unlikely, given the lack of evidence that he did), he has not told the truth about the origin of the 98% figure.
found that the probability of serious injury from an attack is 2.5 times greater for women offering no resistance than for women resisting with a gun. In contrast, the probability of women being seriously injured was almost 4 times greater when resisting without a gun than with resisting with a gun. ...
Men also fare better with guns, but the differences are significantly smaller. Behaving passively is 1.4 times more likely to result is serious injury than resisting with a gun. Male victims, like females also run the greatest risk when they resist without a gun, yet the difference is again much smaller: resistance without a gun is only 1.5 times as likely to result in serious injury than resistance with a gun. The much smaller difference for men reflects the fact that a gun produces a smaller change in a man's ability to defend himself than it does for a woman.
Southwick's description of his findings is very different from Lott's (page 362 of [43])
Table 6 also reports the number of serious injuries received by those who choose each set of actions. The only significant difference here is in the likelihood of receiving an injury if one takes no action.
What Southwick noted and Lott failed to report to his readers is that none of the numbers Lott reported are statistically significant.
Here are 95% confidence intervals (calculated from the data in table 6 of [43]) for the injury rate ratios mentioned by Lott:
| Ratio | value | lower | upper |
| limit | limit | ||
| female: no resist/gun | 2.5 | 0.35 | 17.4 |
| female: resist/gun | 4.0 | 0.57 | 28.1 |
| male: no resist/gun | 1.4 | 0.85 | 2.3 |
| male: resist/gun | 1.5 | 0.91 | 2.4 |
Even if the ratios were statistically significant they would not prove that resisting a criminal attack with gun is a safer choice than not offering resistance. Correlation is not the same as causation. The correlation in the NCVS data could equally well come about if serious injury made with-gun resistance less likely or if there were other differences between with-gun resistors and others. The BJS warns against drawing the conclusions that Lott does [40].
Cook and Ludwig conducted a multi-variate analysis of gun prevalence and burglary in the US [7] and found that where there were more guns the burglary rate was actually higher (that is, more guns, more burglary). Moreover, the ``hot burglary'' rate was not lower where there were more guns.
In a reply [28] to an article by Otis Dudley Duncan [13] that pointed out this error, Lott compounds his error. Rather than admit to making a mistake, he falsely claims that the table contains fifteen polls. It doesn't--there are thirteen polls listed in that table [22].
This is not the only way that Lott has misrepresented Kellermann's study. He claims that the case-control method, as used by Kellermann, was not designed to study these sort of issues because other factors could cause a correlation between gun ownership and homicide. This claim is also false. Lott fails to tell his readers that Kellermann did a multivariate analysis, controlling for dozens of other factors. Earlier (page 4), a simple correlation without controlling for other factors was enough for Lott to conclude causation when it suited his purposes.
. Lott has
obtained his percent changes by multiplying the change in the natural log
of the crime rates by 100. This is only an approximation to the
correct value, and for large changes, not a good one. Every single
percent change given in this table and the other tables giving percent
changes (tables 4.3, 4.4, 4.5, 4.6, 4.8, 4.9, 4.10, 4.11, 4.13, 4.14,
5.1, 5.4, 5.5, 5.6, 5.7, 9.1, 9.3, 9.4, 9.6 and A5.1) is calculated
incorrectly.
Lott arrives at his claim by taking the lowest available estimate for gun crimes (430,000 from the FBI's UCR) and a high estimate for defensive gun uses (An average of the estimates computed by Kleck [21], omitting the NCVS estimate). While that produces a ratio favourable to Lott's position, it is impossible for both estimates to be correct. According to the respondents in Kleck's survey (which is the basis for all the estimates computed in [21]) one fifth of his estimated 2.5 million defensive gun uses were against gun crimes, implying that every single time a criminal committed a gun crime, they encountered an armed victim. This is clearly impossible.
The only clues Lott gives me is references to two papers, one by Bartley and Cohen, and one by Plassmann and Tideman. I don't include any quotations from these papers, so perhaps Lott is trying to say that I didn't accurately report their conclusions.
Bartley and Cohen [4] summarize their conclusions in their abstract:
We find that the deterrence results are robust enough to find them difficult to dismiss as unfounded, particularly those findings about the change in violent crime trends. The substitution effects are not robust with respect to different model specifications.
I wrote:
if we restrict things to just models that include a trend component, homicide and robbery show consistent reductions. For this reason, Bartley and Cohen argued that Lott's results should not be dismissed as unfounded.
Lott also claims:
Bartley has another piece in Economic Letters where he describes how his paper with Cohen provides ``strong support'' for the deterrence hypothesis. [3]What Bartley actually says is:
Bartley and Cohen (1998) find that some evidence does exist from extreme-bounds analysis that violent crimes, such as rape and murder, may be reduced by the passage of these laws.The words ``strong support'' do not appear anywhere in Bartley's article.
Plassmann and Tideman [39] summarize their conclusions in their abstract:
John Lott and David Mustard have argued that their county-level weighted least-squares analysis shows that the right to carry concealed handguns has a statistically significant deterrent effect on crime. However, the number of crimes committed in a county in a year is a non-negative integer that is zero or one in most cases for some important crimes, which makes the estimates of an analysis that assumes a normal distribution unreliable. In a weighted least-squares analysis the conclusions of Lott and Mustard with respect to murders vanish when some plausible changes are made in the specification that is estimated. However, when the data are analyzed as the product of a generalized Poisson process, the average effect of shall-issue laws on the number of murders is even stronger than Lott and Mustard estimated, and the effect is estimated with much greater precision.I wrote:
Plassman and Tideman [39] point out that Lott's analysis technique assumes that crime rates are normally distributed and that this is not even close to being true for low crime counties. When they made some plausible changes to the specification, the effects on murder vanished. However, when they did their own analysis assuming that the murder rate was Poisson distributed, they found an even stronger effect (a 12% decrease).
Readers can judge for themselves whether my summaries are accurate.
I'm afraid that Lott has missed the point of the passage he quotes As indicated by the title, I was considering Lott's claim there had been substitution from violent crime to property crime (page 54). The basis for this claim is that table 4.1 shows a 4.9% decrease in violent crime and a 2.7% increase in property crime associated with the carry law. Lott can argue that looking at before-and-after averages can be misleading, but before-and-after averages is exactly what his claim about substitution is based on. If instead we look at changes in crime trends, table 4.8 shows that the law was associated with a decrease in the trend in the violent crime rate of 0.9% and a decrease in the trend in the property crime rate of 0.6%--not only was there no increase in the property crime trend (contrary to the predictions of the substitution hypothesis), but the change in the trends was similar for property and violent crime, more consistent with a general decrease in crime trends rather than a decrease just in violent crimes that might be caused by the carry law.
Secondly, the survey results did not include transportation of guns and hunting, but only counted guns carried for self-protection. The details of the survey are found in chapter 6 of Targeting Guns [21] which is about the carrying of guns for self-protection. I am surprised to find that Lott is unaware of current research on the frequency with which guns are carried for self-protection.
Thirdly, Lott argues that the percentage of people who get permits will increase in the future. However, Lott's thesis is that the relatively small number of people that got permits in the present caused a relatively large decrease in crime. It is not, I hope, that permits granted in the future are causing crime decreases in the past.
Finally, Lott asks ``why did violent crime rates in neighboring counties without the law increase at the same time that they were falling in neighboring counties with the right-to-carry law?'' The answer is that they didn't.
I'm rather sympathetic to arguments about double standards. I've spent quite a bit of time on talk.politics.guns arguing with pro-gunners and one thing that I've noticed is the way that many of them uncritically accept even the most unlikely pro-gun claim while subjecting pro-control claims to the most searching scrutiny imaginable. For example, many pro-gunners believe that the Japanese count many homicides as suicides, despite there being no evidence whatsoever supporting this claim. While they will claim that the paper by Kellermann at al [19] that found an association between gun ownership and homicide should never have been published because it didn't control for any other factors. (When in fact it controlled for dozens of other factors.)
Anyway, Friedman's case that Teret has a double standard is based on Teret's sympathetic comments an a study by Wintemute et al [] that found that criminal activity was associated with a preference for the purchase of small, inexpensive handguns. Friedman argues that the Wintemute study is markedly inferior to Lott's work because:
However, things are not as clear cut as Friedman believes. Firstly, there is one important way that Wintemute's study is superior--it is at an individual level rather than aggregating things into counties as Lott does. This is better since there is no reason to expect every part of a county to be the same. Secondly, one can well argue with the reason he gives:
Nonetheless, although it is debatable whether Lott's paper is markedly better in some sense, it doesn't seem to be true that it is markedly worse. Hence it seems probable that Teret is operating a double standard.
We could perhaps find a better example of a double standard if we looked at a study that had a similar design to Lott's. A study by Cummings et al [] used a pooled time series design similar to Lott's to study the effect of laws that make gun owners criminally liable if someone is injured because a child gains unsupervised access to a gun. They found that the laws were associated with a 23% reduction in unintentional shooting deaths of children.
Now Steve Milloy's Junk Science site criticizes this study. Here's what Milloy says:
This was an ecologic epidemiology study, meaning the conclusion is based on very "macro" comparisons of groups of people. The study involved no data about individuals, just groups. Traditionally, these studies are only useful for forming hypotheses for further testing, not irrefutable facts.
In particular, no data was collected on compliance with these laws and the relationship of compliance to the decrease in injuries. There may have been fewer unintentional firearm-related injuries in states with safe storage laws, but this study assumed compliance with the laws and assumed that compliance is responsible for the decrease in injuries. A big assumption considering the result.
The reported 23% decrease in injuries is a pretty weak result-probably beyond the capability of the ecologic type of study to reliably detect. Even in the better types of epidemiology studies (i.e., cohort and case-control), rate increases of less than 100% (and rate decreases of less than 50%) are very suspect.
So how much stock can be put in a weak result based on inadequate data?
Now this criticism applies equally to Lott's study, only more so, since the crime decreases found by Lott were much less than 23%. (For the bit that reads ``assumed compliance with the laws'' you need to read ``assumed frequent encounters between criminals and permit holders''.)
Furthermore, elsewhere on his site Milloy gives us six tips on how to spot junk science
So what does Milloy say about Lott's study? Do you think he condemns it as ``a weak result based on inadequate data''? Does he inform visitors to his site that it is junk science? Follow this link to find out.
I did find a study on criminologists' views on the death penalty which found that even 30 years after Ehrlich's work, criminologists remain unconvinced. Whether that is a reflection on them, or on Ehrlich's work, I cannot say.
Friedman's description of the history of the debate about the alleged deterrent effects of privately held guns is rather inaccurate. The pattern has been for some pro-gun person to argue for deterrence by using a simple comparison of a crime rate in a before year with a crime rate in an after year (for example, Kennesaw in the year before it passed its law to make gun ownership compulsory compared with Kennesaw in some suitable chosen year after the law) and then for a criminologist to perform a more sophisticated analysis using more data and find no effect. (In the Kennesaw example, an interrupted time series analysis using about 20 years of data found that the law had no effect.)
In the case of concealed carry, pro-gunners argued that such laws caused a reduction in the homicide rate based on simple two point comparisons. Some criminologists did a study using interrupted time series analysis in five representative cities in three states and found no effect. I'm not sure why Friedman claims that the study found ``that laws permitting concealed carry increase the murder rate''.
The curve fitting applet makes extensive use of Java code written by Bryan Lewis for his Elementary Statistics Applets. My thanks to Mike Huben, John Lott, Daniel Webster, Clayton Cramer, Tim Starr, William Vizard, Dan Black and Terry Austin for comments that have improved this document.
Special thanks to Florenz Plassmann for his thoughtful and extensive comments.