Deltoid

[On Oct 03 2002 I posted this to firearmsregprof.]

Norman Heath writes:

Tim Lambert asked for suggestions as to how John Lott might have formed a belief about the proportion of DGUs that involve gunfire, prior to having conducted a survery. I took this to mean that Lambert was open to those suggestions which involve innocent explanations. But when I offer an innocent explanation, Tim responds by pointing out that a sinister one also exists. I think Lott is owed the courtesy of assuming his innocence, and it still seems to me that there are justifications that Lott might have had, or thought he had, for saying that a “vast majority” of DGUs do not result in gun discharges, prior to his having conducted a survey on the subject.

Well, there certainly are, and I listed them. The most straightforward one is that by “vast majority” he was referring to the 76% that did not fire in Kleck’s survey. Duncan has suggested another possibility — that Lott misread Kleck’s table 1. This is the table that contains the numbers that Kleck derives from several surveys that Lott cites. In that table, directly above the DGU numbers that Lott cites, is a row entitled “% who fired gun”. The numbers in that row range from 2.6% to 16%. It is possible that Lott mistakenly took these as percentages of gun users rather than the percentage of respondents which they are.

One of these is the fact that the number of claimed DGUs looks to me to be much, much higher then the number of shootings that could conceivably be attributed to DGUs. The lowest figure for DGUs in the given poll was 746,000. What is the highest possible number of shootings that could conceivably be attributable to those DGUs, after we subtract known suicides, accidents, etc, from the total of reported shootings? Say the highest conceivable number of shootings attributable to DGUs appears to be 5000 (a wild guess). That would be less than 0.7% of the lowest poll figure for DGUs, and something over 0.1% of the highest. Even if ten or twenty warning shots or misses occurred for every DGU that resulted in a shooting, it would still indicate that a “vast majority” of claimed DGUs do not result in gunfire.

But your calculation is, by your own admission, based on a wild guess. The normal standards of scholarship would require you to tell your readers that and not present it to them as a fact.

An author’s having confidently expressed a belief on a matter prior to confirmation is NOT evidence that he has fabricated anything.

I have explained several times now why I think the matter is significant. You keep ignoring my explanation.

I look out my window and tell everybody in earshot that the vast majority of trees on my property are oaks. Now I will count them … ninety-eight percent of my trees are oaks.

Damn, I lost the paper I wrote the tree-count on. The fact that I expressed a belief while looking out the window at my trees will now be cited as proof that I did not later count my trees.

Moreover, my political enemies will also suggest that looking out my window and claiming that the vast majority of my trees are oaks was a “fabrication” on my part, since I had not yet counted them. That is the reasoning Tim Lambert uses in his last post.

That is not even remotely close to my reasoning. I see no point in repeating my reasoning. I refer you to my previous messages.

I knew that somebody would bring up Bellesiles sooner or later. Imagine the political utility of tainting John Lott just as the Bellesiles scandal comes to a head! Reason enough to pursue the matter on pure speculation!

You object to legitimate questions about Lott’s survey because they carry implications that might reflect on his character. And then you turn around and cast slurs on my character.

For the record: Your allegations as to my motivations are false.

I point out to Mr. Lambert that there is a mountain of evidence against Michael Bellesiles that Bellesiles himself produced. Bellesiles misrepresented the contents of innumerable documents which he then cited, and many of which were available for examination in libraries.

So is this like claiming that Kleck found that 98% of DGUs involved brandishing when, in fact, he made no such finding? Or claiming that “national surveys” gave the 98% figure when, in fact, they didn’t?

The complaint against John Lott is that he has no evidence. Can Tim Lambert see the difference between this and the case above? No evidence means NO EVIDENCE. We cannot contrast Lott’s claim with his evidence, as was done with Bellesiles, because the evidence is not available.

I did not say the cases were the same. The specific parallel I drew was with the probate data:

Bellesiles claims to have collected probate records that gave far lower gun ownership rates than those obtained by anyone else. Some of his results appear to be mathematically impossible. He claims his records were destroyed in a flood.

Lott claims to have conducted survey that gave far lower DGU shooting rates than those obtained by anyone other survey. Some of his results appear to be mathematically impossible. He claims his records were destroyed in a computer crash.

Presumably it was lost. Perhaps Lott diabolically destroyed the data, or perhaps he never had it. I do not know. But I do know that John Lott is entitled to the presumption that the data were lost in a computer crash as he claims.

That’s fine, as long as you give Bellesiles the presumption that his probate data was destroyed in a flood.

Lott’s opponents may dismiss the 98% figure as unverifiable, but they cannot properly accuse him of fabricating it until they have some real evidence. The fact that Duncan does not know what Lott said in legislative testimony several years ago, and does not know how to find out, is not “evidence.” Nor is the fact that Lott made a vague claim prior to nailing down a specific percentage for that claim.

The evidence, yet again, is that all seven other surveys gave a radically different figure. That it is mathematically impossible for a survey of the size of Lott’s to give a number that Lott says that he obtained from his survey. That Lott first claimed (falsely) that the 98% figure came from Kleck, and the first printed mention of this 1997 survey was in 2000 despite Lott citing the 98% figure dozens of times. That no-one, not colleagues, not collaborators, not experts on surveys that he consulted at the time on other matters, that not a single person that Lott can name was aware of a survey that would have required at least 300 person-days of work. That Lott has not provided any of evidence that the survey was carried out, despite having had ample time to do so.

When Lambert/Duncan can examine Lott’s data and demonstrate that he cooked the numbers then they can compare him to that person from Emory.

Why? Has someone examined Bellesiles’ probate data and demonstrated that it was cooked? Oh, wait. They can’t do that since Bellesiles say the data was destroyed.

In the meanwhile, it is perfectly plain from Prof. Duncan’s letter which Lambert forwarded to this list that Duncan is on a vendetta; that he is not merely trying, in scientific fashion, to replicate Lott’s results in the unstated hope that they will prove irreproducible.

For someone who demands that Lott be treated with kid gloves, you are mighty free with the personal attacks. Prof Duncan suspects, as do I, that Lott did not carry out the survey he claims that he did. I have presented several pieces of evidence for this. It is entirely legitimate to look for evidence that will resolve the question one way or the other.

Rather, he is looking for old legislative testimony in what he admits is a “longshot” hope of incriminating Lott personally. He should be embarrassed that Lambert distributed his letter in public. It testifies to Duncan’s lack of objectivity vis-a-vis Lott, and undermines his credibility on related academic matters. The fact that Duncan first made an effort to obtain the data from Lott is irrelevant.

I find it bizarre that you assert that Duncan’s search for evidence on both sides of question proves that he is biased. A biased researcher would only look for evidence that supports one side of the question—apparently you think that Duncan should only look for evidence that exonerates Lott—that is, you want him to be biased, but when he is isn’t you accuse him of bias!

If you have have access to the legislative testimony that Duncan asked about, then I suggest that you look at the testimony, and tell us what you find.

[On Oct 03 2002 I posted this to firearmsregprof.]

Glenn Reynolds writes:

I agree that Mr. Lambert’s “payback for Bellesiles” angle is pretty obvious.

Your allegation is false.

I also note that Lott isn’t accused of publishing fraudulent scholarship, but rather of making public statements that appear to be obviously true, but for which he has not published research as a backup.

Are you actually asserting that it is “obviously true” that “98 percent of the time that people use guns defensively, they merely have to brandish a weapon to break off an attack”? That’s the claim that Lott has made on at least forty (40) occasions.

If it turned out that Lott had pretended to conduct a survey that in fact he never did, that would be serious—not of Bellesiles’, er, caliber, but serious. Of course, if it turned out that Lott were the second gunman at the grassy knoll, that would be serious as well.

At the moment, Lambert appears to have as much evidence for the one proposition as for the other.

The evidence, yet again, is that all seven other surveys gave a radically different figure. That it is mathematically impossible for a survey of the size of Lott’s to give a number that Lott says that he obtained from his survey. That Lott first claimed (falsely) that the 98% figure came from Kleck, and the first printed mention of this 1997 survey was in 2000 despite Lott citing the 98% figure dozens of times. That no-one, not colleagues, not collaborators, not experts on surveys that he consulted at the time on other matters, that not a single person that Lott can name was aware of a survey that would have required at least 300 person-days of work. That Lott has not provided any evidence that the survey was carried out, despite having had ample time to do so.

It would be more constructive if you commented on the evidence instead of pretending that it does not exist.

[On Oct 7 2002 I posted this to firearmsregprof and emailed it to Lott.]

However, that isn’t what I was referring to when I wrote “mathematically impossible”. Lott often goes on to claim that 3/4 of the times the DG User fires the shots are warning shots, that only 0.5% of DG uses involve a shot fired at the offender. Kleck’s survey turned up about 200 incidents. Since Lott’s survey had half the sample size of Kleck’s, the most he could expect to find would be 100 incidents (slightly less if you allow for the slightly lower DGU incidence he found). 0.5% of 100 is 0.5. It is mathematically impossible for half a person to report firing at an attacker.

James Lindgren replies:

Tim, your comments seem to be based on an assumption that figures in national surveys are unweighted.

Yes, I assumed that the percentages Lott gave were based on unweighted numbers.

Most national studies use a multistage stratified sampling design, which typically requires that answers be weighted. Further, a well done sample design by telephone would weight by adult family size (otherwise small households are overcounted relative to large households). A respondent in a household in the US with 2 adults would usually be weighted about .98 and a person in a household with 1 adult would be weighted about .49 (which is awfully close to .5) Lott tells me that he did not do this sort of weighting for adult household size, but he claims to have used what is usually called post-stratification, though Lott didn’t use the term. Lott said he weighted his results to match demographic information from his larger study in the book. That could easily lead to weights of half a person

I agree that it is possible that you could get a weight of one-half that way, so I should not say that the result is mathematically impossible if that is what he did.

(but, of course, one wouldn’t normally report a rate of incidence for behavior for which only one person in a sample reported the behavior unless one had pretty close to 100% of the population in one’s sample, which he doesn’t)

However, that may well be what Lott has done. Another favourite Lott statistic, presented in dozens of opeds, speeches and in his book is that “the probability of serious injury from an attack is 2.5 times greater for women offering no resistance than for women resisting with a gun.” If you examine the NCVS data this is based on, you find that 1 out of 80 women who resisted with a gun was injured, whereas if the injury rate had been the same as for women offering no resistance it would have been about 2 out of 80.

Further, Tim’s computation of significance (not shown above) assumes that the same questions were asked and that Lott’s sample was as efficient as a Simple Random Sample. With demographic weighting, no weighting for household size, and a sample taken from a CD-ROM of phone numbers, Lott’s effective sample size would be considerably smaller than the stated sample size.

I’m not sure I follow you here. Surely the only one of those that effectively reduces the sample size would be the demographic weighting? And if he took a random sample of households, the weights would be mostly close to one, so that the effective sample size would not be much different.

Further, his results would be biased in favor of small households.

I looked at Kleck’s data and the fraction who fired seems to be about the same if I calculate it without weights. (Actually there don’t even seem to be weights in the data set.) So this cannot explain, even in part, Lott’s results.

Lott’s results would be almost impossible if he used the same questions to probe the behavior as other studies, had a sampling design as efficient as a Simple Random Sample, and was not biased against large families, but Lott’s study does not have any of these characteristics, so Tim’s computations of significance would not be accurate.

However, I think I’m within a factor of two, which is still enough to say that his results are almost impossible.

One must be extremely careful in making accusations to avoid error or overstatement (to the extent humanly possible). I’m sure that Tim (who has examined this far more than I have) will correct me if I’m in error–and I urge him to do so. I’d rather that we end up with the right answers and the issues clarified than defend my analysis–which might itself be based on misunderstandings.

I most grateful for your comments. As far as I know, Lott has told you more about the design of his survey than he has told anyone else.

Suspicions have arisen about Lott’s survey because he has failed to answer reasonable questions about it.

compiled by Otis Dudley Duncan and Tim Lambert
revised 23 Oct 2005 by Tim Lambert

Note: With the exception of academic publications, some tapes and some found by LexisNexis search, these were found on the Internet. The web is, of course, not perfectly reliable, and items appearing there can later disappear. This approximately chronological listing is probably incomplete, and we welcome additions and corrections.

This page documents how often Lott has made the false claim that 98% of with-gun defences involve merely brandishing the gun. A summary and the latest update on the controversy is here.

“polls of American citizens undertaken by organizations like the Los Angeles Times and Gallup showing that Americans defend themselves with guns between 764,000 and 3.6 million times each year, with the vast majority of cases simply involving people brandishing a gun to prevent attack.”
Does Allowing Law-Abiding Citizens to Carry Concealed Handguns Save Lives? Valparaiso University Law Review, 31(2): 355-63, Spring, 1997. [The same passage also appears in: John R. Lott Jr “Concealed Handgun Laws Can Save Lives” Agenda 3(4): 499-503, 1996]

“There are surveys that have been done by the Los Angeles Times, Gallup, Roper, Peter Hart, about 15 national survey organizations in total that range from anything from 760,000 times a year to 3.6 million times a year people use guns defensively. About 98 percent of those simply involve people brandishing a gun and not using them.”
Page 41, State of Nebraska, Committee on Judiciary LB465, February 6, 1997, statement of John Lott, Transcript prepared by the Clerk of the Legislature, Transcriber’s Office.

(more…)

Lott’s reply to Duncan’s article raises some disturbing questions about Lott’s honesty. See also James Lindgren’s report on his attempt to find some evidence that Lott actually conducted a DGU survey. (more…)

Joyce Lee Malcolm’s article in Reason online is here

In that article she claims that “And in the four years from 1997 to 2001, the rate of violent crime [in England] more than doubled.” and asserts that this increase was caused by British gun control.

It took me less than five minutes to find the official English crime statistics.

Going to the section on violent crime I find the following:

“Estimates from the BCS reveal large and consistent falls in violent crime overall since 1995.”

“Longer-term trends in violence overall continue to show significant declines. Comparison of results reported to the BCS in 2001/02 with those for earlier years show a 17 per cent decline in BCS violence since 1999, a 22 per cent decline since 1997 and a 33 per cent decline since 1995, all of these decreases being statistically significant”

“The fall in violent crime may seem surprising, given media attention to violent crime. However, the BCS suggests that violent crime in general has been falling for some time. Although BCS estimates present an average experience of violence, it is possible that the very rare but more extreme incidents of violence have increased at the same time. It is the latter that are more often reported in the media.”

I have written to Malcolm asking her if she will withdraw her article and replace it with one attributing the violent crime decrease to gun control.

Joyce Lee Malcolm has an article in Reason online entitled Gun Control’s Twisted Outcome. In that article she claims

“And in the four years from 1997 to 2001, the rate of violent crime [in England] more than doubled.”

and asserts that this increase was caused by British gun control.

However if you look at the official English crime statistics: Crime in England and Wales 2001/2002 and go to the section on violent crime you will find the following:

“Estimates from the BCS reveal large and consistent falls in violent crime overall since 1995.”

“Longer-term trends in violence overall continue to show significant declines. Comparison of results reported to the BCS in 2001/02 with those for earlier years show a 17 per cent decline in BCS violence since 1999, a 22 per cent decline since 1997 and a 33 per cent decline since 1995, all of these decreases being statistically significant”

“The fall in violent crime may seem surprising, given media attention to violent crime. However, the BCS suggests that violent crime in general has been falling for some time. Although BCS estimates present an average experience of violence, it is possible that the very rare but more extreme incidents of violence have increased at the same time. It is the latter that are more often reported in the media.”

(more…)