Deltoid

After Lott claimed that biased news coverage of the shootings at the Appalachian School of Law deliberately omitted a defensive gun use, I did my own analysis of the news stories and found that the alleged bias was the product of Lott’s flawed counting methodology. Lott has posted a spreadsheet listing 295 articles he found on Nexis, and a file containing 249 of those articles. Some of those articles he does not count because they are duplicates. He asserts that the coverage was biased because only 3 out 218 stories mentioned that the attack was stopped by armed students. Some of the differences in our counts are because we used different sources for the articles (Factiva vs Nexis), so I’ll redo my analysis using the articles Lott posted. I’ll count things in the same way if possible to see why we get different results.

I indexed and categorized the articles and placed them here. Lott has not counted stories that are exact duplicates from his count, but if two versions of a story are slightly different he counts both of them. For example, he counts this and this as different stories, even though they are almost the same. In order to be as consistent as possible with Lott’s counts, I will count duplicates the same way as him in the analysis below.

After removing the stories Lott marks as duplicates, I am left with 198 articles. Of these, nine mention a defender’s gun. (Lott counted seven—he seems to have missed two of them.)

Next, I leave out stories about the funerals and students being released from hospital, leaving 124 stories stories that mention how Odighizuwa was apprehended,

Rex Bowman of the Richmond Times Dispatch wrote a story on January 17 that stated “fellow students tackled and subdued him”, and then on January 18 wrote another story that stated “Odighizuwa … was wrestled to the ground by fellow students, one of whom aimed his own revolver at Odighizuwa”. Obviously the reason why Bowman didn’t mention the gun on the 17th wasn’t because he was biased against guns, but because he hadn’t learned about it. The only stories that could potentially exhibit bias against guns are those that appeared on the 18th or later. There are 25 such stories. Some of these stories don’t have any bylines and appear to have just been rewritten from wire service accounts. If reporter’s biases are removing references to defender’s guns, then we need to look at the original stories and not the ones without bylines. That brings us down to 14 stories by eight different sets of authors. I’ll look at each of these authors to see if any show signs of bias.

Chris Kahn

Kahn actually mentioned the defender’s gun in another story, so the claim that he deliberately concealed this information is impossible to sustain.

Mike Oduniyi

This story seems to have been written for Africa News because Odighizuwa was Nigerian. Oduniyi looks to have written his story from the news stories of 17 Jan and doesn’t seem to have talked to anyone himself. No evidence that Oduniyi knew about a defender’s gun.

Michael Beach

This story appeared in an Australian newspaper on the 18th when it was still the 17th in the US. It is based on the wire service reports on the 17th. No evidence that Beach knew about a defender’s gun.

Alfonso A Castillo

This story was about the murdered dean. Castillo just mentions in passing the police report of how Odighizuwa was apprehended. No evidence that Castillo knew about a defender’s gun.

Brian Hicks

Another story about the murdered dean which mentions the police report in passing. No evidence that Hicks knew about a defender’s gun.

Paul Dellinger

This story was about the memorial service. Dellinger talked to Mikael Gross about one of the murder victims and identified Gross as one of the students who tackled Odighizuwa. No evidence that Dellinger knew about a defender’s gun.

Adam Laskar

This story is about one of the murdered faculty. It summarizes the wire service reports of the 17th. No evidence that Laskar knew about a defender’s gun.

Maria Glod

Talked to Gross about the shootings. Did not mention that he was armed.

My basic result does not change. There was only reporter whose account could possibly be construed as biased against guns. Lott makes it appear that there is bias by counting all the reports from the 17th and 16th when the reporters did not know about the defender’s gun, and also counting all the stories that were about completely different aspects of the shootings.

Lott has a posting responding to my comments on his claims that the news coverage of the shootings at the Appalachian School of Law was biased. I wrote:

Unfortunately, Lott’s counting methodology is flawed, his count missed half of the stories that mentioned the armed students, his version of what happened deliberately omits important facts and omits contradictory accounts from other eye witnesses and his version contains details that appear to have been invented by Lott.

Lott has no answer at all to almost all of this, so he just responds to part of the criticism about his counting methodology. He once again deliberately omits mentioning Ted Besen’s contradictory account that strongly suggests that the guns were not used to stop the attack. He also carefully avoids mentioning or linking to my posting so that his readers won’t find out what Besen said. And remember that Lott is well aware that Besen and other witnesses say that Odighizuwa had dropped his gun before the armed off-duty policre officers arrived on the scene—he selectively quoted from Mathews’ article, he talked to Markus Funk who told him the same thing and now he is responding to my posting where I stressed the same fact.

It is hypocritical for Lott to accuse reporters of deliberately concealing facts while deliberately concealing facts himself. In a separate posting I redo my analysis using his set of articles and get the same results as before, but the most important thing to notice is the way Lott keeps avoiding mentioning that the “fact” of defensive gun use that he accuses the media of deliberately suppressing, actually isn’t a fact.

Here’s what I wrote about his counting methodology

I did my own search on Factiva of all the stories appearing in the week following the shootings. I found eight stories (twice as many as Lott found) that mentioned the students having a gun. Next, in his 208 stories, Lott has counted the same stories over and over again. Many papers printed the Associated Press story by Chris Kahn and Lott has counted it each time it appeared. He has also counted stories about completely different aspects of the matter—there is no reason for a story about one of the funerals to have mentioned a defender’s gun.

Lott reponds with

The general claim is that there are nowhere near 208 unique stories and that in order to get that number I must have counted each time an AP story appeared as a separate story.

He pretends that the only criticism of his counting methodology was about counting the same stories multiple times and ignores the part about him counting stories on different aspects of the matter.

Lott counts this story on Today as one of the stories where the media suppressed the “fact” of defensive gun use, but Tracy Bridges, who later claimed to have used his gun to disarm Odighizuwa, was speaking live on TV. It wasn’t that a reporter chose not to report the defensive gun use, but that Bridges never mentioned it. Given that other witnesses insist that there was no defensive gun use, it seems most likely that his first account, told to Today was accurate and that when talking to reporters later that day, he embellished the story and invented the defensive gun use.

Otis Dudley Duncan has sent me some comments on the attempts by pro-gun folks to dismiss criticism of Lott as some sort of payback for Bellesiles:

I have gone out of my way to remark that the Bellesiles case is not helpful for evaluating Lott’s work. My statement is in section 4 of the essay on Lott and surveys on your site.

Moreover, while I may have heard rumors about Bellesiles’ writing earlier, I am willing to testify under oath that the first thing of his I read was his article in Guns in America: A Reader, ed. by Jan E. Dizard et al. (1999). This was after my considerable effort to get a straight story out of Lott and, failing that, to run down the material in my article in the Jan./Feb. 2000 Criminologist. If I had had that book, I would have known about and referred to Lott’s 1997 Valparaiso University Law Review article, which included the erroneous statements about L.A. Times and Gallup polls. Obviously, my article was written without any thought of Bellesiles in mind.

After learning something about the case against Bellesiles, I did some cursory web surfing and got the general drift of the criticism. I have never studied the case in detail, but the evidence seemed overwhelming and there would have been nothing for me to say except—if one must say it—that behavior of the kind he has been found to engage in is totally unacceptable, reprehensible, and beyond the pale.

I do find interesting two comparisons between the two cases. The historians, after the Bellesiles evidence began to come apart, went into the evaluation of his writing with zeal and care. By contrast, American criminologists, while a few of them have written critically of More Guns, Less Crime, have simply had no public role in pursuing the question of Lott’s unethical behavior in attributing his purported survey results to other parties and in promulgating statistics for which absolutely no documentation can be produced. The compilation of that record has been left (so far as I know) to two people, and you know who they are. The historians seem to have different standards from the criminologists who should have been involved in compiling that record but were not. (I apologize if I have overlooked some relevant writing by American criminologists. I am assuming that some mention of it would have found its way into the Lambert web site.)

The other salient difference is that the institution that housed Bellesiles when he did his alleged research ultimately did the responsible thing and got a third-party, totally professional evaluation of it for public distribution. But neither the University of Chicago, nor Yale University, nor the American Enterprise Institute, where at various times Lott did and published his work, has lifted a finger (so far as we now know) to help clarify the record for public consumption.

(more…)

John Quiggin points to an interesting compilation of fake Internet identities.

Tbogg is surprised that there are still media organizations that take Lott seriously. He also wonders where Lott got his numbers from. Well, the defensive gun use numbers come from this survey Lott says he did, while the statistic about public school shootings was manufactured by selective quoting from news stories.

Tim Lambert
Justin Lambert
Daniel Lambert

Objective

To determine the relationship between gun brandishing and concealed carry laws.

Methodology

The frequency that mere brandishing was sufficient to scare off bad guys was obtained from statements made by John R. Lott Jr. The number of states with concealed carry laws at the time each statement was made was obtained from packing.org. The analysis also included demographic controls for age, race and gender using the 36 variables that Lott used in More Guns, Less Crime. Note that because all the statements were made by Lott there was no variation on race and age group and little on gender.

We counted each statement made by Lott as a independent observation. Because there were many repeated values, conventional statistical wisdom would tell us that the standard errors would be underestimated, but following the pathbreaking work of John Lott in More Guns, Less Crime, we ignore the problem. Plus, if we discard the duplicates, we only have six observations and the results aren’t statistically significant.

A quadratic fit was first attempted to the data, but this did not have the policy implications that we wanted, so we went with a linear fit.

Results

A strongly statistically significant association (p<0.000001) was found between the brandishing frequency given by Lott and the number of states with concealed carry laws. The lower the number he stated, the more states adopted such laws.

Policy Implications

To achieve his goal of carry laws in all 50 states, Lott needs to adjust his brandishing number to 68.6705%. He could make this adjustment by conducting another survey, citing the results from other surveys, or just making up a new number.

Future work

Our model predicts that if Lott lowered his number to 60% there would be 55.5649 states with carry laws, implying that states in other countries would adopt such laws. We urge Dr Lott to conduct the experiment to find out where those states turn out to be.

---

* School of Computer Science and Engineering, University of New South Wales
** Year 9, Sydney Boys High
*** Year 6, Woollahra Public School

Tom Spencer, commenting on Duncan’s comments that I posted yesterday, writes:

Unfortunately folks, as you well know, the damage is already done. We’ve got right-to-carry laws in the vast majority of states (mine being one of the notable exceptions of course) and there’s not much we can do about it at this point.

Meanwhile, Eric Rasmusen commenting on John Donohue’s “The Final Bullet in the Body of the More Guns, Less Crime Hypothesis” is more sceptical—he’d like to see more evidence that Lott’s work actually influenced the politicians who passed the carry laws. I agree that would be interesting. Rasmusen then goes on to argue that

I was also thinking that one of the lessons of the episode might be that it takes empirical work to get across theoretical ideas. The theory underlying Lott and Mustard’s work is simple: if more non-criminals are allowed to carry guns, then criminals, fearful of getting shot, will be more reluctant to try to rob people. … What all the scholarly arguments have been about is whether this theory shows up in the data we have available. But the big contribution of that debate may have been to bring the simple theory into the public debate.

The “guns deter crime” theory has pretty well always been part of the public debate about gun laws. Lott’s work did not introduce the theory to the public debate, but was just the latest failed attempt to provide empirical support for that theory. A previous example was Kennesaw, where a law making gun ownership compulsory supposedly almost eliminated burglary. However, careful analysis of the data showed that this was not the case.

Jeff Soyer has a post where he is taken in by a Lott opinion piece. He quotes Lott:

In a new book, The Bias Against Guns, Bill Landes of the University of Chicago Law School and I examine multiple-victim public shootings in the United States from 1977 to 1999 and find that when states passed right-to-carry laws, these attacks fell by 60 percent. Deaths and injuries from multiple-victim public shootings fell on average by 78 percent.

Exactly. I know there has been some minor controversy about his statistics of late but even if we cut in-half his pro-gun figures, it still has to give you pause. And NO ONE is suggesting that he is even NEARLY that far off.

The controversy about Lott isn’t just about his dodgy statistics but his misquoting and misrepresenting the findings of others. If you rely on Lott, you are unlikely to ever discover that a study by Duwe et al found no support for the hypothesis that right-to-carry laws decrease mass public shootings. So, in fact, people are suggesting that his results are more than that far off. And Duwe’s study has been published in a journal while Lott’s study has not been. I think that it is unfortunate that even though he knows Lott’s statistics are suspect, Soyer bases an argument on them.

Kevin Drum points out that a correction that Lott requested in response to this Washington Post item implies that Lott did not use “Mary Rosh” in emails when, in fact, Lott did. Drum thinks Lott is lying, which is certainly quite possible, but since there were hundreds of Rosh Usenet postings and only a few emails, it also possible that he just forgot about the emails. Colour me pedantic, but I wish Lott would quit referring to Usenet as an “Internet chat room”. Unlike Usenet, “chat rooms” have real-time discussions.

Everyone has something about Lott’s latest untruth that I mentioned yesterday. Mark Kleiman says

Isn’t there anyone in the management of the American Enterprise Institute with a modicum of institutional self-respect?

Brad Delong replies:

If they had any institutional self-respect, why would they keep James “Dow 36,000″ Glassman and Kevin “We’re Not Joking: Dow 36,000 by 2003″ Hassett? Why would they keep Charles “I Don’t Know What a Correlation Is” Murray? It’s not just John “I’ve Taken My Own Courses, and I Can Report I’m Really a Great Teacher” Lott.

ArchPundit comments:

Kevin and I have both tried to converse with Lott and it resembles an exercise in nailing jello to the wall.

Also commenting are Tom Spencer and Atrios.

Glenn Reynolds links to this Dave Kopel article and says that it shows that the New York Times reporting on guns was “riddled with amateurish errors and apparent deception”. Speaking of which, here is Kopel:

What if a gun owner does something very unwicked—such as saving dozens of people from a mass killer? Don’t expect to read about it in the New York Times. When a failing law student went on a murder rampage at Appalachian School of Law, Times reporter Francis X. Clines explained that the killing ended when the killer “was tackled by fellow students” (Jan 17, 2002). “Mr. Odighizuwa was subdued by three law students who were experienced police officers, the authorities said,” Cline wrote. What Clines and the Times omitted was that two of the law students who “subdued” and “tackled” the killer had retrieved their own handguns from their cars, and had used those handguns to “subdue” the murderer.

and this is what Clines wrote in the New York Times story (emphasis added):

Mr. Odighizuwa was subdued by three law students who were experienced police officers, the authorities said.

“We’re trained to run into the situation instead away from it,” said one of the three, Mikael Gross, 34, of Charlotte, N.C., who ran to his car for his bulletproof vest and service pistol before tackling the suspect.

That’s right, it was in the very next sentence and Kopel somehow failed to notice it.

None of this is new—Kopel admitted that the error was indefensible and posted a correction on The Corner some three months ago. Trouble is, the NRO hasn’t bothered to add a correction to the article. Is this an amateurish error or an apparent deception?

Update: Glenn Reynolds has an update where he notes that failure to correct errors in articles is a common problem at NRO. He also writes:

Notably, however, Lambert doesn’t try to defend Fox Butterfield.

I didn’t write anything about Fox Butterfield because I don’t know whether the charges are true or false. At the moment I have neither the time nor the inclination to investigate the matter. Given Kopel’s egregious error about another matter in his article, I am not going to take his word for it.

Further update: Tom Maguire examines the first thing Kopel writes about Butterfield and finds it wanting.

The Federalist Society is sending Lott on a speaking tour.  They asked Saul Cornell to debate Lott. This is what he sent them:  

Lott has been accused of research fraud and has lied on a host of other topics related to his research, including his participation in Internet discussions under a false female identity (cyber-cross dressing—very un-Federalist Society if you ask me—can you imagine Publius in drag?) I would gladly debate with any serious academic on this or other topics, but I have a general rule about not sharing the stage with frauds. In fact no serious scholar would even bother with Lott at this moment. He is only being kept afloat because he has never passed a serious tenure review and has jumped around on a series of ideologically funded fellowships. If he were in a regular academic job, he would certainly be the subject of an independent investigation. I would urge you to reconsider the invitation. I could easily suggest a host of more congenial and interesting persons to defend either the individual rights view or concealed carry laws. If Lott comes to town you are apt to make the issue seem silly—just imagine all of the Mary Rosh (Lott’s twisted cyber-sister) street antics you would encourage. Do you really want to make the cause of gun rights look just silly? I certainly would not want to encourage this. The decision is up to you, but I think you are making a mistake.

ArchPundit thinks that Tom Spencer has overstated the influence that Lott has had in getting carry laws passed. One piece of information that I can add is that most of the states that passed carry laws did so before Lott joined the fray. The only ones he could have influenced were Michigan (2001) and Colorado, New Mexico and Minnesota (2003).

Tom Spencer and Atrios reckon that the Federalist Society has embarrassed itself by promoting Lott.

On his blog, Lott offers an excuse for the fact that in his book and on his blog he had not mentioned that Ted Besen contradicts Bridges’ claim to have used a gun to disarm Odighizuwa:

I have gotten an e-mail asking about the role that Ted Besen played in stopping the Appalachian Law School attack during January 2002. While I had seen and referenced a story by Rick Montgomery, a reporter for the Kansas City Star, I hadn’t read down to the last couple hundred words of the 1,400 word piece that he published in March 2002. Montgomery’s piece contains a quote from Besen claiming that the attacker put his gun down before Mikael Gross and Tracy Bridges arrived at the scene. (Montgomery has seen copies of everything that I wrote on the incident and he has never mentioned these facts to me.)

The story about the Appalachian School of Law shootings is Lott’s best example of media bias against guns. He devotes several pages to it in his book and presents his version of what happened whenever he gives a talk promoting his book. Montgomery’s article in the Kansas City Star is one of only two news stories on the allegations of bias in the coverage and Lott quotes extensively from it. The other, longer, article appeared in Legal Times and Lott read to the end of that one, because he quotes the last sentence, and even telephoned the person quoted in that sentence to obtain more information. Lott also wrote a lengthy response to my first analysis where I quoted the entire last part of Montgomery’s article.

Given all of this, is Lott’s claim not to have read the last part of Montgomery’s article even slightly plausible? And even if we believe his claim, why should we believe his claims about new stories being biased when he can’t be bothered to read all of each news story?

Lott goes on to argue that of the news stories that mention Ted Besen “not one of the stories provide information that is consistent with what Besen told the KC Star”. This is not true. Of the 14 stories that mention Besen the only ones that are not consistent with Besen’s account are the three stories that mention that a gun was used to disarm Odighizuwa. Indeed, the whole point of Lott’s discussion is that there were only three such stories put of hundreds of stories covering the matter. One of the stories that is consistent is Bridges’ interview on the Today Show. These are Bridges’ own words, on live TV:

We seen the shooter, started to approach him, stopped at my vehicle, and got out my handgun, and started to approach Peter. At that time, Peter throwed up his hands and throwed his weapon down.

Note that he does not say that he pointed his gun at Odighizuwa—that detail was added later.

Lott also states:

One explanation that appears consistent with both Bridges’ comments and Besen’s later statement in the KC Star is that Besen was closer to the attacker than either Bridges or Gross and wouldn’t have clearly seen what Gross and Bridges were doing.

This is not consistent with Besen’s statement. Besen said that Odighizuwa put the gun down before Bridges and Gross arrived. That is, he saw Odighizuwa put the gun down and he saw Bridges arrive and Odighizuwa put the gun down first. Recall that Besen and Bridges were police officers who were together when the shooting started. Besen would certainly have been on the lookout to see when Bridges arrived to support him.

Note also that none of the accounts from other witnesses like Jody Mitchell, Robert Deatherage, Jack Briggs and Todd Ross mention any defensive gun use. It seems unlikely that they could have all missed it.

Update: In the account he gave on the Larry Elder Show Bridges said:

“We were located in the classrooms just across from where the first professor was shot. We heard the first three shots. At the time, we didn’t really know that it was gunfire. Just a few seconds later, we heard the next three shots, followed by some screams. Another student and I went into the hallway. We ran into a professor and he said that Peter (the gunman) was in the building and that he was shooting. So I ran back to the classroom and, what students were left, I said you all need to get out—there’s a shooter in the building.

“We exited out the back stairwell of our building. As I exited, that day I was running a little late, so I did park in a faculty parking spot, which put my vehicle between me and the shooter. We saw him in the front yard there. I stopped at my vehicle and got my handgun out.

“As we approached Peter, I started giving him a lot of verbal commands. … I told him to drop his weapon, to get on the ground. … His back was to us, and once he turned around and saw that I had a weapon, he laid his weapon down and stuck his hands in the air. At that time we approached him, and there was somewhat of a struggle, but we took him to the ground and handcuffed him until the authorities got there.”

The student he was with was Ted Besen and Bridges is saying that he and Besen approached Odighizuwa together. How could Besen possibily be unaware of what Bridges was doing?

(more…)

Summary: When Lott discovered that correcting his coding errors made his “More Guns, Less Crime” results go away, he refused to admit it. Instead, he tried to resurrect his results by quietly changing his model. When I emailed him, asking him to explain why he had changed his model, he made a clumsy attempt to cover his tracks.

The trouble with the econometric debate about “More Guns, Less Crime” is that it is so impenetrable. Lott has his model that shows that there was less crime, while Ayres and Donohue have their model that shows that there was more crime. You have to be an expert to figure out which model is more likely to be correct. Fortunately non-experts now have a solution—if you correct Lott’s coding errors then, even with Lott’s model there wasn’t less crime. Instead of conceding this, Lott has changed his model and then tried to rewrite history to make it look like he never changed it.

Lott claims that even after fixing the coding errors there were “clear drops in violent crime for murder, rape, and robbery.” To see what he means, we need to look at the numerical results of the regressions Lott, Plassmann, and Whitley computed.

There are lots of different regressions in their paper, but fortunately we only have to look at some representative results to figure out what is going on. I’ve chosen the crime trend using the Spline Model in their table 3a. These are the numbers that Lott usually gives when he describes his results. For example, in his 21 July letter he wrote:

Plassmann and Whitley, who examine three additional years worth of data and find “annual reductions in murder rates between 1.5 and 2.3 percent for each additional year that a right-to-carry law is in effect.”

The first column in the table below shows the annual change in crime rates in the years after the law was passed. For example, the first number says that murder fell by 2% each year after a carry law was passed. The numbers in bold are statistically significant (less than 5% chance of getting a number this large or larger if there really was no trend). The underlined numbers are strongly statistically significant (less than 1% chance). The numbers in parentheses are standard errors. The standard error gives us an idea of how uncertain the estimate is. The conventional rule of thumb is that the true value could be up to two standard errors higher or two standard errors lower. For example, the reduction in murder could be as much as 2.0+20.9=3.8% per year or as little as 2.0-20.9%=0.2% per year. If the model is correct, we can be reasonably sure that there really is a reduction in murder, which is why the number is marked as statistically significant. If the model is not correct, then the standard errors are meaningless. A statistically significant result does not mean that the model is correct.

	Lott, Plassmann & Whitley	Ayres & Donohue’s correction	Lott’s “correction”
Murder	-2.0% (0.9%)	-1.0% (1.0%)	-1.0% (0.3%)
Rape	-2.4% (0.8%)	-1.4% (1.0%)	-1.4% (0.3%)
Robbery	-2.0% (0.8%)	-0.9% (1.0%)	-0.9% (0.3%)
Aggravated Assault	-1.3% (1.0%)	-0.3% (1.2%)	-0.3% (0.3%)

The second column shows the numbers that Ayres and Donohue got when they reran Lott’s regression after correcting the coding errors. Notice that none of the numbers are statistically significant any more. This is why they said that correcting the errors eliminates Lott’s results.

The third column shows the numbers that Lott says he got when he reran the regression after correcting the coding errors. The reductions in murder, rape and robbery are all now strongly statistically significant. In fact, while the reductions are half as big as before, they are more significant. What’s going on here? Why are the results different?

If you look at the numbers you will see that the size of the reductions are not different—for example, they both indicate that there was an annual drop of 1% in murder. Where they differ is in the standard errors, which are much smaller in Lott’s “corrected” version. Lott has put his “corrected” computer programs on his web page, so we can find out what he has done to “correct” the coding errors in a way that doesn’t eliminate his finding. The program has been changed in one respect from what Lott, Plassmann, and Whitley used in their Stanford Law Review article. Lott has eliminated the command “CLUSTER,” which they had used to adjust their standard errors by clustering at the state level. If you don’t correct for clustering, then the standard errors you end up with are much too small, making you think that results are statistically significant when they are not. (Technical details on clustering are here.)

Lott, Plassmann and Whitley clearly state that they corrected for clustering:

Table 3a provides the exact results and significance levels behind these specifications, and reports the robust standard errors that adjust for clustering at the state level.

If you compare the first and second columns, you can see that the standard errors are about the same, indicating that both sets of regressions corrected for clustering. The standard errors in Lott’s “correction” are dramatically smaller even though in his table he claims:

Robust standard errors are shown in parentheses and clustering is assumed by state.

So Ayres and Donohue were correct when they stated that correcting Lott’s coding errors eliminated his results. Lott knows this is true because he must have run the regression after correcting the coding errors and before making the additional change of removing the clustering correction. It is dishonest of Lott to say that their claim is false. Lott can, if he wishes, make an argument that the clustering correction is unnecessary for some reason and offer his new regression as evidence that carry laws reduce violent crime, but it is not proper for him to pretend that he hasn’t changed the way he has calculated his results. Nor is it proper for him to put forward, as he does in his July 21 letter above, results from Plassmann and Whitley that murders fell by 1.5%–2.3% annually when even his own corrected figures contradict this.

The only thing that might suggest that Lott’s claim that his results survive the correction of his coding errors is an honest mistake is the fact that it is easy for anyone with a copy of Stata to run the regressions and see that Lott’s claim is false. But remember that all along Lott has been extremely reluctant to comment on the coding errors. According to his website, he made his corrections on April 18. If he really thought that the coding errors were immaterial, why did he wait four months to say so? Why did he duck all the questions he was asked about the coding errors? Why did he stop citing the research based on the miscoded data? He only claimed that the coding errors didn’t matter when he couldn’t evade the question any more and the only alternative was to admit that they did.

This whole affair also provides an insight into the way that Lott conducts econometrics. He tries lots of different models and just presents the ones that support his position. There are so many different models that could be considered that it is hard to show that Lott looked at a particular one, but here we have proof that he considered this model and when he found out that it didn’t support his position he switched to a different model with no correction for clustering.

Also note that Ayres and Donohue offer persuasive evidence that the model discussed here is not adequate and they go on to a more general model that shows that carry laws are associated with crime increases in most states. The point of all this discussion is that even with the model Lott chose, carry laws do not reduce crime.

I wrote the first part of this post on August 22. When the controversy over the fabricated survey erupted Lott wrote to Jim Henley:

One difference between newspapers and bloggers is that the newspapers at least contact the person and try to talk to them before they write something and put it out.

Fair enough, I thought. So I emailed Lott, asking him to explain what he had done. No answer. Maybe he is blocking emails from me, so I sent another email from another address. No answer. I emailed Lott’s co-authors, Plassmann and Whitley asking them (and CC’d Lott as a courtesy). No reply from Whitley. Plassmann was kind enough to reply. He conceded that no significant results remain after correcting the coding errors and did not know why Lott had removed the clustering correction. I also posted my question on the firearmsregprof list (also CC’d to Lott because I am very courteous). No-one there knew of a reason either.

Then, last week, something happened. The file containing Lott’s “corrected” Table 3a vanished from johnlott.org. Then, a few days later it was back. Only now it contained a different “corrected” Table 3a. How was it different? First, it no longer claimed to correct for clustering. More importantly, the results were computed from the data set with the coding errors. How come? Well, Lott is trying to pretend that he removed clustering before he corrected the coding errors. You see, if you use the dataset with the coding errors, you get significant results whether or not you use clustering. Removing clustering just so you can claim you got significant results is transparently dishonest, so Lott wants you to think that he first removed clustering (when it didn’t change the significance of the results) and then corrected the coding errors. Too bad this version of Table 3a appeared after the other one.

But wait, there’s more! I wondered when Lott had created this new version of Table 3a, so I looked at the modification date on the file. It said January 18, 2004, which is, uh, next year. Of course, the modification date doesn’t have to be when the file was last modified—you can set it to any date you please, but why would Lott set it to next year? It was only when I wrote the date out in numerical form that I was able to figure it out. The modification date was set to 01/18/04. On the page where you download the file, Lott claims that it was “corrected: April 18, 2003″, or, in numerical form, 04/18/03. Bingo! Lott was trying to set the modification date to April 18, 2003 so it would look like he created this version of the table first. He managed to set the day of the month to 18, but screwed up and set the year to 04 instead of setting the month to 04.

Unfortunately, Lott’s behaviour with the dates exemplifies his approach to research as well. It is way, way past time that pro-gun folks cut him loose.

A reader has pointed out that you can find out the real creation date of the latest version of Lott’s “corrected” Table 3a by looking at the File/Document Properties/Summary with Acrobat Reader. The document was created on Sep 2 2003 and not Jan 18 2004. A similar exercise with the earlier “corrected” version of Table 3a shows that that version was last modified on May 6 2003.

Glenn Reynolds calls this the “seemingly interminable John Lott coding error question”. I’m afraid the interminable nature of the question is entirely Lott’s fault. It took a brief exchange of emails with Florenz Plassmann to settle the issue to our satisfaction. If you correct Lott’s coding errors and rerun the regressions his results go away. There is absolutely no way that a reasonable person could dispute this. But Lott does. And he hasn’t stopped trying to get concealed-carry laws passed by advancing results that even he has conceded are based on miscoded data. In a letter published in the The Capital Times & Wisconsin State Journal on September 8 he refers to a

study by Professors Plassmann and Whitley that examines three additional years’ worth of data and finds “annual reductions in murder rates between 1.5 and 2.3 percent for each additional year that a right- to-carry law is in effect.”

That is what the study found, but you only get those numbers if you use the miscoded data. He just won’t stop lying.

Chris Lawrence writes

Tim Lambert does a pretty good job demolishing John Lott’s latest evasions

I think that part of his post is right, but this isn’t:

However—accepting that Ayers and Donahue do it right—there’s still the issue of null results. More guns may not mean less crime, but the results clearly show that more guns don’t mean more crime either, and the signs indicate that concluding “more” has less support than concluding “less,” although you’d have to be an idiot to come to either conclusion based on the Ayers and Donahue results (that’s why we call them “not statistically significant”).

These should not be called the “Ayres and Donohue results”—they are Lott’s results after correcting his coding errors. Ayres and Donohue offer persuasive evidence that the model Lott used is not adequate and they use a more general model that shows that carry laws are associated with crime increases in most states.

Tom Spencer wonders Just how dishonest is John Lott? and says that’s it time for Glenn Reynolds and Clayton Cramer to admit that Lott is a fraud.

Eric Scheie argues that statistics about guns and crime “have no bearing on basic constitutional rights”. I’m afraid that all the pro-gun folks who have based arguments on Lott’s statistics would seem to disagree with him.

Kevin Drum has an excellent summary of Lott’s cheating with his models and his attempt at a coverup. He concludes that Lott should be fired forthwith. Niraj agrees, as Tom Spencer.

Atrios says that Lott’s work on the Florida election is even more dishonest than his “More Guns, Less Crime” work.

skippy is amused by Lott’s apparent use of time travel. Tapped and buzzflash link here and send me many visitors.

Glenn Reynolds is wavering—he now says that he would quite reluctant to rely on Lott’s work.

Dick Dahl has a lengthy article about Lott and Bellesiles. He observes that Bellesiles lost his job bcause of his transgressions, while Lott doesn’t seem to have suffered any consequences.

Lott has a response to my post on the way he switches models. I was hoping he would respond to the question I have been trying to get him to answer for weeks: “Why did he remove the clustering correction from his model?”, but he evades it again. Also of interest would be an explanation of why, after I started questioning about his table that used the corrected data, he removed it and replaced it with one based on the miscoded data. Instead, all he does is attempt to explain the January 18, 2004 date stamp on the latest version of his table.

Lott writes:

I use a Mac using system X and if you were to look at the date setting file, you would see that the months can only be set by name. The year dates are also listed as “2003″ or “2004″ instead of just “03″ or “04″ and in the program the months are listed right above the year in any case. In other words, in order to have the claimed result, I would have had to confuse “April” with “2004″ while simultaneously ignoring the fact that the month listed right above it read “January.” I am not sure what the theory regarding the “01″ at the beginning of the sequence involves. Just to make it easier for to see how Apples work, I have included a pdf picture of the Date & Time setting file from my computer. Sorry, I haven’t always worried about properly setting the dates on all my computers.

I’m afraid this doesn’t wash. If the date on the computer he used to create the file was accidently set to Jan 18, 2004, then the creation date given in the File/Document Properties/Summary would also have been Jan 18, 2004. Instead, it was Sep 2. Also, using the Date & Time control panel isn’t the only way to set the date on a file. You can, for example, use the “touch” command from the console. The “touch” command uses a numerical date format. Indeed, if he had set the computer’s date with the control panel then the Document Properties date would have agreed with the file date.

I checked all the downloads from johnlott.org, and the only file with a bad date is the one with the new table—the one that would have been helpful to Lott if it was created back in April, instead of after I started questioning him about his reasons for dropping the clustering correction. This hardly seems consistent with Lott’s suggestion that his computers might have incorrect date settings.

Andrew Chamberlain observes “Once again, economist John Lott has been busted for lying.”

Julian Sanchez writes about Lott: “It’s long past time for people who care about gun rights to cut this albatross from our necks.”

William Quick says that “it doesn’t look good for Lott at the moment”, but asks for a simple explanation. I recommend Kevin Drum’s summary.

It has now been one year since I asked him for evidence that he had conducted a survey. The original email is here.

In comments James and Rick poke more holes in Lott’s excuse about the file from the future. James points out that Mac OS X machines automatically connect to network time servers to keep the time and date accurate, while Rick shows us that Macs can use a numeric format for months—they don’t always use a name as Lott implied.

Kevin Drum does not think much of Lott’s latest evasion. Tom Spencer agrees that the date excuse is just another way of Lott avoiding the serious questions about his work.

Marie Gryphon writes about Lott:

This former darling of the gun rights movement has surely become its premier enfant terrible. Get us a new economist, please!

John Constantine awards Lott his Psycho Babble Of The Week Award:

I simply stand in shocked awe of John Lott. His inability to face reality and his preternatural ability to make excuse after excuse without tiring of them is truly stunning to behold. His explanations truly worthy of the title “Psycho Babble”.

Meanwhile, Skip was banned from the LiveJournal conservatism forum for daring to criticise John Lott.

Eric Helland and Alexander Taborrak have a new paper “Using Placebo Laws to Test More Guns, Less Crime : A Note”. I commented on their paper back in May, but here is Lott’s take (in full, 8/22/03 blog entry):

A new research paper has an new important approach towards estimating statistical significance. Professors Eric Helland and Alex Tabarrok conclude that:

“the cross equation restrictions implied by the Lott-Mustard theory are strongly supported.”

“Surprisingly, therefore, we conclude, that there is considerable support for the hypothesis that shall-issue laws cause criminals to substitute away from crimes against persons and towards crimes against property.”

Compare the parts Lott quoted with their abstract, so you can see what Lott left out:

A boomlet has occurred in recent years in the use of quasi-natural experiments to answer important questions of public policy. The intuitive power of this approach, however, has sometimes diverted attention from the statistical assumptions that must be made, particularly regarding standard errors (Bertrand, Duflo and Mullainathan 2002, Donald and Lang 2001). Failing to take into account serial correlation and grouped data can dramatically reduce standard errors suggesting greater certainty in effects than is actually the case. We reexamine Mustard and Lott’s important and controversial study on the affect of shall-issue gun laws on crime using an empirical standard error function randomly generated from placebo laws. We find that the in some specifications the effect of shall-issue laws on specific types of crimes is much less well-estimated than the Mustard and Lott (1997) and Lott (2000) results suggest (i.e. placebo shall-issue laws produce estimated real effects at greater rates than suggested by the standard errors in the original studies.) We also find, however, that the cross equation restrictions implied by the Lott-Mustard theory are strongly supported.

Taking into account grouped data is what the clustering correction does. Helland and Tabbarok show that even this correction isn’t enough and it is wrong for Lott to remove it. No wonder Lott didn’t quote that bit. In fact, the only statistically significant decrease that they found was that on murder using the trend model. However, they only used the data up to 1992 for this. Include the data up to 1997 and that result goes away too.

As for the cross equation restriction, that only occurs with the dummy variable model, which Lott abandoned when the addition of more data meant that it stopped supporting his thesis. Is he going to pick it up again?

Alan Schussman has a thoughtful post on the issues of how scientists should treat their data and how it relates to Lott’s conduct.

Jeff Soyer has a lengthy response to my correction of his use of a bogus Lott statistics. Instead of correcting himself he insists that “Alphecca isn’t about statistics but about bias. Lambert ignored that.” Right. He just quotes statistics but he isn’t “about” statistics so it doesn’t matter if the statistics he quotes are correct or not.

Soyer goes on to criticize my statistics by making a pure ad hominem attack on John Donohue. He can’t find anything wrong with Donohue’s work on concealed carry, so he turns to Donohue’s work with Steve Levitt that found evidence that legalized abortion led to large decreases in crime by reducing the number of unwanted children, children who were more likely to be neglected and grow into criminals. He quotes from a page by one Angela Franz at the National Right to Life site, who more or less accuses Donohue and Levitt of being Nazis. Her argument is that allowing poor women to choose abortion is essentially the same as the forced sterilization of “defective stocks” advocated by eugenicists. And the Nazis were eugenicists too, so there you go. It would seem that Soyer has never heard of Godwin’s Law.

Update: Soyer has an update where he writes:

The fact that one of the sources I quote imply that Donahue endorses Nazi solutions leads him to, I presume, his final comment accusing me of using Nazi tactics. His Godwin’s Law comment is typical of all the leftist-liberals who also compare George Bush to being equal to Hitler.

Not only has Soyer not heard of Godwin’s law, he doesn’t understand it, even though I provided a link to the definition. And it is rank hypocrisy for him to complain about “leftist-liberals” calling Bush a Nazi right after he calls Donohue a Nazi.

ArchPundit writes:

The simple question to Lott is why did he stop correcting for clustering in the observations. Hear the silence?

Chris Mooney has a very interesting article about the dubious techniques used by creationists to make it appear that there is strong public support for teaching creationism in schools alongside scientific theories. My favourite was this question:

Texas law requires students to analyze, review, and critique scientific explanations, including hypotheses and theories, as to their strengths and weaknesses using scientific evidence and information. Should the state board of education apply this standard to how evolution is presented in textbooks?

People who agree with that are apparently agreeing that “intelligent design” should be taught beside evolution. I bet you’d get less support if you asked:

Should religion be taught in science classes in schools or should such teaching be left to the churches?

I can add an anecdote to this: A couple of years ago I was visiting my parents and there was a lively disscusion about how certain things had evolved. My brother’s girlfriend listened with rapt attention. When we were finished she spoke up: “Wow! That was amazing! They never taught me that stuff in school.” Turns out that she had gone to an Adventist high school and they had never taught her about evolution. I don’t know how the school pulled this off—presumably evolution was in their syllabus so they could get government funding. My guess is that they just skipped over that bit.

Over at the History News Network, a die hard Bellesiles supporter who posts under the name “Benny Smith” has attacked James Lindgren for, get this, his “ill-fated attempt to defend” Lott. Here’s Smith’s version of what happened:

After serious questions arose regarding a survey that Lott claimed to have done, Lindgren interviewed “at length” a man who claimed to have been interviewed by Lott for the survey. Lindgren pronounced him credible and the Washington Times in a follow-up article credited Lindgren when it said the matter of whether Lott had actually done such a survey was partly resolved. Yet others investigated further and found that the man who claimed to have been interviewed by Lott had extensive connections to the gun lobby, and was himself a former NRA board member. Apparently, this was not significant enough for Professor Lindgren to investigate himself. That Lindgren should naively trust hard core gun rights activists at their word is not at all surprising, however. Lindgren’s scholarship has borrowed liberally from gun rights propagandist and right wing extremist Clayton Cramer, whose own vendetta against Bellesiles rivals Lindgren’s own. It was up to another investigator to discover that Lott himself was masquerading on the internet as “Mary Rosh”, alternately praising and defending himself in various forums. No wonder that Lindgren excused himself from investigating further, claiming ironically enough that “this project detracts from my other scholarly efforts.”

Smith’s attack is both misleading and unfair. It is misleading not to mention Lindgren’s extensive and meticulous report that actually helped raise the serious questions about Lott’s mysterious survey. It is unfair to imply that Lindgren’s investigation was somehow inadequate just because others discovered some of the facts about the matter. Why is one person expected to uncover everything about Lott’s misconduct?

Lindgren also sent me his own reponse to Smith:

Tim, the quote in the Washington Times from last January, as I wrote you privately at the time, left out some of the qualifying language I had used and ignored the concerns that I raised about the faulty nature of the estimate, even if it were based on an actual survey. You noted at the time: “McCain selectively quotes Lindgren to make sure his readers do not learn that there are still unresolved questions about Lott’s claim.”

Remember that this Washington Times story came at the brief time when both you and I thought that it was more likely than not that some sort of study was done in 1997, though not necessarily what was described in More Guns, Less Crime. As more information came out, including my belief that Lott continued to change his story from what he told me and the possibility that Mr. Gross was surveyed by a different survey organization than Lott (something I looked into enough to think it possible), I fairly quickly returned to being highly uncertain whether Lott ever did the survey he claimed to have done in 1997. I have told the couple of reporters who called me since February 1 (including a few weeks ago, Chris Mooney for a forthcoming story) that I still have substantial doubts whether John Lott ever did the supposed 1997 study.

I also agree with almost every point in Ayres and Donohue’s two critiques of Lott’s work in the Stanford Law Review, which I find absolutely devastating to the primary thesis of John Lott’s work. The findings of Ayres and Donohue tend to support the conclusion that more open gun laws either have no effect or lead to slightly higher rates for some crimes, a result that I find plausible even beyond the high quality of their work in that exchange.

One more thing: Smith’s contention that Lindgren’s goal was to defend Lott is the other side of the coin from Lott’s claim that Lindren’s goal was to prosecute Lott. I have corresponded extensively with Lindgren on this matter and I am confident that they are both wrong and his goal was to get to the truth of the matter, just as it was in the Bellesiles affair.

Alex Tabarrok has some more on the question of whether Iraqis were well armed while Saddam was in power. (My earlier comments are here.)

He points to a New York Times article that states: “Mr. Hussein, never one to tolerate competition, forbade private citizens to carry weapons, effectively outlawing the security industry.”, and suggests that contradicts an earlier New York Times article that reported that guns were easy and legal to obtain in Saddam Hussein’s Iraq. Tabarrok concludes:

Clearly, the New York Times is wrong. But where does the truth lie?

However, it is possible that Iraqi civilians might have been allowed to own guns and keep them in their homes, but not to carry them in public, so there isn’t necessarily a contradiction between the two reports.

Lindgren’s latest comments have prompted Glenn Reynolds to write:

I trust Jim Lindgren as a neutral arbiter with expertise in the area, and it seems to me that this time Lott’s critics have him dead to rights, and he’s failed to mount a convincing response.

In an update he writes that he is getting email from Lott supporters complaining that he is picking on Lott. I hope those emails aren’t all coming from the same IP number. Reynolds also calls for an investigation by a disinterested party. I think that is a good idea. Emory had an inquiry into Bellesiles, the AEI should follow their lead. If Lott believes himself to be innocent, then he should support this as well. Chris Lawrence, however, isn’t optimistic. He says that

Anyone motivated enough to “come forward and sort all of this out” would, by definition, no longer be disinterested.

Eugene Volokh writes

Lindgren is a very smart guy who knows a lot about quantative scholarship, and who to my knowledge has no axe to grind on this; remember that he was the most prominent critic of Michael Bellesiles. People who are interested in the More Guns, Less Crime debate should definitely take Lindgren’s views seriously.

Pro-gun blogger Kevin Baker writes:

For me it was the “Mary Rosh” bit that discredited Lott in my eyes. Defending yourself through a pseudonymous alter-ego was, IMHO, dishonest. If he was willing to be mendacious about that, then why should I trust his word on other matters? …

Lott gives the appearance of being the gun-rights Michael Bellesiles, and James Lindgren, who Eugene Volokh notes seems pretty neutral on the entire gun-rights issue, was instrumental in showing Bellesiles’ flaws. That he is doing so with Lott makes his criticism more believable, not less.

Ralph Luker writes:

The balances of opinion about John Lott’s credibility seem now to be tipping heavily against him. … It’s time for Clayton Cramer to belly up to the bar. It isn’t enough to claim that Lott is credible because you want him to be credible.

Eugene Volokh posts links to Lott’s latest defences. Mark Kleiman responds:

Eugene Volokh seems to think that John Lott’s responses to his critics are worth reading. I can’t imagine why, except for those interested in abnormal psychology. He has been detected in so many different lies [], some of them utterly pointless [], that his words have approximately the net information content of the sound coming from your window fan.

Well, put it down to my interest in abnormal psychology, but I examined both of Lott’s responses. First was a response to Michelle Malkin’s op-ed. This response doesn’t contain anything new, but it does collect together all of his various responses on the survey in the one place, so I have written some comments explaining what is wrong with his responses.

His other response is just his August 20 posting where he finally admitted to making hundreds of coding errors. There were many things wrong with that posting. I already corrected his false implication that the errors only made a difference of thousandths of one percent and also corrected his claim that Ayres and Donohue were keeping their data secret. And the biggest problem was his denial of the fact that correcting his errors eliminated his findings, which I explained here.

However, there was a new bit, in an update hidden in the middle of his posting:

It is also not clear that applying STATA’s clustering command to all counties within a state provides an adequate solution to the problem of cross-correlation. While it is possible that error terms are correlated across jurisdictions within a state, the more important correlations may be among neighboring jurisdictions across state lines. Professors Eric Helland and Alex Tobarrok paper also provides an interesting approach that deals with this problem in an novel way and shows that the results are significant even after these correlations are accounted for.

You know, every now and then Lott comes out with something so breathtakingly dishonest that you have to shake your head at the sheer audacity of it. Helland and Tabarrok show that STATA’s correction for clustering is not enough. That means that after STATA’s correction is made, you still might wrongly decide that your results are significant. But if STATA’s correction means that your results are not significant then Helland and Tabarrok’s further correction will not make them significant. Lott’s postion is like someone arguing that an umbrella will keep you dryer than a raincoat and using that as justification for going out into the rain without a raincoat or an umbrella.

It is true that Helland and Tabarrok found that a couple of results from the original Lott and Mustard paper remained significant after correcting, but none of those results were in Lott’s Confirming “More Guns Less Crime” paper. Correcting Lott’s coding errors causes his results to go away whether you use STATA’s clustering correction (as Lott originally did) or Helland and Tabarrok’s technique.

And Lott still has not responded to the question I have been trying to get him to answer for weeks: “Why did he remove the clustering correction from his model?” Also of interest would be an explanation of why, after I started questioning about his table that used the corrected data, he removed it and replaced it with one based on the miscoded data.

Paul Cella writes about Lott:

My amateur and incomplete (and, if you insist, predisposed) sense is that Mr. Lott has roundly disarmed his often-strident critics with the scrupulousness of his research.

Fortunately, Wes Little sorts him out in comments and Cella ends up conceding:

Alright, Wes: you win. I had altogether forgotten about Lott’s shadiness when I posted the link. If I had remembered the scandal, I surely would have looked for someone else to adduce on guns. It is also unfortunate that the reviewer I link to failed to mention it.

Jesse Taylor writes:

does it ever strike anyone else that Glenn’s style of covering dishonest conservatives is akin to the way that the mainstream media tends to cover conservative screw-ups?

“A person levied a charge against a conservative, and proves it. Offering no proof, the conservative’s defenders said this was ‘(some synonym for “bad attack”)’ from his partisan enemies. I’ll wait until a conservative makes a criticism of the conservative before I hop on board, if I ever bring it up again, unless it’s something that I can safely criticize now without tiptoeing around it.”

Site Meter says that I have now had 50,000 visits to this blog. Who could have thought that John Lott would prove to be such an interesting topic?

I’ve actually had more visits to another page that I never update on convex hull algorithms which has the advantage of being the number one hit on Google for “convex hull”.

In a review of The Bias Against Guns, Pat Buchanan claims that Kleck’s survey found that

11 out of every 12 times citizens use their guns in self-defense, they merely brandish them or fire a warning shot.

and that this was “confirmed” by Lott:

Brandishing a gun stops crime 95 percent of the time, Lott learned.

Buchanan doesn’t seemed to have noticed that Lott’s 95% brandishing number, far from confirming Kleck, contradicts Kleck’s 11/12. Buchanan also got the number from Kleck wrong. Kleck found that 84% involved brandishing or a warning shot and 76% involved just brandishing, making the disagreement with Kleck even larger.

Mark Kleiman has posted an email from Michael Maltz with some comments on Lott and his research. An extract:

It seems that most of Lott’s critics and supporters forgot about what I feel is the most damaging lie he told while hiding behind the skirts of his fictitious Internet persona Mary Rosh: s/he described himself as “a chaired professor” at the Wharton School at the University of Pennsylvania. Had s/he forgotten that he was never even awarded as much as a stool? One can be disgusted by his unfairly lashing out at his critics while in drag, calling them liars who hide behind fictitious personas (I wonder where he got that idea from?), but this lie within a lie takes the cake. How can any of his supporters defend this?

He also misrepresented himself in front of the Nebraska Legislature, calling himself a professor when he was the John M. Olin Fellow in Law and Economics; see his testimony here.

Maltz also points out some more suspicious analysis by Lott

Well over a year ago I sent a data file showing the extent of this missing data to Lott, and he countered by saying that I didn’t take into account that most of the counties with high missing data have low populations. He was right: but taking population into account just reduced the number of problematic states from 21 to 15 (just to use some benchmark, I called a state problematic if more than 10 percent of its observations—county-years—had more than 30 percent missing data; weighting by population, 21 drops to 15).

Lott then reanalyzed the data after eliminating the 16 worst states from his analysis. He doesn’t explain why he dropped 16 states (why not 15, or 17? forget sensitivity analysis!), nor does he state why he dropped whole states instead of problematic counties, nor does he make any provision for the remaining errors (which are still plentiful) in the remaining states’ counties.

Now maybe Lott was just being lazy and did not try a different number of states or different ways to deal with the problem, but his conduct in changing his model to bring his results back after correcting the coding errors removed his results suggests that he tried the possibilities and just presented the one that cast his thesis in the best light.

This is why all of Lott’s econometric research is now worthless. It is not enough to check his data to see if he made more coding errors. It is not enough to rerun his regressions to see if he did them right. You have to look at all the other reasonable ways to model the data to see whether he has just cherry-picked the models that give the results he wants to get. At that point, you might as well do the whole research yourself from scratch.

(All of my postings on the shootings at the Appalachian School of law are here.)

Lott has a report of a conversation with Mikael Gross on his blog. In The Bias Against Guns Lott claims that Gross pointed his gun at Peter Odighizuwa:

Only two local newspapers (the Richmond Times Dispatch and the Charlotte Observer) mentioned that the students actually pointed their guns at the attacker.

So does Gross confirm that he pointed his gun at Odighizuwa? Nope. Does he confirm that Bridges pointed his gun at Odighizuwa? Nope. The best he can do is:

What Gross knew of the attack was “consistent” with Bridges’ statements, though Gross was unable to see Bridges during a good portion of the attack.

So, although Bridges claims that he pointed his gun at Odighizuwa, there are no witnesses who saw him doing it. We have eyewitness accounts from Mikael Gross, Ted Besen, Jody Mitchell, Robert Deatherage, Jack Briggs and Todd Ross and none of them say they saw Bridges point a gun at Odighizuwa.

This bit is interesting, too:

Gross never really personally knew Besen when they were both at the law school, but he did get to know Bridges through their membership in the Federalist society

That is the same Federalist Society that is pushing a pro-gun agenda by sponsoring a speaking tour by Lott. It seems that Bridges and Gross have political motives to make the most of the role defensive guns played in the shooting.

Lott continues to insinuate that Besen changed his story, waiting two months and then adding the detail about Odighizuwa putting his gun down before Bridges arrived. In fact, it is Bridges who has changed his story. In his first version, told to the Today Show, Bridges mentions his gun, but says nothing about pointing it at Odighizuwa. Besen’s story has not changed. He didn’t mention that Odighizuwa put his gun down before Bridges arrived, because that fact was not important until Bridges starting claiming to have pointed his gun at Odighizuwa and Bridges did not make that claim until later.