Deltoid

This is David Hemenway’s response to criticism of his book Private Guns Public Health by Kevin Baker at The Smallest Minority.

September 27, 2004

I was asked to respond to what is claimed to be a critique of my book by Kevin Baker. I have neither the time nor inclination to have a detailed response to the many assertions and arguments he makes, many of which are wrong or misleading.

It turns out that Baker isn’t really discussing my book Private Guns Public Health, but a magazine article about it. Unfortunately it seems that Baker may not have read my book (or the hundreds of journal articles that the book summarizes). It does seem silly for him to accuse the journalist who tried to reduce a 300+ page book and 3 hours of interviews into 3 interesting pages of text, as engaging in “bait-and-switch” tactics or not sufficiently discussing what Baker would have liked discussed.

I will talk about one issue, to illustrate the type of problem found in Baker’s discussion.

A dozen case-control studies all find that, in the U.S., a gun in the home is a risk factor for “violent death” (i.e., homicide, suicide or unintentional gun death). Some of the other risk factors accounted for in one or more of these studies include age, gender, community, living alone, education, alcohol illicit drug use, depression medication, and psychiatric diagnosis. Ecological studies also find that, across U.S. states and regions, higher levels of household gun ownership are associated with higher rates of homicide (due to higher gun homicide rates), higher rates of suicide (due to higher gun suicide rates) and more unintentional gun deaths. Some of the other risk factors accounted for in one or more of these studies include poverty, alcohol consumption, unemployment, urbanization, divorce, education, violent crime, major depression, and suicidal thoughts.

Massachusetts, where I live, is a state with (relatively) low levels of household gun ownership, strict gun control laws, and low rates of violent death. I remarked to the journalist, who lives in Massachusetts, that I was glad I lived in Massachusetts and that “It’s nice to have raised my son in Massachusetts, where he is so much safer” than most other states. Baker took this quote, asserted that I live in Boston, which I do not, and made comparisons to violent death in parts of Arizona, a state that has more permissive gun laws than Massachusetts.

So, let’s compare Massachusetts and Arizona. Here are data from 1999–2001, the most recent time period available, easily obtained from the CDC WISQARS website.

Number of Deaths and Mortality Rate Ratio, 1999-2001
	Arizona pop: 5.154 million	Massachusetts pop: 6.356 million	Mortality Rate Ratio, Arizona v. Massachusetts
Homicides	1,374	501	3.4
Gun	909	218	5.2
Non-Gun	465	283	2.0
Suicides	2,317	1,244	2.3
Gun	1,433	330	5.4
Non-Gun	884	914	1.2
Unintentional Gun	47	6	10.0
Total Gun Deaths	2,460	565	5.4

In other words, a resident of Arizona is over 5 times more likely to be murdered with a gun, commit suicide with a gun, and be unintentionally killed with a gun than a resident of Massachusetts. Arizona may be nicer than Massachusetts in many ways (e.g. climate) but it’s difficult to understand how Baker can suggest that Arizona is a safer state in terms of gun deaths, or violent deaths.

In general, Baker seems to believe if he can find an anomaly, then the general associations scientists find between guns and death is disproved. It is analogous to his finding that Abel smokes but Cain doesn’t, and Cain has heart disease but Abel doesn’t, and believing that this proves that smoking does not really cause heart disease. Or believing that the fact that Japanese smoke more than Americans and have less cancer shows not only that cigarettes don’t cause cancer, but may well be protective. But such anecdotal evidence shows only what everyone knows, that there are many factors affecting the likelihood of heart disease or cancer, and smoking is only one of those factors. It is not the only factor. Nor is gun availability the only factor affecting homicide or suicide—but the evidence is quite strong that it is one important factor.

What makes Baker’s arguments even more questionable is that his claimed anomalies are often specious. One can find states with more guns and a lower homicide rate than Massachusetts (look for very rural states, since virtually all crime, including homicide, is much higher in urban areas), but Arizona is not one of them. There are many other examples. Baker also says that about half the households in Finland contain guns. While a UN report did say that, the information appears to be incorrect. Probably the best source for comparative gun ownership is the International Crime Surveys that found that in 1989 23% of Finnish households contained a gun, in 1992 it was 25%, and in 1996 it was about 26%.

Discussions of firearms on Baker’s and many other internet sites, seem primarily to be debates, where each party tries to find evidence to support his already held point of view. These are interesting exercises, but they add little to science, and I am not very interested in them. There seems to be a surprisingly lack of curiosity as to what really is happening in the lives of 300 million American, or the 5-6 billion people on the planet. We can’t rely on news to tell us much. We should rely, not on anecdotes, but on good scientific studies, where the goal is to find the truth rather than support for what one already believes.

David Hemenway

I see that the Sydney Morning Herald is now publishing funky conspiracy theories. Alan Anderson informs us that Kyoto is nothing but a Euro-commie plot:

Of course, everyone who is familiar with the Kyoto Protocol knows what it really is: a brazen attempt by the EU to compensate for its competitive disadvantage (the result of socialist economic policies) by hobbling the United States economy.

I like the way Anderson uses the “everyone who is familiar with X knows” locution to avoid offering any actual arguments or evidence in support of his position. I checked the Wikipedia page on Kyoto and strangely enough found no support for Anderson’s theory. In fact, pretty well all the industrialized countries except Australia and the US are signing. This seems to be inconsistent with Anderson’s position. Why have all these countries fallen for the EU’s dastardly plot if everyone knows about it? It’s a mystery.

Readers may remember Pat Michaels, who authored a paper one that “disproved” global warming by deliberately removing almost one-third of the satellite data from his analysis and co-operated with Ross McKitrick on another paper that managed to “prove” that global warming wasn’t happening by mixing up degrees with radians. Alan Anderson has responded to my criticism of his claim that Kyoto was a dastardly EU plot to cripple the US economy by offering up an article by … Pat Michaels.

I’m afraid that this article is up to Michael’s usual standards. He constructs a measure of carbon efficiency by dividing a country’s CO₂ emissions per unit of GDP by the area of that country. His rationale for this is that larger countries have to consume more energy on transportation. But Michael’s measure makes absolutely no sense at all.

1. While it is true that in a larger country you can travel further inside the country, that distance is not proportional to the area of the country. For instance, if country A is twice as far from North to South and twice as far from East to West as country B, then the area of country A is four times that of country B, but you can only travel twice as far in country A.
2. Imagine dividing the US into two equal sized countries. By Michael’s measure the efficiency of each piece is half that of the US, even though nothing else has changed. The reason is that Michael’s measure does not include international travel at all.

If you wanted to do a competent job of estimating the effects of Kyoto on transport in different countries you would need to show that the fraction of energy consumed by transportation is significantly different in each country, and that transportation would be particularly affected by Kyoto. Michaels does none of this, instead presenting his bogus measure as if it meant something.

Michaels also falsely claims that Kyoto “would cost us about 3 percent of GDP per year”. In fact, an extensive comparison of several studies finds:

All the studies project irreducible losses to the economy that are small (less than 1 percent of GDP in 2010 and 2020) in absolute magnitude

Why did Michaels advance such a nonsensical measure? It couldn’t have anything to do with funding by coal and oil interests, could it?

Update: Anderson has another post where he offers an article by Christopher Horner in another attempt to support his position:

For any other ignorant types out there who prefer factual arguments to rhetoric, here is an interesting article about the next likely step in the EU’s campaign to undermine US economic competitive advantage under the aegis of fighting climate change.

Here are some of the alleged “factual arguments”:

Doubtless accompanied by specious claims of scientific certainty, its plea would claim that the U.S. refusal to follow the EU’s greenhouse gas (Kyoto) path constitutes impermissible protectionism and/or “eco-dumping.” Incredibly, the WTO has indicated a willingness to accept such an argument, also advocated by some as a path to “harmonize” the otherwise incompatible pro-trade and anti-energy pacts….

mindless carbon dioxide suppression….

Any treaty threatening the economic health of nations will ultimately collapse of its own potential harm, though not without first wreaking havoc…

one important step would be to abandon Kyoto once and for all, with its built-in appeasement of ideological extremists seeking to impede global prosperity…

Of course, these aren’t “factual arguments”. For example, Horner offers no evidence that Kyoto will wreak havoc, no evidence that Kyoto is intended to appease “ideological extremists”, no evidence that the people who designed Kyoto are “mindless” and no evidence that the plea to the WTO was accompanied by “specious claims of scientific certainty. Nor is there any reason to believe that these “factual arguments” are accurate—they are nothing more than empty rhetoric, deployed because the actual facts are not helpful to Horner.

And guess who employs Horner to write this rubbish? The Competitive Enterprise Institute. Readers may recall fellow CEI employee Paul Georgia, who told the world that average temperature had no physical meaning. And Iain Murray, another CEI employess, who tried to rewrite basic epidemilogical principles and insisted that the evidence for global warming was cooked up. The CEI warns people that using Linux is legally risky, attacked the FDA when it proposed regulating tobacco, and relentlessly attacks Kyoto. It is no doubt just a coincidence that the CEI receives extensive funding from Microsoft, Philip Morris and Exxon.

For reasons similar to those given by Tim Dunlop, Jason Soon and John Quiggin, I’ll be voting Labor in the election today. Not that it makes a difference, since I live in Kingsford Smith, a safe Labor seat.

I hope that I’m wrong, but I don’t think that Labor will manage to win the election. I expect that there will be a small swing against the government, but that their vote will hold up in the marginal seats where the government has been raining money down, and they will hold onto to enough seats to stay in power.

Update: Howard has been returned, as I expected. There was even a small swing to the government, which I didn’t expect. This is bad news for Australian gun owners.

The warbloggers have been attempting to spin the result of the election here to their advantage. Cori Dauber claims that the election “was a referendum on Australia’s participation in Iraq”, and Glenn Reynolds claims that it was “in no small part as a referendum on the war”. They link to stories that provide no support for their claims and indeed undercut them by reporting that Howard did not even mention Iraq in his victory speech. No, the election was not about Iraq—it was hardly an issue.

They also continue to make hysterical attacks on Diana Kerry. Mike at Cold Fury accuses her of:

a dastardly attempt by Kerry to undermine the war effort by directly undermining the coalition fighting it, hoping to convince Australians to abandon a steadfast will to win in favor of tremulous cowardice and appeasement.

Spoons calls her “traitorous”, while deacon at Powerline continues to maintain that she lives in Australia and was campaigning for Labor.

In fact, she had nothing to do with the election here at all and wasn’t even in the country. All she did was give a truthful answer to a question about whether invading Iraq has increased the terrorist threat to Australia. The attacks on her would appear to be just motivated by a desire for partisan advantage in the US election.

Update: Reynolds responds, offering posts from Tim Blair and Mike Jericho to support his claim. Neither post offers any evidence that Iraq was a major issue. Anyone who has been following the election would know how little it was discussed. This article in the Sydney Morning Herald on the election campaign observes:

Iraq flared briefly after the Jakarta bombings—most notably in the leaders’ debate—but was mostly left alone by the Opposition, even though Mr Howard refused to talk about the issue, betraying his fears the missing weapons of mass destruction and increased terrorist threat could hurt this chances.

Update 2: Tim Blair offers support for his position from, get this, Taiwan News. I wonder if Blair gets all his political news about Australia from Taiwan. Perhaps he just reads Aussie papers for their news about Taiwan.

I wonder what the Australian papers say? The closest thing I could find to support for Blair’s position was in the Australian, where Howard supporter Greg Sheridan opines (my emphasis):

Howard’s triumph has been extensively reported in the US. Almost universally, the international interpretation of the Australian election was to see it as a referendum on the nation’s involvement in Iraq.

In some ways this interpretation is accurate. Even though Iraq hardly figured in the campaign, the fact that Labor did not challenge Howard on the war shows that the Prime Minister had already won that particular argument before the campaign began.

Of course, by Sheridan’s logic the election was also a sort-of referendum on ditching the monarchy because that hardly figured in the campaign either. And also a sort-of referendum on everything else that didn’t figure in the campaign. Heavens knows how we should categorize the vote as to issues the did figure in the campaign. Referendum is taken, so maybe we need a new word?

Update 3: Tim Blair keeps digging himself deeper and deeper.

The relentless spinning of the result of the election in Australia continues. In the New York Post John O’Sullivan’s headline is “Bush wins again”. I didn’t even know Bush was running in the election here. O’Sullivan also writes:

Al Qaeda has received a serious setback, Kofi Annan a rebuke, France and Germany a disappointment—and the media elites a slap in the face so stinging that outside Australia Howard’s victory has been a non-story.

This is just bizarre. Labor is very keen on wiping out Al Qaeda—the only difference with the government seems to be about tactics. I’m pretty sure that Al Qaeda probably doesn’t care about the difference between the two parties. And Kofi Annan? I doubt if there was one single voter who was thinking about Kofi Annan when they voted. France? Germany? Why on earth would they even care? And the result got about as much coverage as Australian elections usually do overseas. The “media elites” (presumably O’Sullivan somehow excludes himself here) are upset at the result so they retaliate by giving the result the usual coverage? Come on.

Oddly enough, in an article written before the election, when he thought Labor might win, O’Sullivan wrote

Iraq has scarcely been an issue.

Somehow he forget to mention that in the article he wrote after the election. Funny, that.

Glenn Reynolds links to O’Sullivan’s piece (not the one where O’Sullivan wrote that “Iraq has scarcely been an issue” but the other one) and complains that if Howard had lost it would have been a treated as a big story. Oh, quite possibly, and Reynolds would have been telling everyone about how Iraq was scarcely an issue in the election campaign instead of claiming that it was a referendum on Iraq.

Tim Blair continues to insist that the election was about Iraq. I’ll look at his arguments in a moment, but first let’s look at what everybody else says about this.

Tom Allard and Mark Metherell in the Sydney Morning Herald:

Iraq flared briefly after the Jakarta bombings—most notably in the leaders’ debate—but was mostly left alone by the Opposition, even though Mr Howard refused to talk about the issue, betraying his fears the missing weapons of mass destruction and increased terrorist threat could hurt this chances.

A “Labor Insider” in Crikey:

Labor’s Iraq policy. Latham’s on the run promise to “bring the troops home by Christmas” cruelled this potential vote winner for Labor. Even opponents of the war think this a bad policy and it cast doubt on his grasp of security issues. Howard wasn’t able to use it during the campaign because of his lies on Iraq, but it meant Latham wasn’t able to use it either.

Greg Sheridan in the Australian

Iraq hardly figured in the campaign

The Associated Press:

Iraq played only a minor role in Australia’s election campaign.

Sally Young in the Herald Sun:

Another factor that went in the Coalition’s favour was that an issue which could have caused some difficulties for the Coalition—the Iraq war—did not turn out to be a major issue in the campaign

John Black in the Courier-Mail:

Error 8: Don’t mention the war. Labor ran no effective campaign on Iraq

John O’Sullivan in NRO:

Iraq has scarcely been an issue.

Australian warblogger Arthur Chrenkoff:

If the issue of Iraq did not seem to have been on the forefront of the Australian election campaign, it’s because by contrast with the US presidential campaign it wasn’t there to anywhere near the same degree.

John Quiggin at Crooked Timber:

Anyone with any knowledge of Australia, or even with the capacity to read Australian papers on the Internet, would know that Iraq was barely mentioned either during the election campaign or in the subsequent analysis.

And there’s plenty more where those came from.

How does Tim Blair attempt to make a case that the election was about Iraq? Well, he gathers together all the mentions of Iraq that he can find. Trouble is, there aren’t that many, so he even has to pad out the list with lots of stuff from March, well before the election campaign. So, what did he find? Well, as mentioned above, Iraq flared briefly as in issue in the leaders’ debate after the embassy bombing. Following the Iraq Survey Group’s finding that Iraq had no WMDs Howard got four questions about Iraq after his Press Club appearance (Blair counts this as 27 mentions). That’s pretty much it.

Probably the best indicators of the relative importance of the issue are Howard’s and Latham’s policy launches. Howard devoted just one out of sixty-five paragraphs in his policy launch to Iraq. Latham only mentions Iraq briefly, almost at the end of his speech. Compare the amount they talk about Iraq with how much they talk about taxes, unemployment, interest rates, health care, education, the environment, industrial relations, childcare, the economy and pensions

So does anyone agree with Blair’s position? Blair claimed that Robert Corr did, but Corr repudiated him:

Tim Blair also puts my name forward as someone who thinks the war on Iraq was a significant election issue. I do not.

Glenn Reynolds, in a heroic leap, has apparently concluded that the election in Australia really was a referendum on Iraq and folks who don’t think so must just be spinning.

One of those spinning must be Prime Minister John Howard, who told CNN that Iraq “wasn’t the dominant factor” in the election victory.

Also spinning must be The Bulletin, which has eighteen pages on the election this week (including a whole page by Tim Blair). In what can only be part of a massive conspiracy by the Main Stream Media, Iraq is mentioned by name a grand total of zero times.

Also spinning must be all the people, left and right, bloggers and journalists, listed before who stated that Iraq was not a significant issue.

Reynolds seems to believe that the only person not spinning is Tim Blair in his blog. But Tim Blair in The Bulletin must be spinning. Can he both be spinning and not spinning? Perhaps some quantum uncertainty is involved.

The grandly named EnviroTruth web site has section that purports to debunk “myths” about climate change. The “myths” include the usual false claims such as satellite measurements don’t show warming, but “myth” number 11 is pretty funny.

Here’s “myth” 11:

Those Who Question Whether Human Activity Contributes in Any Significant Fashion to Climate Change are Secretly Funded by Coal, Oil, Gas and Other “Smokestack” Industries.’

Brandon MacGillis of Ozone Action, a Washington DC-based public interest group, refers to global warming doubters as “part of a handful of skeptics, mostly coal producers and users, who are still trying to debate the scientific certainty of this threat.” David Suzuki, an influential Canadian environmentalist, makes a similar claim and refers to those who oppose his views on this topic as “anti-environmentalists.”

Here’s their debunking:

Despite the condemnations of radical environmentalists, it is a safe bet to conclude that scientists who express skepticism about the likelihood of an imminent, human-caused climate change catastrophe act independently of their funding sources - in other words, they aren’t motivated by money. With all scientists competing for very limited funding resources, and due to the strong media and government interest in this area, it has become an attractive selling point for scientists to be able to associate their research in some way with global warming. As a consequence, reference to global warming tends to be made whenever possible, often for projects that are often only distantly related to that line of research. Global warming skeptics are unable to make this association and, thus, have no covert incentive to oppose the alarmists. More to the point, the vast majority would receive more funding if they did endorse the more politically-correct global warming theory.

I hope you were paying attention there. Did you notice that they never got around to actually saying that the alleged myth was false? In fact they seem to have tacitly admitted that the sceptics are funded by carbon energy companies. Their assertion that these sceptics could get more money is also pretty silly. Scientists are funded to conduct research, not reach pre-defined conclusions. If the sceptics could do work that could meet the standards of the top journals, then they could get funding without taking money from energy companies.

So anyway, who funds Envirotruth? If you follow the links from their page you eventually get to this page, which states:

Our audited figures show that most — 81.5% in 2002, 93% in 2001, 93% in 2000, 88% in 1999 and 80% in 1998 — of The National Center’s funding comes from small gifts from individuals. The remainder comes from foundation/non-profit grants (16% in 2002, 4.6% in 2001, 3% in 2000, 5% in 1999 and 11.6% in 1998), with additional income coming from corporate contributions (2.0% in 2002, 2.6% in 2001, 4% in 2000, 4% in 1999 and 8% in 1998), sales of publications and materials and interest income.

Cool, no mention of funding by energy companies.

However, if you check the always useful Disinfopedia you find that

In 2003 [ExxonMobil] boosted its general operating support to $25,000 with another $30,000 for “global climate change/EnviroTruth website”

Not only is Envirotruth specifically funded by an oil company, it’s secretly funded by an oil company. That was some refutation.

Earlier, Glenn Reynolds accused me of spinning because I wrote that “the [Australian] election was not about Iraq—it was hardly an issue.”. Now he approvingly links to a piece by Greg Sheridan

Labor did not buy a single ad on Iraq. Nor did Latham mention his troops-home-by-Christmas pledge in his policy speech. Indeed Iraq only figured in the last line on page 13 of a 16-page speech by Latham. … It was rather strange that we have troops at war and they were hardly mentioned in the campaign.

Why, that’s what I was saying! Do you think Reynolds accuses Sheridan of spinning? Of course not, because despite the troops being “hardly mentioned”, Sheridan manages to conclude

The other critical conclusion to come out of this election is that it was a total vindication of John Howard over Iraq.

By what tortured logic does Sheridan come to this conclusion?

This is actually a bigger victory for Howard than if the election had been fought on Iraq and he had won. His victory in the Iraq argument was so comprehensive that Labor did not even raise it in the campaign.

Trouble is, the Coalition did not raise it in the campaign either, so you could just as well argue that Howard’s defeat on the Iraq argument was so comprehensive that he did not dare bring it up in the campaign. Both sides do extensive polling, so the fact that neither side wanted to talk about Iraq shows that it was not a positive issue for either side. Sheridan is spinning furiously.

Steven den Beste has looked at a graph of polls of voting intentions and decided:

In September, I think there was a deliberate attempt to depress Kerry’s numbers, so as to set up an “October comeback”. Of course, the goal was to engineer a bandwagon.

This seems rather implausible to me. There are very many organizations do the polling. If all the polls are rigged, a huge number of people would have to know about it, and surely one of them would have leaked the information by now. It also seems unlikely that every single poller (including, for example, Fox News) would happen to support Kerry.

The reason that den Beste believes that the polls are rigged is:

The data for September, however, is clearly an anomaly. The data is much too consistent. Compare the amount of jitter present before September to the data during that month. There’s no period before that of comparable length where the data was so stable.

Well, if the results in September were much too consistent that would certainly be evidence that the results were fixed, but are they? The pink line in the graph to the left is a detail from den Beste’s graph showing Bush’s numbers for September. The pink line is a moving average of the results of six polls each of roughly 1000 people, so I constructed my own graph by simulating the results of polls of that size and assuming that the underlying number was not changing. The result is the green line in the graph to the left. As you can see, it shows about the same amount of varation as the pink line. It is not true that the results are much too consistent—there is no evidence for fraud here.

It is true that there is less variation in September than there was in earlier months, but that could well be because people had not made up their minds and so there really were larger swings in support for each candidate.

Den Beste also draws two trend lines for each candidate. However he seem to have constructed his trends by just eye-balling the graph. This is not a good technique for constructing trends since it is far too easy to find the trend you were expecting or hoping to see. To get trends that are not affected by subjective bias you need to use statistics.

In this column, Richard Muller claims that McKitrick and McIntyre have shown that the hockey stick graph is an “artifact of poor mathematics”. If you have been following the global warming debate this claim should look familiar, because McKitrick and McIntyre made the same claim last year as well. So what’s new? Well, last year they claimed that the hockey stick was the product “collation errors, unjustifiable truncations of extrapolation of source data, obsolete data, geographical location errors, incorrect calculations of principal components, and other quality control defects.” Now they are saying that the hockey stick is the product of improper normalization of the data. This is an improvement on their previous claims, since it seems that it will be reasonably simple to test. William Connolley has looked at the data and thinks M&M are probably wrong:

But (having read their paper) I now think I understand what they think the problem is (aside: they complain about data issues with some series but I think this is beside the point: the main point they are talking about is below), and I think that they are probably wrong, based on reading MBH’s Fortran (aside: Fortran is a terrible language for doing this stuff, they should use a vector language like IDL). But anyway:

Lets for the moment assume for simplicity that these series run from 1000 (AD) to 1980. MBH want to calibrate them against the instrumental record so they standardise them to 1902–1980. 1902–1980 is the “training period”.

What M&M are saying (and Muller is repeating) is (and I quote): the data

“were first scaled to the 1902-1980 mean and standard deviation, then the PCs were computed using singular value decomposition (SVD) on the transformed data…”

they complain that this means that:

“For stationary series in which the 1902–1980 mean is the same as the 1400–1980 mean, the MBH98 method approximately zero-centers the series. But for those series where the 1902–1980 mean shifts (up or down) away from the 1400–1980 mean, the variance of the shifted series will be inflated.”

This is a plausible idea: if you take 2 series, statistically identical, but when one trends up at the end where the other happens to be flat, and you compute the SD of just the end bit, and then scale the series to this SD, then you would indeed inflate the variance of the up trending series artificially. But hold on a minute… this is odd… why would you scale the series to the SD? You would expect to scale the series by the SD. Which would, in fact, reduce the variance of upwards trending series. And also, you might well think, shouldn’t you take out a linear trend over 1902–1980 before computing the SD?

So we need to look at MBH’s software, not M&M’s description of it. MBH’s software is here, and you can of course read it yourself… Fortran is so easy to read…

What they do is (search down over the reading in data till you get to 9999 continue):

1. remove the 1902-1980 mean
2. calc the SD over this period
3. divide the whole series by this SD, point by point

At this point, the new data are in the situation I described above: datasets that trend upwards at the end have had their variance reduced not increased. But there is more…

4. remove the linear trend from the new 1902-1980 series
5. compute the SD again for 1902-1980 of the detrended data
6. divide the whole series by this SD.

This was exactly what I was expecting to see: remove the linear trend before computing the SD.

Then the SVD type stuff begins. So… what does that all mean? It certainly looks a bit odd, because steps 1–3 appear redundant. The scaling done in 4–6 is all you need. Is the scaling of 1–3 harmful? Not obviously.

Perhaps someone would care to go through and check this. If I haven’t made a mistake then I think M&M’s complaints are unjustified and Nature correct to reject their article.

My previous experience with McKitrick gives me no confidence in his work. David Appell is also sceptical of this latest attack on the hockey stick.

There seems to be some confusion about McKitrick’s latest attempt to refute global warming. For instance, Andrew Sullivan thinks that McKitrick’s famous degrees-radians screw up is part of this latest attempt. However, McKitrick claims to have refuted global warming in several different ways and the degrees-radians screw up was a in a different paper to his latest one. I decided to draw up a table to help folks sort them out.

Authors	Summary	Consequences if he is right	Status
Essex and McKitrick	There is no physical basis to average temperature.	No global warming because there is no such thing as global temperature.	Failed—the whole field of thermodynamics has not been thrown out.
McKitrick and McIntyre version 1	The hockey stick graph was the product of “collation errors, unjustifiable truncations of extrapolation of source data, obsolete data, geographical location errors, incorrect calculations of principal components, and other quality control defects.”	The global warming we are seeing might be natural.	Mann et al publish a correction to the supplementary information for the hockey stick graph. They say that the errors do not affect their published results.
McKitrick and Michaels	Surface temperature record is contaminated by economic influences.	No evidence that there is global warming going on	Results go away after errors such as confusing degrees with radians are corrected.
McKitrick and McIntyre version 2	hockey stick is the product of improper normalization of the data.	The global warming we are seeing might be natural.	Jury is still out, but it does not look promising for McKitrick

Meanwhile, James Annan agrees with Connolley’s concerns about M&Mv2:

Having had a quick glance at this and their papers, I think I agree with you. In fact it appears that we can add not knowing the difference between multiplication and division, to the already impressive list of blunders that M&M have made. They even seem to talk about adding the mean to the time series rather than subtracting it too. I might check this more carefully over the next few days if no-one else beats me to it.

Brad DeLong also seems to agree.

But Connolley argues—I think correctly—that McKitrick and McIntyre are simply confused: the normalizations diminish the influence of series that show a recent uptrend.

Orin Kerr writes writes about Wikipedia

My very tentative conclusion, based on a just few sample queries, is that I hope no one relies on Wikipedia for anything very important. Its entries seem to be a strange mix of accurate statements and egregious errors.

My own experience is that Wikipedia is quite accurate and errors get corrected. An erroneous description of the Patriot Act that Kerr pointed to was quickly corrected. It seems counterintuitive that letting anyone edit any page would result in quality information, but that seems to be what has happened.

Kerr argues:

If I understand accurately how Wikipedia works—a big “if,” I should point out—my views of what is in the Patriot Act are no more and no less valued by Wikipedia than the views of any other Internet user. Given the widespread misperceptions about what is in the Patriot Act, some one else is likely to come across my corrected entry and think, “What idiot wrote this? This is totally wrong!” They will then erase my entry and re-enter all the mistakes that I corrected. The “genius” of Wikipedia is that no one is there to resolve the disagreement: the loudest voice eventually wins.

Well, it doesn’t seem to work out that way. For example, someone with IP address 38.118.12.78 has been changing the Wikipedia entry on John Lott to remove any criticism of him. (The scrubbed version is here.) Mr 38.118.12.78 is very persistent—so far he has replaced the page eight times. However, since his changes are obviously unreasonable, each time they have been undone. The loudest voice has not won.

Oh, and IP 38.118.12.78 resolves to americanenterpriseinstitute.org.

Via Pandagon I find an appallingly innumerate article on polls by Michael Barone:

Blogger Steven Den Beste has prepared an interesting chart. Den Beste charges that pollsters “deliberately gimmicked” the results, “in hopes of helping Kerry.” I don’t agree with that at all. But he has made another interesting observation. Eliminating some of the peaks and valleys of the Bush and Kerry percentages in realclearpolitics.com’s average of recent polls, Den Beste shows that Bush’s percentages have tended to rise over time while Kerry’s have risen much less if at all.

He draws the Bush long-term trend line from a low point around 43 percent in May, when the media were full of stories about the Abu Ghraib prison scandal, to higher numbers around 45 percent in July and August, then up to the 49 percent level he has reached today. His long-term Kerry trend line runs through the 44 to 45 percent level in the spring to the 45 to 46 percent level in August, after the Democratic National Convention, to the same 45 to 46 percent level of today.

It seems curious that the percentages of the incumbent should rise while the percentages of the challenger have not risen much if at all. As a general proposition, you expect an incumbent’s standing to change less, because voters already know much more about him than about his opponent. But that hasn’t happened this time.

But what is actually rising is not support for the incumbent but the trend line Den Beste drew on his “interesting chart” (on left). Den Beste did not use any statistical procedure to work out his trend line, he just eyeballed the graph.

Underneath Den Beste’s chart is one I just drew. My trend lines fit the data just as well as Den Beste’s and show exactly the opposite—support for Bush has not risen much if at all, while Kerry’s support has been rising. So whose support is increasing faster? To answer this you have to ignore the suggestive but misleading trend lines and look at the graph of the spread at the bottom of the chart. In March Bush was ahead by about two points, in October he was ahead by about two points. It follows that both Bush and Kerry have increased their support by about the same amount. Duh.

It gets worse. Barone explains the trends:

My tentative explanation is this. Bush’s most effective opposition this year has come not from Kerry and the Democrats but from Old Media, the New York Times and the news pages of the Washington Post, along with the broadcast networks ABC, CBS, and NBC. Old Media gave very heavy coverage to stories that tended to hurt Bush—violence in Iraq, Abu Ghraib, the false charges of Richard Clarke and Joseph Wilson, etc.

Curse that Old Media and their Bush-hating facts!

And check out Barone’s slant here:

The theory about the Democrats can be tested by looking at the 1996 and 2000 elections, when the Democrats were the incumbent presidential party in times of apparent peace and apparent prosperity

Gee, he can’t even bring himself to admit that there was peace and prosperity under Clinton. It was only “apparent” peace? American soldiers were being killed by the score and nobody noticed? It was only “apparent” prosperity? All those people only “apparently” had jobs but they weren’t real jobs?

Barone continues:

George W. Bush is not running this year as an incumbent in a time of apparent peace or, in public perceptions, a time of apparent prosperity. (Actually, the economic numbers are about where they were when Bill Clinton was running for re-election in 1996, but Old Media consistently report economic news more pessimistically when Republicans hold the White House than when Democrats do.)

First of all, “in public perceptions, a time of apparent prosperity”? Can anyone figure what this is supposed to mean? How does something perceived as apparent prosperity differ from something perceived as prosperity and how does it differ from apparent prosperity? Second, the economic numbers have been much worse than the first Clinton term, see here. And third, he seems to be channelling Lott’s bogus study alleging bias against Republicans in reporting economic news.

It is disturbing that someone so innumerate as to not understand that a random line drawn on a graph does not make a trend should be interpreting polls for a major news magazine. I’ve never read anything by Michael Barone before. Perhaps his other columns are higher quality, but this piece was just dreadful.

My earlier post on Den Beste’s chart is here.

Media Matters for America details Lott’s latest bizarre claim about the 2000 election in Florida—on CNN Lott claimed:

I think a lot of the discussion about disenfranchising African-American voters, in particular I think it’s been fairly sad, because I think there have been a lot of myths in Florida, for example. I mean, you have the Commission on Civil Rights did an extensive set of hearings, they weren’t able to identify even one person.

Media Matters for America has the links to the Commission on Civil Rights report that puts the lie to Lott’s claim.

A previous bizarre Lott claim about Florida 2000 is dissected here.

Baseball’s World Series is played over the best of seven games. The first two games are played at the home field of one team (we will call this one team A), the next three at the home field of team B, and the last two at the home field of team A. Given that teams are more likely to win games on their home fields, does this give team A an advantage?

Keith Burgess-Jackson argues that neither team has an advantage:

Every Series goes either four, five, six, or seven games. We don’t know at the outset how many games it will go. Suppose it goes four games. Then there will have been two games in each team’s park. No advantage. Suppose it goes five games. Then there will have been three games in team B’s park and two in team A’s park. Advantage to team B. Suppose it goes six games. Then there will have been three games in each team’s park. No advantage. Suppose it goes seven games. Then there will have been four games in team A’s park and three in team B’s park. Advantage to team A.

Let’s take stock. In two of the scenarios, there is no home-field advantage. In one scenario, the team that begins at home has an advantage. In one scenario, the team that begins on the road has an advantage. It’s a wash! Where’s the overall advantage?

Is he right, or does team A have the advantage? The answer is below the fold.

Team A has the advantage. The easiest way to see this is to imagine what would happen if all seven games were always played instead of stopping when one team gets four wins. Then team A would have the advantage since it plays one more game at home. But once a team reaches four wins, playing the remaining games makes no difference to which teams wins the Series, so team A has the same chance of victory in the “stop at four” and in the “play all seven” formats. So team A has the advantage in the “stop at four” format that is actually used.

A second way to see it is to consider what would happen if home field advantage was absolute. Then each team wins all their home games and team A wins the Series in game 7. So team A has the advantage again.

For people who are not convinced, I have calculated the actual probabilities in the table below (this requires that your browser supports Javascript). The home team has won 57% of the games in World Series play¹, so there is a 57% chance the Series score will be 1-0 after game one and a 43% chance it will be 0-1. The score will be 1-1 after two games if either B wins with the score 1-0 (43% of 57%) or if A wins with the score 0-1 (57% of 43%). The total is 49% (43% of 57% + 57% of 43%). Similarly you can work out the probability of each Series score and add up the ones where A wins to work the chance of A winning the Series and see that A has a 52% chance of winning the seven game Series. The calculations assume that the results of the games are independant—that there is no such thing as “momentum” where the winner of a game is more likely to win the next game.

You can experiment by entering different percentages for home field advantage or different formats for the sequence of home and away games. (Press ‘Enter’ after changing values to have the table update.) Notice that the format makes no difference to A’s chance of winning the Series—it’s the same with BBBAAAA as it is with AAAABBB.

Percentage of games that A wins at home
Percentage of games that B wins at home
Home/Away Format

How well do the calculated probabilities match with what happened in the 78 World Series with the AABBBAA format? The table below shows the actual results. The entry for “2-0″, for example, says that in 27 of the 78 Series (35%) the score was 2-0 after two games. It also shows that in 20 of those 27 Series, team A went on to win. The percentages agree fairly well with the calculated ones in the table above.²

0-0	100% 45/78	1-0	62% 30/48	2-0	36% 21/28	3-0	13% 10/10	4-0	12% 9/9	A wins 58%
0-1	39% 15/30	1-1	49% 21/38	2-1	47% 24/36	3-1	21% 13/16	4-1	12% 9/9
0-2	16% 3/12	1-2	32% 11/25	2-2	44% 21/34	3-2	27% 15/21	4-2	13% 10/10
0-3	9% 0/7	1-3	17% 2/13	2-3	36% 12/28	3-3	42% 17/32	4-3	22% 17/17
0-4	8% 0/6	1-4	6% 0/5	2-4	9% 0/7	3-4	19% 0/15
B wins 42%

Another comparison you can make is between the expected distribution of Series lengths and the actual numbers. These are shown in the table below. For example, there is a 6% chance of a 4-0 win by A and a 6% chance of a 4-0 win by B, so you would expect (6%+6%)*78 Series that end in four games.

Games played	four	five	six	seven
Observed
Expected

What is surprising when you compare the Series lengths with the expected numbers is that there are a fair bit more four and seven game Series than you would expect. Of course, it could just be chance, so we should test to see if the difference is statistically significant. The appropriate test for this is called the Chi-square test. You can see the results of the test below—the p value is the important number and tells us how likely such a large difference could arise by chance. A p value of less than 0.05 is generally considered statistically significant, so this difference is statistically significant.

Chi-square df p value

Now there are some possible explanations for the extra four-game Series. For example, I assumed that the teams are evenly matched except for home field advantage, but if one team is better than the other, that increases the chance of a four game Series. (Try entering 67 and 47 as the percentages that A and B win at home.) Trouble is, that decreases the chance of a seven game Series. (Try it—the p value even goes down.) Similarly, if there is “momentum” and winning one game makes a team more likely to win the next game, that makes four-game Series more likely but seven-game Series less likely.

The only way that there can be a sixth game is if the score is 3-2 after five games. For the Series to go on to seven games, the team that is behind must win the sixth game. Remarkably, that has happened 31 out of 49 times or 63% of the time instead of the 50% you would expect. I can’t think of a good reason why this would happen. If you can, leave a comment.

Update 30 Oct: Included result of 2004 Series.

¹ The results of all the games are available from this site. I collected the results for the 78 world Series that used the AABBBAA format for games (the Series from 1924 to 2004 except for 1943 and 1945) and put them in this file so you do your own calculations if you are so inclined.

² Alan Abramowitz has article on home field advantage. While he finds that the team with home field advantage wins more often, he argues that this is not because they get to play the seventh game at home:

playing game seven at home does not appear to be a significant advantage in the World Series. … Moreover, since the 2-3-2 format was introduced in 1924, the home team has won only 16 of 31 seventh games (52 percent), far below the 57 percent success rate of the home team in all World Series games.

I don’t think 52% is “far below” 57%. In fact, if the home team had won just two more of those seventh games the success rate would have more than 57%. If you do a statistical test (the Fisher Exact Test) you will find that the difference between 52% and 57% is not even close to being statistically significant, so it is wrong for Abramowitz to reject the notion that the advantage comes from the extra home game. (He also miscounts the number of wins for the home team in game seven—it is actually 17 of 32, or 53%.)

Via Ralph Luker I find Andrew Ackerman’s correction of a Boston Globe article that downplayed Michael Bellesiles’ misconduct. The Emory panel rightly found Bellesiles guilty of falsification and other academic misconduct. It is disgraceful that the American Enterprise Institute refuses to conduct a similar investigation into John Lott’s conduct.

Lott has responded to Media Matters criticism of his comments on Florida 2000. Lott writes:

Media matters makes it look like I was talking about “voter disenfranchisement” (which I assume includes the non-voted ballot issue) by adding into what I said the broader statement “[on voter disenfranchisement],” and misconstrued what I was saying. I have written extensively on the myths regarding the Florida vote here, and would have been happy to get into the issue of non-voted ballots, but the amount of time available was just so limited we barely got to talk about the intimidation part of Dobb’s question. Second, even if the point I was making wasn’t clear at first because I was cut off, when I was allowed to continue I clearly stated what I was talking about involved voter intimidation.

But here is what Lott said:

Well, probably no way you can get around that. Different people gain and lose as a result of how well I think the system can operate in different places. I think a lot of the discussion about disenfranchising African-American voters, in particular I think it’s been fairly sad, because I think there have been a lot of myths in Florida, for example. I mean, you have the Commission on Civil Rights did an extensive set of hearings, they weren’t able to identify even one person.

Media Matters did not add the mention of “disenfranchising African-American voters” to what Lott said. Now maybe Lott meant to say voter intimidation but that doesn’t change what he actually said.

Lott continues:

From the US Commission on Civil Rights report on Voting Irregularities in Florida During the 2000 Presidential Election, chapter 2 entitled “First-Hand Accounts of Voter Disenfranchisement”:

As a result of its investigation, the FHP found that some policy violations had occurred, but concluded that no citizen was unreasonably delayed or prohibited from voting as a result of the Oak Ridge Road checkpoint.

Neither of these witnesses’ testimony indicates how their or others’ ability to vote was impaired by these events. . . . Despite this one, highly publicized incident, there has been no evidence whatsoever of police intimidation of voters. . . . Colonel Hall’s testimony conclusively demonstrates that there was no intent by members of the Florida Highway Patrol to delay or prohibit any citizen from voting on Election Day. All pertinent evidence shows that in fact no one was delayed or prohibited from voting by virtue of the equipment checkpoint operation.

Only the first paragraph of Lott’s alleged quote actually comes from chapter 2 of the report and it is describing the findings of a Florida Highway Patrol investigation, not the findings of the Commission on Civil Rights which were:

Regardless of the motivation for the Florida Highway Patrol’s actions on Election Day, it appears that a number of voters perceived, at minimum, that they were negatively affected by the proximity of law enforcement officers to the precincts around Tallahassee.

Lott’s second paragraph does not appear in the CCR report at all, despite the fact that Lott cliamed that it did. In fact, a Google search for it indicates that it only appears on Lott’s blog, so it might have been fabricated by Lott..

Update: Alert reader dmm found the source of Lott’s second paragraph—it’s not from chapter 2 of the Commission on Civil Rights report, it’s from the dissent to that report. So Lott did not fabricate it, he just presented something as the findings of the Commission when it was the opposite of the findings. The Commission did find some intimidation. Lott can certainly argue that the dissent was right and the Commission was mistaken, but it is wrong for him to claim that the Commission found that there ws no intimidation.

An interesting feature of the dissent is their attack on Allan Lichtman’s statistical analysis:

The choice of Dr. Lichtman to carry out this work is problematic. When he appeared at the June 8, 2001, meeting of the commission to present his findings, he took pains to present himself as a scholar above party, who had “worked for Democratic interests… and for Republican interests.” At the time, the American University web site identified him as a “consultant to Vice-President Albert Gore, Jr.”[34] His partisan commitment was evident in his media appearances throughout the campaign and the period of post-election uncertainty.

Instead, the dissent relies on ananlysis by… John Lott, nowhere mentioning that Lott might be partisan. (And a third, independent analysis by Klinkner agreed with Lichtman and not Lott.)