President Bush, methodology expert? Not!
Talk about a numbers game! On October 11 the British medical journal Lancet published a study by a team of American and Iraqi epidemiologists that estimated that 655,000 more people have died in Iraq since coalition forces arrived in March 2003 than would have died if the invasion had not occurred.
The study’s researchers interviewed a random sample of households throughout Iraq. They concluded that there were 655,000 “excess deaths” as a result of the war, equivalent to 2.5 per cent of the population; 601,000 died through violence, usually gunfire. The same researchers had in 2004 already published an estimate of almost 100,000 excess deaths in the first 18 months after the invasion, using a similar technique.
As numbers go that is shockingly high. It is more than 20 times the estimate of 30,000 civilian deaths that President Bush gave in a speech in December. It is more than 10 times the estimate of roughly 50,000 civilian deaths made by the British-based Iraq Body Count research group.
But even more shocking was the reaction of the Bush administration. President Bush said the study’s methodology had been “pretty well discredited.” “I don’t consider this a credible report. Neither does General Casey [the US commander in Iraq] and neither do Iraqi officials,” he said.
The problem is that the White House offered almost no specifics on methodological errors. And neither President Bush nor his press secretary Tony Snow, who has been dutifully spinning the same line are known for their biostatistician or public health credentials.
Now, this is not to say the Lancet study is absolutely correct. It is, after all, an estimate, not a precise tally. Most scientific studies are open to analysis and criticism; that is basic scientific method. And bear in mind that this study is statistical in nature; making confirming its conclusion even more difficult. Those wishing to understand how to correctly interpret mortality data should read this.
And certainly it has not taken long for criticism to come, and not just from Pro-Bush supporters. Iraq Body Count, a group that gathers fatality figures from news reports and posts them online, has published a critique finding that the “authors have drawn conclusions from unrepresentative data.” Useful criticisms have also come from respected independent researchers such as William Arkin and Anthony Cordesman of the Center for Strategic and International Studies. Parsing the Lancet numbers it would mean about 500 unexpected violent deaths per day throughout the country, which seems unlikely, even when taking into account that not all deaths are reported.
Yet the study actually used a standard public health epidemiological tool; one that has been used to assess deaths due to natural disasters, such as the 1995 heat wave in Chicago, described in this book, and war, such as the ongoing civil war in the Congo. It has also been used in Darfur where the data has been widely quoted by the U.S. government. So, then one might naturally wonder whey is the methodology credible in the Sudan but not Iraq?
A point, which has not been talked about much however is this. Even if the Lancet numbers are wildly off, say by half, so that the total is 327,000, it shows that that the US has failed to live up to its responsibilities as occupying power under the Geneva Conventions, and indeed that the situation of the Iraqi people has been deteriorating steadily since the United States invaded. That conclusion, by the way, is confirmed by the study which divided the reported deaths up into four time periods of roughly 13-14 months each:
Period I (pre-war)– 5.5 deaths per 1,000 people/year
Period II– 7.5
Period III– 10.9
Period IV– 19.8
In short, deaths have been growing steeply throughout every year the US has stayed in Iraq. Simply dismissing the study findings because they are politically inconvenient is the wrong move for the Bush administration. The not very subtle message it sends to Muslims everywhere is that it doesn’t really care about how many Iraqis die.
Until the study can be proven wrong, or partly wrong, on scientific grounds, it must certainly stand, until better evidence emerges. And in the meantime the United States should abandon its “we don’t do windows” attitude towards casualty reporting and, in the interest of public accountability promote more accurate and transparent monitoring of the true effects of violence on Iraqi civilian health.
Related posts:
It is interesting that while the Bush administration casually dismisses the Lancet study as “discredited,” it always insists that everyone killed as a result of U.S. forces bombing a target in Iraq are either insurgents or terrorists. Yet they can offer no proof or methodology for making such claims, but expect us to accept their assertions.
Comment on October 17, 2006 @ 8:48 am
| Iraq Body Count … “authors have drawn conclusions from unrepresentative data.”
Without further detail, this claim seems meaningless. The point of statistical inference is to draw inferences about an unknown population by means of sampling. Thus, without knowing the population characteristics, one cannot tell how representative a particular sample is. (If one already knew the population characteristics, there’d be no point in sampling.) This is where statistical theory becomes useful. The drawing of a random sample is the best that one can do without good population knowledge. (Population knowledge does allow one to correct a random sample; for example, if it is known that the response rate to a survey decreases in the respondants’ income, then a random sample will be more representative of poor people.)
| the Lancet numbers … mean about 500 unexpected violent deaths per day
| throughout the country, which seems unlikely.
This may seem unlikely if our standard is the number of deaths reported in the media. However, the Lancet authors note that: “Aside from Bosnia, we can find no conflict situation where passive surveillance [such as relying on media reports] recorded more than 20% of the deaths measured by population-based methods.”
| the United States should … promote more accurate and transparent
| monitoring of the true effects of violence on Iraqi civilian health.
Can we upgrade this to “must demand” from “should … promote”? A credible study by established researchers, supported by top research universities, and published in a top medical journal, has claimed that our project of liberation has killed two thirds of a million people – placing this among the most murderous periods of Iraqi history. If we wish to claim that we care for Iraqis, we cannot simply dismiss this off-handedly. Why has the independent assessment called for by the researchers in 2004 still not been conducted?
Comment on October 17, 2006 @ 9:01 am
I think David has missed something key here. Essentially, that the administration doesn’t care about Iraqi civillian deaths. Everything is keyed towards minimising US service deaths, which are reported.
The US force protection strategy really doesn’t help in many cases resulting in excessive civilian deaths and prevents US troops from acting effectively to supress the sectarian violence. Muslim or otherwise, Iraqi deaths get reported simply as numbers and there are no yellow ribbons or empty plots in Arlington for the US electorate to see and grieve over.
Comment on October 17, 2006 @ 2:28 pm
Ok, so David knows all that stuff, I think he left it out for reasons of brevity. I also forgot to say I thought it was a good post.
Comment on October 17, 2006 @ 11:53 pm
What you don’t say, David, apropos the Lancet estimates, is that statistical estimates of this sort produce distributions with associated probabilities. It is natural enough to take the middle of the distribution as a rough indication but it is a shorthand that should be set in the context of its measures of confidence. From the data I’ve seen, these are quite wide. Which is to say that the central number is not very meaningful.
Your half the estimate is safer. Methodology apart – and everyone shd know the limits of statistical estimation – to treat it as a hard number is to mislead. Given the sophistication of the people at the White House, I assume this is intentional. Another lie.
The other thing you don’t t quite say is that any death is unacceptable, let alone the smaller numbers, LET ALONE even half the horrendous Lancet estimate. Is this how we improve the lives of the Iraqis? You do touch on it in your concluding point abt breaching the Geneva Convention… as if those tough, straight-shooting Texans care about such pansyish concerns. (Real men don’t worry about other people.)
Of course, every such revelation exposes all the more cruelly the lies told by the US and UK about the reasons for the invasion. Bring democracy to the Iraqis? Gimme a break! Re-colonise the Middle East and subordinate local concerns to American priorities, more like.
Joe Roeber
Comment on October 18, 2006 @ 11:00 pm