Civilian Deaths in Iraq: Bad Methodology on Parade
Published January 09, 2005
Opponents of the War in Iraq have been circulating the depressing figure that 100,000 civilians have been killed in the past year of fighting in Iraq. Anyone can make up a figure and claim it's real, but the opponents of the war have been stating this figure with great authority because they claim it is supported by a report in the British medical journal Lancet, arguably the most respected medical journal in the world.
The truth is that the original report published there was a purely statistical analysis of death rates in Iraq based not on detailed data or reported deaths, but on a gross comparison of the relative rate of death in the year before the invasion and the year after it using only a limited statistical sampling taken from 10 selected locations. In addition the samples are drawn overwhelmingly from the more violent southern and central regions of the country with only two of the ten samples coming from the north - perhaps because both of those samples actually show a substantially lower rate of violent death after the end of the war, perhaps because Saddam Hussein wasn't around to randomly massacre Kurds anymore.
The specificity of the results matches the poor quality of the methodology. All they can tell us in the report is that somewhere between 8000 and 194,000 more people died in Iraq in the year after the invasion than in the year before. The enormous spread of possible additional deaths is a product of the statistical nature of the analysis, which doesn't actually differentiate civilian from military casualties or casualties from war-related violence from increases in violent death from any of a number of possible sources such as riotingm, damaged roads or other hazardous conditions. The 100,000 dead figure which people are throwing around is the average of the high and low range numbers in this figure, and it is essentially meaningless. In addition, since there are about 16,000 documented Iraqi military deaths which resulted from the invasion, that number should be subtracted from the total, leaving a range of -8000 deaths to 182,000 deaths and making it statistically possible for there to have been no civilian deaths at all during the period after the invasion.
This demonstrates that counts of civilian deaths based purely on statistics aren't a good way to analyze the human cost of the war. Counts based on actual reported deaths and people identified as dead from causes related to the war are much better. Despite the daily reports of ongoing terror in Iraq the truth is that the basic social and government structure is functioning and people are keeping track of who dies and where, when and how they die. In addition, families know when members go missing and make an effort to see that they are either found or identified as being dead. It's not necessary to fall back on vague statistical analysis when there are real records to look at.
Although they are war opponents and looking for all the deaths they can find to count, there's a much more sensible count - about 16,000 reported deaths and about 3,000 identified civilians dead - at Iraq Body Count. The interesting thing there is that they are counting all civilian deaths related to the war as part of the casualty count and not differentiating between deaths caused by coalition troops and those caused by terrorist/insurgent activity. From their perspective anyone who dies from violence in Iraq gets blamed on the coalition. But if you look at their actual database of the 3000 fully documented civilian deaths you discover that the overwhelming majority of the deaths they are counting were caused by terrorist/insurgent attacks and assassinations, not by coalition forces. They don't make it entirely clear, but when they attribute gunfire as the cause and Iraqi police as the victims, you can bet it's not the US or the UK shooting them. In actuality less than 500 of the 3000 fully documented deaths can be clearly attributed to the actions of coalition forces. Based on that relationship, a reasonable high estimate of deaths actually caused by coalition military forces, including all of those claimed but not fully documented would be about 2700. This is not far from the estimate of about 4000 civilian deaths released by the US Department of Defense, and even that figure is suspect because when enemy combatants don't wear uniforms it's awfully hard to be sure that all of those non-uniformed casualties were genuine civilians, so the 2700 documented deaths looks like the most reliable figure to work with.
Dave Nalle has been a magazine editor, freelance writer, capitol hill staffer, game designer and taught college history for many years. He is a Liberty Republican and former Libertarian. He now designs fonts for a living and lives with his family and pets just outside Austin. You can find his writings on politics and culture at Republic of Dave and works on designs and fonts at The Scriptorium.
- Civilian Deaths in Iraq: Bad Methodology on Parade
- Published: January 09, 2005
- Type:
- Section: Politics
- Writer: Dave Nalle
- Dave Nalle's BC Writer page
- Dave Nalle's personal site
- Spread the Word
- Like this article?
- Email this
Save to del.icio.us
Comments on this articleComments
Excellent piece and maybe the question that should be ask is how a leading medical journal would allow political bias to interfere with science?
The real purpose of the piece is to support the notion that the Iraqis are better off with a manic at the helm instead of slowly rebuilding a new democratic state.
>>You don't have a !@#$% clue of the number dead, yet you are offended as if you were a member of the family.<<
I think family members and anyone with a conscience is troubled by any civilian casualties regardless of the number - as I said in the original article.
>>You disagree - therefore false methodology.<<
Exactly backwards. Anyone with half a lick of sense can look at the results of that 'analysis' and realize the methodology is terrible. Garbage in, garbage out.
>>I've seen many more people try and tear apart the Iraq Body Count Web site and methodology. I guess merely until a higher figure comes along to affront your senses.<<
Did you actually READ my article? I pointed to Iraq Body Count as a better example, not perfect, but closer to how a count should be done.
>>Excellent piece and maybe the question that should be ask is how a leading medical journal would allow political bias to interfere with science? <<
I'm not positive that the skewed sample in the study was intentional. The actual tone of the article in Lancet is pretty cut and dried and not blatantly polemic. It's just not terribly well done - certainly not reliable enough for all the credence people are placing in it.
Dave
So now you're backing off everything you wrote? Yikes. Forget it then.
The way you come across - without any support that I can see is you are smarter than many scientists. Gotta prove that kind of thing otherwise you're just a bloviator. Which is fine - for what it is.
Also when you write this, whic is what really provoked me to start with
>>In a nation with Iraq's population of 22.6 million the number of actual civilian casualties is really remarkably low even if you take Iraq Body Coount's most inflated figure of 17,000. That's only a tiny increase over the normal rate of death for a population that size.
... I find it incredibly hard to believe you are a clearer thinker than a scientist. Or that you really thought about your piece. I'm just discussing this in methodological terms - as you should have tried in your post instead of a wide collection of random opinions.
>>So now you're backing off everything you wrote? Yikes. Forget it then.<<
Not as far as I can tell.
>>The way you come across - without any support that I can see is you are smarter than many scientists. Gotta prove that kind of thing otherwise you're just a bloviator. Which is fine - for what it is.<<
I'm a historian with a background in demography. That may or may not make me smarter than some scientists, but it does mean that I can tell what kind of results are useful and a numerical result with a spread as large as they got in the lancet study is meaningless.
As for support, the article includes something called 'hyperlinks'. You may have heard of them before. If you click on them they take you to the data and source documents. In this case to the PDF of the Lancet article which shows where there samples came from and what the results were, and also a link to Iraq Body Count which I believe goes to the page where they provide the specifics on the 3000 deaths they've got hard data on. That's support.
>>
>>In a nation with Iraq's population of 22.6 million the number of actual civilian casualties is really remarkably low even if you take Iraq Body Coount's most inflated figure of 17,000. That's only a tiny increase over the normal rate of death for a population that size.
... I find it incredibly hard to believe you are a clearer thinker than a scientist. Or that you really thought about your piece. I'm just discussing this in methodological terms - as you should have tried in your post instead of a wide collection of random opinions.<<
It really isn't an opinion to say that the body count is small compared to previous wars when I give the example from WW2 and provide the relative percentages. That's not an opinion. That's a fact.
Dave
http://www.diablog.us
there's a much more sensible count - about 16,000 reported deaths and about 3,000 identified civilians dead - at Iraq Body Count.
Actually, the man behind the project also conducted a similar count of casualties in Afghanistan. That was debunked. Same accusations were made against the current project.
If you check the iraqbodycount's methodology, you'll see it gets it's info from places like news sources (CNN, AP) and morgue records. During a time of war and when huge sections of a country still experience lawlessness, I don't think that's an accurate way to determine deaths. I'm not sure how many cities Christiane Amanpour visits every day.
How many warring tribes are taking advantage of the occupation and eliminating opposing tribes members? How many warring tribes disappear, get taken to a basement and never heard from again? Shouldn't we consider that a percentage of 'missing' would qualify as dead? Would all this get counted as a war casualty if it would have been less likely to happen, but still possible of happening under Saddam's rule?
I'm not saying the number's high or low, I don't know. I hope it's low. What I am saying is that your source has been debunked much more effectively than you are debunking Lancet.
From their perspective anyone who dies from violence in Iraq gets blamed on the coalition.
Um, well, yeah, it's unfortunate but it's because the terrorists wouldn't be killing civilians if they weren't trying to drive an occupying force out of the country. So the occupying force gets blamed. Not saying it's right, but it's the way it is. The damage that is being done to the perception of the West over there is catastrophic and will take generations to heal, and may never be healed, thanks to the war-is-profit conservatives who feel they own this country.
The most important question here is: with your methodology, who are you trying to convince, the American people or the rest of the world?
My particular point was this from you
>>That's only a tiny increase over the normal rate of death for a population that size
.. is absolutely nonsensical. By saying that, you now include all methods of death - presumably all natural causes as well. That is a wildly inaccurate and meaningless comparison.
And especially if you just stuck to the murder rate - then the rate is more than double.
I say that thinking only of the approx. 10,000 gun-related murders in America a year (and that in a country 11/12 times the pop. size of Iraq). So even 17,000 - the low figure - would more than double the murder rate and I'd have to say it would be closer to 50 or 100 times the previous rate.)
I was wrong to say you were running away from all you were saying, but you are making basic mistakes and starting to slip and slide on some of your definitions.
Dave, fiddling with 'civilian death methodology' is a diversionary smoke screen.
*Tell us how swell Iraq is going.
I'm especially interested in the upcoming election (a Shiite theocracy), the exit strategy, and what will constitute an "American Victory".
* And do you believe in Santa Claus?
Subject: Dave fiddles while Baghdad burns.
re. the sentence:
"...So the story here is not really the unsupportable claim of 100,000 casualties, but the less alarming but really much more impressive fact that the war and pacification of the country has been carried out with so amazingly few civilian casualties and that in fact the vast majority of the civilian casualties outside of full-scale military operations have been caused not by coalition forces, but by terrorists and insurgents indescriminately bombing and shooting their own population or deliberately assassinating specific civilian targets. Wouldn't it be nice to see that mentioned in the media or on a liberal weblog for a change."
1) Proust, you're not.
2) What the hell are you smoking?
"[The story ] ...is the war and pacification of the country has been carried out with so amazingly few civilian casualties..." --??!!
Wha?
Dave, ya think at least part of "the story" might be that the "war" was not followed by the "pacification" of the country? Or that we have no EXIT STRATEGY? Or that under the goal of 'fighting terrorism', we've created a few hundred thousand more terrorists? Or that the 'war' and occupation are bankrupting our nation? Or that are military are stretched beyond their limits? Or that the upcoming 'election' will probably result in a Shiite Theocracy wild about Iran?
Folks, I didn't say anything about how I feel about our involvement in Iraq, lack of pacification, the upcoming election, shiite theocracies or Santa Claus. I merely objected to blatantly bogus misrepresentation of the number of civilian casualties and the mischaracterization of casualties caused by terrorists/insurgents as the direct responsibility of coalition forces. That's it. Trying to tar me with the brush of defending the war isn't going to work because that's not what I was doing.
And just for the record, I do believe in Santa Claus. See my recent essay at http://www.diablog.us/comments.php?id=A90_0_1_0_C
Dave
Dave, I think your post is very sensible and is an important tonic to the 100K figure that I hear thrown around with authority all over the place. I bleieve that was your purpose and you did a good job of chipping away at the figure from several angles.
Yes, ALL civilians deaths are to be grieved and avoided at all reasonable cost, but I agree with you that the coalition forces have done a remarkable job of avoiding civilian casualties and it should be clear to anyone who cares to see that we are not the enemy of Iraq: the insurgents are and they are the ones PURPOSELY killing civilians.
There is also the issue of hostile "civilians" who once they are armed and take action are no longer "civilians" as far as I am concerned
Honor the Fallen:
"Spc. Glenn J. Watkins, 42, of Carlsbad, Calif.; assigned to the 1st Battalion, 161st Infantry, Washington Army National Guard; killed April 5 when a vehicle born improvised explosive device detonated near his military vehicle in Bahgdad."
www.militarycity.com/valor
Watkins, a husband and father of four, passed up an opportunity to return stateside this March and volunteered for another six months in Iraq. Two weeks later while on patrol in a Humvee, he was manning the rooftop machine gun when a bomb exploded nearby. Four other soldiers were injured in the blast.
Watkins has an adult son serving in the 82nd Airborne Division at Ft. Bragg.
U.S. military killed to date since the invasion of Iraq...1,587...
"Some gave all, all gave some" (VVA slogan)
- MCH, Vietnam era vet
You have to admire Watkins dedication to the work being done in Iraq and mourn the loss to our nation. It's another reminder of how important it is to follow through so that 1587 men and women will not have died in vain. We must not waver in our support, because every weakness we show here at home is encouragement to the murderers in Iraq - our weakness validates their violence.
Dave
Re: >>All they can tell us in the report is that somewhere between 8000 and 194,000 more people died in Iraq in the year after the invasion than in the year before. The enormous spread of possible additional deaths is a product of the statistical nature of the analysis ... The 100,000 dead figure which people are throwing around is the average of the high and low range numbers in this figure, and it is essentially meaningless. <<
The range of a population distribution is the least reliable statistic that you can derive from a sample. What the study tells us is that if you repeated the survey, it is likely that you would come up with a distribution with a mean of about 100,000.
Re: >> since there are about 16,000 documented Iraqi military deaths which resulted from the invasion, that number should be subtracted from the total, leaving a range of -8000 deaths to 182,000 deaths and making it statistically possible for there to have been no civilian deaths at all during the period after the invasion.<<
When a transformation like the proposed subtraction yields meaningless numbers like -8000 excess deaths, it is time to examine the validity of the technique.
Mark
Exactly, MDE. Their extremely hypothetical analysis is incapable of being compared meaningfully to any actual statistics which apply to the same subject matter, so what use is it?
DAve
It is the proposed subtraction of 16,000 from the mean that is illegitemate.
re: >>Their extremely hypothetical analysis is incapable of being compared meaningfully to any actual statistics which apply to the same subject matter, so what use is it?<<
There are no 'actual statistics which apply to the same subject matter'. We can go around about the passive counts going on; they do not apply to the same subject matter, nor do they claim to.
As for usefulness, I quote the Economist from last November http://www.economist.com/science/PrinterFriendly.cfm?Story_ID=3352814 : 'The study is not perfect. But it does not claim to be. The way forward is to duplicate the Lancet study independently, and at a larger scale.'
Mark
We all come to the table with an agenda here; right-winger wants to minimize the number of casualities, left winger wants to inflate them, moderate wants to sit on the fence...
Common sense tells us that a hundred thousand dead civilians would create a lot of gravesites...How about a statistical sampling of graveyard burials, instead of interviews with sunni muslims? The manner in which this number has become fact is an embarrassement to liberals everywhere. The truth is supposed to be more important than our political goals and agenda.
re:"We all come to the table with an agenda here; right-winger wants to minimize the number of casualities, left winger wants to inflate them, moderate wants to sit on the fence..."
Some of us simply want a no bullshit assesment of the 'cost' of this war in order to evaluate tactics employed. I suspect that 'Shock and Awe' was more bloody than our press portrayes it to have been.
Mark
Whether you are for the war or not, bogus numbers about the number of casualties isn't going to help you. If you are pro-war, then you cannot have a substantive debate about the merits of the war if they accept the numbers as fact because it is emotional terrorism. If you are against the war then your credibility is shot by the sort of posters above who are busy trying to kill the messenger. This attitude is simple; "if you don't agree with everything I say and believe, you must be pro-war.". The polarization of this nation has led to new lows for the right and the left. We are all supposed to ascribe to Michael Moore or Donald Rumsfeld. Great.
http://livefromblogdahd.blogspot.com/
Add your comment, speak your mind
(Or ping: http://blogcritics.org/mt/tb/24062)
This post. Bad taste on parade.
You don't have a !@#$% clue of the number dead, yet you are offended as if you were a member of the family.
You disagree - therefore false methodology.
I've seen many more people try and tear apart the Iraq Body Count Web site and methodology. I guess merely until a higher figure comes along to affront your senses.
You seem to be saying only war opponents are worried and offended by the civilian deaths and therfore should be discounted out of hand. Are you? Because that's not true.