Dave,

Never forget what Samuel Clemens said many years ago.

"There's lies, there's damned lies, and there's stastistics."

Dave - I think you go overboard in debunking the Lancet's figures, but I largely agree with you. On the other hand, the most thorough mainstream debunking of these figures that I recently heard was on a public radio station, supposedly the heart of the left, so I don't think that you can simplistically proclaim that the left will universally embrace this dis-information. Rather, there is a pseudo-progressive core that foolishly believes that if you can demonstrate that a war kills some magically big number of innocents or civilians, then everyone will equally magically agree that war is bad and end all hostilities.

On the other hand, working through the data to conclude that the number of firearm deaths of men suggests "active involvement in combat" is not necessarily correct by any means. For example, it is clear that there are factions that target males who attempt to volunteer for the army or the police force, and that Sunni or Shiite males are targeted by one side or the other in an attempt to eliminate any potential future opposition, whether or not these people ever actively take up arms as insurgents. Kurds have more often been victims of bomb attacks since there is already a degree of Kurdish separation from areas that have mixed Sunni and Shiite enclaves. It is not quite ethnic cleansing, since women and children are not yet equally targeted, but still, any attempt to draw simple distinctions between civilian and combatant just plays the same game as the Lancet.

Dave - I think you go overboard in debunking the Lancet's figures, but I largely agree with you. On the other hand, the most thorough mainstream debunking of these figures that I recently heard was on a public radio station, supposedly the heart of the left, so I don't think that you can simplistically proclaim that the left will universally embrace this dis-information.

I have some confidence in the honesty of public radio. But was this a debunking of these figures or the 2004 study? They're already crowing about these new numbers at DailyKos and DU, but I haven't heard anyone criticising them yet.

Rather, there is a pseudo-progressive core that foolishly believes that if you can demonstrate that a war kills some magically big number of innocents or civilians, then everyone will equally magically agree that war is bad and end all hostilities.

That appears to be the state of mind which is motivating the authors of the study who seem to believe that if they can throw out a really, really big number as a possible outcome of the study then it will make everything better.

What I didn't point out specifically in my analysis is that my reduced amount based on their research is almost exactly the same as the low-end version of their conclusions adjusted for their (unstated) margin of variation. Despite the fact that their conclusion states that 655,000 people died, their data suggests that what they really mean is that somewhere between 131,000 and 655,000 died, but they aren't promoting that lower number and it's not the one that's' going to get all the press.

Dave

It seems the media is already picking up the magic number after only 24 hours. It's showing up on the BBC, CNN, ABC and most of the newspapers as well, pretty much uncontested and unqualified.

My favorite so far in the media is this editorial which refers to this pile of claptrap as a 'highly credible study' and compares Bush to Pol Pot.

Dave

Today's International Herald Tribune has an article discussing the study with a quote from the Iraqi government rejecting the study an saying that it's 10 times higher than any realistic casualty estimate.

Dave

Dave - love the transformations and adjustments...keep up the analysis and you soon might be able to 'show' that no one died in Iraq today

the improvement in your work this time out is that you don't rely on Iraq Body Count methodology for your 'corrected' estimate

...but please let us know on what studies/evidence you base your claims for example that the Iraq countryside has been significantly 'less violent' than the cities or that underrepresented areas have experienced significantly fewer excess deaths than overrepresented ones - I've heard that information about much of the country is inaccessible to observers and government employees...that there is no central agency visiting morgues and counting death certificates

and why should anyone believe such convenient claims in the absence of data - ?

this brings us back to the problem of how to improve on cluster study data collection methodologies in violent environments which is what these researchers have been working on around the world for years

rather than accusing them of being political hacks and liars I suggest that you offer them your ideas on methodological improvements for their third time out in Iraq

Dave even though I agree with your article on what it says I couldn't help but notice that the Bush team works the same formula..
IF YOU TELL THE SAME LIE OVER AND OVER PPL WILL BELIEVE IT ....this formula has worked for Bush for the last four years and the ppl bought it..So if the Dems want to jump on this and use it to their gain I will give them credit for it..who cares whats right or wrong anymore, Bush has lied so much already that Dems would be smart to take this flawed info and go with the big number and then say it over and over again even run ads with the big number HELL it will take four years for the American ppl to figure it out.

Dave, I am echoing what Troll said. In the US, rural and urban crime/murder rates are generally pretty equal. I need data to support your assertion that the rates are different in Iraq; do you have any? I am willing to bet that most foreign reporters hang out in cities, and that the Iraqi media and government pay more attention to urban events than to rural ones. Today, fourteen bodies were discovered in an orchard outside Baghdad... would it have made the news at all if they were farther from the big city?
BTW everyone, "methodology" is the study of methods. It is pure academic pretension to tack the "ology" on when we are really just discussing methods. Thank you, and stay skeptical.

"By all means let's stop the deaths in Iraq, but let's do it for legitimate reasons and not based on questionable research."

Because if the murder of innocent Iraqi men, women and children by the US was stopped while it was permissible it would be a fucking horrific tragedy, right Dave!

We've dropped 240,000 cluster bombs, 3 dead for each bomb seems like a decent average to me.

No matter what you think about statistics, this survey team visited 1,840 homes and found 547 post invasion deaths. 1 dead for every 4 houses randomly selected is sad no matter how you look at it.

this brings us back to the problem of how to improve on cluster study data collection methodologies in violent environments which is what these researchers have been working on around the world for years

IMO cluster studies should probably be abandonned alltogether. I also think it's significant to note that none of the groups they had gathering data were attacked or had any significant problems collecting their data.

But even putting the cluster system aside, how about as a working principle - you don't count the dead with a fucking opinion poll.

Hell, huge numbers of Iraqis have cell phones now - why not just call them all on their cell phones and question them. That sounds pretty damned safe.

But for a start, if you're going to do cluster studies, you have to break the sources down proportional to where the population lives. You can't just poll the cities and ignore the villages and rural areas.

rather than accusing them of being political hacks and liars I suggest that you offer them your ideas on methodological improvements for their third time out in Iraq

As someone earlier suggested, why not just count the dead? This time out they went to great pains to make sure that almost all the people they questioned provided death certificates for the people they claimed had died - and about 80% had them. If those death certificates exist, why not go to the issuing body and get a total and then add 20% to it as a 'fudge factor'? That would be a hell of a lot more accurate than this effort.

Dave

Dave even though I agree with your article on what it says I couldn't help but notice that the Bush team works the same formula..
IF YOU TELL THE SAME LIE OVER AND OVER PPL WILL BELIEVE IT

Georgio. Did I say one word about Bush in this article?

Dave

CANDIDATE-VETERAN ATTACKS BUSH ON IRAQ
By DENNIS CONRAD, Associated Press Writer
Sep 30, 2006

WASHINGTON - An Illinois congressional candidate who lost both her legs during combat in Iraq said Saturday that President Bush has no real strategy for securing the war-ravaged nation, just political talk designed to appeal to voters.

"Instead of a plan or a strategy, we get shallow slogans like 'mission accomplished' and 'stay the course,'" former U.S. Army Captain Tammy Duckworth said in the Democrats' weekly radio address. "Those slogans are calculated to win an election. But they won't help us accomplish our mission in Iraq."

Duckworth, who co-piloted a Black Hawk helicopter that crashed while under a rocket grenade attack almost two years ago, also criticized Bush and others in his administration for accusing anyone who challenges the president's policies of "cutting and running."

"Well, I didn't cut and run, Mr. President. Like so many others, I proudly fought and sacrificed," Duckworth said. "My helicopter was shot down long after you proclaimed 'mission accomplished.'"

At a GOP fundraiser Thursday in Alabama, Bush said, "The party of FDR and the party of Harry Truman has become the party of cut and run."

Duckworth is seeking the suburban Chicago seat against Illinois state Sen. Peter Roskam.

In her address, Duckworth, now a major in the Illinois National Guard, also lashed out at the GOP-led Congress for refusing to do its job of holding the Bush administration accountable for its flawed Iraq policy.

"We need a Congress that will ask the tough questions and work together for solutions rather than attacking the patriotism of those who disagree," she said. "It is time to encourage Iraqi leaders to take control of their own county and make the tough choices that will stop the civil war and stabilize the country."

The estimate of 600k seems extraordinary on the face of it, but the article has failed to point out failings in the methods. The Lancet report has not been refuted, merely had curses thrown at it.

Your analysis is incorrect in many ways. At this time I'd like to point out two ways.
You suggest that high violence areas were oversampled by 20% because of rounding error. I've assembled the data from Table 1 of both the Lancet articles to attempt to determine the over- and under-sampling. I assigned each governate to High, Medium and Low violence categories according to Figure 3 of the Lancet article. These assignments are congruent with the ones you made, except you mentioned only the overrepresented high violence areas and the underrepresented low violence areas. There also happen to be under-represented high violence areas and over-represented low violence areas. I then calculated the under/over representation for each of the 3 categories. I did this with and without weighting the governates by their proportion of the total population, but the results are nearly identical. Here are the results (2006 sample only):

High violence: overrepresented by 4.1%
Medium violence: underrepresented by 2.6%
Low violence: underrepresented by 1.5%

I note that although you and I both include the low violence areas Dahuk and Muthanna that Burnham et al actually omit from their analyses because they did not sample it (that is, the ~1 million people in those provinces were essentially subtracted from the total population. If we drop these two provinces from consideration, as the study does, and weight each province by its proportion of the total population, then we obtain:

High violence: over-represented by 3.2%
Medium violence: under-represented by 4.3%
Low violence: *over*-represented by 1.1%

Net result: its unclear that there is any over-representation at all of violent areas.

Even if you're adjustment percentages were correct, you applied them incorrectly. Your calculation was:
654,965 * (-.3 + -.2 + -.3) = 130,993

In fact you should multiply:
654,665 * .7 * .8 *.7 = 256,746

If we correct for your error regarding over-representation of high violence areas by changing the -.2 to -.04, then you have
654,665 * .7 * .96 *.7 = 308,000

Three hundred and eight thousand excess civilian casualties!
And that's assuming your other adjustments are justified, which as others have pointed out, they are not.

Dave - RE: I have some confidence in the honesty of public radio. But was this a debunking of these figures or the 2004 study? They're already crowing about these new numbers at DailyKos and DU, but I haven't heard anyone criticising them yet.

The public radio report very clearly debunked the recent Lancet study, and also noted the deficient methodology of the earlier study. I didn't pay much attention to the report at the time, didn't even note which public radio program it was, because I under-estimated the degree to which people would gloam onto this stuff and pass it off as valid analysis.

OUTSTANDING article, Dave. This should not only be a "Blogcritic pick of the week," but it should also be picked up and echoed by other fair-minded bloggers.

Great job!

"The Lancet" is peer reviewed, isn't it?

MCH, #13 is off-topic and a violation of copyright. Take it somewhere else, perhaps.

Dave

You suggest that high violence areas were oversampled by 20% because of rounding error.

Keep in mind that just as the study conclusions are an 'estimate', I'm making an estimate as well.

I've assembled the data from Table 1 of both the Lancet articles to attempt to determine the over- and under-sampling. I assigned each governate to High, Medium and Low violence categories according to Figure 3 of the Lancet article.

That's more or less what I did as well, except I gave the medium regions a pass.

These assignments are congruent with the ones you made, except you mentioned only the overrepresented high violence areas and the underrepresented low violence areas.

Actually, I only mentioned those which were over or underrepresented by at least 25%.

There also happen to be under-represented high violence areas and over-represented low violence areas.

Yes, but under or overrepresented at a much smaller percentage than the ones I noted.

I then calculated the under/over representation for each of the 3 categories. I did this with and without weighting the governates by their proportion of the total population, but the results are nearly identical. Here are the results (2006 sample only):

You have to weight the governorates by population or at the very least by number of clusters as well.

High violence: overrepresented by 4.1%
Medium violence: underrepresented by 2.6%
Low violence: underrepresented by 1.5%

I note that although you and I both include the low violence areas Dahuk and Muthanna

I actually didn't factor those areas in, I merely note that they were not reported in the study. With no data at all to go on it didn't seem fair to include them.

And as I said, my final number was merely an estimate. I don't see how you could possibly come up with a more accurate number working only with the tables and the data in the report, because the report as published does not break casualties down by governorate. To get an accurate count you'd need to have that data.

that Burnham et al actually omit from their analyses because they did not sample it (that is, the ~1 million people in those provinces were essentially subtracted from the total population. If we drop these two provinces from consideration, as the study does, and weight each province by its proportion of the total population, then we obtain:

High violence: over-represented by 3.2%
Medium violence: under-represented by 4.3%
Low violence: *over*-represented by 1.1%

Net result: its unclear that there is any over-representation at all of violent areas.

I'd have to go through and do all the math and come up with somethinng more precise than my original estimate, but I don't see how this result can possibly be correct when some of the largest provinces were under or overrepresented by as much as 30% because of rounding.

Even if you're adjustment percentages were correct, you applied them incorrectly. Your calculation was:
654,965 * (-.3 + -.2 + -.3) = 130,993

In fact you should multiply:
654,665 * .7 * .8 *.7 = 256,746

A different way of looking at it, certainly. Not necessarily more valid unless you assume that the geographical underrepresentation and the rounding somehow apply more to military age men than to women. At the very least you have to do the removal of military age men as a separate calculation.

If we correct for your error regarding over-representation of high violence areas by changing the -.2 to -.04, then you have
654,665 * .7 * .96 *.7 = 308,000

So, if we accept your approach, the study's estimate is only off bya factor of 100%? Wow, that's great news.

Three hundred and eight thousand excess civilian casualties!

In the course of three years, of course.

And that's assuming your other adjustments are justified, which as others have pointed out, they are not.

Maybe I should point out again that even my 130,000 casualties estimate is double any of the figures reached by more verifiable methods. If anything my adjustments were super-conservative.

And even you don't have the temerity here to try to claim the the study is in any way accurate or anything but horribly misrepresentative.

Dave

"The Lancet" is peer reviewed, isn't it?

From what I can tell their medical articles are peer reviewed, but some of their other material is not. There is no indication that this study has been peer reviewed in any way. They seem to publish two sorts of articles. Straight science or medicine articles are peer reviewed and appear primarily in their print edition. Articles of a mostly political or policy nature appear in their online edition without peer review.

Dave

I read some very good comments on the study over at pajamas media in response to an emotionall critique from Iraq by Omar Fadil.

A couple of points were raised there which didn't even occur to me, including the idea that if there are 650,000 dead, where are the 2 million corresponding wounded and where are the graves for all these people?

But my favorite comment is from one fellow who points out that if he'd done a paper in college where the results had a margin of error of +/- 40% he'd have been laughed out of class.

Dave

I'm not sure about the methods of the study, but the number they reached is obviously wrong. If there were that many dead people in Iraq there would be no way to minimize it or cover it up.

I'm most bothered by how the media is just running with it and not thinking twice. I did a search for 'iraq casualties' on google news and got over 1000 entries for news stories in the past 24 hours, adn they're all the same thing just repeating that big number and wringing their hands.

Dave, you haven't been paying attention. Just turn on your television and start surfing US news channels if you want to see dead and wounded Iraqis. There are scores of news crews all over Iraq filming US military personnel and US contracted death squads butchering innocent Iraqi men, women and children and Eye Witless News can't wait to beam it into the living rooms of the nation.

McNab, you live in a truly bizarre alternate reality. I watch the news from time to time. And yes, I've seen reports direct from Iraq. But I certainly haven't seen mountains of bodies or anything resembling 'death squads'.

In three years we have a handful of incidents where coalition soldiers stepped off the reservation - and this while faced with endless terrorist attacks which violate every one of the rules of war. What's shocking about Iraq is the forbearance which our forces have shown.

Dave

Dave and McNab 24 & 25 - you are both babbling about that which you have limited to no evidence...tasty bullshit but bullshit none the less

the history of US support for death squads in South America does not mean the same is going on in Iraq...but even the MSM reports regularly on death squad activity there

and I've heard stories from soldiers who have been 'in country' of numerous acts of small time socio-pathology inflicted on the civilian population by US soldiers...tank drivers playing bumper cars for no reason...pointless slaughter of farm animals on which families depend...that kind of 'I'm gonna get some' shit

Iraq is a tragic mess - a death factory - not a American tale of good guys and bad guys

Troll, I've seen nothing in the media about government and/or US-sponsored death squads in Iraq, which is what we sponsored in central america and what McNab implied was going on in Iraq. All the info I've seen on death squads is about sectarian ones which are not directly US or government sponsored.

As for good guys and bad guys, regardless of whatever small misbehaviors there may be, the US and coalition forces are still the good guys when compared to death squads, roadside bombers and hostage beheading kidnappers. In fact, there's really no comparison.

Dave

May I offer this quote from Stratfor.com, one of the most respected strategic research groups there is:

"Allegations that Shiite death squads are operating out of the Iraqi Interior Ministry and Iraqi police force are completely false, U.S. Maj. Gen. Joseph Peterson, who is in charge of training the Iraqi police, said Sept. 20. Shiite militiamen unaffiliated with the Interior Ministry, notably members of Shiite leader Muqtada al-Sadr's Mehdi Army, have been most frequently detained in relation to the killings"

Dave

#25,"eye Witless News", sarcasm, Dave. Once in a very, very, very blue moon will you see US atrocities aired on US television.

Dave writes:
Keep in mind that just as the study conclusions are an 'estimate', I'm making an estimate as well.

Bishop replies:
The Burhnam study's estimate is based on a statistical method have been tested and found to give reasonable estimates in hundreds of other circumstances. Your estimate is based on wild speculation and poor mathematical ability.

Let me help you with the math.

To estimate total over/under representation of high violence areas, you can add up the population for all governates in the "high violence" category. These total 6394508 and they constitute 24.5% of the country's population (omitting Dahuk and Muthanna). Then total the number of clusters in those governates: 13 out of 47, which equals 27.7%. So high violence areas are over-represented by 27.7-24.5 = 3.2%.

Your method, incorrect for coming up with a correction factor, was to take a governate like Anbar representing 4.9% of the population and that had three clusters, representing 6.4% of clusters, and calculate (6.4-4.9)/4.9=30.6% (though you saw fit to "round up" to 36%). You then correct the total body count by the average of such percentages. To illustrate how wrong this is, notice that if you had a high violence governate of only 270 people (0.001% of the total population) sampled with one cluster (2.1% of clusters), your method would tell us we've oversampled (2.1%-0.001%)/0.001% = 2099%. Average that in to your correction factor, and you'll have no deaths at all, which not even you seem to have the temerity to suggest.

Bishop wrote:
Even if you're adjustment percentages were correct, you applied them incorrectly. Your calculation was:
654,965 - 654,965 * (-.3 + -.2 + -.3) = 130,993

In fact you should multiply:
654,665 * .7 * .8 *.7 = 256,746

Dave replied:
A different way of looking at it, certainly. Not necessarily more valid unless you assume that the geographical underrepresentation and the rounding somehow apply more to military age men than to women. ...

Bishop replies:
Dave, these aren't two equally valid ways of oing the math. In fact, under your method if you had found one more error to correct for by -30%, you would have 654,965 - 654,965 * (-.3 + -.2 + -.3 + -.3) = - 65,497! But not even you have the temerity to suggest that there have been negative deaths in Iraq.

Dave writes:
So, if we accept your approach, the study's estimate is only off bya factor of 100%? Wow, that's great news.

John replies:
Ok, so you can't do math, and you can't read. I showed why a single one of your factors and an additional one of your calculations were wrong and its consequence. In fact, your other corrections are baseless as well. One of them isn't even a correction, its just the assertion that we shouldn't include "combatants" along with wild speculation about how many dead were combatants.

Dave writes:
Maybe I should point out again that even my 130,000 casualties estimate is double any of the figures reached by more verifiable methods. If anything my adjustments were super-conservative.

Bishop replies:
What you think are "more verifiable methods" have in fact been shown time and again to vastly underestimate the number of deaths in other conflicts and humanitarian tragedies. The Burnham et al study is not perfect, but its methodology is far sounder than anything else used in Iraq to date.

I haven't seen you call for a more thorough investigation by an independent group, therefore it appears that you are not actually interested in the truth about how many people have died in Iraq.

- John

Dave, you wrote: "Despite the fact that their conclusion states that 655,000 people died, their data suggests that what they really mean is that somewhere between 131,000 and 655,000 died, but they aren't promoting that lower number and it's not the one that's' going to get all the press." That is not correct-the article estimates 654,000 and that is in the middle of their 95% confidence interval (392,000 to 942,000). If they or the media wanted to exaggerate or "promote" a bigger number they could have used the 942,000 upper limit, but they didn't.

As someone who reads plenty of medical journals, I did not find this article to be poorly conceived or executed, and to me it looks like they made every effort to be as complete as possible. I do think the rural/urban point is a good one, and would ideally be better addressed(though I had heard 75% of Iraqi's were urban). I also think a morgue or "news report" death count is much more likely to be too low than this is to be too high...
Gary

I must not be able to read either because when I look at the report I don't see one word about any possible total other than the 655000 figure. No lower or higher figure is mentioned at all.

The defects of this survey are obvious. You can play all the math games you lke, but at the end of the day the basic points in this article seem solid. The subjet here is important enough that it deserves better than such a gross and implausible mess of guesswork and speculation.

And Bishop, your assertion that other methods underestimate seems ridiculous in the face of the fact that 80 percent of the deaths in the survey were verified by death certificates.

That being the case then it seems entirely reasonable to conclude that the Iraqi government figures represent a 20 percent undercount. That seems to be one of the few hard facts shown here. So that means a more accurate total might be around 75000 - a hell of a lot less than 655k.

Here's the quote:
"We estimate that between March 18, 2003, and June, 2006, an additional 654?965 (392?979-942?636) Iraqis have died above what would have been expected on the basis of the pre-invasion crude mortality rate as a consequence of the coalition invasion. Of these deaths, we estimate that 601?027 (426?369-793?663) were due to violence." The standard thing is to put the 95% confidence intervals in parentheses.

I don't think this study is a "gross and implausible mess" at all, to me it seems more plausible than not. I also think the media hype is deserved, or at least on par with what we got for vioxx and all the "excess deaths" that happen from medical errors.
G

your vioxx comparison says it all. When the statistics for vioxx deaths were finally studied it became clear that the death rate for vioxx users was statisticqlly indistinguishable from the death rate for non users.

I think you hit the nail on the head, Bialy. I never intended to do a real mathematical analysis. There's plenty wrong with the study without crunching numbers.

I may have to go back and do a bit of a rewrite to minimize the opportunities for misdirection.

Dave

The Burhnam study's estimate is based on a statistical method have been tested and found to give reasonable estimates in hundreds of other circumstances.

Cluster sampling is notorious both for its unreliability and for the ease with which the methodology can be manipulated to slant the results to a particular bias. I refer you to SocialResearchMethods.net which observes: "Though very economical cluster sampling is very susceptible to sampling bias."

Your estimate is based on wild speculation and poor mathematical ability.

Let me help you with the math.

My estimate was almost pure guesswork. I never intended to do a mathematical analysis and should have realized including any numbers at all was a mistake which would let someone like you try to distract from the issues with the methodology by trying to turn it into an argument about numbers.

Your method, incorrect for coming up with a correction factor, was to take a governate like Anbar representing 4.9% of the population and that had three clusters,

No, my method was to point out that there were problems with rounding off and then just pick an arbitrary number. I ddin't look at the variances for any but the most problematical provinces and didn't do any detailed calcuations at all. I wasn't trying to figure out the exact amount of error. To do so would be ridiculous.

Remember, they don't even have specific counts for the individual provinces in the report and there's a margin of error of +/-40%. Given these things fidding with 'exact' calculations is totally pointless. There's nothing there to work with and all your number crunching is completely meaningless. I made a terrible blunder in opening the door to it.

Even if you're adjustment percentages were correct, you applied them incorrectly. Your calculation was:
654,965 - 654,965 * (-.3 + -.2 + -.3) = 130,993

In fact you should multiply:
654,665 * .7 * .8 *.7 = 256,746

I agree, your method of calculation is more accurate, except that you're starting with meaningless numbers so the result is equally meaningless.

One of them isn't even a correction, its just the assertion that we shouldn't include "combatants" along with wild speculation about how many dead were combatants.

This is one of the main points you're missing. The media is representing the study as reporting CIVILIAN casualties, but the fact is that a good third of the deaths are clearly not civilian deaths, as admitted by the study authors. That's important.

I haven't seen you call for a more thorough investigation by an independent group, therefore it appears that you are not actually interested in the truth about how many people have died in Iraq.

The truth is obvious - too many have died. An accurate investigation wouldn't be a bad idea once we acknowledge that this study is NOT accurate and that releasing such a misleading report is counterproductive.

Dave

Dave & Co - if it were accurate that between 400 and 900 thousand people had died in Iraq (in excess of the 'norm - etc) since the invasion would it make any difference to the way you view the war - ?

just wondering

Reply to Lucky Frog #33
Lucky Frog - you have a good point about the fact that a large percentage of the dead were represented by death certificates. You would think that if all those death certificates were issued, then someone could tally them. However, the system for keeping track of them seems to have broken down, and in addition the Iraqi government will not be transparent about the information they have. None of the other estimates to date are based on a systematic, country wide sampling of death certificates - but that is essentially what the study we're debating has done!
The fact that these certificates exist does suggest a) an even better job of tallying the dead is still possible, and b) the Burnham study was not misled by people lying about their dead.

BTW the actual number for the certificates is that they were provided in 92% of the cases requested.

Dave & Co - if it were accurate that between 400 and 900 thousand people had died in Iraq (in excess of the 'norm - etc) since the invasion would it make any difference to the way you view the war - ?

60,000 - 130,000 - 400,000 - 900,000

Even the smallest of those is one hell of a lot of people.

On the one hand I'm troubled that ANY civilians have died in Iraq, though I realize that civilian casualties are impossible to avoid.

But on the other hand, the raw number of casualties has very little to do with whether or not the involvement in Iraq is a wise strategy in the war on terror.

Then there's also the problem that most of whatever deaths there are in Iraq are not caused by the coalition forces there. They predominantly kill hostile insurgents and terrorists. Most of the other deaths are faction on faction and secterian 'cleansing' type operations. There really ought to be more of the former so that there will be fewer of the latter.

But as I see it ALL of this is what you get with a situation such as exists in Iraq, but the big problem is that there's ZERO reason to think things there would get one bit better if we pulled out. In fact, they're likely to get much worse in the short term, followed by some sucky long term outcome for the people living there.

As you might surmise, I have lots of concerns but no answers. But what I do know is that inflating civilian casualties and running around wringing your hands doesn't actually solve any of these problems.

Dave

But Dave, if an invading power, who really has no place being there in the first place, obliterates a system that was keeping sectarian violence in check (even if that system was itself a feared dictatorship) without replacing it, thereby allowing sectarian violence to mushroom, then doesn't that occupying power bear some responsibility? And if the occupying power makes numerous blunders that further exacerbate or enable sectarian vioelence, then doesn't it bear even more responsibility? And finally, wouldn't you want to fire the management team that ineptly led you into this situation and, you know, come up with a new management team? And maybe consider some alternative strategies? Or would you just want to "stay the course"? I think its clear we have to find something other than both "cut and run" and "stay the course".

I challenge your implication that we have no responsibility for a large portion of these deaths. Regardless of who is killing whom, the death rates reported in the Burnham et al article are *excess* over what they were before we invaded. That implies that they would not be happening if we had not invaded. In other words, our invasion created a situation that allowed that violence to happen, even if American bullets and bombs are only directly responsible for 31% of the casualties.

"It is irresponsible and despicable for academics and scientists to use their trusted position and their credentials to spread deception for partisan purposes."

Wow. How do you know this is true?

But Dave, if an invading power, who really has no place being there in the first place,

So, you're a strict non interventionist? Brave stance. No foreign financial aid, no humanitarian aid in Darfur or the Congo or Uganda or anywhere else. No military deployments outside our borders? Takes some guts to make a stand like that on principle. Or is it just the involvement in Iraq you object to?

obliterates a system that was keeping sectarian violence in check (even if that system was itself a feared dictatorship) without replacing it,

Last I checked there was a government in Iraq which has replaced Saddam's regime. In fact it's one the people there chose for themselves.

thereby allowing sectarian violence to mushroom,

Let's say we hadn't invaded and had managed to topple saddam through internal overthrow or assassination or some other means. Do you think there would have been no sectarian conflict then?

then doesn't that occupying power bear some responsibility?

Absolutely, but the question is where does that responsibility end. Clearly you think we ought to be caretakers for Iraq indefinitely and probably increase our military and financial presence, but I think there has to be a point at which we decide we've done all we can and they have to solve their own problems. The internal conflicts are NOT of our making, we just opened the door to the inevitable.

And if the occupying power makes numerous blunders that further exacerbate or enable sectarian vioelence, then doesn't it bear even more responsibility?

I don't see how the factions in Iraq were encouraged by anything we did. We certainly attracted Al Qaeda, but we're no longer the focus of most of the hostility. We're mainly targeted now because we get in the way and try to stop some of the violence.

And finally, wouldn't you want to fire the management team that ineptly led you into this situation and, you know, come up with a new management team? And maybe consider some alternative strategies? Or would you just want to "stay the course"? I think its clear we have to find something other than both "cut and run" and "stay the course".

I'm all for new strategies. And we do have an electoral process to get rid of our management team every few years. As for what those strategies are and whether they'll actually work, that's a whole different debate, as is the question of whether experimentation is really the best approach to Iraq at this point when we have no reason to think that a different approach won't make things worse.

I challenge your implication that we have no responsibility for a large portion of these deaths.

Didn't I say no direct responsibility? We're not the ones planting the IEDs, sending the suicide bombers, running the death squads and otherwise being indescriminately violent. For the most part our violence is reasonably well targeted.

Regardless of who is killing whom, the death rates reported in the Burnham et al article are *excess* over what they were before we invaded. That implies that they would not be happening if we had not invaded. In other words, our invasion created a situation that allowed that violence to happen,

You make the assumption that civil war would never have broken out there without our intervention. Even the most repressive dictatorship does eventually fall, and we didn't create the ethnic and sectarian divisions in Iraq.

even if American bullets and bombs are only directly responsible for 31% of the casualties.

I find it interesting how that number almost exactly corresponds to the additional deaths of military age males.

Dave

Wow. How do you know this is true?

How could you possibly not agree with it, Bliffle?

Dave

Dave,
I do have to agree with you that many of the newspapers have misreported the excess deaths as being all civilians, and this is really a major error. This seemed to start with the New York Times, "A team of American and Iraqi public health researchers has estimated that 600,000 civilians have died". This was then copied by many other prominent papers.

This is not what was stated by the Lancet article, whose authors stated clearly that they were sampling all deaths and could not distinguish combatants from others. Nor was it misstated by the Johns Hopkins press release regarding the article.

I know they didn't make that represenation in the report, Bishop. But this reaction from the media is EXACTLY what happened with their 2004 report. In this case as was the case back then, the media immediately latched onto a high number, treated it as an absolute fact rather than as an estimate and presented it as a count of civilian casualties.

It's certainly not the fault of the study authors, but they ought to have been sensitive to it given that the exact same thing happened last time. They should have made an exra effort to qualify and explain the study and they certainly shouldn't have released it right before the election. They opened themselves up for accusations of bias and deservedly so.

Dave

"So, you're a strict non interventionist? Brave stance. No foreign financial aid, no humanitarian aid in Darfur or the Congo or Uganda or anywhere else. No military deployments outside our borders? Takes some guts to make a stand like that on principle. Or is it just the involvement in Iraq you object to?"
- Dave Nalle

Another "brave stance" is typing bellicose pro-interventionast slogans from the safety of a fortified compound 10,000 miles from the invasion, while someone else risks life-and-limb in the "intervention."

Interesting that this week, the new British Army chief, the Chief of the General Staff, Sir Richard Dannatt, has called for the pullout of British soldiers as soon as possible, believing that the continued presence of foreign troops is now causing more problems than solutions.

Dannatt rightly says that we kicked down the door in Iraq, rather than being invited in, and while the removal of a dictator is a worthwhile achievement, we are now wearing out our welcome.

Tony Blair has backed him and one would hope that any such British pullout would involve also the removal of Australian troops (although I won't be holding my breath under the current Aussie government) and ultimately those of the US.

Once those three decide to go, there's not much point in anyone else being there.

This is part of the edited text of Dannatt's interview as reported on CNN:

He said Operation Sinbad - a security and reconstruction effort in Basra - "is trying to make Basra better and a lot of British soldiers are doing a really good job. In that regard, their presence is helping. But there are other parts where our mere presence does exacerbate and violence results."

However, he said, "that is not a reason for us to leave."

"I am on record publicly saying we're standing shoulder to shoulder with the Americans. I am on the record from a speech three weeks ago saying that I'm planning force packages in Iraq through 2007 into 2008. I'm a soldier - we don't do surrender, we don't pull down white flags. We will remain in southern Iraq until the job is done. We're going to see this through."

However, Dannatt, who took over as the Army's chief of general staff in August, had been quoted in the newspaper as saying that the U.S.-led coalition's plan to establish a democracy in Iraq that would be an "exemplar for the region" was unlikely to happen.

"That was the hope. Whether that was a sensible or naive hope, history will judge," he said. "I don't think we are going to do that. I think we should aim for a lower ambition."

But with the country edging nearer to civil war - if not already immersed in it - Dannatt said the strategy for implementing an Iraqi democracy was ill-prepared.

"I think history will show that the planning for what happened after the initial, successful war-fighting phase was poor, probably based more on optimism than sound planning," he said.

Now, he said, Britain has essentially overstayed its welcome in Muslim Iraq.

"The military campaign we fought in 2003 effectively kicked the door in," he said, noting that was a far cry from being invited into the country. "Whatever consent we may have had in the first place may have turned to tolerance and has largely turned to intolerance."

Perhaps he's right.

And you're absolutely right Dave. I too don't think civilian casualties caused by us rather than the insurgency have been too bad (although as you say, any is no good) ... but the PR campaign in my view started going awry the moment a young and naive US soldier with no idea of the local sensibilities planted the US flag (against orders) on the statue of Saddam outside the 14th Ramadan mosque opposite the Palestine Hotel in Baghdad.

Not really his fault, as young men are wont to do such things, but the law needed to be laid down very clearly from day one from higher up. That has been a big part of the problem.

And I'm going to end on this.

Dave Nalle wrote: "It's higher than estimates from nations which are not beset by terrorism and internal conflict ... Ireland."

A slip of the keyboard old boy???

If the war is wrong, even "only" 10,000 excess civilian deaths shouldn't make it any less wrong.

If the war is right, even a million excess civilian deaths shouldn't make it any less right.

In either case I think we have a responsibility to face the war and know the costs, in financial, political, and especially human terms. I think these guys are at least trying to measure the human costs (unlike our govt). I feel we have a right and moral obligation to try to know these costs for both current and future reference.

Thanks for the lively and civil discussion Dave!
Gary

I agree that the timing of this publication so close to an election is questionable...it makes it difficult to focus on the 'science' of the study

I smell the partisan 'conspiracy' that Dave claims exists but have no insider info to prove it

Do you really believe that: "... academics and scientists ... use(d) their trusted position and their credentials to spread deception for partisan purposes."

Maybe you've just become inured to authorities lying systematically in your experiences with government officials.

The Lancet controlled when the results of a study were released, the authors have no control over it. The authors submitted the article to the journal several months ago, which was as soon as they possibly could given that they conducted the research in June/July. They had trouble actually getting the data out of Iraq, and given the amount of time it takes to analyze and write, I can't see them submitting any earlier than late August. It then typically takes several weeks to obtain peer reviews (this article was peer reviewed by four scientists). It then takes several more weeks (sometimes months, in fact) to format it for the journal.

The journal may have timed the release relative to the elections, but it is also very plausible that this was simply the soonest they could get it out. If they had published it a month earlier, which would be lighteneing fast turnaround for a journal, you still would have claimed it was too close to the election.

Bishop, they presumably had past experience and knew how long Lancet takes to get something from submission to publication, so they were at the very least aware of when it would be released.

Dave

and the issue remains: if what Alex claims in #2 (*there is a pseudo-progressive core that foolishly believes that if you can demonstrate that a war kills some magically big number of innocents or civilians, then everyone will equally magically agree that war is bad and end all hostilities.*) is true and correctly describes the researchers' belief/motive it's hard to see how it wouldn't affect the outcome

hi

question is who is the bigger killer : Bush & Co or ____ . . . ?

and for what, . . . the Party ?

tch, tch, tch !

best

Hi! I'm new to this forum, and pleasantly surprised by the objectivity and apparent good will of the author. This is definitely a cut above the average blog I've visited recently.

Dave Nalle (#12) wrote: As someone earlier suggested, why not just count the dead? This time out they went to great pains to make sure that almost all the people they questioned provided death certificates for the people they claimed had died - and about 80% had them. If those death certificates exist, why not go to the issuing body and get a total and then add 20% to it as a 'fudge factor'? That would be a hell of a lot more accurate than this effort.

The problem is, the "issuing body" is probably the local coroner, or maybe just a medical technician who happens to be the only thing resembling a doctor in the immediate vicinity when the death occurs, and there is no centralized repository for reporting deaths in Iraq right now. Remember, the Coalition Provisional Authority established its policy of not counting Iraqi deaths way back in March of 2003. No doubt the new government would like to maintain vital statistics at a central location, but keeping the members of parliament alive (and out of the hands of kidnappers) is a more pressing concern, from what I hear.

Death certificates are generally produced in at least triplicate -- one for the family of the deceased, one for the coroner / doctor, and one for the bureau of vital statistics. In the case of Iraq, there's a good chance that the two "official" copies have already been lost or destroyed in the havoc and confusion, so that the one copy remaining in the hands of family members may be the only written record of the deceased that still exists. The family would tend to hold onto it for quite a while, as a memento, in all likelihood.

Oh, I read the Lancet report, and one detail seems to be garbled here. At 80% of the interviews where a death was reported, the interviewers actually asked to see the death certificate. And in 92% of those cases the death certificate was actually produced. So the researchers only saw certificates for some 73% of the reported deaths. Apparently, in the other 20% of the interviews, the researchers either forgot to ask the question, or deemed that particular question to be impolitic.

and the issue remains: if what Alex claims in #2 (*there is a pseudo-progressive core that foolishly believes that if you can demonstrate that a war kills some magically big number of innocents or civilians, then everyone will equally magically agree that war is bad and end all hostilities.*) is true and correctly describes the researchers' belief/motive it's hard to see how it wouldn't affect the outcome

I find it hard to believe you can't see how it coudl affect the outcome. Cluster studies are made for manipulation. The people asking the questions pick and choose which doors to knock on. The designers of the study pick where the clusters go based mainly on convenience rather than balance, or they can pick the locations to get a particular slant to the sample. Then you get some very rough numbers which can be massaged pretty easily and interpreted in any of a number of ways. The methodology of this sort of study is just not reliable at all. There are too many arbitrary factors and choices involved.

Dave

"It is irresponsible and despicable for academics and scientists to use their trusted position and their credentials..."

Wow! You're actually accusing the "academics and scientists" at Johns Hopkins of deliberate deception! A calculated and conscious plan to deceive people. Amazing. How do you KNOW they did that? That they set about to contrive this elaborate deception? Are you familiar with the participants? Do you recognize their names? Have you worked with them in the past? Have you read their other works?

"... to spread deception for partisan purposes."

Partisan? Do you know the party they belong to?

How did you learn all this stuff?

Wow! You're actually accusing the "academics and scientists" at Johns Hopkins of deliberate deception! A calculated and conscious plan to deceive people. Amazing. How do you KNOW they did that?

Bliffle, did you miss the heading on this article that reads 'opinion'?

That they set about to contrive this elaborate deception? Are you familiar with the participants? Do you recognize their names? Have you worked with them in the past? Have you read their other works?

Actually, I've read their 2004 report which used the same methodology, so I'm familiar with their work.

"... to spread deception for partisan purposes."

Partisan? Do you know the party they belong to?

Partisan doesn't just mean loyalty to a political party, it can be loyalty to an ideology or any group. In this case they appear to be partial in their generic opposition to war on any and all grounds.

The neutrality of the report and the research is clearly in question, and which side of the issue they come down on is evident when they write:

We continue to believe that an independent international body to monitor compliance with the Geneva Conventions and other humanitarian standards in conflict is urgently needed. With reliable data, those voices that speak out for civilians trapped in conflict might be able to lessen the tragic human cost of future wars.

... this sort of study is just not reliable at all. There are too many arbitrary factors and choices involved.
Speaking as a professional statistician (degree in mathematics from CalTech, and over 20 years experience as a life insurance actuary) I have to disagree. While bias is alway possible in any statistical study, it's not necessarily the result of "arbitrary choices." And "this sort of study" is widely used to measure exposures to toxic chemicals, and the incidence of disease, and even to plan multi-million dollar advertising campaigns. If sampling techniques are so unreliable, why do so many professional decision makers depend on them every day of the week?

The authors of this study aimed to sample 50 clusters, each consisting of 40 households. They went to elaborate lengths to make the sample of 50 clusters reasonably representative of the population of Iraq (based on the best available census data), and they were careful to randomize the center of each cluster, to the extent that was possible. They ended up with 47 clusters and 1,849 households ... in other words, they had to exclude data from 3 of the clusters they had targeted, and they didn't complete the survey at some 31 households sprinkled among the 47 clusters where they did get good data.

Now a random survey based on 1,849 independent data points and including 300 "excess" deaths by violent means would be highly reliable, yielding a mortality estimate that would be good (at the 95% level of confidence) to within some 6% or 7%, plus or minus. In fact, the authors recognized that their sample only consisted of 47 data points, and this fact is reflected in the rather large 95% confidence interval they reported (plus or minus 30%, roughly).

In short, Dave, you're correct to criticize the "cluster" method of gathering data as being inherently unreliable. The authors of this study recognized that, and accounted for it in their statistical analysis. That's why the "error bars" are so big. To go farther, and imply that they deliberately skewed their sample -- especially when they have described in detail the techniques they used to assure randomness -- is to express a personal prejudice not supported by the facts.

Your description of the sampling method is completely at odds with the protocol described in the paper. Are you accusing them of fraud?

Clusters were not picked in major urban areas of each governate, but in districts picked randomly, weighted by population. Within each district, streets were picked randomly. That last step is unclear and could bias things, either way, but it is simply false to say they avoided villages.

Where do you get the claim that the three Anbar clusters were in Ramadi and Fallujah?

In short, Dave, you're correct to criticize the "cluster" method of gathering data as being inherently unreliable. The authors of this study recognized that, and accounted for it in their statistical analysis. That's why the "error bars" are so big.

Which raises the basic issue of why we should even consider a study with a margin of error so incredibly huge. If this were a political poll or a census study the entire data set would be thrown out as unreliable.

To go farther, and imply that they deliberately skewed their sample -- especially when they have described in detail the techniques they used to assure randomness -- is to express a personal prejudice not supported by the facts.

The fact that the even published a report based on methods where they had to compensate for a sample which is so incredibly unreliable suggests at a minimum that they were biased towards giving the report an endorsement of legitimacy which it doesn't deserve.

The truth is that because of the lengths they went to in order to compensate for their sampling methods the results are more the result of mathematical fiddling than of actual data.

Dave

Your description of the sampling method is completely at odds with the protocol described in the paper.

Odd, since it's based directly on what they say in the paper.

Are you accusing them of fraud?

I'm accusing them of being overly enthusiastic and arrogant. Whether it's intentional fraud or not is a matter of perspective and intent.

Clusters were not picked in major urban areas of each governate, but in districts picked randomly, weighted by population.

In comments made outside the report they say specifically that they went from house to house and from apartment to apartment and that when someone wasn't home or didn't answer the door they went down the hall to the next apartment.

Within each district, streets were picked randomly. That last step is unclear and could bias things, either way, but it is simply false to say they avoided villages.

Do you have access to more information about the study than what's actually in the published text. If so, please share your source. They say that they followed the same methodology as the first study, in which they did overwhelmingly draw their data from urban areas. Nothing in the study says that they went to small villages, in fact a number of statements imply that for purposes of access and speed they did not. For example they say that they were "confining the survey to a cluster of houses close to one another," which suggests tightly packed areas of settlement - urban areas.

Where do you get the claim that the three Anbar clusters were in Ramadi and Fallujah?

As I said before, they followed the methodology of the last study, and it's been established that the first study drew ALL of its data for Anbar and a province they paired with it from Fallujah.

Dave

Let's add some fuel to the fire:

Here's a video of one of the authors of the Lancet report publicly admitting that the timing of the release was political.

Here's a video of the editor of the Lancet making inflammatory anti-US and anti-war statements.

And here's a relevant bit expressing the skepticism of some experts and also confirming the urban nature of the data sources from a NYT article from last week:

Robert Blendon, director of the Harvard Program on Public Opinion and Health and Social Policy, said interviewing urban dwellers chosen at random was "the best of what you can expect in a war zone."

But he said the number of deaths in the families interviewed -- 547 in the post-invasion period versus 82 in a similar period before the invasion -- was too few to extrapolate up to more than 600,000 deaths across the country.

Donald Berry, chairman of biostatistics at M. D. Anderson Cancer Center in Houston, was even more troubled by the study, which he said had "a tone of accuracy that's just inappropriate."

...why ... even consider a study with a margin of error so incredibly huge. If this were a political poll or a census study the entire data set would be thrown out as unreliable.
Speaking as a statistician again, this simply is not so.

Every statistical study is up against Gauss' Central Limit Theorem. In essence, this says that to cut ths size of the probable error in half, you have to collect four times as much data. This is a mathematical fact embedded in every random process. It's kind of like the law of gravity -- there's just no way to get around it.

The ultimate question that decides the size of a statistical sample comes down to economics. How much money are you willing to spend collecting the data? Do you want to lay out 4 times as much money to cut the probable error in half? Or have you spent enough money already?

Typical public opinion polls have "error bars" in the range of 4% to 5%, corresponding to a sample size in the range of 500 to 600. Since such data are easily collected (say 5,000 phone calls all together, assuming that some 90% of the calls are not completed, or the guy who picks up the phone doesn't want to participate, etc.), we've grown accustomed to this size of error bars. Pollsters could reduce the reported error to the 2% range, but then they'd have to make 20,00o phone calls to complete a survey. They figure it's just not worth it.

Conditions in Iraq are not nearly so favorable. To obtain comparable precision (say 5%, plus or minus), the researchers would have to collect 36 times as much data (30% / 5% = 6; 6 * 6 = 36). This would mean 72,000 face to face interviews. Since the data are only meaningful if they're collected within a relatively short period of time, in practice they'd have to employ 36 times as many people.

The actual study employed 10 people (2 teams of four interviewers, with one field supervisor for each team). To obtain the sort of precision you'd like to see, they'd have to put 360 people in the field for two months. Apparently the authors of this study simply didn't have that much money available. It's also likely that they couldn't find 360 Iraqi doctors who would be willing to participate, at any rate of pay.

In summary, one ought not confuse accuracy with precision. A well-designed measurement of any description is as accurate as it can be, but it is always imprecise. That's what confidence intervals are all about. They do not say the measurement is wrong -- they only indicate the precision with which that measurement has been made.

Dave: Your characterization of the study as unreliable is incorrect. You are referring to accuracy, I think. And, yes, the broad range of figures indeed seems large. But, keep in mind this estimate represents a 95% confidence interval wherein the true number of deaths is most likely near the 655000 figure. Assuming a normal distribution for the sampling statistic of interest, the lowest and highest estimates are less likely to be correct than is the estimate in the middle of the distribution.

The authors address the issue of reliability, in part, when they compare results from this study with the previous study and find consistent trends.

First, David Bryant's comments above are correct -- the issue is precision (rather than accuracy as I mentioned. My bad).

Second, as to the timing of the release of the study being political--so what? Releasing the data weeks before the election allows for an informed electorate. Contrast this behavior with that of the James Baker group who is not going to release their recommendations on Iraq until after the election. Which decision is more politically useful to the electorate?

Guys, I can say I made a study by interviewing one Iraqi that suggests that between 0 and 27 million Iraqis have died in the last three years and it would be 'accurate', but releasing it as if it's meaningful would be incredibly irresponsible. The same is true with this study which has a margin of error of +/- 40%.

And contrary to what you guys are suggesting, if you saw a poll in a newspaper which said 'margin of error 40%' in small print at the end of it you'd consider it meaningless. They expect us NOT to make that same judgement of this report, and the media is obliging them. They're reporting the number without reporting how incredibly imprecise that figure is.

The way I figure it, a 95% confidence that a number is accurate to within 40% either way is useless. They release this number and present it as accurate, when the truth is that it's more likely to be any number BUT that number based on their total lack of precision.

Dave

But, keep in mind this estimate represents a 95% confidence interval wherein the true number of deaths is most likely near the 655000 figure.

If they have a 95% confidence in that result then they're not being objective, because as has been mentioned numerous times before on this thread, if the deaths were that high there would be overwhelming physical evidence which could not be ignored. That evidence does not exist despite several governments and multiple NGOs looking for it.

The UN has been tracking violent deaths in Iraq almost since the end of the war and although their numbers are fairly high, they still don't total more than 120,000. What this report asks us to believe is that there are 530,000 unreported deaths, all of whom were buried privately and not one of which was reported in a newspaper.

Let's apply some common sense here. Do you think that no one would notice if 7 million extra people died in the US in a 3 year period? That's what you're asking us to believe. We're talking almost 1 in 50 people here. It's such a huge proportion that it wouldn't take a cluster study to find it.

Dave

Second, as to the timing of the release of the study being political--so what? Releasing the data weeks before the election allows for an informed electorate.

Or a misinformed electorate.

Contrast this behavior with that of the James Baker group who is not going to release their recommendations on Iraq until after the election. Which decision is more politically useful to the electorate?

I'd argue that the release of accurate information in a timely manner is desirable. Rushing out questionable and controversial data to impact the election is not.

Dave

Guys, I can say I made a study by interviewing one Iraqi that suggests that between 0 and 27 million Iraqis have died in the last three years and it would be 'accurate', but releasing it as if it's meaningful would be incredibly irresponsible. The same is true with this study which has a margin of error of +/- 40%
Just three things.

1, You wouldn't have to interview any Iraquis at all to make your hypothetical estimate.

2. The study's margin of error is not +/- 40%. It's +/- 30%.

3. Misrepresentations are not arguments. They are simply misrepresentations.

Here's the methodology, from page two, left column:

At the second stage of sampling, the Governorate's constituent administrative units were listed by population or estimated population, and location(s) were selected randomly proportionate to population size. The third stage consisted of random selection of a main street within the administrative unit from a list of all main streets. A residential street was then randomly selected from a list of residential streets crossing the main street.

Douglas. Do you see where it says 'main street' and 'residential street'? Camps in the desert don't have streets. One-road villages don't have residential and non-residential areas. Farms don't have streets at all.

As for the +/- factor. By my calculations 942,000 is 43% more than 655,000 and 392,000 is 41% less than 655,000. But feel free to calculate it yourself. I'll be amazed if you come up with different answers. Perhaps if you borrow a calculator from someone on the study team...

Dave

"the rofecoxib group had an increased risk of confirmed thrombotic events (relative risk, 1.92; 95 percent confidence interval, 1.19 to 3.11)" Dave, based on the study this quote was taken from, Vioxx was withdrawn from the market by Merck.

As you can see, a very wide confidence interval is present, and the lower and upper limits are almost 40% from the mean. So clearly all studies with wide confidence intervals shouldn't be considered useless! Please tell me if you'd like I could provide dozens of similar examples.
Gary

Where do you get the claim that 37% of Iraqis don't even live in villages?

It's not clear what the part about streets means, but the previous study said that they sampled in villages.

" So clearly all studies with wide confidence intervals shouldn't be considered useless! "

Sounds reasonable to me. That much of an uncertainty factor renders the figures unusable foe anything except stirring up trouble, which is clearly the intent here. did you take a look at those video quotes? These guys are fanatics on a mission, not objective scientists.

Two GOP senators demand new strategy in Iraq
By The Associated Press - 10/16/06

WASHINGTON (AP) -- Two leading Republican senators called Sunday for a new strategy in Iraq, saying the situation in getting worse and leaving the United States with few options.

Sens. Chuck Hagel of Nebraska and John Warner of Virginia are part of the growing list of Republicans who are speaking out against President Bush's current plan for Iraq as U.S. casualties rise.

"The American people are not going to continue to support, sustain a policy that puts American troops in the middle of a civil war," Hagel said on CNN's "Late Edition."

Hagel said he agreed with Warner, chairman of the Senate Armed Services Committee, who said after a recent visit to Iraq that Iraq was "drifting sideways." Warner has urged consideration of a change of course if the Iraq government fails to restore order over the next two months or three months.

Warner said Sunday he stands by that assessment, and even in the week since his trip to Iraq, there has been an "exponential increase in the killings and the savagery that's going on over there."

"You can see some movement forward, but a lot of movement back," Warner said on Face the Nation on CBS. "We have to rethink all the options, except any option which says we precipitously pull out, which would let that country fall into a certain civil war."

Bush told reporters last week that he invites a change in strategy if the plan isn't working. But he also said the U.S. will not leave until the job is done.

Hagel said it is time to change course, but "our options are limited."

"We need to find a new strategy, a way out of Iraq, because the entire Middle East is more combustible than it's been probably since 1948, and more dangerous," Hagel said. "And we're in the middle of it."

Dave - #70 assumes that the country is open for inspection by the improperly so called central authority - NGOs - the UN - and media types...it's not

if it were then a direct count would be possible

half a million deaths unreported beyond the local level does not seem so unlikely...anarchy is hell on bureaucrats

---

another way to look at the confidence interval is as a statement about the study rather than about the 'real' number...if this study were repeated numerous time then the result would fall within the interval 95 out of 100 times in the long run

---

I heard Burhnam on cspan the other day where he said that this study was carried out on the cheap for $50k...I would like to see $250k invested for the next one

Dave, you've been attacked for your assertion that the authors oversampled urban regions. There's an easy way to check whether you're right: look at a relatively non-controversial figure that is reported by the authors and would differ between urban and rural areas. Then compare it with some well-grounded stats from a presumably independent body.

One such figure is the infant mortality rate. Infant mortality in urban areas is lower than in rural areas in countries like Iraq, just as most measures of non-violent death would be. Burnham et al, over the entire time period of their survey, report 40 infant deaths out of 1474 live births, or 27 per 1000 live births. This is half the number (50/1000) estimated by the US government(which some would accuse of bias towards underreporting infant mortality) and only a quarter of the UN Population Division's number of 100/1000. Incidentally, the UN's mortality rates, whether infant, child, or total, have remained essentially unchanged since the mid-90's.

If you look at the birth rate, the authors' numbers also indicate a total Iraqi birth rate far lower than the actual, well-established number. This, too, argues in favor of your assertion of urban oversampling.

Unfortunately, it is unlikely that anyone will ever find out which neighborhoods the authors sampled, and thus be able to verify their representative nature. This is because the authors 1) did not collect "unique identifiers" such as addresses or names out of concern for participants' safety, and 2) did not conduct the survey themselves. They assigned this task to survey teams, natives on the ground in Iraq who picked neighborhoods more-or-less in accordance with the methodology on which Les Roberts instructed them. The teams apparently had quite a bit of discretion, and the authors (to their credit) admit that, "the potential exists for interviewers to be drawn to especially affected houses through conscious or unconscious processes." That these teams were themselves biased against the coalition, and/or cultivated additional biases based upon Les Roberts' own outspoken views on the war (and, by extension, expectations of the survey's outcome) is entirely possible.

Not only will interested parties never know which houses were surveyed, they will never be able to verify any of the data whatsoever, again because of the author's decision not to collect any unique identifiers.

This brings us to the issue of death certificates, raised above in the comments. Death certificates were listed as "present" in 501 cases out of 629 total deaths--just under 80%. The authors make the misleading statement that 92% of deaths were verified by certificate...because the survey teams, at their discretion, did not ask for death certificates in at least 80 cases. What exactly the "verification" entailed is not at all clear from the paper. I get the impression that death certificates may have been a very touchy subject for the surveyed households; this may have been the reason for neglecting to ask for them in some cases. The methods are somewhat hazy on how the survey teams assessed the certificates they did ask for: "At the conclusion of household interviews where deaths were reported, surveyors requested to see a copy of ANY death certificate and its PRESENCE was recorded" (my emphasis). Were all "present" certificates actually viewed? Were names matched? Was authenticity assessed? These questions are left unanswered. The methods then go on to say that, "Where differences between the household account and the cause mentioned on the certificate existed, further discussions were sometimes needed to establish the primary cause of death." At least a subset of the certificates were apparently viewed, then, but it is not clear what fraction of the total this represents.

As for the indignant commenter who was so outraged by supposedly detrimental reference to Johns Hopkins or its researchers: the party affiliation of Les Roberts is available for anybody with an internet connection. Les Roberts ran a campaign in New York as a Democrat candidate for Congress in 2006. His platform had one plank: anti-Iraq war. He raised $160,000 before dropping out of the race in the spring. Who contributed to his campaign? About $90K was raised from individual contributions alone, some of them from his colleagues. These include his co-author Gil Burnham and people who could potentially serve as his "reviewers." So the political party of Roberts is well-established, and the circumstantial evidence leans towards placing Burnham in that category as well. (Professors at Johns Hopkins tend to vote Democrat, in any case. Or, to rephrase, I know and know of a lot of people at Hopkins, and I haven't heard of a single Republican above the rank of grad student, although there might be a few.) In a sense, though, party affiliation isn't so important for bias in this paper as political passion on the issue of Iraq. Both Roberts and Burnham have demonstrated it in multiple venues. Here's an interesting quote from Burnham, interviewed by The World Today: "we wouldn't go to the effort of doing something like this if we didn't feel that here was a situation that was egregious and, you know, there really needs to be some attention to what we can do to better protect the civilians." This is as plain a statement as one can make: Roberts and Burnham conducted the 2004 and 2006 surveys because they already presumed what the outcome would be and wanted to make a political point and, ideally, effect changes in foreign policy, perhaps by helping, through publication timing, to influence the US elections. But why take their word for it? Look in the paper itself. The authors link the coalition forces to execution-style killings and assassinations of innocent civilians in the following two curiously-juxtaposed sentences: "The circumstances of a number of deaths from gunshots suggest assassinations or executions. Coalition forces have been reported as targeting all men of military age." How do the authors reference this astounding suggestion, when, to an approximation, all reported execution-style deaths in Iraq have been ascribed to sectarian and political infighting? They endnote two newspaper articles, both describing a single case where four US soldiers were accused of killing three Iraqi insurgents who were in their custody and may or may not have escaped and attempted to attack the soldiers prior to the killings. According to the soldiers' lawyers, the soldiers were told to kill all military-aged men in a house being used by al-Qaeda and to secure the surrounding houses. The Army, of course, disputes this purported order. In any case, the authors show their propensity, indeed, their desire, to lay blame for all Iraqi deaths at the feet of all coalition forces by turning the case of a) a single order which may or may not have been given; b) concerning a single al-Qaeda-occupied house on an island in a lake in central Iraq; c) involving the deaths of three insurgents who may or may not have been escaping from/attacking the soldiers who shot them; and d)
a military trial that, far from sweeping this under the rug, takes the charge of murder so seriously that the soldiers involved may face the death penalty if convicted....into a sweeping indictment of all coalition forces, US and otherwise, in the execution and assassination of innocent Iraqi civilians. If that's not oozing, pustulent bias, then I'm a Baltimore Raven.

Like you, Dave, I don't wish to accuse Burnham and Roberts of intentionally falsifying data. It is quite possible to produce utter trash like this paper simply by the action of biased "unconscious processes." As a point of fact, although I consider Les Roberts to be a special case, I have quite a bit of respect for Gil Burnham and much of his work. I do not doubt that he is a very conscientious and well-intentioned individual, and I applaud his efforts on behalf of public health. We all have a certain capacity for self-delusion, and of being led astray by people we respect, so while I criticize this bit of his work, I respect Burnham himself.

It is unfortunate, however, that The Lancet chose to disregard their own conflict of interest disclosure policy in publishing this paper. The author's ties to the Democrat party and their close affiliations with the anti-war movement constitute a perfect match for "conflict" as described in The Lancet's author's guide: "A conflict of interest exists when an author or the author's institution has financial or personal relationships with other people or organisations that inappropriately influence (bias) his or her actions. Financial relationships are easily identifiable, but conflicts can also occur because of personal relationships, academic competition, or intellectual passion. A conflict can be actual or potential, and full disclosure to The Editor is the safest course. Failure to disclose conflicts may lead to publication of a Department of Error. All submissions to The Lancet must include disclosure of all relationships that could be viewed as presenting a potential conflict of interest."
Of course, since the Editor of the journal is himself farther to the left than even Les Roberts, this inconvenient bit of advice was ignored when the paper was fast-tracked for publication ahead of the 2006 US Congressional elections and when the Editor himself wrote an inflammatory commentary on the piece.

My colleagues in science sometimes wonder why the Bush administration doesn't seem to like them. Call this one 'Exhibit A.'

I'll respond at more length later, but to back up what Ernest says in the last post about Roberts, part of the problem is that The Lancet is overseen by Richard Horton who is about as rabidly partisan as they come. As you suggest, full disclosure to the editor and editorial oversight become meaningless when the editor is the one with a history of bias.

Dave

So now we all know why you question the numbers in the first place Dave. Your own paritsan reasons are not disimilar to what you claim the study was about. You accuse everyone else on here that tries to refute your fuzzy math with their own fuzzy math of getting OFF TOPIC and the topic to you is partisan politics driving outrageous if not false body counts. Is that really the point, or are the body counts the point? You claim that any innocent loss of life is a tragedy yet you spend your whole article and time refuting these NUMBERS. Is that all they are to you? Numbers piped in over the wire into your modest suburban compound somewhere outside of Austin?
Why don't you write your next article on how the analysis of the number of Jews killed during WWII was blown out of porportion? Come on let's see that one Dave. Are you scared that Ruvy wouldn't compliment you anymore?

Mr. Bishop, you feel into Dave's trap, once you trap him, he dismisses the facts, once you don't have the facts, he asks for them. He goes to great pains NOT to mention Bush, yet he consistently writes articles that back him up. He claims the LEFT/ists are all about manipulating the media, yet he prowls around the rags waiting for things he can refute claiming all the while he's a libertarian with no agenda.

I say put Bush in power in Iraq Dave.
Put that mother fucker in the palace and make him and all his neocons butt-buddies clean this mess up one home at a time, with no child left behind. Let the Iraqi's have his blood and then maybe the rest of us good Americans can live in peace.

I always thought the practise of body counting to gauge the success or otherwise of an operation to be an extremely flawed piece of methodology, particularly in Vietnam ... and it was a favourite trick of the US general staff there but fooled no one.

Civilian and military casualties just can't be used to argue the toss either way here: In my book, you're either winning, or you're getting spanked.

Right now, we're all getting royally spanked, no matter how much Vietnam-style spin is put on it.

To paraphrase, there's not much point winning the war if you can't win the peace. Time to get out of the bastard of a place and let the locals sort it out amongst themselves ... while we work on the development of the hydrogen-powered engine.

Dittos 82 and 83!!

Another phoney stat is when the war-wimps, in an effort to justify the cluster fuck over there, compare traffic fatalities to being killed in combat.

ErnestD, from what I've read, infant mortality is commonly significantly underestimated in surveys such as this. Also my calculations for birth rate gave me about 33 births/1000 people per year, which is about the birth rate I've seen as a current Iraqi national average(32.5).

Also, I think the word "any" as used in the sentence you mentioned can only refer to the pertinent death certificate, otherwise the meaning of the sentence becomes absurd(and would have intentionally been written absurdly). Also the word "presence" would be commonly used to report whether data(in this case the relevant certificate) were "present" or "absent".

"Where differences between the household account and the cause mentioned on the certificate existed, further discussions were sometimes needed to establish the primary cause of death." -- To me this implies that they at least had it in their study design to examine each certificate (otherwise the statement would indeed be dishonestly misleading).

Lastly, as opposed to some of us on the left, I don't feel that a well-designed study must be thrown out just because of bias. There is inherent bias in many, many medical and public health trials(e.g. drug company funded trials, other trials where you REALLY want your intervention to help people), but they can still be of value, even if they have to be taken with a grain of salt. The key is whether the author's were honestly trying to control for all of the biases that they could, and I really do think these guys made that effort. I also think the authors' biases, base on listening to them talk, lie not so much in knee-jerk anti-war sentiment as they do in honestly grieving over and wanting to know what the true human costs are. I do agree they should have put their strong feelings about the needless deaths caused by the war in the conflict of interest statement at the end of the article(but I don't agree that it negates the entire validity of the study).

To me it seems obvious that this kind of number is likely to be much closer to the truth than the 45K counted by Iraq Body Count, especially when someone I consider a media-savvy Iraqi blogger writes:

"I have personally witnessed dozens of people killed in my neighbourhood over the last few months (15 people in the nearby vicinity of our house alone, over 4 months), and virtually none of them were mentioned in any media report while I was there. And that was in Baghdad where there is the highest density of journalists and media agencies. Don't you think this is a common situation all over the country?"
(from HealingIraq - BTW the guy thinks 300K is a better estimate!).
Gary
I promise if I ever write again it will be 4 lines tops.

So now we all know why you question the numbers in the first place Dave. Your own paritsan reasons are not disimilar to what you claim the study was about.

Actually, Martin. I wasn't aware of the political histories of the people involved in the study until that information was brought to my attention as a result of this study. I suspected they leaned left