It is irresponsible and despicable for academics and scientists to use their trusted position and their credentials to spread deception for partisan purposes.
Much to my regret, I find myself forced to revisit old territory on the subject of civilian casualties in Iraq and the questionable means being used to estimate them. In 2004 a study by researchers from Johns Hopkins University was published in The Lancet. It purported to be a statistical analysis of civilian deaths in Iraq during the first year after the invasion. It was immediately latched onto by the left and they began talking about 200,000 civilian casualties and other alarming numbers despite the fact that the study was poorly conceived, shoddily executed and a classic example of bad methodology.…
Article comments
— go to most recent comments26 - troll
Dave and McNab 24 & 25 - you are both babbling about that which you have limited to no evidence...tasty bullshit but bullshit none the less
the history of US support for death squads in South America does not mean the same is going on in Iraq...but even the MSM reports regularly on death squad activity there
and I've heard stories from soldiers who have been 'in country' of numerous acts of small time socio-pathology inflicted on the civilian population by US soldiers...tank drivers playing bumper cars for no reason...pointless slaughter of farm animals on which families depend...that kind of 'I'm gonna get some' shit
Iraq is a tragic mess - a death factory - not a American tale of good guys and bad guys
27 - Dave Nalle
Troll, I've seen nothing in the media about government and/or US-sponsored death squads in Iraq, which is what we sponsored in central america and what McNab implied was going on in Iraq. All the info I've seen on death squads is about sectarian ones which are not directly US or government sponsored.
As for good guys and bad guys, regardless of whatever small misbehaviors there may be, the US and coalition forces are still the good guys when compared to death squads, roadside bombers and hostage beheading kidnappers. In fact, there's really no comparison.
Dave
28 - Dave Nalle
May I offer this quote from Stratfor.com, one of the most respected strategic research groups there is:
"Allegations that Shiite death squads are operating out of the Iraqi Interior Ministry and Iraqi police force are completely false, U.S. Maj. Gen. Joseph Peterson, who is in charge of training the Iraqi police, said Sept. 20. Shiite militiamen unaffiliated with the Interior Ministry, notably members of Shiite leader Muqtada al-Sadr's Mehdi Army, have been most frequently detained in relation to the killings"
Dave
29 - McNab
#25,"eye Witless News", sarcasm, Dave. Once in a very, very, very blue moon will you see US atrocities aired on US television.
30 - Bishop
Dave writes:
Keep in mind that just as the study conclusions are an 'estimate', I'm making an estimate as well.
Bishop replies:
The Burhnam study's estimate is based on a statistical method have been tested and found to give reasonable estimates in hundreds of other circumstances. Your estimate is based on wild speculation and poor mathematical ability.
Let me help you with the math.
To estimate total over/under representation of high violence areas, you can add up the population for all governates in the "high violence" category. These total 6394508 and they constitute 24.5% of the country's population (omitting Dahuk and Muthanna). Then total the number of clusters in those governates: 13 out of 47, which equals 27.7%. So high violence areas are over-represented by 27.7-24.5 = 3.2%.
Your method, incorrect for coming up with a correction factor, was to take a governate like Anbar representing 4.9% of the population and that had three clusters, representing 6.4% of clusters, and calculate (6.4-4.9)/4.9=30.6% (though you saw fit to "round up" to 36%). You then correct the total body count by the average of such percentages. To illustrate how wrong this is, notice that if you had a high violence governate of only 270 people (0.001% of the total population) sampled with one cluster (2.1% of clusters), your method would tell us we've oversampled (2.1%-0.001%)/0.001% = 2099%. Average that in to your correction factor, and you'll have no deaths at all, which not even you seem to have the temerity to suggest.
Bishop wrote:
Even if you're adjustment percentages were correct, you applied them incorrectly. Your calculation was:
654,965 - 654,965 * (-.3 + -.2 + -.3) = 130,993
In fact you should multiply:
654,665 * .7 * .8 *.7 = 256,746
Dave replied:
A different way of looking at it, certainly. Not necessarily more valid unless you assume that the geographical underrepresentation and the rounding somehow apply more to military age men than to women. ...
Bishop replies:
Dave, these aren't two equally valid ways of oing the math. In fact, under your method if you had found one more error to correct for by -30%, you would have 654,965 - 654,965 * (-.3 + -.2 + -.3 + -.3) = - 65,497! But not even you have the temerity to suggest that there have been negative deaths in Iraq.
Dave writes:
So, if we accept your approach, the study's estimate is only off bya factor of 100%? Wow, that's great news.
John replies:
Ok, so you can't do math, and you can't read. I showed why a single one of your factors and an additional one of your calculations were wrong and its consequence. In fact, your other corrections are baseless as well. One of them isn't even a correction, its just the assertion that we shouldn't include "combatants" along with wild speculation about how many dead were combatants.
Dave writes:
Maybe I should point out again that even my 130,000 casualties estimate is double any of the figures reached by more verifiable methods. If anything my adjustments were super-conservative.
Bishop replies:
What you think are "more verifiable methods" have in fact been shown time and again to vastly underestimate the number of deaths in other conflicts and humanitarian tragedies. The Burnham et al study is not perfect, but its methodology is far sounder than anything else used in Iraq to date.
I haven't seen you call for a more thorough investigation by an independent group, therefore it appears that you are not actually interested in the truth about how many people have died in Iraq.
- John
31 - Gary Kunkel
Dave, you wrote: "Despite the fact that their conclusion states that 655,000 people died, their data suggests that what they really mean is that somewhere between 131,000 and 655,000 died, but they aren't promoting that lower number and it's not the one that's' going to get all the press." That is not correct-the article estimates 654,000 and that is in the middle of their 95% confidence interval (392,000 to 942,000). If they or the media wanted to exaggerate or "promote" a bigger number they could have used the 942,000 upper limit, but they didn't.
As someone who reads plenty of medical journals, I did not find this article to be poorly conceived or executed, and to me it looks like they made every effort to be as complete as possible. I do think the rural/urban point is a good one, and would ideally be better addressed(though I had heard 75% of Iraqi's were urban). I also think a morgue or "news report" death count is much more likely to be too low than this is to be too high...
Gary
32 - Lucky Frog
I must not be able to read either because when I look at the report I don't see one word about any possible total other than the 655000 figure. No lower or higher figure is mentioned at all.
The defects of this survey are obvious. You can play all the math games you lke, but at the end of the day the basic points in this article seem solid. The subjet here is important enough that it deserves better than such a gross and implausible mess of guesswork and speculation.
33 - Lucky Frog
And Bishop, your assertion that other methods underestimate seems ridiculous in the face of the fact that 80 percent of the deaths in the survey were verified by death certificates.
That being the case then it seems entirely reasonable to conclude that the Iraqi government figures represent a 20 percent undercount. That seems to be one of the few hard facts shown here. So that means a more accurate total might be around 75000 - a hell of a lot less than 655k.
34 - Gary Kunkel
Here's the quote:
"We estimate that between March 18, 2003, and June, 2006, an additional 654?965 (392?979"942?636) Iraqis have died above what would have been expected on the basis of the pre-invasion crude mortality rate as a consequence of the coalition invasion. Of these deaths, we estimate that 601?027 (426?369"793?663) were due to violence." The standard thing is to put the 95% confidence intervals in parentheses.
I don't think this study is a "gross and implausible mess" at all, to me it seems more plausible than not. I also think the media hype is deserved, or at least on par with what we got for vioxx and all the "excess deaths" that happen from medical errors.
G
35 - Lucky Frog
your vioxx comparison says it all. When the statistics for vioxx deaths were finally studied it became clear that the death rate for vioxx users was statisticqlly indistinguishable from the death rate for non users.
36 - Dave Nalle
I think you hit the nail on the head, Bialy. I never intended to do a real mathematical analysis. There's plenty wrong with the study without crunching numbers.
I may have to go back and do a bit of a rewrite to minimize the opportunities for misdirection.
Dave
37 - Dave Nalle
The Burhnam study's estimate is based on a statistical method have been tested and found to give reasonable estimates in hundreds of other circumstances.
Cluster sampling is notorious both for its unreliability and for the ease with which the methodology can be manipulated to slant the results to a particular bias. I refer you to SocialResearchMethods.net which observes: "Though very economical cluster sampling is very susceptible to sampling bias."
Your estimate is based on wild speculation and poor mathematical ability.
Let me help you with the math.
My estimate was almost pure guesswork. I never intended to do a mathematical analysis and should have realized including any numbers at all was a mistake which would let someone like you try to distract from the issues with the methodology by trying to turn it into an argument about numbers.
Your method, incorrect for coming up with a correction factor, was to take a governate like Anbar representing 4.9% of the population and that had three clusters,
No, my method was to point out that there were problems with rounding off and then just pick an arbitrary number. I ddin't look at the variances for any but the most problematical provinces and didn't do any detailed calcuations at all. I wasn't trying to figure out the exact amount of error. To do so would be ridiculous.
Remember, they don't even have specific counts for the individual provinces in the report and there's a margin of error of +/-40%. Given these things fidding with 'exact' calculations is totally pointless. There's nothing there to work with and all your number crunching is completely meaningless. I made a terrible blunder in opening the door to it.
Even if you're adjustment percentages were correct, you applied them incorrectly. Your calculation was:
654,965 - 654,965 * (-.3 + -.2 + -.3) = 130,993
In fact you should multiply:
654,665 * .7 * .8 *.7 = 256,746
I agree, your method of calculation is more accurate, except that you're starting with meaningless numbers so the result is equally meaningless.
One of them isn't even a correction, its just the assertion that we shouldn't include "combatants" along with wild speculation about how many dead were combatants.
This is one of the main points you're missing. The media is representing the study as reporting CIVILIAN casualties, but the fact is that a good third of the deaths are clearly not civilian deaths, as admitted by the study authors. That's important.
I haven't seen you call for a more thorough investigation by an independent group, therefore it appears that you are not actually interested in the truth about how many people have died in Iraq.
The truth is obvious - too many have died. An accurate investigation wouldn't be a bad idea once we acknowledge that this study is NOT accurate and that releasing such a misleading report is counterproductive.
Dave
38 - troll
Dave & Co - if it were accurate that between 400 and 900 thousand people had died in Iraq (in excess of the 'norm - etc) since the invasion would it make any difference to the way you view the war - ?
just wondering
39 - Bishop
Reply to Lucky Frog #33
Lucky Frog - you have a good point about the fact that a large percentage of the dead were represented by death certificates. You would think that if all those death certificates were issued, then someone could tally them. However, the system for keeping track of them seems to have broken down, and in addition the Iraqi government will not be transparent about the information they have. None of the other estimates to date are based on a systematic, country wide sampling of death certificates - but that is essentially what the study we're debating has done!
The fact that these certificates exist does suggest a) an even better job of tallying the dead is still possible, and b) the Burnham study was not misled by people lying about their dead.
BTW the actual number for the certificates is that they were provided in 92% of the cases requested.
40 - Dave Nalle
Dave & Co - if it were accurate that between 400 and 900 thousand people had died in Iraq (in excess of the 'norm - etc) since the invasion would it make any difference to the way you view the war - ?
60,000 - 130,000 - 400,000 - 900,000
Even the smallest of those is one hell of a lot of people.
On the one hand I'm troubled that ANY civilians have died in Iraq, though I realize that civilian casualties are impossible to avoid.
But on the other hand, the raw number of casualties has very little to do with whether or not the involvement in Iraq is a wise strategy in the war on terror.
Then there's also the problem that most of whatever deaths there are in Iraq are not caused by the coalition forces there. They predominantly kill hostile insurgents and terrorists. Most of the other deaths are faction on faction and secterian 'cleansing' type operations. There really ought to be more of the former so that there will be fewer of the latter.
But as I see it ALL of this is what you get with a situation such as exists in Iraq, but the big problem is that there's ZERO reason to think things there would get one bit better if we pulled out. In fact, they're likely to get much worse in the short term, followed by some sucky long term outcome for the people living there.
As you might surmise, I have lots of concerns but no answers. But what I do know is that inflating civilian casualties and running around wringing your hands doesn't actually solve any of these problems.
Dave
41 - Bishop
But Dave, if an invading power, who really has no place being there in the first place, obliterates a system that was keeping sectarian violence in check (even if that system was itself a feared dictatorship) without replacing it, thereby allowing sectarian violence to mushroom, then doesn't that occupying power bear some responsibility? And if the occupying power makes numerous blunders that further exacerbate or enable sectarian vioelence, then doesn't it bear even more responsibility? And finally, wouldn't you want to fire the management team that ineptly led you into this situation and, you know, come up with a new management team? And maybe consider some alternative strategies? Or would you just want to "stay the course"? I think its clear we have to find something other than both "cut and run" and "stay the course".
I challenge your implication that we have no responsibility for a large portion of these deaths. Regardless of who is killing whom, the death rates reported in the Burnham et al article are *excess* over what they were before we invaded. That implies that they would not be happening if we had not invaded. In other words, our invasion created a situation that allowed that violence to happen, even if American bullets and bombs are only directly responsible for 31% of the casualties.
42 - Bliffle
"It is irresponsible and despicable for academics and scientists to use their trusted position and their credentials to spread deception for partisan purposes."
Wow. How do you know this is true?
43 - Dave Nalle
But Dave, if an invading power, who really has no place being there in the first place,
So, you're a strict non interventionist? Brave stance. No foreign financial aid, no humanitarian aid in Darfur or the Congo or Uganda or anywhere else. No military deployments outside our borders? Takes some guts to make a stand like that on principle. Or is it just the involvement in Iraq you object to?
obliterates a system that was keeping sectarian violence in check (even if that system was itself a feared dictatorship) without replacing it,
Last I checked there was a government in Iraq which has replaced Saddam's regime. In fact it's one the people there chose for themselves.
thereby allowing sectarian violence to mushroom,
Let's say we hadn't invaded and had managed to topple saddam through internal overthrow or assassination or some other means. Do you think there would have been no sectarian conflict then?
then doesn't that occupying power bear some responsibility?
Absolutely, but the question is where does that responsibility end. Clearly you think we ought to be caretakers for Iraq indefinitely and probably increase our military and financial presence, but I think there has to be a point at which we decide we've done all we can and they have to solve their own problems. The internal conflicts are NOT of our making, we just opened the door to the inevitable.
And if the occupying power makes numerous blunders that further exacerbate or enable sectarian vioelence, then doesn't it bear even more responsibility?
I don't see how the factions in Iraq were encouraged by anything we did. We certainly attracted Al Qaeda, but we're no longer the focus of most of the hostility. We're mainly targeted now because we get in the way and try to stop some of the violence.
And finally, wouldn't you want to fire the management team that ineptly led you into this situation and, you know, come up with a new management team? And maybe consider some alternative strategies? Or would you just want to "stay the course"? I think its clear we have to find something other than both "cut and run" and "stay the course".
I'm all for new strategies. And we do have an electoral process to get rid of our management team every few years. As for what those strategies are and whether they'll actually work, that's a whole different debate, as is the question of whether experimentation is really the best approach to Iraq at this point when we have no reason to think that a different approach won't make things worse.
I challenge your implication that we have no responsibility for a large portion of these deaths.
Didn't I say no direct responsibility? We're not the ones planting the IEDs, sending the suicide bombers, running the death squads and otherwise being indescriminately violent. For the most part our violence is reasonably well targeted.
Regardless of who is killing whom, the death rates reported in the Burnham et al article are *excess* over what they were before we invaded. That implies that they would not be happening if we had not invaded. In other words, our invasion created a situation that allowed that violence to happen,
You make the assumption that civil war would never have broken out there without our intervention. Even the most repressive dictatorship does eventually fall, and we didn't create the ethnic and sectarian divisions in Iraq.
even if American bullets and bombs are only directly responsible for 31% of the casualties.
I find it interesting how that number almost exactly corresponds to the additional deaths of military age males.
Dave
44 - Dave Nalle
Wow. How do you know this is true?
How could you possibly not agree with it, Bliffle?
Dave
45 - Bishop
Dave,
I do have to agree with you that many of the newspapers have misreported the excess deaths as being all civilians, and this is really a major error. This seemed to start with the New York Times, "A team of American and Iraqi public health researchers has estimated that 600,000 civilians have died". This was then copied by many other prominent papers.
This is not what was stated by the Lancet article, whose authors stated clearly that they were sampling all deaths and could not distinguish combatants from others. Nor was it misstated by the Johns Hopkins press release regarding the article.
46 - Dave Nalle
I know they didn't make that represenation in the report, Bishop. But this reaction from the media is EXACTLY what happened with their 2004 report. In this case as was the case back then, the media immediately latched onto a high number, treated it as an absolute fact rather than as an estimate and presented it as a count of civilian casualties.
It's certainly not the fault of the study authors, but they ought to have been sensitive to it given that the exact same thing happened last time. They should have made an exra effort to qualify and explain the study and they certainly shouldn't have released it right before the election. They opened themselves up for accusations of bias and deservedly so.
Dave
47 - MCH
"So, you're a strict non interventionist? Brave stance. No foreign financial aid, no humanitarian aid in Darfur or the Congo or Uganda or anywhere else. No military deployments outside our borders? Takes some guts to make a stand like that on principle. Or is it just the involvement in Iraq you object to?"
- Dave Nalle
Another "brave stance" is typing bellicose pro-interventionast slogans from the safety of a fortified compound 10,000 miles from the invasion, while someone else risks life-and-limb in the "intervention."
48 - stan
Interesting that this week, the new British Army chief, the Chief of the General Staff, Sir Richard Dannatt, has called for the pullout of British soldiers as soon as possible, believing that the continued presence of foreign troops is now causing more problems than solutions.
Dannatt rightly says that we kicked down the door in Iraq, rather than being invited in, and while the removal of a dictator is a worthwhile achievement, we are now wearing out our welcome.
Tony Blair has backed him and one would hope that any such British pullout would involve also the removal of Australian troops (although I won't be holding my breath under the current Aussie government) and ultimately those of the US.
Once those three decide to go, there's not much point in anyone else being there.
This is part of the edited text of Dannatt's interview as reported on CNN:
He said Operation Sinbad - a security and reconstruction effort in Basra - "is trying to make Basra better and a lot of British soldiers are doing a really good job. In that regard, their presence is helping. But there are other parts where our mere presence does exacerbate and violence results."
However, he said, "that is not a reason for us to leave."
"I am on record publicly saying we're standing shoulder to shoulder with the Americans. I am on the record from a speech three weeks ago saying that I'm planning force packages in Iraq through 2007 into 2008. I'm a soldier - we don't do surrender, we don't pull down white flags. We will remain in southern Iraq until the job is done. We're going to see this through."
However, Dannatt, who took over as the Army's chief of general staff in August, had been quoted in the newspaper as saying that the U.S.-led coalition's plan to establish a democracy in Iraq that would be an "exemplar for the region" was unlikely to happen.
"That was the hope. Whether that was a sensible or naive hope, history will judge," he said. "I don't think we are going to do that. I think we should aim for a lower ambition."
But with the country edging nearer to civil war - if not already immersed in it - Dannatt said the strategy for implementing an Iraqi democracy was ill-prepared.
"I think history will show that the planning for what happened after the initial, successful war-fighting phase was poor, probably based more on optimism than sound planning," he said.
Now, he said, Britain has essentially overstayed its welcome in Muslim Iraq.
"The military campaign we fought in 2003 effectively kicked the door in," he said, noting that was a far cry from being invited into the country. "Whatever consent we may have had in the first place may have turned to tolerance and has largely turned to intolerance."
Perhaps he's right.
And you're absolutely right Dave. I too don't think civilian casualties caused by us rather than the insurgency have been too bad (although as you say, any is no good) ... but the PR campaign in my view started going awry the moment a young and naive US soldier with no idea of the local sensibilities planted the US flag (against orders) on the statue of Saddam outside the 14th Ramadan mosque opposite the Palestine Hotel in Baghdad.
Not really his fault, as young men are wont to do such things, but the law needed to be laid down very clearly from day one from higher up. That has been a big part of the problem.
And I'm going to end on this.
Dave Nalle wrote: "It's higher than estimates from nations which are not beset by terrorism and internal conflict ... Ireland."
A slip of the keyboard old boy???
49 - Gary Kunkel
If the war is wrong, even "only" 10,000 excess civilian deaths shouldn't make it any less wrong.
If the war is right, even a million excess civilian deaths shouldn't make it any less right.
In either case I think we have a responsibility to face the war and know the costs, in financial, political, and especially human terms. I think these guys are at least trying to measure the human costs (unlike our govt). I feel we have a right and moral obligation to try to know these costs for both current and future reference.
Thanks for the lively and civil discussion Dave!
Gary
50 - troll
I agree that the timing of this publication so close to an election is questionable...it makes it difficult to focus on the 'science' of the study
I smell the partisan 'conspiracy' that Dave claims exists but have no insider info to prove it
51 - Bliffle
Do you really believe that: "... academics and scientists ... use(d) their trusted position and their credentials to spread deception for partisan purposes."
Maybe you've just become inured to authorities lying systematically in your experiences with government officials.
52 - Bishop
The Lancet controlled when the results of a study were released, the authors have no control over it. The authors submitted the article to the journal several months ago, which was as soon as they possibly could given that they conducted the research in June/July. They had trouble actually getting the data out of Iraq, and given the amount of time it takes to analyze and write, I can't see them submitting any earlier than late August. It then typically takes several weeks to obtain peer reviews (this article was peer reviewed by four scientists). It then takes several more weeks (sometimes months, in fact) to format it for the journal.
The journal may have timed the release relative to the elections, but it is also very plausible that this was simply the soonest they could get it out. If they had published it a month earlier, which would be lighteneing fast turnaround for a journal, you still would have claimed it was too close to the election.
53 - Dave Nalle
Bishop, they presumably had past experience and knew how long Lancet takes to get something from submission to publication, so they were at the very least aware of when it would be released.
Dave
54 - troll
and the issue remains: if what Alex claims in #2 (*there is a pseudo-progressive core that foolishly believes that if you can demonstrate that a war kills some magically big number of innocents or civilians, then everyone will equally magically agree that war is bad and end all hostilities.*) is true and correctly describes the researchers' belief/motive it's hard to see how it wouldn't affect the outcome
55 - gazelle
hi
question is who is the bigger killer : Bush & Co or ____ . . . ?
and for what, . . . the Party ?
tch, tch, tch !
best
56 - David Bryant
Hi! I'm new to this forum, and pleasantly surprised by the objectivity and apparent good will of the author. This is definitely a cut above the average blog I've visited recently.
Dave Nalle (#12) wrote: As someone earlier suggested, why not just count the dead? This time out they went to great pains to make sure that almost all the people they questioned provided death certificates for the people they claimed had died - and about 80% had them. If those death certificates exist, why not go to the issuing body and get a total and then add 20% to it as a 'fudge factor'? That would be a hell of a lot more accurate than this effort.
The problem is, the "issuing body" is probably the local coroner, or maybe just a medical technician who happens to be the only thing resembling a doctor in the immediate vicinity when the death occurs, and there is no centralized repository for reporting deaths in Iraq right now. Remember, the Coalition Provisional Authority established its policy of not counting Iraqi deaths way back in March of 2003. No doubt the new government would like to maintain vital statistics at a central location, but keeping the members of parliament alive (and out of the hands of kidnappers) is a more pressing concern, from what I hear.
Death certificates are generally produced in at least triplicate -- one for the family of the deceased, one for the coroner / doctor, and one for the bureau of vital statistics. In the case of Iraq, there's a good chance that the two "official" copies have already been lost or destroyed in the havoc and confusion, so that the one copy remaining in the hands of family members may be the only written record of the deceased that still exists. The family would tend to hold onto it for quite a while, as a memento, in all likelihood.
Oh, I read the Lancet report, and one detail seems to be garbled here. At 80% of the interviews where a death was reported, the interviewers actually asked to see the death certificate. And in 92% of those cases the death certificate was actually produced. So the researchers only saw certificates for some 73% of the reported deaths. Apparently, in the other 20% of the interviews, the researchers either forgot to ask the question, or deemed that particular question to be impolitic.
57 - Dave Nalle
and the issue remains: if what Alex claims in #2 (*there is a pseudo-progressive core that foolishly believes that if you can demonstrate that a war kills some magically big number of innocents or civilians, then everyone will equally magically agree that war is bad and end all hostilities.*) is true and correctly describes the researchers' belief/motive it's hard to see how it wouldn't affect the outcome
I find it hard to believe you can't see how it coudl affect the outcome. Cluster studies are made for manipulation. The people asking the questions pick and choose which doors to knock on. The designers of the study pick where the clusters go based mainly on convenience rather than balance, or they can pick the locations to get a particular slant to the sample. Then you get some very rough numbers which can be massaged pretty easily and interpreted in any of a number of ways. The methodology of this sort of study is just not reliable at all. There are too many arbitrary factors and choices involved.
Dave
58 - Bliffle
"It is irresponsible and despicable for academics and scientists to use their trusted position and their credentials..."
Wow! You're actually accusing the "academics and scientists" at Johns Hopkins of deliberate deception! A calculated and conscious plan to deceive people. Amazing. How do you KNOW they did that? That they set about to contrive this elaborate deception? Are you familiar with the participants? Do you recognize their names? Have you worked with them in the past? Have you read their other works?
"... to spread deception for partisan purposes."
Partisan? Do you know the party they belong to?
How did you learn all this stuff?
59 - Dave Nalle
Wow! You're actually accusing the "academics and scientists" at Johns Hopkins of deliberate deception! A calculated and conscious plan to deceive people. Amazing. How do you KNOW they did that?
Bliffle, did you miss the heading on this article that reads 'opinion'?
That they set about to contrive this elaborate deception? Are you familiar with the participants? Do you recognize their names? Have you worked with them in the past? Have you read their other works?
Actually, I've read their 2004 report which used the same methodology, so I'm familiar with their work.
"... to spread deception for partisan purposes."
Partisan? Do you know the party they belong to?
Partisan doesn't just mean loyalty to a political party, it can be loyalty to an ideology or any group. In this case they appear to be partial in their generic opposition to war on any and all grounds.
The neutrality of the report and the research is clearly in question, and which side of the issue they come down on is evident when they write:
Not frothing, but it's very clear where they stand and what they're trying to achieve, and why they might inflate their figures.
How did you learn all this stuff?
By paying attention and not being a total dumbass?
Dave
60 - David Bryant
... this sort of study is just not reliable at all. There are too many arbitrary factors and choices involved.
Speaking as a professional statistician (degree in mathematics from CalTech, and over 20 years experience as a life insurance actuary) I have to disagree. While bias is alway possible in any statistical study, it's not necessarily the result of "arbitrary choices." And "this sort of study" is widely used to measure exposures to toxic chemicals, and the incidence of disease, and even to plan multi-million dollar advertising campaigns. If sampling techniques are so unreliable, why do so many professional decision makers depend on them every day of the week?
The authors of this study aimed to sample 50 clusters, each consisting of 40 households. They went to elaborate lengths to make the sample of 50 clusters reasonably representative of the population of Iraq (based on the best available census data), and they were careful to randomize the center of each cluster, to the extent that was possible. They ended up with 47 clusters and 1,849 households ... in other words, they had to exclude data from 3 of the clusters they had targeted, and they didn't complete the survey at some 31 households sprinkled among the 47 clusters where they did get good data.
Now a random survey based on 1,849 independent data points and including 300 "excess" deaths by violent means would be highly reliable, yielding a mortality estimate that would be good (at the 95% level of confidence) to within some 6% or 7%, plus or minus. In fact, the authors recognized that their sample only consisted of 47 data points, and this fact is reflected in the rather large 95% confidence interval they reported (plus or minus 30%, roughly).
In short, Dave, you're correct to criticize the "cluster" method of gathering data as being inherently unreliable. The authors of this study recognized that, and accounted for it in their statistical analysis. That's why the "error bars" are so big. To go farther, and imply that they deliberately skewed their sample -- especially when they have described in detail the techniques they used to assure randomness -- is to express a personal prejudice not supported by the facts.
61 - Douglas Knight
Your description of the sampling method is completely at odds with the protocol described in the paper. Are you accusing them of fraud?
Clusters were not picked in major urban areas of each governate, but in districts picked randomly, weighted by population. Within each district, streets were picked randomly. That last step is unclear and could bias things, either way, but it is simply false to say they avoided villages.
Where do you get the claim that the three Anbar clusters were in Ramadi and Fallujah?
62 - Dave Nalle
In short, Dave, you're correct to criticize the "cluster" method of gathering data as being inherently unreliable. The authors of this study recognized that, and accounted for it in their statistical analysis. That's why the "error bars" are so big.
Which raises the basic issue of why we should even consider a study with a margin of error so incredibly huge. If this were a political poll or a census study the entire data set would be thrown out as unreliable.
To go farther, and imply that they deliberately skewed their sample -- especially when they have described in detail the techniques they used to assure randomness -- is to express a personal prejudice not supported by the facts.
The fact that the even published a report based on methods where they had to compensate for a sample which is so incredibly unreliable suggests at a minimum that they were biased towards giving the report an endorsement of legitimacy which it doesn't deserve.
The truth is that because of the lengths they went to in order to compensate for their sampling methods the results are more the result of mathematical fiddling than of actual data.
Dave
63 - Dave Nalle
Your description of the sampling method is completely at odds with the protocol described in the paper.
Odd, since it's based directly on what they say in the paper.
Are you accusing them of fraud?
I'm accusing them of being overly enthusiastic and arrogant. Whether it's intentional fraud or not is a matter of perspective and intent.
Clusters were not picked in major urban areas of each governate, but in districts picked randomly, weighted by population.
In comments made outside the report they say specifically that they went from house to house and from apartment to apartment and that when someone wasn't home or didn't answer the door they went down the hall to the next apartment.
Within each district, streets were picked randomly. That last step is unclear and could bias things, either way, but it is simply false to say they avoided villages.
Do you have access to more information about the study than what's actually in the published text. If so, please share your source. They say that they followed the same methodology as the first study, in which they did overwhelmingly draw their data from urban areas. Nothing in the study says that they went to small villages, in fact a number of statements imply that for purposes of access and speed they did not. For example they say that they were "confining the survey to a cluster of houses close to one another," which suggests tightly packed areas of settlement - urban areas.
Where do you get the claim that the three Anbar clusters were in Ramadi and Fallujah?
As I said before, they followed the methodology of the last study, and it's been established that the first study drew ALL of its data for Anbar and a province they paired with it from Fallujah.
Dave
64 - Dave Nalle
Let's add some fuel to the fire:
Here's a video of one of the authors of the Lancet report publicly admitting that the timing of the release was political.
Here's a video of the editor of the Lancet making inflammatory anti-US and anti-war statements.
And here's a relevant bit expressing the skepticism of some experts and also confirming the urban nature of the data sources from a NYT article from last week:
Dave
65 - David Bryant
...why ... even consider a study with a margin of error so incredibly huge. If this were a political poll or a census study the entire data set would be thrown out as unreliable.
Speaking as a statistician again, this simply is not so.
Every statistical study is up against Gauss' Central Limit Theorem. In essence, this says that to cut ths size of the probable error in half, you have to collect four times as much data. This is a mathematical fact embedded in every random process. It's kind of like the law of gravity -- there's just no way to get around it.
The ultimate question that decides the size of a statistical sample comes down to economics. How much money are you willing to spend collecting the data? Do you want to lay out 4 times as much money to cut the probable error in half? Or have you spent enough money already?
Typical public opinion polls have "error bars" in the range of 4% to 5%, corresponding to a sample size in the range of 500 to 600. Since such data are easily collected (say 5,000 phone calls all together, assuming that some 90% of the calls are not completed, or the guy who picks up the phone doesn't want to participate, etc.), we've grown accustomed to this size of error bars. Pollsters could reduce the reported error to the 2% range, but then they'd have to make 20,00o phone calls to complete a survey. They figure it's just not worth it.
Conditions in Iraq are not nearly so favorable. To obtain comparable precision (say 5%, plus or minus), the researchers would have to collect 36 times as much data (30% / 5% = 6; 6 * 6 = 36). This would mean 72,000 face to face interviews. Since the data are only meaningful if they're collected within a relatively short period of time, in practice they'd have to employ 36 times as many people.
The actual study employed 10 people (2 teams of four interviewers, with one field supervisor for each team). To obtain the sort of precision you'd like to see, they'd have to put 360 people in the field for two months. Apparently the authors of this study simply didn't have that much money available. It's also likely that they couldn't find 360 Iraqi doctors who would be willing to participate, at any rate of pay.
In summary, one ought not confuse accuracy with precision. A well-designed measurement of any description is as accurate as it can be, but it is always imprecise. That's what confidence intervals are all about. They do not say the measurement is wrong -- they only indicate the precision with which that measurement has been made.
66 - notajungian
Dave: Your characterization of the study as unreliable is incorrect. You are referring to accuracy, I think. And, yes, the broad range of figures indeed seems large. But, keep in mind this estimate represents a 95% confidence interval wherein the true number of deaths is most likely near the 655000 figure. Assuming a normal distribution for the sampling statistic of interest, the lowest and highest estimates are less likely to be correct than is the estimate in the middle of the distribution.
The authors address the issue of reliability, in part, when they compare results from this study with the previous study and find consistent trends.
67 - notajungian
First, David Bryant's comments above are correct -- the issue is precision (rather than accuracy as I mentioned. My bad).
Second, as to the timing of the release of the study being political--so what? Releasing the data weeks before the election allows for an informed electorate. Contrast this behavior with that of the James Baker group who is not going to release their recommendations on Iraq until after the election. Which decision is more politically useful to the electorate?
68 - Dave Nalle
Guys, I can say I made a study by interviewing one Iraqi that suggests that between 0 and 27 million Iraqis have died in the last three years and it would be 'accurate', but releasing it as if it's meaningful would be incredibly irresponsible. The same is true with this study which has a margin of error of +/- 40%.
And contrary to what you guys are suggesting, if you saw a poll in a newspaper which said 'margin of error 40%' in small print at the end of it you'd consider it meaningless. They expect us NOT to make that same judgement of this report, and the media is obliging them. They're reporting the number without reporting how incredibly imprecise that figure is.
The way I figure it, a 95% confidence that a number is accurate to within 40% either way is useless. They release this number and present it as accurate, when the truth is that it's more likely to be any number BUT that number based on their total lack of precision.
Dave
69 - Dave Nalle
But, keep in mind this estimate represents a 95% confidence interval wherein the true number of deaths is most likely near the 655000 figure.
If they have a 95% confidence in that result then they're not being objective, because as has been mentioned numerous times before on this thread, if the deaths were that high there would be overwhelming physical evidence which could not be ignored. That evidence does not exist despite several governments and multiple NGOs looking for it.
The UN has been tracking violent deaths in Iraq almost since the end of the war and although their numbers are fairly high, they still don't total more than 120,000. What this report asks us to believe is that there are 530,000 unreported deaths, all of whom were buried privately and not one of which was reported in a newspaper.
Let's apply some common sense here. Do you think that no one would notice if 7 million extra people died in the US in a 3 year period? That's what you're asking us to believe. We're talking almost 1 in 50 people here. It's such a huge proportion that it wouldn't take a cluster study to find it.
Dave
70 - Dave Nalle
Second, as to the timing of the release of the study being political--so what? Releasing the data weeks before the election allows for an informed electorate.
Or a misinformed electorate.
Contrast this behavior with that of the James Baker group who is not going to release their recommendations on Iraq until after the election. Which decision is more politically useful to the electorate?
I'd argue that the release of accurate information in a timely manner is desirable. Rushing out questionable and controversial data to impact the election is not.
Dave
71 - David Bryant
Guys, I can say I made a study by interviewing one Iraqi that suggests that between 0 and 27 million Iraqis have died in the last three years and it would be 'accurate', but releasing it as if it's meaningful would be incredibly irresponsible. The same is true with this study which has a margin of error of +/- 40%
Just three things.
1, You wouldn't have to interview any Iraquis at all to make your hypothetical estimate.
2. The study's margin of error is not +/- 40%. It's +/- 30%.
3. Misrepresentations are not arguments. They are simply misrepresentations.
72 - Douglas Knight
Here's the methodology, from page two, left column:
At the second stage of sampling, the Governorate's constituent administrative units were listed by population or estimated population, and location(s) were selected randomly proportionate to population size. The third stage consisted of random selection of a main street within the administrative unit from a list of all main streets. A residential street was then randomly selected from a list of residential streets crossing the main street.
73 - Dave Nalle
Douglas. Do you see where it says 'main street' and 'residential street'? Camps in the desert don't have streets. One-road villages don't have residential and non-residential areas. Farms don't have streets at all.
As for the +/- factor. By my calculations 942,000 is 43% more than 655,000 and 392,000 is 41% less than 655,000. But feel free to calculate it yourself. I'll be amazed if you come up with different answers. Perhaps if you borrow a calculator from someone on the study team...
Dave
74 - Gary Kunkel
"the rofecoxib group had an increased risk of confirmed thrombotic events (relative risk, 1.92; 95 percent confidence interval, 1.19 to 3.11)" Dave, based on the study this quote was taken from, Vioxx was withdrawn from the market by Merck.
As you can see, a very wide confidence interval is present, and the lower and upper limits are almost 40% from the mean. So clearly all studies with wide confidence intervals shouldn't be considered useless! Please tell me if you'd like I could provide dozens of similar examples.
Gary
75 - Douglas Knight
Where do you get the claim that 37% of Iraqis don't even live in villages?
It's not clear what the part about streets means, but the previous study said that they sampled in villages.