Chasing Smoke

imageMy simple question – where did the author of a recent science/medical book get her startling facts? – ended up with a deep dive into sources, a very enjoyable phone conversation, and a resolution of sorts.

Readers are directed to earlier posts here and here, which spell out how it all started.

Story begins with this startling line from Sonia Shah’s new book Pandemic: “90 percent of epidemiologists said that a pandemic that will sicken 1 billion, kill up to 165 million, and trigger a global recession that could cost up to $3 trillion would occur sometime in the next two generations.”

Pretty scary. So where did her numbers come from? All signs pointed to a guy named Larry Brilliant, coauthor of the paper Shah mistakenly cited, and a man with one of the best CVs ever, including stints as Ken Kesey’s traveling companion, Jerry Garcia’s personal doctor, world traveler, smallpox eradication pioneer, Google executive director, and head of various global health initiatives. A fascinating career.

I expect that Brilliant is very busy. So I was mildly surprised a few days ago when I got a phone message and an email from the man himself. He wanted to talk about the numbers that I had questioned.

We spoke yesterday. And I really enjoyed our talk, which ranged from the question at hand to digressions on everything from movies to think tanks to rock. Larry Brilliant seems like a great person.

But when it came to the source of Shah’s numbers, well . . .

Turns out that the figures she used were floated first at a private meeting ten years ago, during a time when bird flu (H5N1) was just starting to hit, jumping from poultry to humans and looking like something that might become the next Spanish Influenza, killer of millions. The meeting Brilliant convened in 2005, called Pandefense 1.0, was a private gathering of around 50 thinkers from various fields who wanted to get their hands around the potential scope of the threat. Among them was a investment expert from Canada who was among the first to project the economic impact of a pandemic. Her rough numbers for the U.S. and Canada were back-of-the-envelope projected to global levels to get to the $3 trillion figure. None of this was ever published in a professional journal.

As for the other numbers? They also came from Pandefense 1.0, via a survey of participants. The only published paper from the survey (the one cited by Shah as her source) does not include the numbers she used. Brilliant agrees her citation was wrong. Instead, he said, she should have cited a TED talk he gave in 2006. And — now we’re getting down to it — the numbers he used in that talk came from discussions at Pandefense. They were nothing more than educated guesses thrown around at a private meeting. They were never peer-reviewed. They never appeared in a published paper.

They might turn out to be right, but they are not science. (And they might turn out to be wrong: After years of frightening bird flu headlines and disaster scenarios, I was more than slightly surprised to find out that the actual global death toll from H5N1, to date, is 440.)

Brilliant plans to ask for a new survey that might really assess what epidemiologists think of the risk of a pandemic. I agree that one is needed. And until that’s done, the statement made in Shah’s book should be corrected.

(you can read the followup to this post here)