Vaccines: Learning as we go

Vaccine news has me thinking about a book I read a couple of months back, The Flaw of Averages: Why we Underestimate the Risk in the Face of Uncertainty, Sam L. Savage.

As the recent collapse on Wall Street shows, we are often ill-equipped to deal with uncertainty and risk. Yet every day we base our personal and business plans on uncertainties, whether they be next month’s sales, next year’s costs, or tomorrow’s stock price. In The Flaw of Averages, Sam Savage­known for his creative exposition of difficult subjects­ describes common avoidable mistakes in assessing risk in the face of uncertainty. Along the way, he shows why plans based on average assumptions are wrong, on average, in areas as diverse as healthcare, accounting, the War on Terror, and climate change.

The first article is from the New York Times:

A Nation Battling Flu, and Short Vaccine Supplies

[excerpts]

By ANDREW POLLACK and DONALD G. McNEIL Jr.

Dr. Anthony S. Fauci was talking with fellow federal officials in September, a month before swine flu vaccinations were to begin, when it became clear they had a bigger problem than they feared with supplies.

“As we got closer and closer, they said, ‘Oh, my God, we’re not going to make it,’ ”

Many of the projections originated from the Biomedical Advanced Research and Development Authority, or Barda, a part of Health and Human Services, which is responsible for vaccine contracting for emergencies. In April, an official with the agency predicted that as many as 600 million doses could be available by January, if all went well. From May through September, Barda signed contracts worth $1.5 billion for about 250 million doses of vaccine with five companies. The contract volumes were based basically on what each company said it could provide.

Dr. Fauci, the director of the National Institute of Allergy and Infectious Diseases, said in an interview.

Earlier this month, the government was forced to announce that only about 28 million doses would be available by the end of this month, about 30 percent below the 40 million it had previously predicted. That is not enough to satisfy panicky people who are lining up for vaccine around the country or desperately phoning their doctors and public health departments.

But the October shortfall was not the first. Indeed, since the outbreak of the H1N1 swine flu occurred in April, federal projections have been consistently and wildly overoptimistic and have had to be ratcheted down several times. As recently as late July, the government was predicting having 160 million doses by this month.
[snip]
But, these experts say, the government’s accomplishments, and its credibility, are being undermined by overly rosy projections that did not take account of the vagaries of vaccine production, making it look as if the vaccine effort is failing.

“To my mind, it was over-promising what there would be based on our historic experience with flu vaccines,” said Michael T. Osterholm, director of the Center for Infectious Disease Research and Policy at the University of Minnesota.

“When you plant corn in May in Iowa, you have no idea what your harvest is going to be in October,” Dr. Osterholm said. “You have to factor in hailstorms, floods and whatever. They put out a very high yield estimate early on. Every time they had to back off, they lost credibility.”

An introductory scenario from Sam Savage’s book gives us an easily grasped glimpse into the likely flawed thinking that fed those estimates..

Imagine that you and your spouse have an invitation to a ritzy reception with a bunch of VIPs.  You must leave home by 6 p.m. or risk being late.  Although you work in different parts of town, each of your average commute times is 30 minutes.  So if you both depart work at 5:30, then you should have at least a 50/50 chance of leaving home together for the reception by 6 o’clock.

This thinking sounds right.  But your instinct warns that you will probably be late.  Which is correct: your brain or your gut?

Your gut is correct, but not being particularly good with words, it may have difficulty winning the argument intellectually.  So here, in terms that even a brain can understand, is why you’ll probably be late.

Suppose there really is a 50/50 chance that each of you will make it home by 6:00.  Then the trip is like a coin toss in which heads is equivalent to arriving by 6:00 and tails to arriving after 6:00.  Four things can happen:

• Heads/tails: You are home by 6:00 but your spouse isn’t.
• Tales/heads: Your spouse is home by 6:00 by you aren’t.
• Tails/tails: Neither of you is home by 6:00.
• Heads/heads: Both of you are home by 6:00.

The only way you can leave by 6:00 is if you flip two heads, for which there is only one chance in four.

Now imagine that your brother, who also works 30 minutes away, is going to join you. The chance of your all leaving on time now drops to one in eight.

Back to the New York Times article..

Federal officials say they factored such difficulties into their projections. They say they were pressed to make the projections by state and local health officials and by the news media.

I have it a lot lately, we botched our response to this pandemic — thankfully providence was kind and handed us one that was mild.  Though I sort of chuckled when I read that last line from the NYT.  “Don’t blame us — it’s YOUR fault!”  Ah, the Blame Game, it’s only human nature I suppose.

Still, they said, they relied on assurances from the vaccine manufacturers, some of whom might have been overconfident of their ability to resolve production problems.

“I think it’s fair to say that some were overly optimistic, thinking the fix was just around the corner,” said Dr. Nicole Lurie, the assistant secretary for preparedness and response at the Department of Health and Human Services.

Given how badly we botched the vaccine issue, and I must say – due to long known potential issues, though not specific to H1N1-2009,rather novel influenza strains in general, it is only natural to question other official assumptions.  When I started down that road I suddenly found myself asking exactly how much trust do we invest in our government and public officials anyway?  Not much was the answer I came up with, which explains why I also laughed when I read this bit…

But, these experts say, the government’s accomplishments, and its credibility, are being undermined by overly rosy projections that did not take account of the vagaries of vaccine production, making it look as if the vaccine effort is failing.

Two things: A) The vaccine effort IS failing, and B) We are learning as we go.  Mistakes will be made.  That too is only human nature.  As much as we would wish our public officials to be perfect – they are not – they are fallible human beings dealing with other fallible human beings, sometimes in the course of their duties a lot of fallible human beings in the course of the performance of their duties.

Let’s not get twisted up in the “Blame Game”, instead, let’s deal with the realities, and yes, the vagaries, of the situation we find ourselves in.

But more importantly, let’s learn from our mistakes so that next time — and there will be a next time at some point in the future — we will do better.  Doing better means less human misery [even if only temporary misery] and fewer very preventable deaths.  Preventable, that is, if one has access to a vaccine.

The stakes may be high but the learning curve is not all that steep.