Robert L. Scharff, with the help of funding from the Produce Safety Project at Georgetown University, recently published a study analyzing the financial impact of foodborne illness in the United States. Employing the same methods used by Food and Drug Administration (FDA) economists, Scharff estimates that the total cost of foodborne illness in the United States is a staggering $152 billion per year. This headline-grabbing number certainly calls attention to the significant yearly expenditures attributable to illnesses caused by foodborne pathogens. It is my belief, however, that the number is based on speculative data that, unfortunately, calls into question the accuracy of the results.
Before I begin my own analysis, let me be clear: I commend Scharff’s efforts in publishing this study. Without a doubt, a significant amount of research went into this project, and the questionable results are not a product of the methodology employed. Rather, it seems that the real problem lies in the data. Specifically, the data sourced from the oft-quoted “Food-Related Illness and Death in the United States” study conducted in 1999 by Paul S. Mead. The use of data from the Mead study unnecessarily discounts the results of Scharff’s paper. Indeed, if Scharff were to have published the same study using numbers of actual confirmed, reported illnesses, the results would have shown dollar costs per illness that were just as impressive, yet the results would leave far fewer lingering questions as to accuracy.
The 1999 Mead study is perhaps best known for one sentence: “We estimate that foodborne diseases cause approximately 76 million illnesses, 325,000 hospitalizations, and 5,000 deaths in the United States each year.” (See http://www.cdc.gov/ncidod/eid/vol5no5/mead.htm). This sentence is widely quoted in articles that discuss foodborne illness. Heck, I have even used the sentence in my own writings on the topic as a means to show the widespread nature of foodborne illness. Interestingly, some of the numbers from the text of the study are even higher than the above-quoted figures, notably the death estimate, which the study estimates is actually 5,194 persons per year.
Critics of the Mead study, however, point out that roughly 3,400, or 65 percent, of the approximately 5,200 estimated deaths were attributed to “unknown foodborne agents.” Therein lies the twist. As detailed by Paul D. Frenzen, the problem with the Mead study is that the estimates of deaths attributed unknown foodborne agents were generated from hospital records and death certificates that stated a cause of death as “gastroenteritis of unknown cause.” But, as Frenzen points out, “the death estimate consequently omitted deaths from unknown foodborne agents that do not cause gastroenteritis and likely overstated the number of deaths from agents that cause gastroenteritis.” (See http://www.cdc.gov/ncidod/eid/vol10no9/03-0403.htm).
The problems with the Mead study go beyond the death cases discussed in the Frenzen critique. The Mead study employed the same symptom-based data to determine non-death cases of foodborne illness caused by unknown foodborne agents. All said, the study attributed 62 million cases of foodborne illness to unknown agents, or fully 82 percent of the total estimate of 76 million foodborne illness cases per year. Given the imprecise means that were used to obtain the estimates of illnesses attributed to unknown agents, such a large percentage discounts the credibility of the entire data-set.
All of this is not to say that Mead was trying to hide the ball. In the last paragraph of his study, Mead points out that “further refinements of foodborne disease estimates will require continued and improved active surveillance.” (See http://www.cdc.gov/ncidod/eid/vol5no5/mead.htm). Nonetheless, this caveat is often ignored by those who quote Mead’s figures.
The shortcomings of data from the Mead study became even more amplified in the context of the recent Scharff study. The Scharff study directly applied the Mead data for illnesses caused by unknown agents to its own cost calculations, simply adjusting the numbers for population growth. As a result, Scharff attributed 67 million cases of foodborne illness to unknown agents, resulting in an annual cost to United States residents of almost $96 billion. This accounts for 63 percent of the total estimated annual cost of $152 billion. (For those of you who are wondering why the figure does not account for 82 percent of the total cost, Scharff weighted the various dollar costs according to the varying cost of treatment for each pathogen studied.)
As a result of applying the Mead data for illnesses caused by unknown foodborne agents, Scharff discounted the credibility of his recent study. That is not to say that his figures are inaccurate. As far as anyone knows, the dollar figures he has assigned to foodborne illness might coincidentally be quite precise. The problem, however, is one of reliability. Followers of Scharff’s study cannot rely on the accuracy of the results, because as Frenzen pointed out, “the methods used by the Mead study have several shortcomings.” (See http://www.cdc.gov/ncidod/eid/vol10no9/03-0403.htm).
All of this is not meant to imply that Scharff’s numbers are inflated. Rather, as emphasized by Frenzen, because of the methodology used to collect the data, the number of foodborne illness victims exhibiting gastroenteritis may be overinflated, and the victims of foodborne illnesses that do not exhibit gastroenteritis are unaccounted for. For all we know, Mead’s figures may in fact be underinflated. At this point, however, there is simply not enough data to support a reliable conclusion.
Like many others who work in the world of foodborne illness, I am personally very interested in the annual costs associated with victims of foodborne infections. The Scharff study followed the right approached. It sought to account for the totality of costs, ranging from quality of life losses to the cost of lost life expectancy. Indeed, as those with a bit of economic knowledge can attest, the societal costs of severe illnesses are much greater than simply the sum of all medical bills. Nonetheless, correct methodology can only take one so far when the majority of the data used to generate the final results is based on speculative assumptions.
The Scharff study would have proved more useful if it would have focused on individual average costs incurred by victims of various foodborne illnesses. This would have shown the severe economic impact that sufferers of foodborne illness endure, yet would have alleviated the problems associated with estimating aggregate costs based on the results of the Mead study. Perhaps a figure like $152 billion generates a greater wow factor, but realistic figures of actual costs would generate a more lasting impact.