By Marcia Herzberg Lee, LP.D
Adjunct Professor NEU
For the complete study with corresponding tables and graphs, please click here.
ABSTRACT
Foodborne Illness variables can be broken down into 3 major categories controllable through policy intervention: 1) Risk Factors; 2) Inspection (including standardization of inspections); and 3) Food Laws. Finding the right combination of interventions to help minimize the risk of illness is important in public policy for the reduction of foodborne illness.
The Food and Drug Administration assigned risk factors used in the current Retail Food Code, which inspectors mark in reports to identify trends in license holder’s food safety performance. The assignment of value to each item/observation marked has changed through different food codes, some moving from ‘critical’ items subsequently downgraded to ‘core’ (low level) observations. This poses the questions “is the assignment of value placed upon items a true representation of the potential risk and prevention of foodborne illness especially when risk levels change with state and local laws?”; “Does the FDA recommended frequency of inspection per risk level of establishment result in a reduction of foodborne illness events, and are the levels achievable with the available resources?”; “ Do risk factors and frequency of inspection function independently or in unison as a deterrent to foodborne illness?”
FDA established risk levels to facilities as a means for retail food agencies to assign sparse resources in a manner that best utilizes those resources for inspection and provide a recommended frequency of inspection for facilities based upon risk; however, is providing accommodations to do ‘the best with what agencies have’ enough to ensure the safety of retail foods for public consumption? Industry wants less reporting on publicly available inspection reports, the public wants to know inspection report scores placing politicians in a position of keeping both segments of their constituency happy while maintaining economic stability. Is it feasible to keep everyone satisfied and protected? What impact does frequency of inspection, risk factors, and change in food law have in foodborne illness if any?
This study builds upon previous studies to add to the empirical literature for food safety interventions. Reviewing studies on regulatory inspections, recalls, and food laws can provide insight to legislators for viable solutions to our food safety system. By examining the relationship related to food safety laws, foodborne illness risk factors, and frequency of inspection over a three-year period or longer the intent of this study is to identify individual variables that when collectively analyzed will provide a broader picture for plausible policy change corresponding to the reduction of foodborne illness events.
SYNOPSIS OF STUDY
The study Inspection, Laws, Risk Factors, & Foodborne Illness is designed to examine variables that might contribute to the control of foodborne illness in the Retail Food Program – the one section of the food safety system that notably is not federalized and lacks formal federal jurisdiction. The FDA Model Food Code is voluntary with many versions in use across the country – sometimes within the same state.
The questions posed in this study are on the frequency of inspection and change in food code laws, subsequent effect of risk factors levels observed during regulatory inspections of retail food establishments, and any affect foodborne illness:
• Is there a correlation between frequency of inspection and foodborne illness?
• Is there a correlation in changes of risk factors as laws change?
• Does a change in food code/law correlate to risk factor changes/ foodborne illness changes independently or in conjunction with frequency of inspection?
An important question that must be answered is how the current regulatory system can uphold its responsibility to the public enforcing the law and food safety without adequate funding, with an increasing number of food establishments, and insufficient numbers of food inspectors. With the budget constraints for regulatory agencies and pandemic of foodborne illness, the U.S. needs to re-evaluate what policies will lead to better outcomes for consumer protection.
This study adds to the empirical literature on foodborne illness by examining the relationship between frequency of inspection, changes in food law, risk factors, and foodborne illness events in the Retail Food Programs in the three comparable metropolitan areas of Atlanta (Fulton County), Boston, and San Francisco.
Food Study Framework
Looking at past studies helps to understand the foundation on which this current study is built. Many diligent researchers have compiled data, which this study and others add to in the hopes of finding plausible solutions to improve our food safety system and reduce foodborne illness.
Understanding the inherent strengths and weaknesses of the United States food safety model, it is easier to conceptualize the cause of gaps in food safety structure, oversight, and funding that permit adulterated foods to enter commerce and cause foodborne illness events at various stages in the food supply chain from the farms to the retail segment. Because of the variables associated with foodborne illness outbreaks past studies have examined different variables to look for correlation between specific variables and foodborne illness. Few have examined the relationship between frequency of inspection as a metric and none have examined the relationship between change in food law and risk factors.
Timothy Jones, Boris Pavlin, Bonnie LaFleur, Amanda Ingram, and William Schaffnert conducted a study in 2004, Restaurant Inspection Scores and Foodborne Disease using data from statewide restaurant inspection in Tennessee from January 1993 through April 2000 to determine if a causal relationship between inspection scores and foodborne illness existed. The report stated, “The limited data available on outbreaks in Tennessee suggests that restaurant inspection scores alone do not predict the likelihood of a foodborne outbreak occurring in a particular establishment” (Jones, 2004). “While the two most common critical violations (proper storage of toxic items and good hand washing and hygienic practices) were more likely to have been cited during the two routine inspections before an outbreak occurred at a restaurant, the number of reported outbreaks is small, and the conclusions that can be drawn from this observation are limited” (Jones, 2004). The authors suggested that further study be considered to determine if the frequency of inspection improved food safety.
A small, but important study in Washington State’s King County – A Study of Food Service Establishment Sanitation by Max Bader (1978) – examined the correlation between frequency of inspection and food safety. The study randomly selected establishment couples forming a control group and experimental group to measure the differences observed by sanitarians in the control group undergoing inspection four times a year excluding complaints and foodborne illness events and an experimental group undergoing inspection once annually with the same exclusions. The study noted a 47% higher demerit score in the experimental group with 15 foodborne illness complaints associated with the experimental group in comparison to the three complaints against the control group (Bader, 1978). While the study results are impressive, potential for bias existed in the study which was designed and conducted by the inspectors responsible for inspecting the same facilities in their district for the study. A broader sample of historical data in other regions would avoid a potential claim of this bias and could validate, add to the weight, and demonstrate the significance of the findings in the King County study. Furthermore, increased study on the frequency of inspection for wholesale and retail foods would be beneficial considering modern technologies and regulations in both segments not available in 1978.
A variable that is useful to measure the importance of regulatory inspection is the ability of FDA to remove adulterated food from commerce, by removing unsafe foods from the market foodborne illness is prevented. The USA commissioned a report in 2018 to determine if a causal relationship existed between recalls and frequency of inspection. Elina Tselepidakis Page authored a report summary from the Economic Research Service titled ‘Trends in Food Recalls: 2004-13’, in which an increase in recalls was attributed to an increase in inspection. Information collected from FDA and FSIS sources was charted and tabulated using the types of recalled Foods. Recalls were averaged from the first five years and the last five years and tested for significance according to the study (Page, 2018). “Several possible factors may explain the upswing in recalls, but conclusively stating a cause is difficult” (Page, 2018). Food expenditures increased annually by an average of 1.4 percent; however, the total number of recalls increased annually by 18 percent indicating other factors also contributed to the recalls (Page, 2018). The frequency of FDA inspection was indicated as being a possible variable. Analyzing the total number of FDA recall events against the total number of FDA inspections of domestic facilities from year to year suggested that inspections may be positively associated with recalls with results in a Pearson correlation coefficient of 0.5 (Page, 2018).
The importance of inspection and recall is significant because removing unsafe foods will reduce outbreaks. With frequency of inspections association with improved inspection scores and increase in recall, it is prudent to examine this relationship further and any preventative mechanism impacting foodborne illness resulting from increased inspection. Also, USDA has the ability to meet its mandated frequency of inspection with its workforce, it is prudent to examine the benefit of increased frequency of inspection by FDA with an increase in field inspectors comparable to the ratio held by USDA. FDA is scheduled to do high-risk facilities at least once every five years; the Food Safety Inspectional Service (FSIS) does daily inspection. The ratio of FSIS inspectors to FDA inspectors is significantly higher annually, and they inspect facilities under their jurisdiction with a higher frequency than FDA. The Federal Food Safety System: A Primer written in 2014 and 2016 provide statistics gathered from government sources on the number of inspections conducted by USDA and FDA between the Fiscal Year 2004 -2015 (Johnson, 2014-16). The lack of field inspectors is prohibitive to FDA’s meeting its mandated inspection numbers under FSMA at this time and will continue to worsen without a meaningful change in the number of inspections or registered facilities. In compiling the data excluding foreign inspections and USDA and FDA state inspections, the FSIS has approximately 1.37 field inspectors per facility in comparison to FDA’s approximately .038 field inspectors per facility. This ratio is important to consider in the number of recalls and foodborne illness events under each jurisdiction and the ability of each agency to reduce risk of foodborne illness events. Underfunded and understaffed food safety agencies have been a historic problem in the United States making timely monitoring and enforcement of foods difficult and creating the reactive rather than proactive approach to food safety currently in place.
Study Design
The study is based upon three comparable metropolitan areas as subject areas with retail food inspectional data spanning a period of three or more years examining the retail food segment of the United States food safety system. The metro areas were chosen because of the availability of data for the food inspections, the variations of food code laws during the span of years in the study, different approaches in retail food inspection, and frequency of inspection as required by each city’s laws governing retail food.
The study uses inspections of facilities licensed over the specified time frames between 2014 and 2018 with food law differences in comparable metropolitan areas.
• Assigned Violation Risk Level’s per year are tallied in each subject metro area to determine the frequency of each item observation annually (per metro area) under the state assigned risk level.
• Coded Risk Factor Items per year in each subject metro area are tallied to determine the frequency of each item observation annually (per metro area) under a common coding system (using the FDA 44 item sheet numbering system) using Boston’s risk levels as the control to determine the pattern of item observations in each state and collectively in comparison to local assignment of risk factor levels. This will identify if laws affect risk factor assignment.
• The target sample in each of the three metropolitan areas are licensed retail food establishments serving non-prepackaged foods produced on site in all licensed full service and fast service food facilities only.
• Retail Food Facilities excluded from the study models are retail markets (grocery stores, supermarkets) because Georgia assigns inspection of these facilities to the DAG for retail food inspections. All other types of retail food service not open to the public are excluded. Hospitals, correctional facilities, schools, daycares, nursing homes, assisted living, private clubs, bars not serving food, religious facilities and events, mobile food service, kiosks and carts, camps, elderly care facilities, and temporary events are excluded.
• Unplanned routine inspections only are included in the study, excluded are complaints, follow ups, or other forms of inspections.
Theoretical Framework: Correlation Relationship Studies
Further study has been called for to build upon and expand the body of empirical works to understand the influence of variables in foodborne illness. Foodborne pathogens affect ingredients from the farm to the fork resulting in the need to look at controls for the various stages in the supply chain. Food is supposed to be free of adulterants (21 U.S. Code § 342);However, inherent pathogenic risks from the environment follow the product through the food chain. At the final stages in the supply chain prevention can minimize risk of outbreak by foods that have been contaminated from the source through some form of treatment (i.e., heat treatment, freezing, or chemical intervention) or by control through risk factors at the retail food level. Consumers’ shift in habits from eating food mostly at home to eating food prepared outside of the home or simply eating food at restaurants or hybrid retail establishments has corresponded to an increase in foodborne illness from the retail food sector (DeWaal & Dahl 1996) (Jones, Pavlin, LaFleur, Ingram, & Schaffner, 2004).
FDA established risk factors to reduce the risks at the production side in the retail sector and accommodate for the lack of staffing in retail food inspectional programs (FDA, 2013 Annex 5, p. 588) and assigned different frequency of inspection based upon establishment risk categorization as classified by the FDA (for the purposes of this study as found in the 2013 Retail Food Code). Whether the illnesses are a result of primary source contamination or production contamination, risk factor controls should help to minimize foodborne illness. The shift in foodborne illness to the retail sector in restaurants has occurred prompting scholarly examination of factors for controlling foodborne illness. As outlined previously, different theoretical frameworks exist for examining food safety. The correlation relationship study is the chosen approach to look for interventions through food law assigned risk factors and frequency of inspection as the variables in food inspection that may reduce foodborne illness specific to retail food laws.
Food law defines risk factors and risk levels of items. Reassignment of risk factor levels and values due to variance in state and local law may result in the misrepresentation of risk factor patterns that contribute to foodborne illness. Past studies that researched frequency of inspection are limited with varied results; also, past studies used single jurisdictions that would not naturally demonstrate a change in frequency of inspection time frames. Time frames ranged from a one-year period for data collection, with the longest a collection of three-years of data in a single jurisdiction.
This study uses diverse populations in different geographic locations in multiple cities with different food laws over longer time frames. The broader sample size provides additional regional differences that might broaden the variable ranges for testing and analysis on possible effect of the chosen independent variables on foodborne illness. By reviewing the risk factors marked as violations with change in food law, the frequency of inspection, and incidence of foodborne illness reported between 2014 – 2018 in the subject cities respectively, it is possible to determine if a correlation between one or more variables exists.
A consideration for the choice of subject cities is the development of food service inspection programs; inspector employment and training requirements; foodborne illness data in comparable metro areas; availability of access to data for retail food inspections in the subject cities, and the number of years of reporting for the available data. Analyzing inspection data and food code regulation versions from the subject city retail food laws using descriptive and inferential statistics this study examines the frequency of inspection in years with food code changes, the risk factors marked as violations in those timeframes (assigned and coded as applicable so all risk levels are equal) and reported incidence of confirmed foodborne illness. patterns of risk factors and items observed, the focus is not on assigning causation to any specific risk factor for foodborne illness events as in other studies but the change in risk factor level assignment as laws change and overall high-risk factor effect on foodborne illness.
Different states use different versions of the food code during the same time period or a unique state food law (as in San Francisco) that complicate comparison of some inspectional data sets; therefore, coding is done to match violations under the same item number in all subject cities for comparison to the subject cities assigned risk levels by local retail food law and regulation. Using the Boston inspection form from 2018 (based on the historical Federal 44 Item Inspection Form) as the standard for coding observations it is possible to standardize the item numbering and risk factors, so all things are equal and identify differences in observation practices between the subject areas. In this manner it is possible to test for differences if any in risk factors posed by change in law.
States may arbitrarily determine what retail food inspection data to collect and risk factors are not always captured as data. Retail food reports vary from state to state and may vary from county to county or town to town. Violation Items vary in reports in different jurisdictions; some retail food reports only hold values of “In” or “Out” of compliance, others include “Not Observed”, or “Not Applicable”; some states coded violations as “Blue” and “Red” violations for critical and non-critical items, others as ‘critical’ and ‘non-critical violations risk factors. Retail Food agencies may use a grading system in inspections either numerical or alphabet, some do not use either. Inspection forms vary in all the subject states as is typical across the country, even within states (as seen in California and Massachusetts).
Reporting of foodborne illness complaints differ from city to city. San Francisco separates illness reports by confirmed Foodborne illness events where 3 or more people are confirmed ill from the same pathogenic organism (Terrence Wong Supervisor, personal correspondence, 2019). Other cities adhere to the CDC standard of 2 or more people with exception to members of the same household. CDC statistics for foodborne illness events are reviewed. A weakness in the CDC data is that it does not include all illnesses, only illnesses that are reported; also, it does not separate illnesses by cities or counties so subject city reporting from epidemiological departments is included in the study.
Rationale and Significance for the Study
Food can and does kill children, teens, adults, the elderly, and the immune compromised population in this country. The United States is currently experiencing an average of 22 foodborne illness events annually that cause illness and death to its citizens. No-one should have to become seriously ill or die because they consumed adulterated food. It is imperative we examine our food safety system to find solutions to reduce foodborne illness. This research will not be an end all solution, but rather a continuation in the ongoing effort to find solutions to improve food safety.
Methodology
The methodology is quantitative using descriptive and inferential statistics; the approach for the study is a comparative correlation/causal relationship study. The approach was chosen because of the availability of pre-existing inspection data capturing the risk factors from the multiple subject states/cities, frequency of inspection across years with changes in food code and reported foodborne illness during the same time frames for each subject state/city.
Correlation/Causal Hypothesis
By looking at correlation/causal relationship studies we can look at individual variables that when collectively analyzed can give a fuller picture of approaches for policy change that could beneficially impact food safety in the U.S. and correspond to the reduction of foodborne illness events. Looking at studies on regulatory inspections, food codes/laws, and recalls provides insight to legislators on plausible solutions for our fractured and ineffective food safety system in the U.S. that are viable options.
Hypothesis
An increase in the independent variables, frequency of food establishment inspection and/or change in food law will cause a change in the dependent variable demonstrating a change in foodborne illness rates and risk factor assignment. The null hypothesis (H0) is, “An increase in frequency of inspection and/or a change in food law will not change food establishment risk factor assignment or demonstrate a change in foodborne illness rates”.
Data and Method
Data for facility inspections in Atlanta, GA (Fulton County, most of Atlanta), Boston, and San Francisco are public record and available for analysis upon request or through public portals. The different cities’ data is downloaded as csv files and pivot tables are created to create tables, graphs, and analyze information in excel. Risk factors are coded using Boston’s item numbering system to standardize observations in Atlanta and San Francisco for a comparison between levels all things being equal. Risk levels are also tabulated as assigned per state law for comparison.
Coding provides a means of comparison between different food codes; violation observations are assigned different item numbers in different systems of food law. Subsequently true differences from state assigned risk levels and a standardized risk level can be identified. The violation items are ranked according to risk level and measured annually in each metro subject area per year.
Data from the CDC on foodborne illnesses and state epidemiology in the sample states will be cross referenced. Comparing the frequency of inspection and changes in food law identify if either variable separately or combined demonstrates a reduction of foodborne illness and change in risk factors that has been identified by FDA as having a causal relationship to foodborne illness.
Analyzing the Data
The inspection frequency for each metro area which is naturally modified by staffing levels in inspection programs will be one variable. To control for the differences in training, experience, rules applied, and local policy and practice the same jurisdictions are used over a period of three years or longer the data collected is measured and analyzed. A correlation coefficient, regression analysis test, test of average mean, in addition to other tests are used to examine the relationship between the dependent and independent variables.
Analysis & Subject City Summary
The three cities in this study Atlanta, Boston, and San Francisco represent three different geographical areas of the United States. Atlanta is in the southeastern part of the county, Boston the northeastern part of the county, and San Francisco the southwest seacoast. Along with geographic differences, the cities have political boundary division differences with both Atlanta and San Francisco being part of the county system of government and Boston part of a commonwealth system.
. . .
Analysis and Findings
Question 1. Is there a correlation between frequency of inspection and food born illness?
The Correlation Coefficient test provides statistical analysis correlation between variables. The closer the number is to one the stronger the correlation. At -0.794414077 the negative correlation between frequency of inspection and Illness per capita is very high. This would indicate that as the frequency of inspection drops the incidence of foodborne illness increases, and as the frequency of inspection rises the foodborne illness rates decrease.
Question 2. Is there a correlation in changes of risk factors as laws change?
The answer to question two is yes, there is a correlation in change of risk factors as laws change. The difference in all risk categories per law in the subject cities is demonstrated by the coded and assigned values by law as shown in table 15. The graph shows the dramatic differences based upon assigned law and coded law; Atlanta and San Francisco’s risk factors levels were coded to compare with Boston’s assigned risk factor levels as the control. Each had different food laws in place and assigned different risk values to their risk levels per violation item. A few easily identifiable examples of the differences are visualized in San Francisco’s use of three different risk levels for rodent and pest observations while Boston and Atlanta use one. Also, Atlanta assigns wiping cloth violations a moderate level assignment while San Francisco and Boston assign it a low-risk level value. San Francisco food law does not place an emphasis on managerial control to the extent that Boston does, the San Francisco Food Code was not as stringent on that item. These violation items show in different positions, representing the different emphasis assigned law placed the items at in the subject states.
Question 3. Does a change in food code correlate to risk factor changes/ foodborne illness changes independently or in conjunction with frequency of inspection?
The answer to this question is yes. The difference of means test was conducted for Atlanta, and San Francisco’s coded risk levels and the assigned risk level by state food law. The Difference in mean in Atlanta and San Francisco shows a significant difference in high-risk factors based upon assignment by a different version of food law (Boston, MA risk level assignment based upon Boston’s version of the FDA Model Food Code). This would indicate that law does impact the difference in risk factors all things being equal.
. . . The difference between coded and assigned risk levels is significantly different in Atlanta and San Francisco, with Boston as the control. San Francisco’s difference was equally as significant with an increase in 2016 from 304 to 972 high-risk factors, in 2017 a rise from 1490 to 4017, even with a less dramatic increase in 2018 from 1349 to 1593, it was still significant. All categories of risk shifted as when coded by the difference in law.
The difference in assignment of risk values is indicative of a correlation in change of risk factors as laws change; the difference noted in the mean number of coded risk items to assigned in Atlanta . . . demonstrates how significant the change of risk factors by law can be.
Furthermore, Boston has the highest mean frequency of inspection and the lowest mean Illness (per capita), next Atlanta, then San Francisco; Boston has the highest mean violations and lowest mean Foodborne Illness rates (per capita), next Fulton, then San Francisco.
For question three, using the correlation testing function in Excel with the mean high-risk data and frequency of inspection collected from the subject cities databases. . . . (There is) a high negative correlation between frequency of inspection and Illness per Capita; as the frequency of inspection rises the rate of illness decreases and as the rate of inspection decreases the rate of illness increases.
While High Risk Factors show a strong negative correlation to foodborne illness per capita it shows a strong positive correlation to frequency of inspection. This would suggest that the more inspection the more likelihood of capturing high-risk violations for corrective action with high-risk factors acting as a preventative mechanism toward preventing foodborne illness. . . . The relationship between high-risk factors responsible for foodborne illness, complaints, and frequency of inspection; the tables demonstrate how the preventative mechanism of reporting of risk factors for correction impacts foodborne illness when an increase or decrease in the reporting is observed that hinders or assists the function of the preventative mechanism of risk factor reporting. There is a strong negative correlation between frequency of inspection and foodborne illness; as frequency of inspection goes down foodborne illness increases. The data indicates that high-risk factors independently and in conjunction with frequency of inspection factor into foodborne illness rates.
Null Hypothesis
The answers to questions 1 – 3 were all yes so, we can reject the null hypothesis (H0), “An increase in frequency of inspection and/or a change in food law will not change food establishment risk factor assignment or demonstrate a change in foodborne illness rates”.
Summation
The analysis demonstrates that frequency of inspection has a high negative correlation relationship with foodborne illness as frequency of inspection increases or decreases; foodborne illness will travel in the opposite direction. Every year that the frequency of inspection rose in the subject cities the per capita foodborne illness rate decreased. In the years where frequency of inspection was stagnant the foodborne illness rates were relatively unchanged in the subject cities; this is reflected in the correlation coefficient test. Frequency of Inspection and High-Risk factors have a high positive correlation relationship, the more frequent the inspections, the higher the incidence of high-risk factors was noted. Risk factors are used to reduce foodborne illness, they are a preventative control. By correcting these critical items an establishment can theoretically prevent foodborne illness.
High-risk factors had a high negative correlation with foodborne illness in the years that foodborne illness decreased demonstrating it functioned as the preventative control it is meant to be. Risk factors were coded by assigned values by state law and under a common numbering system. The design was to see if food law changed the assignment of the risk factors. The coded (control) law and assigned law risk factors did demonstrate a significant difference of means in each year of the study establishing that food law does influence risk factor assignment.
Recommendations for Future Study
One of the leading drivers to the success of this study was the ability to access research material through databases. The retail food system would benefit from a study on a viable national electronic inspection platform and information database system, accessible across all states by multiple agencies in all retail food program nationally. The sharing of information in real time is key to identifying factors that can mitigate risks of foodborne illness events, not only at the local level but nationally.
It is through data sharing that we can create a dynamic food safety system that can impact food borne illness, necessary policy change, and make the most of scares resources in the public health sector. Good policy is always data driven.
A recommendation for future study would be examining different models for Retail Food Programs and State Food Protection Programs and their interactions to determine the most effective and efficient model that could be implemented across the states, territories, and districts.
To correct for differences in food law, one possibility may be a stand-alone federal Act or amendment to the Food Drug & Cosmetic Act to include Retail Food. In the subject cities there were different model versions in the different states. With the FDA Model Retail Food Code changing every four years with amendments every two years it can be burdensome for states to constantly have to go through the legislative process to change versions of the Food Code. This is subject to state legislative processes that may not be bypassed. It can take several years to update versions and is a costly and time-consuming process for departments that are already facing challenges from scarce resources. Several Senators including Senator Durbin have proposed a single food agency over decades to correct for redundancies, deficiencies, and create a uniform food code; the Retail Food program could be a part of that in the future.
A key issue brought up in the GAO MWD-76-42 report in 1975 was the lack of support from the federal to the state level to the local level for retail food programs; this problem has not been addressed to date rendering the same ineffective retail sanitation as found in 1975. It is time to provide data to create solutions to integrate federal, state, and local retail programs and determine the necessary funding required to present to Congress as outlined in the 1975 report.
(To sign up for a free subscription to Food Safety News,click here)