Questions and Answers on the FDA Report on the Occurrence of Foodborne Illness Risk Factors in Selected Institutional Foodservice, Restaurant, and Retail Food Store Facility Types (2009)
The U.S. Food and Drug Administration’s (FDA) National Retail Food Team has released a report on certain food preparation practices and employee behaviors in 9 food facility types that are most frequently reported to the Centers for Disease Control and Prevention as contributing factors to foodborne illness outbreaks. The following questions and answers may help provide a better understanding of the report and its contents.
1. What is the purpose of this study?
This 2009 report is the third report in a series and presents findings based on data collected in 2008. The first report in the study was issued in August 2000 and presented the findings from the first data collection effort in 1998. A second report was issued in 2004 and presented data collected in 2003. The study is designed to collect data in selected foodservice and retail food establishments at five-year intervals. These data collection efforts are designed to get an accurate picture of the extent to which foodservice and retail food operations have active managerial control over contributing factors to foodborne illness. Using data from multiple collection periods, FDA hopes to evaluate trends and determine whether progress is being made toward the goals of reducing the occurrence of food safety practices and employee behaviors that could contribute to foodborne illness.
2. How does FDA define a foodborne illness risk factor?
In developing the design for this study, the U.S. Food and Drug Administration's National Retail Food Team used information from the Centers for Disease Control and Prevention's (CDC) Surveillance Report for 1988 - 1992 to identify which food safety practices should be the focus of its data collection efforts. This CDC surveillance report identified food safety practices and employee behaviors most associated with foodborne illness outbreaks. Some of these reported contributing factors relate to food safety concerns within foodservice and retail food store facility types and include: food from unsafe sources, improper holding/time and temperature, inadequate cooking, poor personal hygiene, and contaminated equipment/prevention of contamination.
For the purposes of this study FDA designated these five areas as foodborne illness risk factors. FDA added another category to assess control of potential chemical hazards. It is important that one not confuse the term risk factor as it is used in this FDA study with another use of the same term which is to describe characteristics of the person, food, or location that are statistically linked to an illness by an epidemiological investigation. The results for the FDA 2009 Report should not be used to infer a correlation between the occurrence of a foodborne illness risk factor with the actual occurrence of human illness.
3. What is the scope of this study?
The 2009 Report provides a summary of not one FDA study but 9 separate studies, one for each of the nine facility types. These 9 facilities types are drawn from 3 targeted industry segments - Institutional Foodservice, Restaurants and Retail Food Stores.
- Nursing Homes
- Elementary Schools
- Fast Food
- Full Service
Retail Food Stores
- Deli Operations
- Meat and Poultry Departments/Markets
- Seafood Departments/Markets
- Produce Departments/Markets
The results from the data should not be used to compare one facility type with another. Different food safety practices and employee behaviors are observed within each of the 9 facility types.
The data presented in the 2009 study were collected in 2008 during 850 inspections of foodservice and retail food facilities across the U.S as part of a 10-year plan for data collection during 1998, 2003, and 2008.
4. How are observations of food safety practices and employee behaviors recorded for later analysis?
The data collection tool used to record the data for the study includes 42 individual data items sorted among the 5 foodborne illness risk factor categories and the other chemical category. Dividing each of the foodborne illness risk factors into individual data items allows the study to assess what specific food safety practices and employee behaviors are in most need of attention.
For each individual data item, the data collector determined whether it was:
- IN - In Compliance with 1997 Food Code provisions;
- OUT - Out of Compliance with 1997 Food Code provisions;
- N.O. - Not Observed. The N.O. marking is used when an item is a usual practice in the foodservice or retail food operation, but the practice was not observed during the time of inspection; or
- N.A. - Not Applicable. The N.A. marking was used when an item was not part of the foodservice or retail food operation.
The same 42 data items were assessed for each of the data collection periods in the study (1998, 2003, and 2008). The critical limits and standards contained in the 1997 FDA Food Code were used to assess each of these 42 data items for all 3 data collection periods.
5. Since the FDA Food Code is updated on a periodic basis, why continue to assess the 42 data items using the 1997 Food Code criteria?
One of the purposes of the project is to track changes related to industry's efforts to control foodborne illness risk factors over time. For this type of study, the standard of measurement used to evaluate these comparisons - in this case the 1997 Food Code - must remain constant for each of the data collection periods.
Suppose the recommended standard in the Food Code was relaxed between data collection periods in the study and the data item was changed accordingly. Let's say that the next data collection shows an improvement in the IN Compliance percentage for that data item. One would not know if the improvement was due to an improvement in employee food safety practices or whether it was due to the change in the Food Code standard.
The study is designed to track changes in industry's active managerial control of food safety practices and employee behaviors; not simply compliance with regulatory standards.
6. Won’t changes to the Food Code influence the compliance percentages for the risk factors and therefore the study results?
FDA has assessed what impact changes to the Food Code may have had on the compliance percentages for individual data items and foodborne illness risk factors.
The data collection form contains a section titled, Supplemental Items. These additional items were added to assess some industry food safety practices not addressed in the first report and determine what impact, if any, changes in the Food Code may have on the data results. Only a couple of data items from the original study contain standards or critical limits that have been changed since the release of the 1997 Food Code.
7. How is the data collected from field observations used to determine the compliance percentages presented in the report?
Three levels of data analysis are conducted for each of the 9 facility types:
- An Out of Compliance percentage is determined for each of the individual data items;
- An Out of Compliance percentage is determined for each foodborne illness risk factor; and
- An overall IN Compliance percentage for all the 42 data items is determined.
A. Out of Compliance Percentage - Individual Data Items
The Out of Compliance percentage for an individual data item is calculated by dividing the total number of observations that are Out of Compliance by the total number of observations (both IN and OUT). Not Observed and Not Applicable markings are not used to determine Out of Compliance percentages. This is because the study is designed to be an assessment of actual observations.
Note that an observation is defined as the totality of the evaluations of a single data item for one establishment. For example, if handwashing was evaluated 10 times in an establishment and one or more of these evaluations was "out of compliance" then the observation would be considered "Out of Compliance". If all 10 evaluations were "in compliance", then the observation would be considered "IN Compliance".
The Out of Compliance percentage for a data item represents the proportion of establishments where that data item was found Out of Compliance at least once when the practice or procedure could be observed. For example, the report shows a 52% Out of Compliance percentage for proper, adequate handwashing in deli operations. This percentage should NOT be interpreted to mean that food workers were not washing their hands properly 52% of the time.
What the 52% Out of Compliance percentage means is that a little over 5 out of 10 deli establishments where handwashing was observed, there was at least one instance of improper or inadequate handwashing.
B. Out of Compliance Percentage - Foodborne Illness Risk Factors
The second level of analysis is to determine the Out of Compliance percentage for each of the foodborne illness risk factors for each of the facility types. Each of the risk factors is comprised of a number of individual data items. The Out of Compliance percentage for each foodborne illness risk factor is determined by dividing the total number of Out of Compliance observations of all the data items that comprise a risk factor by the total number of observations (both IN and OUT) made of those data items.
For example, the Poor Personal Hygiene risk factor is comprised of 5 data items. The Out of Compliance percentage is determined by dividing the total Out of Compliance observations of these 5 data items by the total number of observations (both IN and OUT) of the 5 data items.
C. Overall IN Compliance percentage - each of the 9 facility types
The third level of data analysis is to determine the Overall IN Compliance percentage for each of the 9 facility types. This is done by dividing the total number of IN Compliance observations of all 42 data items by the total number of observations (both IN and OUT) made of those data items. The 1998 Overall IN Compliance percentage represents the study's Baseline measurement for each facility type.
8. What are the limitations of this study?
Two limitations were imposed by the nature of the inspection process. First, the timing of the inspection could not always coincide with some of the procedures FDA wanted to observe, such as an establishment receiving food items from their suppliers or the end-of-the-day breakdown and storage of food products.
Second the length of inspection time did not always provide an opportunity to observe completion of an important food process. This is particularly true of the cooling process for cooked foods.
Although the data on foodservice and retail food establishments throughout the country have been collected for this study, it is not designed to support comparisons of the occurrence of foodborne illness risk factors across states, counties, cities, or regions of the U.S. Similarly there is not enough data to justify comparison of chains of restaurants or chains of grocery stores.
9. Since some of the same recommendations were issued in the 2000, 2004 and this report, does this mean the industry has not followed past recommendations on proper food handling?
The data presented in this 2009 report indicate that some of the same risk factors and data items identified as problem areas in the 2000 and 2004 Reports remain in need of priority attention. In general, Out of Compliance percentages remained high for data items related to the same risk factors identified in the 2000 and 2004 Reports and they are:
- Improper Holding/Time and Temperature
- Poor Personal Hygiene
- Contaminated Equipment/Protection from Contamination
Results from the 2008 data collection indicate that the recommendations made to foodservice and retail food operators and regulators in the 2000 and 2004 Reports need to be reemphasized. Foodservice and retail food store operators must ensure that their management systems are designed to achieve active managerial control over the risk factors. Likewise, regulators must ensure that their inspection, education, and enforcement efforts are geared toward the control of the risk factors commonly found to be Out of Compliance.
Many establishments with the foodservice and retail food segment of the industry have taken proactive measures to enhance the effectiveness of their food safety management systems. At the same time, more and more regulatory agencies are incorporating a risk-based inspection methodology into their food inspection programs. For more information on regulatory agencies that have adopted the FDA Voluntary National Retail Food Regulatory Program Standards, go to: http://www.fda.gov/Food/FoodSafety/RetailFoodProtection/ProgramStandards/ucm121796.htm
10. Are Certified Food Protection Managers helpful in a food establishment?
This study was not designed to definitively determine whether having a Certified Food Protection Manager on-site has an effect on the overall IN Compliance of establishments. However, the results of the study indicated that the presence of a Certified Food Protection Manager is positively correlated to the overall IN Compliance percentages in certain facility types. Four facility types had overall IN Compliance percentages (all 42 data items combined) that were significantly higher in establishments with a Certified Food Protection Manager than in establishments without a Certified Food Protection Manager. These facility types include:
- Full Service Restaurants
- Seafood Departments/Markets
- Produce Departments/Markets
In the five remaining facility types, the differences were not statistically significant.
Poor Personal Hygiene, Improper Holding/Time and Temperature, and Contaminated Equipment/Protection from Contamination appear to be the risk factors for which the presence of a certified manager had the most positive correlation. Full Service Restaurants with a certified manager showed a positive correlation for the all three of these risk factors. Delis showed a positive correlation for the Poor Personal Hygiene and Improper Holding/Time and Temperature risk factors. There was no risk factor for which the IN Compliance percentage for establishments without a certified manager exceeded the percentage for establishments with a certified manager in a statistically significant manner.
11. What should I do with this report?
For many food safety professionals there will be a normal and understandable urge to focus on the out of compliance percentages for the risk factors and data items. This fixation on compliance percentage numbers can, however, draw attention from one of the more important uses of the data results. The 2009 Report should be used as a strengths/weaknesses assessment of a regulatory retail food protection program or an industry food safety management system.
For industry, when the study identifies a food safety practice or employee behavior as having a high out of compliance percentage, the question to ask is what procedures or processes are currently in place to ensure active managerial control of that specific item? Assess existing food safety management systems to ensure there are not any gaps in control measures for the specific item identified.
For regulatory food safety professionals, an examination should be made of current food inspection protocols to ensure that food safety practices and employee behaviors with high out of compliance percentages are inspection priorities. Is time being taken to assess industry's active managerial control of these areas?
The 2009 Report is intended to be read, and the data interpreted, as a stand alone study. It provides important insights into the effectiveness of regulatory and industry efforts to enhance the level of public health protection within foodservice and retail food store facility types. If the nation's foodservice and retail food protection system is to be significantly improved, the industry and regulatory communities must remain focused in their efforts to reduce the individual data items identified in the 2009 Report as having the most prominent Out of Compliance percentages. FDA intends to publish a report in 2010 that analyzes the data from the three data collection periods to determine if any statistically significant improvement and/or regression trends in the control of foodborne illness risk factors occurred during the 10-year study period.