Evaluation and Definition of Potentially Hazardous Foods - Chapter 7. Comparison of NSF and ABA Protocols to Determine Whether a Food Requires Time/Temperature for Safety

Table of Contents 

1. Introduction

Both the American Bakers Association (ABA) and the NSF International (NSF) have written protocols describing microbiological testing to determine whether certain foods require time/temperature control for safety (ABA 2000; NSF 2000. The ABA document is strictly devoted to testing pumpkin pie, while the NSF document addresses breads with vegetables or cheese added before baking, breads filled after baking, pies filled before baking, and toppings destined for use in other products. The ABA protocol can be obtained by calling ABA at 1-202-789-0300. The NSF protocol may be ordered by calling NSF at 1-800-NSF-MARK or via the website at

Both the ABA and the NSF testing protocols suffer from significant weaknesses that hamper their usefulness in determining whether a food can be safely stored at room temperature. The NSF protocol takes an overly stringent approach, whereas the ABA protocol is sometimes overly permissive. The two most significant differences between the two protocols are 1) the consideration (or lack of consideration) of the process the food did or will undergo, and 2) the selection of microorganisms used or not used to inoculate the food. Table 1 of this chapter presents a comparison between the features of the testing protocols, including the protocol developed by the panel (see Chapter 6).

2. Consideration of process

A significant difference between the two protocols is the consideration given to the processing method in the ABA protocol and the lack of consideration of process in the NSF protocol. A given process/packaging combination may serve to eliminate a particular pathogen from a food product. The post-process reintroduction of this pathogen in a challenge test may represent an artificial situation and not what may actually happen. A challenge test that inoculates a pathogen into a processed food may be unduly challenging if post-processing contamination is not likely. It should be noted that some non-PHF foods on the market today might not be able to pass such stringent test criteria. For example, while currently excluded from consideration as a PHF under the NSF protocol, if required to undergo the NSF protocol, white bread might not be able to pass such test criteria despite a well established safety record.

3. Microorganisms used

A second significant difference between the two protocols is the use of an inoculum. The ABA protocol uses only the natural microflora present in the product, and requires testing for aerobic plate count (APC), coliforms, Staphylococcus aureus, and Salmonella spp. The NSF protocol requires the use of five strains each of Bacillus cereus, Escherichia coli O157:H7, Listeria monocytogenes, Salmonella spp., S. aureus, and Clostridium perfringens, depending on the pH and aw of the product. The advantage of the ABA approach is its simplicity, but it is probably too simple, and relies on natural or accidental contamination events occurring in those batches of product produced for testing in order to detect a problem. Let us assume a product being produced for evaluation by the ABA protocol is inappropriately handled such that it is recontaminated with S aureus on a recurring but infrequent basis (for example, 1/100 containers). The ABA protocol will not appropriately evaluate this potential problem since the product used for the study is not deliberated inoculated and the chance of using the contaminated product is very low (for example, 1/100). If thousands of containers are produced on a daily basis, this may be enough to present a health concern to consumers.

The NSF protocol can be criticized from the opposite stanpoint: it is unduly stringent. A food must be inoculated with the appropriate pathogens among those listed above, depending on the pH and aw of the food. Inoculation is required even if none of the pathogens are commonly found in any of the product ingredients, or if one or several would be eliminated by processing. Both approaches, however, suffer from the lack of inclusion of Clostridium botulinum as a test organism. Inclusion of C. botulinum in a challenge study greatly increases its cost and complexity, but with these increases there is a concomitant increase in confidence that the appropriate organism is being used. While the NSF protocol includes C. perfringens for certain products due to concerns in baked goods, it is not meant to be used as a surrogate for C. botulinum. The panel agrees that because these organisms differ in cold sensitivity, heat resistance, rate of growth at various temperatures, oxygen tolerance, and toxin mode of action, C. perfringens should not be used as a surrogate for C. botulinum.

4. Pass/fail criteria

Given the differences in microbial testing between the ABA and NSF protocols, differences in pass/fail criteria are expected. A product will fail the ABA protocol if it contains detectable S. aureus, Salmonella spp. or coliforms, or if it contains more than 1000 CFU/g within 24 h of packaging, or more than 100,000 CFU/g at the end of its shelf life. Testing for the presence of pathogens in an uninoculated product is not sufficient to determine whether the product requires time/temperature control for safety. Aerobic plate count (APC) data may be useful in determining product quality during shelf life, but these data are of limited value as indicators of safety. Aerobic plate counts may be useful if there is enough product history to suggest that no, or minimal, increases in APC always indicate no pathogen growth, and all pathogens of concern would be counted on APC. Conversely, an increase of APC on uninoculated product does not indicate that pathogens could not grow if present.

Failure criteria for the NSF protocol are based on increases in counts of any of the six pathogens tested. A food fails in the NSF protocol if it supports more than 1 log CFU/g increase by the end of its shelf life, or more than a 1 log CFU/g increase for any two consecutive time points. The 1 log CFU/g increase criterion for infectious pathogens is appropriate for two different reasons. First, a 1 log CFU/g increase is likely to be detected in spite of the inherent variability known to exist in microbial testing methods today. To use a smaller value might invite concerns regarding whether a particular increase was of statistical significance. Second, given our current level of understanding regarding human dose-response for enteric pathogens, a 1 log CFU/g increase probably constitutes a measurable increase in risk. It is not clear why the NSF protocol allows a 1 log CFU/g increase that subsequently declines during the shelf life, but does not allow a 1 log CFU/g increase that is observed at two time periods, except perhaps to allow for inherent sample and analytical observation variations. The NSF protocols are too stringent with respect to S. aureus, B. cereus or C. perfringens because a 1 log CFU/g increase in any of these organisms is unlikely to result in a public health concern (see discussion in Chapter 6).

5. Number of sampling times

Sampling time differences also exist between the two protocols. Neither of the protocols proposes the presence of toxin in the food as a valid criterion. The ABA protocol advocates microbiological testing at only two times (within 24 h post bake and at the end of shelf life) while the NSF protocol advocates 1-10 testing times depending on shelf life. The appropriate number of test observation times is dependent upon the failure criteria. If the failure criterion is detectable toxin, it may be sufficient to simply test a suitably inoculated product at the end of its shelf life. If no toxin is detected, the product passes the challenge test. It might also be appropriate to sample at additional times and adjust the shelf life of the product such that toxin production does not occur during this time. The appropriate number of observations in challenge tests with vegetative cells is significantly more complex. A product should be tested at a sufficient number of time points to insure that a one log CFU/g increase has not occurred. It should also be noted that tests must continue until the end of the test period, even if the inoculated organism declines below the level of detection, to insure against the "Phoenix" phenomenon (Jay 1996).

6. Replication

Both protocols require six replicates: all from one production run for the ABA protocol; two samples each from three lots for the NSF protocol. The decision about the appropriate number of samples and lots must be based on the characteristics of the food and microbes in question, but two samples each from three lots is probably a reasonable minimum. In worst-case scenarios and considering variation and process capability and tolerance, it may be more appropriate to test a greater number of random samples from each lot.

7. Oxidation-reduction potential

The ABA protocol requires evaluation of oxidation-reduction potential (Eh) as a means of controlling risk of C. botulinum, whereas the NSF protocol does not. While the ABA protocol proposes a stringent value for Eh (+100 mv or greater), there are still some important limitations to this approach. Eh values of +100mv or greater are not inhibitory to C. botulinum type E. Although this organism is not expected to be found in pumpkin pie, it might be encountered in marine foods that require time/temperature control for safety. Eh is also notoriously difficult to measure accurately, and erroneous measurements may lead to a false sense of security. Finally, the Eh of the micro-environment may not be reflected by standard measurements. If C. botulinum is a concern, the only reliable means of determining the safety of a particular food are challenge studies using this organism.

8. Methodology

The ABA protocol advocates the use of FDA's Bacteriological Analytical Manual (BAM) and Association of Official Analytical Chemistry (AOAC) methodology, while the NSF protocol uses the Compendium of Methods for the Microbiological Examinations of Foods. The differences between these methodologies are largely inconsequential. It is critical not that one method be used over another, but that some reproducible, commonly accepted, and widely used method be employed. AOAC, BAM, and the Compendium methods all satisfy this requirement.

9. Inoculum

Since the ABA protocol does not use inoculated organisms, the question of inoculum preparation and position is irrelevant. The NSF protocol takes an overly stringent approach by requiring each component and each unique component interface to be inoculated. This requirement ignores the fact that in many cases a properly processed product should not contain contamination with vegetative cells on any internal surfaces. The NSF approach, however, may be appropriate for products in which post-processing contamination may occur at internal surfaces. Other problems include the use of a phosphate buffer that may modify the food microenvironment, and the use of high levels of challenge microbes that could locally overwhelm the preservative system.

10. Duration of test

The two protocols use similar criteria to establish the duration of the test. The ABA protocol tests the product up to the "use by" date, which is 1.3 times the "sell by" date, while the NSF protocol requires that a test last 1.3 times as long as the time period that the product will be outside temperature control. A useful and valid test protocol should last slightly longer than the time period of concern. In the absence of any scientifically valid documentation on this matter, 1.3 times as long as the time period that the product will be outside a temperature control seems as reasonable to use as any criterion.

11. Product categories

Neither protocol addresses all of the product categories the panel was asked to consider by the FDA. The ABA document has a narrow focus (evaluation of pumpkin pie), while the NSF document is somewhat broader (evaluation of breads with vegetables or cheese added before baking, breads filled after baking, pies filled before baking, and toppings destined for use in other products). Neither protocol include such food items as cheeses or fruits and vegetable products. A testing protocol should be flexible and robust enough to use with any food product where safety out of time/temperature control is questioned. However, a universal protocol may be impossible to develop. In some instances, different challenge study protocols will need to be used for different foods. A well thought-out generic protocol should satisfy the desired criteria of flexibility and robustness to the greatest extent possible.

12. Summary

Both the ABA and NSF protocols have some significant weaknesses. An alternative protocol that considers the complementary strengths and weaknesses of the ABA and NSF methods, with the few minor additions noted above, can be used to determine which foods require time/temperature control for safety. The panel's recommendations, summarized in Table 1 below, can be seen as an alternative protocol.

Table 1. Summary of comparison of NSF, ABA and expert panel protocols to determine if a food requires time /temperature control for safety.
Item ABA NSF Panel's Alternative Protocol
Type of product Pumpkin pie Four groups: bread with vegetables and cheese pre-bake, filled post-bake, filled pre-bake, toppings. Traditional and other products excluded. Any food product proposed to be stored outside temperature control.
Consideration of process Yes (Good Manufacturing Practices, [GMP's], baking temperature, cooling, and packaging) No Yes. Additional information for validation of process also required.
Microorganisms tested Aerobic Plate Counts (APC), Staphylococcus aureus, coliforms, salmonellae Bacillus cereus, Escherichia coli O157:H7, Listeria monocytogenes, Salmonella spp., S. aureus, Clostridium perfringens, depending on pH and aw Organisms should be selected based on history of safety, formulation, storage atmosphere environment and packaging of the food.
Inoculation type None (indigenous only) Composite of 5 strains of each organism. Each composite inoculated into the product separately. Composite of multiple strains of each organism. Each composite inoculated into the product separately.
Inoculation method Not applicable Prescribed in phosphate buffer. Prepared in system that mimics the product: Previously mixed with buffer or water, directly added to product, aseptically injected, mixed powder product, or lyophilized, depending on the product.
Inoculum preparation Not applicable Aerobes cultured in tryptic soy broth, C. perfringens cultured in fluid thioglycolate (sp?) broth Cultures grown in suitable media under either optimal or food-adapted conditions. Spores are washed and heat-shocked before or after inoculation.
Inoculum position Not applicable Each unique component and each unique interface between components at both internal and external surfaces Each component, and each unique interface between components, but only where the organisms of concern would survive the process or be reintroduced post-processing.
Inadvertent product modification Not applicable Addition of the inoculum in buffer has a potential to change product water activity. Additional measurements of aw should be taken to insure that inoculation technique does not influence product aw.
Inoculum technique Not applicable No consideration for relative component weights given when splitting inoculum between components. Not applicable
Sampling times Two: within 24 h of packaging, and at end of shelf life One to ten: depending on intended shelf life A minimum of 5 times over shelf life, including zero time.
Pass criteria No pathogens detected, APC less than 1,000 CFU after bake, and less than 100,000 CFU at end of shelf life Less than 1 log CFU increase for any pathogen by the end of the study and not to exceed 1 log CFU for any pathogens at two consecutive time points before the end of the study. Depending on the pathogen, less than 1 or 4 log increase at any point in shelf life for vegetative pathogen(s) of concern and no detectable toxin at the end of the shelf life for toxin-forming microbes.
Other tests Oxidation/Reduction potential, pH, aw pH, aw pH, aw, pH/aw interaction,
Process Process is considered by use of natural inoculum. Process is not considered, since pathogens are inoculated into the food after processing. Process should be considered in the selection of appropriate microbes for use in the challenge study. Data to validate the process should be provided.
Methods Association of Official Analytical Chemistry/Bacteriological Analytical Manual Compendium of Methods for the Microbiological Examinations of Foods Any reproducible, validated method is acceptable.
Duration of study The study lasts until the use by date, which is calculated by multiplying 1.3 times the sell by date. The study lasts 1.3 times the time the products will be out of temperature control. The study should last for at least the shelf life of the product, but 1.3 times the intended shelf life is recommended.
Spoilage Addressed indirectly with APC Not applicable Testing of inoculated sample for background bacteria
Replication 6 samples at beginning and 6 at end of one production run 3 lots, 2 samples/lots, over shelf life Minimum of 3/sampling time unless this is a revalidation study or control sample (less samples are needed)
Anaerobes Only an O/R potential measurement is made, no microbial tests are done. C. perfringens C. botulinum itself is used, with toxin production as the definitive measure of safety.
Microbial growth modeling Not applicable Not applicable Properly validated growth models can be used alone or in combination with microbial challenge studies.
History of safe use Not applicable Not applicable A long history of safe use can be considered in combination with appropriate scientific rationale instead of challenge studies.


[ABA] American Bakers Association. 2000 Jan 18. Industry protocol for establishing the shelf stability of pumpkin pie [final version plus executive summary]. Washington (DC): ABA. 18 p. Available from:

Jay JM. 1996. Modern food microbiology. 5th ed. New York: Chapman & Hall. Chapter 10, Culture, microscopic, and sampling methods.

[NSF] NSF International. 2000 Nov. 10. Non-potentially hazardous foods. Ann Arbor (MI): NSF International. Report nr ANSI/NSF 75-2000. 12 p.

Page Last Updated: 08/10/2015
Note: If you need help accessing information in different file formats, see Instructions for Downloading Viewers and Players.
Language Assistance Available: Español | 繁體中文 | Tiếng Việt | 한국어 | Tagalog | Русский | العربية | Kreyòl Ayisyen | Français | Polski | Português | Italiano | Deutsch | 日本語 | فارسی | English