Analysis of Results for FDA Food Defense Vulnerability Assessments and Identification of Activity Types
This report is required under Section 106 of the Food Safety Modernization Act. Over the past several years, FDA has conducted vulnerability assessments (VA) on more than 50 products or processes, leading to the identification of processing steps of highest concern, and potential mitigation strategies to reduce vulnerabilities. The current study utilized the results from 25 VAs to determine if a potential “threshold” score for the implementation of mitigation strategies could be identified.
Section 106 of the Food Safety Modernization Act (FSMA), requires FDA, among other things, to conduct a vulnerability assessment (VA) of the food system, A VA is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system.
From 2005 to 2008, under the Strategic Partnership Program Agroterrorism (SPPA), the U.S. Food and Drug Administration (FDA), along with the United States Department of Agriculture (USDA), Federal Bureau of Investigation (FBI), and Department of Homeland Security (DHS) conducted vulnerability assessments (VAs) on products, processes, or commodities in the food and agriculture sector. In keeping with the requirements of Homeland Security Presidential Directive 9 (HSPD-9), Defense of Agriculture and Food as well as the requirement under FSMA, FDA reassessed VAs conducted from 2005 to 2008 and continued this assessment process for products not previously assessed.
To date, FDA has conducted vulnerability assessments on more than 50 products or processes, which has led to the identification of processing steps of highest concern, potential mitigation strategies that may reduce these vulnerabilities, as well as research gaps.
The methodology used to conduct these VAs is called CARVER+Shock. CARVER is an acronym for the following six attributes used to evaluate the attractiveness of a target for attack:
- Criticality - measure of public health and economic impacts of an attack
- Accessibility - ability to physically access and egress from target
- Recuperability - ability of system to recover from an attack
- Vulnerability - ease of accomplishing attack
- Effect - amount of direct loss from an attack as measured by loss in production
- Recognizability - ease of identifying target.
A seventh attribute, “Shock,” was added to the original six attributes to assess the combined health, economic and psychological impacts of an attack within the food industry. CARVER+Shock is a tool that can be used to assess the vulnerabilities within a system or infrastructure. By conducting a CARVER+Shock assessment of a food production facility or process, the user can determine the most vulnerable points in the infrastructure, and focus resources on protecting the most susceptible points in the system.
The current study utilized the results from 25 VAs to determine if a potential “threshold” score for the implementation of mitigation strategies could be identified. The analysis of Criticality, Accessibility, and Vulnerability (CAV) scores showed that since CARVER+Shock is a relative risk ranking tool, there is no equivalence between a score value of a processing step in one industry to the same score value for another processing step in a different industry. The data set was then reevaluated to determine what common attributes or activities occurred between processing steps in the data set. Through this review, it was determined that processing steps could be grouped according to the type of activity occurring at a particular point in an operation.
The key activity types in most production environments are:
- Ingredient Staging/Prep/Addition
- Liquid Receiving/Loading
- Liquid Storage/Hold/Surge Tanks
By identifying and defining key activity types, a CAV score is no longer needed as a threshold. The activity types and their descriptions can be publicly disseminated, industry can objectively map processing steps into activity types and any mitigation strategies associated with those activity types could be audited.
FDA, with support from Battelle Memorial Institute, reviewed the Criticality, Accessibility, and Vulnerability (CAV) scores from 25 vulnerability assessments to determine a potential “threshold” score for the implementation of food defense mitigation strategies. Other CARVER+Shock scoring components (Recuperability, Effect, Recognizability, Shock) were not included. The data set includes scoring results derived from the following vulnerability assessments:
- VAs conducted under the original SPPA initiative when an updated VA was not conducted;
- Updated VA results from a previous SPPA assessment, and;
- New VAs conducted since the conclusion of the SPPA initiative.
The CAV score results for each assessment were reviewed and only included processing steps for further analysis if they were part of the top twenty-five percent of CAV scores within a vulnerability assessment. Where there were ties at the bottom of the quartile, all processing steps with the same score were included. This selection constituted the data set for further analysis and consisted of 141 of the 465 scored processing steps (or approximately 30% of all scored processing steps from 25 assessments).
When the data set is sorted by CAV score, some processing step types repeatedly rise to the top. Forty-seven processing steps had CAV scores of 26 or above (top quartile of the analyzed population). Fourteen of the 47 processing steps involve mixing, grinding, or coating as the primary function, thus resulting in probable homogeneous distribution of a threat agent into the product. Twelve of the 47 processing steps involved the staging, preparation, or addition of minor ingredients. Six of the 47 processing steps involved receiving and five of the 47 processing steps involved storage. The 10 remaining processing steps were an assortment of other activities. When ordering by CAV score, the processing steps where mixing occurs or secondary ingredients are staged, prepped or added, prove to be critical processing steps in many assessed products. Even though assessment scores are independent from other products/assessments, the high CAV scores in these types of processing steps indicate that attention should be placed on these areas when considering food defense mitigation strategies.
Additionally, when comparing across industries, it was found that 65% of products were assigned a high CAV score of 26 or above (17 of 25 products). It should also be noted that several products had multiple processing steps tied for highest rank. However, for industries where the high CAV score was not 26 or above, the highest scoring processing step(s) in the assessment of these products should still be regarded as highly sensitive within that assessment and should not be considered less risky for intentional contamination.
It was also found that the higher range of CAV scores (26 or above) was dominated by assessments of manufacturing facilities generating processed, consumer-ready foods. This may help in targeting industries where more attention should be focused when considering the development of mitigation strategies.
CAV scores obtained and used in this analysis were generated during the conduct of the assessments with experienced, unbiased facilitators who were highly trained in the CARVER+Shock process. In addition, the CARVER+Shock tool is a relative ranking tool and was never intended to be used to compare disparate industries. Since CARVER+Shock is a relative ranking tool there is no equivalence between a score value of a processing step in one industry to the same score value for another processing step in a different industry. CARVER+Shock and CAV scores can vary between repeat assessments of the same product, as well. During a repeat or update assessment, the relative ranking may remain the same, but assessments conducted at different times with different participants frequently receive different scores values. This could result in a processing step being scored above a CAV score threshold in one assessment and fall below the threshold in a later reassessment.
Also, when conducting the assessments, the facilitators and participants did not have a financial interest in targeting scores for processing steps. Were a CAV score threshold to be used as a factor for requiring industry to institute mitigation strategies, CAV scores could be manipulated to fall below any threshold. Also, since the CARVER+Shock Software tool was developed to be used by individual companies, answers to the questions with the software tool could be changed to generate lower scores.
The data set was reevaluated to determine what common attributes or activities occurred between processing steps in the data set. A review was conducted of the processing step descriptions and CAV score spreadsheet for all upper quartile scoring processing steps within an assessment. Through this review, it was determined that processing steps could be grouped according to the type of activity occurring at the processing step (e.g. Coating, Mixing, Grinding; Ingredient Prep/Staging/Addition; Liquid Surge/Holding/Storage tanks; Rework; Dry Receiving; etc). These activities types helped to refine the processing step analysis by focusing on the nature of the activity at the processing step and not the name that processing step has been assigned by either industry assessment participants, or the CARVER+Shock Software tool. A list of four yes/no questions, and the answers to each question for each processing step were included. The processing steps were then ranked first by the number of yes answers to the four questions and secondarily by the CAV score. The questions were developed to identify processing steps which contained characteristics that have consistently resulted in processing steps being assigned a high CAV score. The four questions were as follows:
- Does mixing occur at or immediately after this processing step?
- Are minor ingredients added at this processing step?
- Are minor ingredients involved with this processing step?
- Are liquids or partially liquid mixtures dealt with at this processing step?
Processing steps which satisfied all or most of the questions above were almost universally assigned high CAV scores. The processing steps that satisfy the questions above resided in the Coating/Mixing/Grinding/Rework activity, the Ingredient Staging/Prep/Addition activity, the Liquid Receiving/Loading Activity, or the Liquid Storage/Hold/Surge Tanks activity. As a result, it was determined that these activity types, when present in a facility, should be given priority consideration for the implementation of food defense mitigation strategies.
As with the previous analysis, when the top quartile of processing steps were organized based on the nature of the activity being performed, it became clear that certain activities in a production process should be given priority consideration for the implementation of food defense mitigation strategies. When sorting by CAV scores, processing steps in the Coating/Mixing/Grinding/Rework activity group and the Ingredient Preparation/Staging/Addition activity group were consistently very highly ranked both within and between assessments. The grouping of various processing step names into these two activity groups only reinforced the importance of what activity occurs at a processing step. In addition, it became apparent that processing steps involving liquid handling carry more risk than handling or storage of other types of ingredients. There was a distinct separation in CAV scores between receiving and storage of liquid vs. dry ingredients. Liquid Receiving/Loading and Liquid Storage/Hold/Surge Tanks activities routinely ranked higher than their counterparts for dry products. By focusing on activities being conducted, the data provided insight into the processes where mitigation strategies and food defense measures should be focused.
It was necessary to find a solution that would reliably identify processing steps which contain key attributes or characteristics so that mitigating actions can be taken. In seeking such a solution, three requirements must be satisfied in order for the solution to be usable:
- The solution must be able to be publicly disseminated and thus not contain sensitive information about a commodity, facility, agent, or CARVER+Shock scores.
- The solution must be able to be assigned in an objective manner.
- The solution must provide an ability to be verified by audit or inspection.
Once the types of activities which take place at certain processing steps were defined and grouped, it was possible to identify the common factors that caused a processing step to receive the CAV scores assigned. A draft was then developed with the four activity types which most commonly ranked high in the vulnerability assessments conducted. By developing detailed descriptions of the key characteristics of these activities, it would be possible to draft mitigation strategies specific to the processing steps in a production environment which are associated with these activities.
Based on this review, the key activity types in most production environments are:
- Ingredient Staging/Prep/Addition
- Liquid Receiving/Loading
- Liquid Storage/Hold/Surge Tanks
Detailed descriptions of these activities are contained in Appendix A. The benefit of focusing on activity types and applying relevant mitigation strategies to those activities rather than establishing a CAV Score threshold is that there is no burden on the industry to attempt to objectively self-score their process. Moreover, there is no burden on FDA or other government agencies to attempt to establish a specific CAV score as the threshold for the implementation of mitigation strategies.
An activity description can be disseminated publicly as the information contained therein is not sensitive. Industry then must only develop a process flow diagram and map their processing steps into activity types; this can be completed objectively as the activity types will have specific characteristics associated with them and processing steps matching these characteristics would be easily identifiable. Mitigation strategies could also be provided to industry members if they have any of the key activity types contained in their production process.
There could also be several tools that could be developed to help the industry understand what mitigation strategies would be most effective and relevant to key processing steps/activities within their process. A “decision tree” with yes/no answers could be developed to help industry members 1) detail and compare a particular processing step in their process against the characteristics of the activity type and 2) identify potential mitigation strategies which would be most appropriate to reduce the vulnerabilities associated with a particular processing step in their operation.
The solution detailed above complies with the three requirements that needed to be satisfied for a successful approach. By identifying and defining key activity types, a CAV score is no longer needed as a threshold. The activity types and their descriptions can be publicly disseminated, industry can objectively map processing steps into activity types and any mitigation strategies associated with those activity types could be audited. Additionally, the processing step/activity type map and any mitigation strategies a facility currently employs could easily be added to existing or future facility registration requirements or food defense plans.
Moreover, tools can be developed by the FDA to assist industry members define and specify the activity conducted at a particular processing step in their process and select effective mitigation strategies unique to their process.
There are tools, such as the CARVER+Shock Vulnerability Assessment Software, that industry members can use to perform a private, custom vulnerability assessment. It is important and worthwhile for industry members to conduct a vulnerability assessment of their production process and facility so they gain a more comprehensive understanding of the vulnerabilities that exist in their process with regard to intentional contamination of their product. The awareness generated by conducting a vulnerability assessment helps industry members understand the need for mitigation strategies and assists them in identifying where process improvements can be made to reduce the likelihood of intentional contamination. The method detailed above does not seek to undermine or replace existing vulnerability assessment efforts; to the contrary, it complements the vulnerability assessment process by helping industry members understand on a more detailed and comprehensive basis why a processing step in their process should be evaluated more closely and where mitigation strategies would most effectively reduce the likelihood of intentional contamination.
For more information regarding FDA’s Food Defense tools and resources, including the Vulnerability Assessment Software and Mitigation Strategies Database, please visit: www.fda.gov/fooddefense.
APPENDIX A. Activity Type Descriptions
Activity Type I: Coating/Mixing/Grinding/rework
This activity type refers to any processing step where the primary purpose or result of the processing step is
- Coating: Evenly coat a solid product with a powder or liquid coating, batter, breading, flavoring, or other ingredient or ingredient mixture where any coating ingredients that did not adhere to the product are recycled and used again in the coating process;
- Mixing: Homogeneously mix a powder, dough, or liquid ingredient mixture;
- Grinding: Reduce the particle size of a solid ingredient to a medium or fine granularity in a manner that would result in widespread mixing of a threat agent among the processed ingredient.
- Rework: Means the practice of using previous batches of product in production runs of other products.
The effect of any of these processes is that an agent added to the process would be evenly mixed throughout the product batch and contaminates the total servings produced from the contaminated batch. Processing steps and equipment associated with this activity include but are not limited to: mixer, blender, homogenizer, cascade breader, mill, grinder, pulverizer, etc.
Activity Type II: Ingredient Staging/Preparation/Addition
This activity type refers to any processing step where ingredients are manipulated prior to or during addition to the product stream by human contact. Computer metering or automatic weighing, sizing, batching, or measuring is not included in this activity so long as the process does not involve the active involvement of a person. Specifically,
- Staging is defined as the act of moving ingredient from medium or long term storage to the production area and any tamper evident packaging is breached.
- Preparation is defined as any act of measuring, weighing, premixing, or otherwise manipulating the ingredient prior to addition to the product stream.
- Addition covers any act of physically adding ingredient directly into the product stream or into surge/metering hoppers in any way that is not remotely or automatically carried out.
The effect of any of these actions is that ingredients are generally open and accessible at processing steps where ingredient manipulation occurs. Also, mixing frequently occurs immediately after ingredients are staged, prepared and/or added to the product stream. Whereas mixing would homogeneously distribute an agent in a batch, the activities of ingredient staging/preparation/addition provide a point of access to introduce the agent into the product stream.
Activity Type III: Bulk Liquid Receiving/Loading
This activity type refers to any processing step where a liquid ingredient is being received and unloaded at a facility or a liquid intermediate or finished product is being loaded into an outbound shipping transport vehicle. This activity type incorporates the actions of opening the transport vehicle, attaching any pumping equipment or hoses, and opening any venting hatches. The characteristics associated with these activities involving bulk liquid receiving/loading are a high probability of an agent mixing within the liquid due to significant sloshing, movement, and turbulence associated with receiving/loading. Also, the actions of the worker associated with these processing steps provides access to hoses, the transport vessel, and potentially the product as it is being received or loaded.
- Bulk liquid receiving refers to the inbound shipping of liquid product into a facility for its use in the food production process.
- Bulk liquid loading refers to the outbound shipping of liquid product from a facility for further processing or use by an end customer/consumer.
Activity Type IV: Liquid Storage/Holding/Surge Tanks
This activity type refers to any processing step where liquid ingredient or intermediate/finished liquid product is stored in either bulk storage tanks or smaller secondary holding tanks or surge tanks. Specifically liquid storage can be broken down into two broad categories,
- Bulk liquid storage refers to any medium-long term storage silo or tank where liquid product may be stored prior to introduction into the product stream or to hold finished product prior to loading for outbound shipping.
- Non-bulk liquid holding and surge tanks refer to any storage tanks used to hold product for a short period or surge tanks. Non-bulk tanks can be used to store non-bulk liquid ingredients, hold liquid product for sample testing and other QC activity, or to control flow rates of liquid ingredients/product through the production system.
Both categories of this activity type can be considered key processing steps because many liquid storage/hold/surge tanks are agitated to prevent any separation or inconsistency within the liquid. Also, many times, tanks are located in isolated parts of the facility where human observation is infrequent. Access hatches may not be locked or alarmed. With regard to surge tanks in the production area, there may not be lids present or locking hatches which would limit accessibility to the liquid ingredient/product.