U.S. flag An official website of the United States government

U.S. Food and Drug Administration
  1. Home
  2. Inspections, Compliance, Enforcement, and Criminal Investigations
  3. Compliance Actions and Activities
  4. Application Integrity Policy
  5. POINTS TO CONSIDER FOR INTERNAL REVIEWS AND CORRECTIVE ACTION OPERATING PLANS JUNE 1991 FOOD AND DRUG ADMINISTRATION
  1. Application Integrity Policy

POINTS TO CONSIDER FOR INTERNAL REVIEWS AND CORRECTIVE ACTION OPERATING PLANS JUNE 1991 FOOD AND DRUG ADMINISTRATION

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES
PUBLIC HEALTH SERVICE
FOOD AND DRUG ADMINISTRATION
OFFICE OF ENFORCEMENT
Division of Compliance Policy
(HFC-230)
5600 Fishers Lane Rockville, MD 20857

POINTS TO CONSIDER FOR INTERNAL REVIEWS AND
CORRECTIVE ACTION OPERATING PLANS
JUNE 1991
FOOD AND DRUG ADMINISTRATION

I. INTRODUCTION

Food and Drug Administration (FDA) Compliance Policy Guide 7150-09 establishes the agency's policy on, "Fraud, Untrue Statements of Material Facts, Bribery, and Illegal Gratuities" (fraud policy). The fraud policy sets forth FDA's regulatory approach regarding applicants who seek to subvert the agency's review and approval processes for premarket applications. The policy also outlines recommended corrective actions by which applicants may seek to restore FDA's confidence in their integrity and permit the agency to proceed with substantive scientific review of their premarket applications.

This "POINTS TO CONSIDER FOR INTERNAL REVIEWS AND CORRECTIVE ACTION OPERATING PLANS" (document) describes actions applicants may take to affirm the validity of data that have been called into question by FDA. This document is intended to permit flexibility of application to the wide range of scientific activities, products, and processes associated with applications submitted to FDA. This document does not establish criteria, standards, or requirements that bind FDA or any applicant. This document also does not create or confer any rights, privileges, or benefits on or for any person.

As discussed in Compliance Policy Guide 7150.09, FDA intends to conduct validity assessments when the reliability of data in an application has been called into question because of wrongful acts. As part of the validity assessment, the applicant should undertake a comprehensive effort to determine the scope of the wrongful acts and their effects on the reliability of data submitted to FDA.

When an applicant finds it necessary to take comprehensive corrective actions based on notification by FDA of questions regarding the validity of data in one or more applications, the applicant should, after conferring with the agency, provide FDA with a written commitment that describes the corrective actions to be taken. As noted in the fraud policy, the corrective actions generally should include full cooperation with ongoing federal investigations, identification of all individuals who were or may have been associated with or involved in the wrongful acts, a credible internal review, and plans to develop and implement an effective corrective action operating plan. The objective of the internal review described in this document is to identify all instances of fraud and of untrue statements of material facts associated with applications submitted to FDA. However, as a matter of policy, the FDA will not rely on the applicant's internal review as a substitute for its own investigations, but will use the applicant's findings to supplement the agency's own investigations and audits. The objectives of the applicant in implementing a corrective action operating plan should be to ensure the reliability of data in applications, to ensure the safety, effectiveness, and quality of marketed products, and to take steps to preclude future instances of wrongful acts and noncompliance with regulatory requirements for approved applications.

Depending on the circumstances, FDA may request different or additional measures. In acting to remedy violations of law and establish reliability of data in applications submitted to FDA, applicants may find it appropriate to take measures other than those described herein. Applicants are encouraged to discuss their corrective action operating plan with the agency to prevent expenditure of money and effort on activities that may later be determined to be unacceptable by FDA.

II. CONDUCT OF THE INTERNAL REVIEW

The fraud policy encourages applicants to conduct a credible internal review, which involves an outside consultant, in order to identify all instances of wrongful acts associated with applications submitted to FDA, including any discrepancies between conditions identified in approved applications and manufacturing conditions during actual production.

Adequate internal reviews are crucial to assisting FDA in identifying all instances of wrongful acts in a reasonable period of time and to the development by the applicant of a useful corrective action operating plan. In general, internal reviews should include: (1) an analysis of the problems identified by FDA in its notice to the applicant, (2) a suitable and specific audit plan, (3) audits conducted in accord with the audit plan, and (4) audit reports that accurately reflect the audit findings. Each of these steps is described below:

  1. Analysis of Problems Identified by FDA - Upon receipt by an applicant of FDA's notification that the agency has reason to believe that the applicant or its employees have engaged in wrongful acts that may undermine the integrity of the applicant's product applications, the applicant should examine each of the problems and deficiencies identified by FDA to determine the cause and scope of the wrongful acts and to establish the best approach to conducting an audit that will identify all instances of wrongful acts. In determining the cause and scope of the wrongful acts, the applicant should examine its organizational structure, personnel and their responsibilities, the nature and extent of managerial or other personnel's participation in activities related to the development or approval of the implicated product applications, the applicant's operating procedures, and any contractual agreements with third parties such as outside testing facilities or clinical investigators. The circumstances surrounding instances of wrongful acts may involve more than one person. For example, if data falsification were attributed to one employee in the Quality Control Laboratory, the applicant should determine whether the employee's supervisor and coworkers were involved in the falsification. In addition, all work conducted by employees involved in the wrongful acts and associated with any other FDA regulated products or applications should be examined.
  2. The Audit Plan
    1. Purpose - The audit plan describes the strategies and procedures to be used to identify all instances of wrongful acts associated with applications submitted to FDA, including any discrepancies between manufacturing conditions identified in approved applications and manufacturing conditions during actual productions. This includes all data or other information generated in support of an application, those submitted with the application, and those generated subsequent to approval.
    2. Who Develops the Audit Plan - The applicant and/or the outside consultant may develop the audit plan. If the applicant or the outside consultant independently develop the audit plan, both the applicant and the outside consultant should agree to the plan before an audit commences.
    3. Content - The audit plan should include a detailed description of the strategies and procedures that will be used to conduct the audit. The audit plan should be sufficiently flexible to permit the consultant to conduct all verifications necessary to assure validity of the data or other information associated with the application. The audit plan also should be sufficiently flexible to permit the consultant to revise the strategy and procedures for the audit if information obtained during the audit indicates there is a need to broaden or shift the focus of the audit. The plan should be comprehensive, thorough, and complete, including an identification of all records, applications, and other documentation that will be examined and all personnel (past and present) that will be interviewed. The audit plan also should include a diagram of the applicant's major organizational structure with key positions associated with the development and submission of an application highlighted. Additional information about audit processes may be found in the references listed under section IV of this document.

      FDA recommends that the applicant submit the audit plan to the agency for review prior to implementation. Generally, the following information should be included in the plan:

      1. Scope of the Audit - The audit process seeks to validate events that occurred in the past. Accordingly, validating information may be obtained through interviews with personnel who were involved with developing the data included in an application and through examination of documentation, specimens, and samples that were prepared at the time of the past events. Therefore, the scope of the audit should encompass not only those activities, applications, and personnel identified by the agency in its notification to the applicant, but also those activities, applications, and personnel identified by the applicant in its preliminary analysis of the problems identified by FDA, described at section II.A. Therefore, the audit plan should identify all personnel to be interviewed, the functions, specific activities, and products that may have been affected, and all associated data, records, samples, specimens, and other documentation that could reveal information about the wrongful acts.
      2. Controls to Assure Impartiality - The audit plan should provide a complete description of all controls that will be used to assure impartiality of the audit process. For example, FDA recommends that the applicant specifically hire an outside consultant to conduct the audit and to lead the audit team, if a team is necessary. In addition, the consultant should be provided complete freedom and authority to conduct a thorough and meaningful audit. The consultant's reports (including draft reports) should not be altered by the applicant and recommendations should be considered and implemented as soon as possible. Applicants should voluntarily cooperate in the consultant's investigations and promptly provide all pertinent and requested documents.
      3. Qualifications of the Consultant - The audit plan should include the name and qualifications of the consultant and of each member of the audit team, if a team is used. The qualifications required to assure that a consultant is appropriate for a specific audit will depend on the nature of the investigation, the products involved, and other circumstances in the case. The consultant generally should be familiar with the product and operations to be audited, should possess the proper mix of education, training, and experience to conduct an appropriate audit and should be free of any past involvement in the wrongful acts or other activities being audited. The background normally associated with consultants includes work experience in quality assurance, in regulatory affairs, and/or with regulatory agencies. Quality control, laboratory, scientific, and manufacturing personnel who have auditing experience may also qualify. Generally, a criminal investigator, or a

         

        Certified Public Accountant or an attorney may not be qualified to conduct an audit independently, but may possess specific skills necessary to assist an audit team.

      4. Projected Time Frames for Conducting the Audit - The audit plan should include a timetable for the initiation, conduct, and completion of the audit. The timetable should identify interim deliverables that the auditor will furnish to the applicant at specific times documenting that progress is being made to comply with the plan.
      5. Specifications for the Final Audit Report - The audit plan should include specifications for the content and format of the final report of the audit, including a scheme for analyzing, describing, and presenting the audit findings. Because the audit report will be used by the applicant to develop the corrective action operating plan, the audit plan may instruct the consultant to include in the audit report recommendations for necessary corrective actions. The plan should state that a complete audit report, identical to the report submitted to the applicant, should be submitted to FDA.
  3. Conduct of the Audit - The audit should be conducted in accordance with the audit plan. When the elements described below are identified in the plan for auditing, the following guidance may be useful to the consultant examining an application.
    1. Nonclinical and Clinical Studies
      1. Protocol - The consultant should scientifically evaluate the study protocol and any amendments to the protocol and determine their adequacy for achieving the study objectives. Flaws should be identified. In addition, the consultant should interview scientific, supervisory, and support personnel involved in the study to determine that they fulfilled their respective roles in the study and to confirm that study operations were conducted as specified in the protocol and in accordance with standard operating procedures and applicable FDA regulations.
      2. Study Report - The consultant should compare the protocol and its amendments, as well as any applicable standard operating procedures, to the methods and procedures used for conducting the study as described in the study report to determine if all of the study objectives were properly attained and that there was no evidence of wrongful acts. A check of study data and documentation for completeness and consistency is a measure that suggests the study objectives were achieved. However, an independent collaboration should be obtained to demonstrate that the study data and associated documents were not a part of any wrongful act. An evaluation of the equipment as well as the capabilities of the staff also should be made to determine that, in fact, the methods and procedures described in the protocol and standard operating procedures could have been followed.

         

        The consultant should compare the final study report with the raw data and other study records to confirm that the conclusions reached in the final report are fully and accurately substantiated by the study records. The consultant also should assure that none of the. information was derived through wrongful acts. Accountability should be determined for all data derived from the experimental units. Further, when the tabular conclusions are the result of calculations applied to raw data, the appropriateness and accuracy of the calculations should be determined and verified. If study data are generated by automated data collection systems, the consultant should verify that computer hardware and software were properly validated and are being properly maintained.

      3. Specimen Examination - The consultant should randomly examine selected and representative specimens generated as a result of a study and required to be retained for confirmation of the number and kind of specimens referred to in the final study report. In addition, pathology data should be reviewed for total animal and tissue accountability including a reconciliation of tissues lost to histological processing. Summary tables and narrative reports should be examined for consistency in terminology and in the appropriate use of topographies and morphologies. All gross observations recorded at necropsy or on individual animal necropsy records should be compared with histopathological diagnoses. All wet tissues should be inventoried. Slides and blocks from all tissues should be compared and matched. If neoplasms and target tissues need to be re-examined microscopically, an independent pathology consultant may be necessary. Differences in diagnoses between the original and consulting pathologists should be resolved.
    2. Manufacturing Processes - When indicated by the audit plan, the consultant should review the manufacturing processes for the product that is the subject of an application, and such review should include, but not be limited to, the following elements:
      1. Personnel - The consultant should interview employees to verify that actual work was performed and to confirm that procedures were followed. Employees should be requested to examine signatures/initials on documents, to verify their work, and to identify their signature or initials to be their own. To further verify an employee's participation, the consultant should examine employee time cards, vacation records, employment history and the like. The consultant or a member of the consultant team who performs employee interviews should be well versed in identifying and recognizing criminal and/or other wrongful acts.
      2. Raw Materials, Components, and Ingredients - The consultant should review and verify receiving records for raw materials, components, and ingredients to confirm that the items were received, the date received, and the amount received and to verify that they were received from approved sources. Lot and serial numbers should be used to document the use of raw materials, components, and ingredients. Raw material verification should be cross referenced with raw material test records. The consultant should conduct interviews with suppliers of raw materials, components, and ingredients to determine the validity of records, chain of custody, and other pertinent information.
      3. Test Records - The consultant should review test records for raw materials, components, and ingredients to verify that the item was received, the date received, and the amount received and to verify that they were received from approved sources. The consultant also should review in-process and finished product test records to assess the validity of the product being manufactured. Stability test records, reserve sample records, and other records of in-house procedures may provide further information pertinent to production verifications and lot number traceability. All of these records should be thoroughly reviewed to determine that no wrongful acts were committed. Analytical testing records, work books, and charts should be examined. The consultant should also determine if any unofficial or private records were (or are) kept by any past or present individuals and identify any discrepancies between the unofficial records and the official records.
      4. Production and Process Records - The consultant should review the master batch records, production history, and batch records for inconsistencies, determine if these records accurately reflect the FDA approved procedures and practices for the specific product, and review appropriate records to verify all aspects of the manufacturing process. All of the records also should be reviewed to assure that appropriate FDA approvals were received for any changes or alterations made in the manufacturing procedures and practices. Further, all these records should be reviewed to assure that no wrongful acts were committed by past or present individuals. The consultant also should determine if any private or unofficial records were (or are) being kept. The consultant should review packaging and shipping records to provide assurance that the amount of product manufactured is documented and can be verified by the batch record. Interviews should be conducted with appropriate vendors and shippers to determine the validity of records, chain of custody, and other pertinent information.
      5. Equipment - The consultant should verify the use and existence of all equipment (manufacturing, packaging, laboratory, etc.) by examining equipment use logs, cleaning records, calibration records, servicing records, lease records, etc. Equipment purchase documents, for example, can be useful in verifying the existence of the equipment identified in manufacturing records. The capacity, speed, and function of the equipment should be consistent with the manufacturing records. This information should be used to verify that the amount of finished product manufactured is consistent with the equipment use. For example, certain equipment is needed to manufacture a drug product with a wet granulation as opposed to one with a dry granulation. Equipment use logs, calibration records, servicing records, cleaning records, and similar other records should be thoroughly reviewed for down-time, individual use time, etc., to assure the validity of records and equipment. The consultant also should determine if any private or unofficial records were (or are) being kept.
    3. In Vivo Bioequivalence and Bioavailability Studies - The consultant should assure that the batches being tested were legitimate, manufactured, and tested in accordance with the application. Further, the consultant should audit the records as recommended in 2(a),(b),(c),(d), and (e). If the product is subject to an in vivo bioequivalence or bioavailability study, the consultant should verify the accuracy of the data provided to the applicant by the test facility. The verification should extend to both the clinical and analytical aspects of the study.
  4. Audit Report The consultant should prepare a report of the audit findings in accordance with the audit plan, submit the audit report to the applicant, and submit a copy of the audit report to FDA. The audit report should properly identify all the members of the audit team along with their qualifications and the specific audits they participated in. The audit team should be signatory to the final report. The consultant's report should also indicate the number of draft reports prepared before the final report and where they are available.

III. THE CORRECTIVE ACTION OPERATING PLAN

The corrective action operating plan is the essential instruction by which applicants plan to achieve correction of violative, unacceptable, and improper practices and to preclude future occurrences of wrongful acts. The operating plan is derived from the information gathered during the internal review. FDA recommends that the applicant submit the corrective action operating plan to the agency for review prior to implementation.

  1. Content of the Corrective Action Operating Plan - The corrective action operating plan should include: (1) an analysis of the audit findings, including an analysis of-all identified instances of wrongful acts associated with applications submitted to FDA, identification all individuals who were or may have been associated with or involved in the wrongful acts, and identification of defective practices, procedures, products, and applications; (2) the disposition of any recommendations made by the consultant; (3) a description of the actions taken and to be taken to achieve correction of fraud and other wrongful acts or deficiencies identified by FDA and discovered by the internal review; (4) a timetable for implementation of the corrective actions; (5) identification of the persons responsible for taking and assuring the satisfactory completion of each of the corrective actions according to timeframes; (6) a comprehensive ethics program that describes standards for employees and procedures for educating employees about the program and for enforcing the program; and (7) the procedures for monitoring the effectiveness of the operating plan and to assure that the applicant can be expected to manufacture products in compliance with current good manufacturing practices and application requirements.

     

    In addition, the corrective action operating plan should establish a mechanism for assuring that supervisory, technical, and scientific employees are made aware of the requirements of the Federal Food, Drug, and Cosmetic Act in their area of responsibility, as well as the requirements under 18 USC 1001.

  2. Implementation of Corrective Actions - The effectiveness of the applicant's corrective actions will depend on effective management control of their implementation and the exercise of in-process quality assurance practices (monitoring) during the corrective actions process. Management should ensure that any new personnel hiring and training practices, standard operating procedures, documentation of activity changes, organizational rearrangements, and other major shifts in operations are done properly. Adherence to the timetable for implementing corrective actions should be enforced and monitored by responsible personnel.
  3. Verification of the Adequacy of the Corrective Actions - Applicants should not assume that the implementation of the operating plan has cured all problems. applicants should establish an on-going program to monitor all activities related to the development and submission of applications to FDA to assure that the applicant's records are reliable and that the applicant can manufacture products in compliance with current good manufacturing practices and application requirements.

IV. References

Baldwin, J.K. and B.K. Hoover. Data Audit as a Management Tool. Accountabil. Res.: Pol & Qual. Assur. 1 71 (1989).

Carson, P.A. and N.J. Dent. Good Laboratory and Clinical Practices. Techniques for the Quality Assurance Professional. Neinemanne Newnes, Publisher. Oxford, England (1990).

Cocchetto, D.M. et al. Accuracy of Data Collection in a Clinical Trail with Guidelines for Case Report Auditing. Jour. Clin. Res. Drug Devel. 1 187 (1987).

Food and Drug Administration. Compliance Program Guidance Manual Program 7438.811. Clinical Investigators (1989).

Food and Drug Administration. Compliance Program Guidance Manual Program 7438.808. Good Laboratory Practice (Nonclinical Laboratories) (1989).

Hoover, B.K. and J.K. Baldwin. Meeting the QA Challenges of the 1980's: Team Auditing by Toxicologist and QA Professionals. Jour. Amer. Col. of Tox. 3 129 (1984).

Loeb, S.E. and A.E. Shamoo. Data Audit: Its Place in Auditing. Accountabil. Res.: Pol. & Qual. Assur. 1 22 (1989)

Pincus, K.V. Financial Auditing and Fraud Detection: Implications for Scientific Data Audit. Accountabil. Res.: Pol. & Qual. Assur. 1 53 (1989).

 
Back to Top