How Well Is The Agency's Quality Control System Working?
Although the process differs to some degree from Center to Center and from product type to product type, the FDA approval process usually involves a complex, and thorough investigation of the data submitted in a medical product application. In the review of pharmaceuticals, for example, the goal of this investigation is to determine (1) if the results of well-controlled studies provide substantial evidence of effectiveness and (2) if the results show the product is safe under the conditions of use in the proposed labeling (i.e., the benefits of the product outweigh the risks).
Under current law, all new drugs must be shown to be effective, as well as safe, before they can be approved for marketing. After a manufacturer (or sponsor) has completed certain preliminary testing of the new product in animals (preclinical testing), it can seek approval from the FDA to begin limited testing of the product in humans. FDA estimates that on average it takes 8.5 years to study and test a new drug before the Agency can approve it for the general public, including early laboratory and animal testing, as well as later clinical trials using humans.
FDA's review of the application, including site inspections and other interactions with the product's sponsor, takes approximately 12 months.
When the application is received, it is assigned to a review team based on the type of drug and intended use. The typical review team (see the box below) for a new drug evaluates test results submitted in the application. The documentation required in an application is supposed to tell the drug's whole story, including what happened during the clinical tests; how the drug is constituted; the results of animal studies; how the drug behaves in the body; how it is manufactured, processed, and packaged; and any other available material. FDA also requires samples of the drug and its proposed labeling (prescribing information or package insert). Using the submitted data, the FDA review team decides whether the studies submitted by the product's sponsor show it to be safe and effective for its intended use. The purpose is to determine whether the drug is safe enough to be marketed and what its labeling should say about directions for use, side effects, and warnings.
Typical Agency Drug Review Team
Chemists focus on how the drug is made and whether the manufacturing process and packaging are adequate to ensure the identity, strength, quality and purity of the product.
Pharmacologists and toxicologists evaluate the effects of the drug on laboratory animals in short-term and long-term studies.
Physicians evaluate the results of the clinical tests, including the drug's adverse as well as therapeutic effects, and whether the proposed labeling accurately reflects the effects of the drug.
Clinical pharmacologists evaluate the rate and extent to which the drug's active ingredient is made available to the body and the way it is distributed, metabolized, and eliminated.
Statisticians evaluate the designs for each controlled study and the analyses and conclusions for safety and effectiveness based on the study data.
Microbiologists also participate in the review of anti-infective drug products and of products that occur as solutions or as injectables.
In the case of medical devices, teams of review scientists include engineers, biological scientists, materials experts, and clinicians. These experts review the valid scientific evidence submitted by sponsors to determine the safety and effectiveness of devices either by comparison to a previously marketed (predicate) device or through a premarket approval application (PMA) that establishes reasonable assurance of safety and effectiveness.
After FDA primary reviewers finish their evaluation, additional review is provided by supervisory personnel. Regulations require that the decision process be appropriately documented in an administrative record.7 And an employee or someone outside the Agency can request an internal Agency review of a decision.8
One key part of FDA's mission is to ensure that manufacturers identify potential risks during drug development and advise prescribers about them in product labeling. Failure to identify and advise the healthcare community about even comparatively rare risks is unacceptable if their identification could reasonably have been expected. This is not to say a perfect understanding of a drug's safety can be attained during drug development. In the end, approval decisions are based on the judgments of Agency experts, resulting from their review of data collected during clinical trials.
The data from trials are always limited by the size of the trial. But, if the trials are well-designed, the data should be adequate to permit reasonable judgments concerning a product's safety. For this reason, FDA focuses considerable attention on the quality control of its premarketing program. The Task Force undertook an audit of the current quality control system to see how well it was functioning.
To measure the adequacy of the Agency's quality control system, the Task Force compared it to the International Standards Organization (ISO) quality framework. ISO 9001 is a generic, worldwide quality management system standard promulgated by the International Organization for Standardization (ISO). ISO is currently completing a revised edition of this standard, ISO 9001:2000.
ISO 9001:2000 sets forth the quality management system requirements an organization should have in place to demonstrate its capability to meet customer (stakeholder) needs. The process-based structure envisioned by ISO 9001:2000 is built around four key areas of concern:
- Management responsibility (policy, objectives, planning, quality management system, management review);
- Resource management (human resources, information, facilities);
- Process management (customer satisfaction, design, purchasing, production); and
- Measurement, analysis, and improvement (audit, process control, continual improvement).
The ISO framework (see figure below) does not impose a uniform quality management system, but instead provides criteria an organization can use. FDA has wide-ranging public health responsibilities, and its premarketing review process seeks to meet the needs of the public, industry, and healthcare professionals. The creation and application of a comprehensive quality control system helps ensure FDA's premarketing review process meets its public health responsibilities.
FDA mapped its premarketing review quality control functions according to the elements of the ISO 9001:2000 framework to ensure that FDA's system contains the necessary elements. Each of the three Centers prepared a detailed inventory of those procedures and processes that meet the specific elements in the ISO system. (See the general listing of relevant ISO elements in Appendix A.)
Because of the diverse products, legal requirements, and organizational structures that were to be audited, the audit focused on the existence and application of control systems. In every Center, an extensive clearance process is relied on as part of the control system.
In addition, samples of recent product approval records from FDA's Center for Drugs, Center for Biologics, and Center for Devices were reviewed for their conformance to the existing quality control standard operating procedures.
The Task Force audit found that there is substantial conformance in FDA's quality control system to the principles of quality assurance (QA) and quality control (QC) as described in ISO 9001:2000. The FDA applies management responsibility, resource management, process management, measurement analysis and improvement, and documentation of control activities to its premarketing review and risk assessment process.
Reviews continue to be subject to 100 percent quality control
Premarketing reviews continue to be subject to 100-percent quality control. At least one level (and regularly, three levels) of expert subject-matter supervisors must concur with, or make documented revisions to, every primary review.
Several QC areas need enhancing
By undertaking this review of FDA's quality control system, the Task Force was able to fulfill one of the elements previously missing from its quality control system: a comprehensive executive management quality review of the Agency's premarketing review program. The Task Force found that, in several quality control areas, the Agency has begun development of processes to satisfy a particular ISO QA/QC element, but has not yet fully implemented them.
One of the most obvious underdeveloped areas was in establishing, administering, and documenting explicit training requirements for review staff. The Agency has relied on professional training, employment qualifications, supervisory mentoring, and annual performance reviews to ensure that the review staff possess the knowledge, skills, and abilities to perform their assigned duties. Additional professional development, attendance at internal and external professional educational meetings, in-house core competency training, and in-house Staff College training have been offered by all three Centers. But continuing education has not been systematically required of all review staff as a means of ensuring eheir continued qualifications to perform their assigned work.
A second area that FDA needs to focus efforts on is compiling explicit, detailed standards against which FDA reviewers are to evaluate new medical products. The goal of the ongoing Good Review Practice project is to systematize existing knowledge of what to look for when conducting a review. The project will document in a reviewer guidance the parts of an application that are most important to evaluate and what to expect when evaluating them. Documenting review expectations at this level is unusual, as it involves analyzing complex, knowledge-based activities. But once the guidance is completed, the Agency can be more confident that consistent and high-quality reviews are being performed.
During the audit of FDA's quality control system, the Task Force paid particular attention to whether differences of scientific opinion among primary reviewers and supervisors are being addressed within a quality control framework. In addition to the regulations mentioned above requiring the documentation of decisions and supervisory review of decisions upon request of an employee or person outside of the Agency, all FDA offices have written procedures on the specific processes for making and documenting scientific review opinions.9 These procedures delegate responsibility for final decisions to individuals with extensive regulatory experience who are supported by multidisciplinary scientific teams. The procedures contain explicit steps for resolving differences in scientific opinion among the reviewers and require the official responsible for the decision to document the reasons for rejecting dissenting recommendations.
The QC system is regularly followed
The Task Force found that, overall, although some elements of FDA's quality control system could more closely comply with the proposed ISO framework, the key elements of the ISO QA/QC system are in place and are regularly followed. We believe that FDA's premarketing review and decision processes are being managed to produce high-quality decisions on new medical product applications. However, in the course of the audit, the Task Force identified factors in the current medical product development process that could be affecting the Agency's ability to minimize unknown risk