• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services


  • Print
  • Share
  • E-mail

What Factors In The Development Process Could Affect Risk Identification?

Table of Contents: Managing the Risks from Medical Product Use: Creating a Risk Management Framework

Previous Section : How well is the Agency's Quality Control System working?


The clinical community and FDA have long recognized that, even given the flawless functioning of both the manufacturer's product development process and the FDA's review process, factors inherent in the current medical product development system will continue to limit FDA's ability to identify all potential risks in new medical products. In interviews with the directors of the Center for Drugs, Center for Biologics, and Center for Devices and with the premarketing biostatistics office directors about their experience with premarketing reviews and about the product development process and its limitations, we identified the following factors that could be affecting the Agency's ability to identify potential risks prior to medical product marketing.

Trials expose only a relatively small number of people to a product

The number of patients exposed to a product increases 1,000-fold or more when the new product moves from the clinical trial setting to the real world setting. For example, clinical trials for most pharmaceuticals enroll and follow between 1,000 and 10,000 patients. For products intended to treat chronic, non-life-threatening conditions that occur in large populations, the International Conference for Harmonisation (ICH)10recommends a baseline safety database that typically involves at least 1,500 patients, usually achieved over multiple trials. The patient sample size and 6-month exposure time recommended by ICH are designed to reliably (95 percent of the time) identify events happening at the 1-percent level and are not expected to identify more rare events. Yet, in the first year of marketing, a successful new medical product can easily reach millions of Americans. This means that for an adverse event that occurs only once in 1,500 patients (considered rare), the chance of seeing such an adverse event goes from a 50-50 chance of seeing one in a clinical trial to seeing as many as 1,000 adverse events in the first year the product is on the market. In addition, preapproval trials rarely gather long-term experience with a product (more than 6 months) because often no more than a few hundred individuals use the product for 6 months or longer.

As a result, information on risks that occur only rarely may be entirely absent from the premarketing database, or may be represented by, at most, only a very few cases. If a risk is novel and not an expression of common patterns of toxicity, there may be insufficient evidence to identify the product as the cause of the event. In addition, adverse events that occur at a background rate in the treated population may be difficult to detect.11 When considering long-term exposure to a medical product, there is little expectation that adverse events that don't occur until 2, 3, or more years of use will be identifiable based on the premarketing data. This is true even for medical products that will be used for a lifetime.

Despite these drawbacks, the size and duration of clinical trials were not determined by chance. Protocol designs for trials reflect decades-long experience studying failures to detect adverse biological effects, identifying statistical design issues, analyzing what can reasonably be achieved during clinical investigations, and carefully considering the practical ability of manufacturers and clinical investigators to regularly conduct large-scale trials. Clinical trial investigators expect the majority of severe toxicities to be detected through a combination of high-exposure animal studies and the current profile of trial size and duration. Under the current clinical trial design, the common modes of major toxicity -- bone marrow suppression, hepatocellular damage, renal damage, neuropathy, and alteration of CNS function -- are regularly detected and, except for economic decisions, account for the majority of medical products not progressing from clinical trial to the marketplace. Novel risks are harder for the clinical development system to find, yet once identified, they are also evaluated in trials. For example, because practolol (a beta-blocker whose IND application was discontinued in 1977) caused cataracts, clinical trials for pharmaceuticals now require that some patients be monitored for cataracts.

Clinical trial patients aren't real world patients

During the development of a new medical product, clinical trial designers want to demonstrate effectiveness clearly. They want to find the greatest achievable therapeutic effect consistent with safe use. To accomplish this, they seek a homogeneous study population with the goal of preventing statistical noise from obscuring the treatment effect. To achieve homogeneity, clinical trials often systematically exclude special populations -- patients with concurrent diseases, with concurrent drug use, or at age extremes, or who otherwise are felt to be at risk of noncompliance. Trials also often exclude people with any other factors that may make it more difficult to measure effectiveness. Yet, the people being excluded may be precisely the people who ultimately will be using a product and in whom toxicities are most likely to occur.

Clinicians and FDA regulators have long recognized this exclusion problem in designing clinical trials. During the last two decades, the Agency has moved to ensure the inclusion of reasonable numbers of both genders, individuals with the ethnic profile of the population that will be using the product (the target population), and individuals at the extremes of age. More important, the Agency has encouraged the design of large simple trials that more accurately reflect normal, real world use patterns. Unfortunately, large simple trials remain uncommon in human medical product development. Clinical trial populations still do not completely reflect the population who will be using a product once it goes on the market.

Clinical trial patients are carefully screened

Another disparity exists between patients in the clinical trial setting and the real world setting because clinical trial screening practices ensure that essentially all patients have the condition being investigated. Once a product goes on the market, less stringent diagnostic criteria are applied than were applied during trial screening. In addition, some patients will be given a product that was intended to treat a different disease (poor product choice). Off-label drug use also may rapidly proliferate. In the case of both poor product choice and off-label use, patients may differ greatly from the trial population; they may be receiving different doses for different lengths of time; and they may be facing very different risks.

The extent to which patients in clinical trials differ from patients in the general population compounds the problem created by the disparity between the size of the clinical trials and the vastly larger population to whom a medical product is marketed. Not only are 1,000 times as many people exposed to a product in the real world, but many of them will differ significantly from the clinical trial patients for whom safety data have been collected.

Clinical trial patients are closely monitored

Another way the clinical trial setting differs from the real world setting is that, in addition to effectiveness, clinical investigators are studying safety. They are on the lookout for toxicity and promptly cease using a product if a toxic event occurs. Clinical trial patients are seen by their clinician for reevaluation at regular intervals; they are advised how to self-monitor for symptoms that may reflect toxicity and to seek prompt medical attention if symptoms occur. Finally, they receive regular laboratory tests for early evidence of target organ toxicity.

Most healthcare practitioners follow the recommendations for monitoring in the Precautions and Warnings sections of the labeling. But even highlighted, explicit warnings sometimes go unheeded. One result of the shift from the clinical trial setting to the real world setting is that toxic effects are less likely to be detected early, when they are most reversible. For example, assume a given product produced biochemical evidence of hepatocellular injury in 3 percent of trial patients, but caused no observed cases of overt hepatitis in the clinical trials (i.e., there were fewer than the 0.3-percent incidence of overt hepatitis that the clinical trials were designed to detect). Transferred to the real world setting, the same 3-percent hepatocellular injury rate could well produce a 1-percent incidence of overt hepatitis and occasional (0.5 percent) liver failure. The higher incidence and more severe consequence would result because the liver damage was not detected early and drug use was not stopped before producing irreversible damage. In cases where not a single incidence of significant liver damage occurred in the clinical trial setting, undetected hepatitis, liver failure, and death (at a one-in-5,000 patient rate) could easily produce dozens of deaths during the first year that a product is on the market.

Market rollout affects risk identification

Once products enter the market, one might think that use patterns would minimize the occurrence of such events, and any adverse event would be detected quickly and its impact mitigated. Yet, experience has shown that, once approved, new products reach consumers so quickly in the U.S. market that often dozens to hundreds of adverse events can occur before they are recognized and action is taken to reduce their effects. (FDA's programs to detect such events are addressed in more detail in Part 3 of this report.) Some of the effects of market rollout are discussed in the following paragraphs.

Market rollout often targets a broad population

Market rollout often targets a broad population that does not resemble the population in which the product was tested and for which the product was intended. Use patterns show that a product is often quickly and widely positioned in the marketplace even if time-tested alternatives to the new product are available. The potential consequences of this practice are clear when one retrospectively looks at products for which unknown risks and serious adverse events led to market withdrawals (e.g., Omniflox [temafloxacin], Duract [bromfenac]). In most of these cases, a patient who suffered a major adverse event could have been prescribed any one of a number of alternative products with established safety records.12

Sometimes, a new medical product has offered treatment for patients where essentially nothing else was available (e.g., AZT), or where existing alternatives are very unsatisfactory (e.g., erythropoietin). When a medical product is developed that demonstrates a clear therapeutic benefit, FDA recognizes its importance by expediting the review. However, most new products are incremental or niche improvements over alternatives that are already on the market. For example, the improvement may be having a better treatment effect (i.e., an incremental improvement), or it may even be limited to dosing convenience (i.e., a niche improvement). Many medical products lack even this often modest demonstration of clear benefits since they are molecular mimics developed and marketed based on evidence showing only that they are effective compared with a placebo.

In these cases, earlier-approved products to treat the same indication, having stood the test of time, will be less likely to produce previously unrecognized toxic effects. When time-tested alternatives to a new medical product are available, it may be prudent to consider how quickly to expose which patients to the new medical product, especially if it has exhibited serious risks during development.

Economic and organizational factors drive market rollout strategies

To better understand the general approach to medical product development and rollout strategies, it is useful to consider some of the economic and organizational factors that influence decisions made by pharmaceutical manufacturers. The development and testing of a new medical product can be difficult and prolonged and typically require an enormous front-end financial investment. If manufacturers are to survive, thrive, and make profits, they must recover the front-end costs of product development and testing. Included among these costs are the losses associated with products that are abandoned during development. When similar products are under development by different manufacturers, a manufacturer is particularly driven to achieve rapid, widespread market penetration and prescriber loyalty. Consequently, firms tend to rollout new products rapidly and market them aggressively.

Manufacturers usually recognize that their long-term goal is to ensure high-quality products while minimizing the frequency of serious adverse events. Yet, they may not always recognize when the incentives they offer -- salary increases, bonuses, or stock options tied to milestones or to the sales volume of a new product --conflict with this goal. Once a new product is approved by FDA, the responsibility for launch and marketing usually falls on entirely different personnel than were involved in developing the medical product. Marketing personnel may not always completely recognize how their product placement and positioning decisions affect the use of a product and the risks that accompany that use.

Although FDA has little direct influence over a manufacturer's product marketing choices, the Agency should consider potential marketing approaches when designing and implementing its QA/QC system.

The FD&C Act standards also affect market rollout strategies

The reality is that a rapid rollout strategy usually results in a new medical product moving quickly from testing in a few thousand clinical trial patients to, perhaps, millions of new prescriptions. This leap in numbers takes place just months after launch, before there is time for feedback through postmarketing reporting. Although driven to a great extent by economic issues, this pattern is also, in part, a consequence of the way the Federal Food, Drug, and Cosmetic Act (FD&C Act) establishes standards for medical product approval.

The FD&C Act standards for safety and effectiveness permit approval based on an independent demonstration of safety and effectiveness for each product. Pharmaceutical manufacturers have strongly objected to the use of review criteria or practices that they believe might lead to a comparative effectiveness standard. Yet, if use of a new product were evaluated comparatively, the potential extent of injury from an unknown risk might be reduced because the product's initial postmarketing use could be limited to those patients who have been shown to experience a clear therapeutic benefit over an alternative product. In such a case, a smaller segment of the population would be exposed to any unknown risks during the early postmarketing phase. The merits and liabilities of changes (both more and less stringent) to FDA's approval standards have been the subject of an ongoing public policy debate. Any impact on product safety is but one aspect of that debate.

Next Section : Conclusions and Recommendations