February 2002, the FDA presented issues before the Anti-Infective Drugs
Advisory Committee (AIDAC) related to the selection of non-inferiority margins,
or “deltas”, for non-inferiority (also know as equivalence) trials. In November 2002, the Agency also
co-sponsored a workshop with the Infectious Diseases Society of America (IDSA)
and the Pharmaceutical Research and Manufacturers of America (PhRMA) to
continue discussions on this topic and relate it to several specific infectious
diseases. The issues of selecting a
non-inferiority margin were also discussed on an international scale in a
document released by the International Conference on Harmonization (ICH), a
body composed of regulators from the
The document, ICH E-10, discusses the implications of selection of non-inferiority margins on clinical trials. In non-inferiority trials, there is not a direct measurement of the benefit of the test drug relative to placebo. Such a measurement is indirect and based upon assumptions that the control drug in the trial is more effective than placebo. It is important not only to know that the control drug is more effective than placebo, but also to know the magnitude of this benefit when selecting a non-inferiority margin. ICH E-10 recommends that investigators consider trial designs other than a non-inferiority design when there is not adequate information to select a non-inferiority margin. This is because without information to appropriately select a non-inferiority margin, proof of comparable efficacy in a non-inferiority trial between test drug and control drug may mean that both or neither drug is more effective than placebo.
The results of the previous workshop and AIDAC discussions cited above were that a single value for a non-inferiority margin for all clinical trials, say 10%, was not scientifically appropriate. To select a scientifically justifiable non-inferiority margin, one must examine the data from previous placebo-controlled trials with similar endpoints in a given disease. The non-inferiority margin should be based on an examination of all relevant placebo-controlled trials in that disease, not just those that show benefit or the largest benefit of antimicrobials in a given disease. One must also take into account the other design features of the placebo-controlled trials and their relevance to the proposed study as well.
Since that time, FDA has asked drug sponsors to submit information from previous placebo-controlled trials to justify the non-inferiority margin they seek to use in their proposed clinical trials. Also, FDA has begun an internal review of these placebo-controlled trials for the purposes of formulating guidance on the design and conduct of clinical trials for various disease states. In reviewing prior placebo-controlled trials, several limitations have become apparent that often make it difficult, if not impossible, to select a scientifically justifiable non-inferiority margin for a trial. Some of these issues include: 1) a lack of adequate definition of the population with bacterial disease for inclusion in the trial, 2) the use of non-validated outcome scoring systems as endpoints, 3) the lack of demonstration of a correlation between microbiological and clinical outcomes, and 4) the timing of the assessment of endpoints. In addition, other factors such as changes in adjunctive therapies over time and changes in resistance patterns of common organisms involved in these infections may impact the interpretation of previous placebo-controlled trials.
Therefore, it has become apparent that these other clinical trial design issues deserve further discussion as well. The Agency will ask the AIDAC to discuss these issues as they relate to the design and conduct of clinical trials in acute bacterial sinusitis.
Over the last year, the FDA has held several discussions with drug sponsors wishing to pursue development of drugs for acute bacterial sinusitis (ABS). Several common themes emerged from these discussions. First, the selection of non-inferiority margin based on prior placebo-controlled trials is problematic. This raises the issue of whether one can perform placebo-controlled trials or other trial designs in studying ABS. Second, the selection of patients most likely to have bacterial disease based on clinical and/or radiological criteria may also be problematic based on data from currently available literature. This raises issues regarding obtaining an adequate microbiological diagnosis in patients with ABS to ensure that patients enrolled in the trial truly have bacterial disease. Third, there are issues with how and when to measure clinical and microbiological endpoints in ABS based on knowledge of the natural history of the disease.
The questions for discussion by the committee on this topic include:
1. How does one ensure that patients in clinical trials of acute bacterial sinusitis have bacterial disease? Please discuss the methods of obtaining microbiologic data including sinus punctures and nasal endoscopy.
2. Please discuss the issues of trial design in the study of acute bacterial sinusitis. Please include in your discussion:
a. The strengths and limitations of placebo-controlled trials and non-inferiority trials. Please discuss how one determines a non-inferiority margin in non-inferiority trials for this indication.
b. The strengths and limitations of comparative microbiologic data.
Please discuss the issues of measuring outcomes in patients in trials of acute bacterial sinusitis. Please include in your discussion measuring time-to-resolution of symptoms as an endpoint compared to fixed endpoints and the use of surrogate markers.
The current FDA guidance for clinical trials in ABS suggests that drug sponsors conduct two trials for licensure of drugs in this indication. The first suggested trial is a non-inferiority trial using clinical or radiographic criteria and endpoints to demonstrate the safety and effectiveness of the test drug relative to the control drug. The Agency has not required sinus puncture at baseline to define the population with bacterial disease in these trials. The second trial is a non-comparative trial that does require sinus puncture at study entry. The outcomes in this trial are based on resolution of clinical signs and symptoms of disease in patients with positive cultures for causative pathogens in ABS at baseline. The Agency has encouraged, but not required, post-therapy sinus punctures in patients judged to have clinical failure in these trials. Microbiological cures were based on presumed eradication of organisms in patients who were clinically well at the test of cure visit. The test of cure visit in both types of trials was set at one to two weeks after completion of therapy.
In reviewing the placebo-controlled trials in acute bacterial sinusitis as well as other relevant literature on the natural history of ABS, several issues with the approach of using a “clinical-only” study and a non-comparative microbiologically based study became apparent. First, the correlation of clinical signs and symptoms or radiological criteria with defined bacterial disease is not well known. There are few studies that directly compare various clinical criteria of combinations of signs and symptoms and/or radiological findings with the subsequent isolation of pathogenic bacteria from sinus punctures. The previous FDA guidance as well as treatment guidelines recommend that symptoms of disease persist for longer than 7 days to more accurately select patients with bacterial disease. Studies that used this approach appeared to have higher rates of positive cultures from sinus punctures, but there does not appear to be any study that directly compares the rates of bacterial isolation in patients with sinus puncture at less than 7 days with patients with greater than 7 days of symptoms.
It is estimated that the number of patients with viral or allergic sinusitis in the population far outnumbers the proportion of patients with bacterial disease. This raises the question of whether all patients entering clinical trials in ABS should have some microbiological assessment to ensure that they, in fact, have bacterial disease. The committee will hear presentations on the epidemiology and natural history of ABS, and a presentation on an evaluation of the literature attempting to correlate signs and symptoms and radiographic findings with the results from sinus punctures. The committee will also see a video presentation of a sinus puncture procedure and a discussion of potential alternate means of making a microbiological diagnosis. The Agency will ask the committee to discuss the options in selecting patients with bacterial disease at baseline for inclusion in trials of ABS.
As discussed above, a review of placebo-controlled trials in ABS revealed a wide range of study designs, outcomes measurements, and results. The trials span from 1970 to a recent trial in published in August, 2003. The results of the placebo-controlled trials show potential benefits of antimicrobials over placebo ranging from no benefit to 37%. However, one must take into account the design of such studies when evaluating the results. Of the 14 placebo-controlled trials in ABS, only two obtained microbiological data at baseline, one from sinus punctures and one from nasal cultures. The one study with sinus punctures used an endpoint in the trial of “ostial patency” rather than clinical and/or microbiological outcomes. The outcome assessment in the other trials was also variable, ranging from radiological resolution to more traditional measurements of clinical resolution of illness. Some of the trials used time to resolution of symptoms as an endpoint. The measurement of clinical endpoints was not standardized, and some trials used clinical scoring scales that were not validated. Other design features of the studies were also highly variable including lack of blinding, lack of randomization, lack of documentation of concomitant medications, duration of symptoms prior to enrollment or use of prior antibacterial agents.
In summary, the variability of the prior placebo-controlled trials in ABS makes it difficult to ascertain the benefit of antimicrobials and the magnitude of that benefit in ABS. Several drug sponsors have suggested using an average of the benefits from the various placebo controlled trials, yielding a non-inferiority margin of 10%-12%, however the variability in the designs of these trials would seem to preclude such an analysis.
Without the ability to determine a scientifically justifiable non-inferiority margin, according to ICH E-10, one should consider other trial designs in evaluating therapies in ABS. Other trial designs could include a dose-response trial or a placebo-controlled trial. Several drug sponsors have indicated reluctance on the part of investigators to enroll patients in placebo-controlled trials in ABS, despite the lack of knowledge about the true benefit of antimicrobials in this disease. These sponsors cite concerns about the potential complications of withholding treatment in patients with ABS, especially in patients with documented bacterial disease.
The issue of placebo-controlled trials in ABS raises the further question of the complications of ABS. Serious complications in ABS appear to be rare, and seem to be less common in patients with maxillary sinusitis as opposed to other sites such as frontal, sphenoidal, and ethmoidal sinusitis. Serious complications may also be less common in patients with acute, as opposed to chronic disease. An estimated rate of bacterial sinusitis in the U.S. of approximately 20 million cases per year and an annual rate of brain abscess in the U.S. of 1500 cases, gives a rate of 0.007% for brain abscesses in ABS. However, these numbers are estimates and there is little data on the actual incidence of either true bacterial sinusitis or brain abscess. This calculation also assumes that every case of brain abscess is associated with ABS. A recent Cochrane review called for placebo-controlled trials in ABS and also concluded that there is no evidence currently showing that antimicrobials prevent complications such as brain abscess. Prevention of complications in ABS may require studies with an exceedingly large sample size, so absence of data in this case is not evidence of absence of benefit of antimicrobial in preventing complications of ABS. The Agency will ask the committee to discuss the issues of placebo-controlled trials in ABS, the potential for other trial designs, the selection of a non-inferiority margin based on the available data, and selection of inclusion and exclusion criteria to minimize the potential for complications.
As mentioned above, when reviewing clinical trials of ABS, it becomes apparent that clinical trials in this disease use a variety of endpoints when attempting to measure the effect of antimicrobials. The overall goal of administering any drug is to improve or cure the signs and symptoms of the disease. Therefore, it would seem the clinical endpoints would be most relevant. The timing of the measurement of clinical endpoints, however, should be relevant to the natural history of the disease. For instance, in diseases with a low mortality rate and a high propensity for spontaneous resolution, selecting the timing of the assessment of clinical endpoints beyond the time at which patients would recover with placebo alone would not allow one to differentiate the effects of any drug from placebo. In self-resolving diseases, then, it may be more appropriate to measure the time to resolution or improvement of symptoms to most accurately demonstrate a difference between an effective drug and placebo. Investigators have utilized this approach in infectious diseases such as influenza and traveler’s diarrhea. In both these diseases, patients tend to have resolution of symptoms within several days of initiation of therapy in both the placebo and antimicrobial arms of the trial. If one were to evaluate resolution of clinical signs and symptoms at day 10 in influenza or traveler’s diarrhea, the time course of illness in the previous placebo-controlled trials would indicate that no drug would appear more effective than placebo. However, time to resolution of clinical signs and symptoms of disease showed a clinically meaningful shortening of disease in both influenza and traveler’s diarrhea. The Agency will ask the committee to discuss whether such an approach could be utilized in clinical trials of ABS.
Several clinical trials in ABS have measured clinical endpoints and used an assessment of time to resolution of signs and symptoms. However, review of these trials shows that the trials used non-validated scales for measuring clinical outcomes. Clinical trials, such as those used in the evaluation of drug for influenza, used validated patient reported outcomes measures and twice daily diaries to assess time to resolution of symptoms. The Agency will ask the committee to discuss whether such an approach could be utilized in clinical trials of ABS.
Surrogate markers for clinical outcomes may also be useful in the measurement of outcomes for infectious diseases when those markers have been validated and when the use of surrogate markers is appropriate. Surrogate markers are most appropriate in clinical trials where the measurement of the clinical outcomes may take months to years to measurement of surrogate markers may be accomplished in a shorter period of time, allowing the public health benefit of the drug to be realized more quickly. The utility of surrogate markers depends upon the correlation of the specific surrogate and the clinical outcome of interest. Clinical trials in ABS have used surrogate markers such as radiological resolution of disease. More recently, other investigators have raised the issue of whether microbiological outcomes could be the endpoints in clinical trials for ABS. There appears to be little information on how the time course of resolution of radiographic abnormalities or the time course of suppression or elimination of pathogenic bacteria correlate with clinical outcomes in ABS. The Agency will ask the committee to discuss surrogate endpoints and their potential utility in clinical trials for ABS.