DATE: August 16, 2001
TO: Members of the Anti-Infective Advisory Committee and
FROM: Dianne Murphy, M.D., Director, Office of Drug Evaluation IV
Renata Albrecht, M.D., Acting Deputy Director, Office of Drug
SUBJECT: Background Material for the September 13, 2001, Advisory
On Thursday, September 13, the Anti-Infective Advisory Committee will hear presentations on the history and development of approaches to the selection of deltas in clinical trials of antimicrobial drug products. This is part of an ongoing process in the Office of Drug Evaluation IV (ODE IV) to provide guidance to industry, to facilitate development of antimicrobial drug products and to promote quality and consistency in clinical trial design. This discussion of delta is part of the process begun in July 1998 when the committee heard presentations on a series of draft guidance documents, including a presentation on the general statistical considerations for the conduct of clinical trials of antimicrobial drugs. (The transcripts from this meeting can be found at
The present meeting will focus on the statistical considerations (and study implications) in the selection of delta for comparative studies. Delta is defined as the "the largest clinically-acceptable difference between the test and control drug." The majority of clinical studies submitted to FDA to support approval of new antimicrobial drug products are active control studies. In these trials the test drug is compared to another antimicrobial, typically a drug approved by the FDA for that indication. In most of these studies, the intent is to show that the efficacy of the test drug is similar to the active control. Thus, the terms equivalence or non-inferiority are used to describe these trials.
The delta is the margin selected to demarcate whether the drugs are judged equivalent or not. A placebo arm is typically not included in these studies because it is considered unethical or unacceptable to withhold antimicrobial treatment in many or most infectious diseases. The absence of a placebo arm, however, means that when equivalence is shown in a trial, it may mean the two drugs are similarly effective or it may mean they are similarly "ineffective". The importance of knowing placebo rates (or natural resolution rates) and active control efficacy rates in determining deltas will be covered in detail during the statistical presentation.
In 1992, the Division of Anti-Infective Drug Products in ODEIV published recommendations in the Points to Consider document, Clinical Development and Labeling of Anti-Infective Drug Products, on setting delta in studies of infectious diseases. To establish non-inferiority, the document recommended a "two-tailed 95% confidence interval around the difference in outcomes" approach. When the efficacy rate was 90% or greater, the lower bound of the 95% confidence interval should be no less than –10% [i.e., the delta should be 10%]; when the efficacy rate was 80%, the delta should be 15%; and when the efficacy rate was 70%, the delta should be 20%. The document also stated that this is just one possible approach and this approach may not be appropriate for serious diseases. Nevertheless, the success-rate-designated delta was used in the vast majority of comparative trials to evaluate non-inferiority. Over time, it became apparent that this approach had inherent risks and limitations.
Because the delta was based on the efficacy rate, at times the delta would be changed after the study analyses were finished and the outcome rate was known. For example, if the expected efficacy was 81%, the recommended delta applied to the study was 15% [relative to the confidence interval around the difference in outcome], whereas if that rate dropped to 79% when all the analyses were completed, a 20% delta would be applied. In some studies, this kind of change from 81% to 79% could result from just a few patients, and possibly mean the difference between approval and non-approval.
Because drugs with lower efficacy rates could be approved with wider deltas, (up to -20%) and because one could sequentially compare and approve slightly inferior products to an approved standard, the real possibility of "biocreep" was introduced, where over time a product could be approved that would not be better than placebo.
Success-rate-based deltas also did not take into consideration the specific indication under study or the seriousness of the disease. For example, in diseases with high mortality a chance that a drug might be 20% less effective may not be acceptable. Conversely, in diseases with a high natural rate of resolution without antimicrobial therapy, a delta that overlaps the "placebo" or natural resolution rate could mean approval of an ineffective drug.
We also believe that it is a fundamental element of FDA’s public health mandate is to ensure the safety and efficacy of marketed products. To do this effectively, we must review our processes and standards as advances in science provide additional understanding and insights into the development of therapeutics.
Thus, ODE IV began to re-evaluate the step-function approach. In 1998, with the publication of the draft Guidance to Industry document, Developing Antimicrobial Drugs – General Considerations for Clinical Trials, in Section XX, Statistical Considerations, ODEIV requested that the method for calculating sample size and determining delta should be specified in the protocol. (The guidance document can be found on the FDA web site, www.fda.gov/cder/guidance/index.htm, Clinical/Antimicrobial, draft.)
The Agency has been addressing the issue of non-inferiority studies and deltas both internally and with international partners. The topic has been a focus of the International Conference on Harmonisation (ICH), an international group with representation from the United States, Europe and Japan that is undertaking the standardization of requirements for drug approval by regulatory bodies throughout the world. The US Food and Drug Administration (FDA) and the Pharmaceutical Research and Manufacturing Association (PhRMA) are US members of this ICH effort. The ICH has published numerous guidance documents, two of which are applicable to the September 13th meeting and are included in this background packet. These are the ICH documents E-9: Statistical Principles for Clinical Trials and E-10: Choice of Control Group and Related Issues in Clinical Trials. The documents address principles of statistical methodology applied to clinical trials for marketing applications and issues of clinical trial design, active control studies and selection of delta. (More information regarding ICH and ICH documents is available at www.ich.org.) In addition, copies of two recent articles from the Annals of Internal Medicine by Drs. Robert Temple and Susan Ellenberg that complement the ICH documents are included in the packet.
On the basis of ODE IV discussions, and in keeping with the ICH documents, in February 2001, the FDA posted a disclaimer to the Points to Consider document, revising the section on deltas, and stating that the sliding scale for selecting delta was no longer in use. (The revised document can be found on the FDA web site – www.fda.gov/cder/guidance.index, Clinical/Antimicrobial.) A copy of the original 1992 document, and the revised pages 20-21, are included in the background material.
A few words about the regulatory perspective on deltas. Delta, the statistical limit or margin, is set or applied to the clinical outcome of the study as a whole. That is, for the test drug to be considered non-inferior to the active control drug, the lower bound of the 95% confidence interval (around the difference in outcome between test and control drug) for the complete study population should be "above" or less negative than the delta. If the lower bound of the confidence interval of the clinical or efficacy outcome "makes the delta," this information is considered supportive of approval, whereas if the lower bound of the confidence interval is outside of the delta (because it is more negative), this result does not typically support approval. However, on occasion, antimicrobial products have been approved for an indication when the delta failed to show non-inferiority, and on occasion, an indication was not approved even though the 95% CI was within the limit of the delta. In those cases, where the product may offer a benefit not available from other therapeutics (safety profiles, ease of administration, etc.), this information was considered in assessing the risk/benefit. The seriousness of the disease for which the therapy is intended and the availability of alternative therapies are also factors. A question for the committee to consider is whether there may be other factors or characteristics of drug products that may be included in the risk/benefit equation when making a regulatory decision about approving a given indication. FDA is aware of the potential impact on sample size of changing deltas and the issue will be addressed in our presentation.
The delta has not, however, been applied by ODEIV to any of the subsets of a clinical study. In particular, ODEIV has not requested that efficacy for a given subset of patients within a study or efficacy for a given pathogen in the study be able to meet a delta. A drug study would not be expected to show that two drug products are statistically "equivalent" relative to each of the pathogens that are isolated during the conduct of the clinical study, unless, of course, the entire indication is attributable to a single pathogen, e.g., Helicobacter pylori. This approach is also true for the study of drug activity against resistant pathogens -- selection of delta would not usually be expected to apply to resistant pathogens. The issues surrounding the subject of developing drugs for resistant pathogens, including the scientific and regulatory implications, are many, are complex and are currently under discussion within ODE IV; we intend to bring this topic to a future advisory committee meeting.
As stated above, this meeting is part of an ongoing process. Thus, we would like to handle the selection of delta as at least a two-part process. The September 13 meeting would serve as the first step in the process of discussing standards for active control trials. The meeting has been designed to give committee members and guests a clear, thorough overview of the statistical methodology for active control studies being adopted by FDA and the implications of these principles for clinical studies of antimicrobial drugs. During this meeting, we invite the committee members and others (through the open public hearing venue) to provide input regarding the issues raised by the E-10 approach. The FDA established a docket on July 29, 1998, in the Federal Register notice announcing the availability of the draft Guidance to Industry documents. We now invite interested parties to submit written comments and suggestions on the E-10 issues and on selection of deltas for individual indications to this docket,
98D-0548. This first meeting is not intended to set specific deltas for specific antimicrobial indications. The advice from the September 13 Advisory Committee meeting and the written comments submitted to the docket will be reviewed and deltas for clinical trials proposed. These proposed deltas would then be the subject of a subsequent advisory committee meeting and would ultimately be incorporated into the relevant guidance to industry documents on antimicrobial drug development.
* * * * * * * * *
At the September meeting, Dr. Robert Temple, Associate Director of Policy for the Center for Drug Evaluation and Research (CDER) will review the history of the ICH process, provide an overview of the E-10 document, and discuss clinical trials and delta selection perspectives and policies. Dr. Temple is one of the authors of E-10, and other ICH documents.
Dr. Renata Albrecht of ODEIV will review the broad implications of changing or tightening deltas in keeping with E-10 principles for antimicrobial drug studies and compare these to the 1992 policy.
Drs. Daphne Lin and Erica Brittain will provide an in-depth statistical perspective on active control studies and the statistical methodology underlying these designs. Their presentation will address the fundamental issues regarding non-inferiority studies that emerge from the E-10 document,
These concerns are central themes of this meeting. Do certain infectious disease indications exist for which efficacy has not been established or where any benefit of active treatment is so marginal that placebo-controlled studies should be considered? When effective therapy is known to exist for certain indications, what are the appropriate limits of non-inferiority that FDA should accept for new drug products?
Drs. Susan Thompson and John Powers will discuss how the principles elucidated by Drs. Lin and Brittain could be applied to two indications, Acute Bacterial Exacerbations of Chronic Bronchitis (AECB) and Hospital-Acquired (Bacterial) Pneumonia (HAP). Dr. George McCracken of the University of Texas will address issues in the study of Acute Bacterial Meningitis, which raise similar issues.
The materials enclosed in this binder are intended to serve as background information regarding the design of active control studies. Copies of the slides used in the presentations will be provided shortly before the meeting.
* * * * * * * * *
Finally, we wish to thank committee members and guests in advance for their time and effort in this important issue for FDA and one that will help address fundamental issues in the design of clinical trials of antimicrobial drug products.