• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services

Science & Research

  • Print
  • Share
  • E-mail

Challenges and Opportunities Report - March 2004

This report replaces the version posted on March 16, 2004. It contains a revised Figure 2, which now reflects fiscal year data for both BLAs and NMEs, and minor editorial changes.


Innovation or Stagnation:  Challenge and Opportunity on the Critical Path to New Medical Products

 

U.S. Department of Health and Human Services
Food and Drug Administration
March 2004

pdf

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

INNOVATION OR STAGNATION?

NEGOTIATING THE CRITICAL PATH

SCIENTIFIC AND TECHNICAL DIMENSIONS ALONG THE CRITICAL PATH

A BETTER PRODUCT DEVELOPMENT TOOLKIT IS URGENTLY NEEDED

TOOLS FOR ASSESSING SAFETY

Towards a Better Safety Toolkit

Getting to the Right Safety Standards

TOOLS FOR DEMONSTRATING MEDICAL UTILITY

Towards a Better Effectiveness Toolkit

Getting to the Right Effectiveness Standards

TOOLS FOR CHARACTERIZATION AND MANUFACTURING

Towards a Better Manufacturing Toolkit

Getting to the Right Manufacturing Standards

A PATH FORWARD

The Orphan Products Grant Program

The Next Steps

LIST OF TABLES AND FIGURES

Figure 1: 10-Year Trends in Biomedical Research Spending
Figure 2: 10-Year Trends in Major Drug and Biological Product Submissions to FDA
Figure 3: Investment Escalation per Successful Compound
Figure 4: The Critical Path for Medical Product Development
Figure 5: Research Support for Product Development
Figure 6: Working in Three Dimensions on the Critical Path
Figure 7: Industry - FDA Interactions During Drug Development
Figure 8: Problem Identification and Resolution During the FDA Review Process
Table 1: Three Dimensions of the Critical Path

 

 Executive Summary

This report provides the Food and Drug Administration's (FDA's) analysis of the pipeline problem -- the recent slowdown, instead of the expected acceleration, in innovative medical therapies reaching patients.

Today's revolution in biomedical science has raised new hope for the prevention, treatment, and cure of serious illnesses. However, there is growing concern that many of the new basic science discoveries made in recent years may not quickly yield more effective, more affordable, and safe medical products for patients. This is because the current medical product1 development path is becoming increasingly challenging, inefficient, and costly. During the last several years, the number of new drug and biologic applications submitted to FDA has declined significantly; the number of innovative medical device applications has also decreased. In contrast, the costs of product development have soared over the last decade. Because of rising costs, innovators often concentrate their efforts on products with potentially high market return. Developing products targeted for important public health needs (e.g., counterterrorism), less common diseases, prevalent third world diseases, prevention indications, or individualized therapy is becoming increasingly challenging. In fact, with rising health care costs, there is now concern about how the nation can continue to pay even for existing therapies. If the costs and difficulties of medical product development continue to grow, innovation will continue to stagnate or decline, and the biomedical revolution may not deliver on its promise of better health.

What is the problem? In FDA's view, the applied sciences needed for medical product development have not kept pace with the tremendous advances in the basic sciences. The new science is not being used to guide the technology development process in the same way that it is accelerating the technology discovery process. For medical technology, performance is measured in terms of product safety and effectiveness. Not enough applied scientific work has been done to create new tools to get fundamentally better answers about how the safety and effectiveness of new products can be demonstrated, in faster time frames, with more certainty, and at lower costs. In many cases, developers have no choice but to use the tools and concepts of the last century to assess this century's candidates. As a result, the vast majority of investigational products that enter clinical trials fail. Often, product development programs must be abandoned after extensive investment of time and resources. This high failure rate drives up costs, and developers are forced to use the profits from a decreasing number of successful products to subsidize a growing number of expensive failures. Finally, the path to market even for successful candidates is long, costly, and inefficient, due in large part to the current reliance on cumbersome assessment methods.

A new product development toolkit -- containing powerful new scientific and technical methods such as animal or computer-based predictive models, biomarkers for safety and effectiveness, and new clinical evaluation techniques -- is urgently needed to improve predictability and efficiency along the critical path from laboratory concept to commercial product. We need superior product development science to address these challenges -- to ensure that basic discoveries turn into new and better medical treatments. We need to make the effort required to create better tools for developing medical technologies. And we need a knowledge base built not just on ideas from biomedical research, but on reliable insights into the pathway to patients.

The medical product development process is no longer able to keep pace with basic scientific innovation. Only a concerted effort to apply the new biomedical science to medical product development will succeed in modernizing the critical path.

Many accomplished scientists in academia, government, and industry are working on these challenges, and there has been much success in recent years. But the fact remains that the pace of this development work has not kept up with the rapid advances in product discovery. The result is a technological disconnect between discovery and the product development process -- the steps involved in turning new laboratory discoveries into treatments that are safe and effective.

Although the FDA is just one participant in advancing development science, we have an important role to play. Because FDA's standards are often used to guide development programs, we need to make sure that our standard-setting process is informed by the best science, with the goal of promoting efficient development of safe and effective new medical treatments.

Because FDA is uniquely positioned to help identify the challenges to development, we need to work with the larger scientific community on developing solutions. Directed by Congress to promote and protect the public health, FDA is responsible for ensuring that safe and effective medical innovations are available to patients.2 As part of its regulatory role, FDA must use available scientific knowledge to set product standards. During clinical testing, FDA scientists conduct ongoing reviews of emerging data on safety, efficacy, and product quality. Agency reviewers see the complete spectrum of successes and best practices during clinical trials, as well as the failures, slowdowns, barriers, and missed opportunities that occur during product development. When serious problems emerge in the development process or common problems continue to recur, FDA scientists attempt to address them by bringing them to the attention of the scientific community, or by conducting or collaborating on relevant research. As an example of such work, the Agency often makes guidance documents publicly available that summarize best practices in a development area and share FDA insights into specific issues or topics. Sponsors report that the availability of guidance documents has been shown to foster development and innovation in areas of therapeutic need, to improve the chances of initial success of a marketing application, and to shorten the time it takes to get safe and effective treatments to patients. But much more needs to be done.

The product development problems we are seeing today can be addressed, in part, through an aggressive, collaborative effort to create a new generation of performance standards and predictive tools. The new tools will match and move forward new scientific innovations and will build on knowledge delivered by recent advances in science, such as bioinformatics, genomics, imaging technologies, and materials science.

FDA is planning an initiative that will identify and prioritize (1) the most pressing development problems and (2) the areas that provide the greatest opportunities for rapid improvement and public health benefits. This will be done for all three dimensions along the critical path -- safety assessment, evaluation of medical utility, and product industrialization. It is critical that we enlist all relevant stakeholders in this effort. We will work together to identify the most important challenges by creating a Critical Path Opportunity List. Concurrently, FDA will refocus its internal efforts to ensure that we are working on the most important problems and intensify our support of key projects.

Through scientific research focused on these challenges, we can improve the process for getting new and better treatments to patients. Directing research not only to new medical breakthroughs, but also to breakthrough tools for developing new treatments, is an essential step in providing patients with more timely, affordable, and predictable access to new therapies. We are confident that, with effective collaboration among government, academia, and the private sector, these goals can be achieved.

 

 Introduction

The mission of the U.S. Food and Drug Administration (FDA) is, in part, to protect the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, and medical devices. The FDA is also responsible for advancing the public health by helping to speed innovations that make medicines more effective, safer, and more affordable; and helping the public get the accurate, science-based information they need to use medicines to improve their health.

In keeping with its mission, FDA is issuing this report to address the growing crisis in moving basic discoveries to the market where they can be made available to patients. The report evaluates how the crisis came about and offers a way forward. It highlights examples of Agency efforts that have improved the critical path and discusses opportunities for future efforts. Finally, the report calls for a joint effort of industry, academia, and the FDA to identify key problems and develop targeted solutions.

 

 Innovation or Stagnation? Challenge and Opportunity on the Critical Path to New Medical Products

The sequencing of the human genome four years ago raised widespread hope for a new era in the prevention and treatment of disease created by the ongoing investment in biomedical research (Figure 1). But that new era has not yet arrived. Instead, 2000 marked the start of a slowdown in new3 drug and biologic submissions to regulatory agencies worldwide (Figure 2). The submission of innovative medical device applications has also slowed recently.4 This means fewer new products can be approved and made available to patients. At a time when basic biomedical knowledge is increasing exponentially, the gap between bench discovery and bedside application appears to be expanding. There is great concern about the ability to bring the hoped-for outcomes of basic research advances -- much awaited new treatments -- to patients. There is concern that hoped-for advances in medicine and new treatments for diseases may never materialize.

Current costs of bringing a new medicine to market, estimated by some to be as high as $0.8 to 1.7 billion,5 are a major barrier to investment in innovative, higher risk drugs or in therapies for uncommon diseases or diseases that predominantly afflict the poor. Product development in areas crucial to public health goals, such as antibiotics, has slowed significantly during the past decade. Inventors of candidate artificial organs, bioengineered tissues, and other novel devices face serious challenges and uncertainties. A viable path for developing many preventive therapies (e.g., some types of cancer chemoprevention) has not been elucidated.

Recent basic science achievements promise significant payoffs in human health, but these potential benefits are threatened by low productivity -- measured by the high costs and high risks of failure in the current development processes and the declining number of successful products reaching patients. Often, developers are forced to rely on the tools of the last century to evaluate this century's advances. And the situation does not appear to be improving. Recent data suggest that the investment required to launch a new drug has risen 55 percent during the last five years (Figure 3). Pharmaceutical, biotechnology, and medical device productivity appears to be declining at the same time that the costs to develop a small number of treatments are rising.

If biomedical science is to deliver on its promise, scientific creativity and effort must also focus on improving the medical product development process itself, with the explicit goal of robust development pathways that are efficient and predictable and result in products that are safe, effective, and available to patients. We must modernize the critical development path that leads from scientific discovery to the patient (Figure 4).

In response to the widening gap between basic biomedical knowledge and clinical application, governments and the academic community have undertaken a range of initiatives. After decades of investment in basic biomedical research, the focus is widening to include translational research -- multidisciplinary scientific efforts directed at "accelerating therapy development" (i.e., moving basic discoveries into the clinic more efficiently).6 Notable are:

  • National Institutes of Health (NIH) Roadmap, announced in September 2003. This is a series of initiatives intended to "speed the movement of research discoveries from the bench to the bed side"7
  • National Cancer Institute's (NCI) Specialized Programs of Research Excellence (SPOREs)8
  • MdBIO, a private nonprofit corporation that supports the growth of bioscience in Maryland9
  • The European Organization for the Treatment of Cancer (EORTC) is committed to making translational research a part of all cancer clinical trials10
  • The British government announced the National Translational Cancer Research Network to facilitate and enhance translational research in the United Kingdom11

Although necessary for product development, these translational research efforts will not yield the hoped-for results without an analogous focus on downstream development concerns. As one group has observed, "Massive investments in one part of the network are likely to be at least partly wasted unless the other links are strengthened as well."12 A third type of scientific research is urgently needed, one that is complementary to basic and translational research, but focuses on providing new tools and concepts for the medical product development process -- the steps that must be taken to get from selection of a laboratory prototype to delivery of an effective treatment to patients. We call this highly targeted and pragmatic research critical path research because it directly supports the critical path for product development success (Figure 5).

 Negotiating the Critical Path

To get medical advances to patients, product developers must successfully progress along a multidimensional critical path that leads from discovery or design concept to commercial marketing.

Currently, a striking feature of this path is the difficulty, at any point, of predicting ultimate success with a novel candidate. For example, a new medicinal compound entering Phase 1 testing, often representing the culmination of upwards of a decade of preclinical screening and evaluation, is estimated to have only an 8 percent chance of reaching the market. This reflects a worsening outlook from the historical success rate of about 14 percent.13 In other words, a drug entering Phase 1 trials in 2000 was not more likely to reach the market than one entering Phase 1 trials in 1985.14 Recent biomedical research breakthroughs have not improved the ability to identify successful candidates.

The main causes of failure in the clinic include safety problems and lack of effectiveness: inability to predict these failures before human testing or early in clinical trials dramatically escalates costs. For example, for a pharmaceutical, a 10-percent improvement in predicting failures before clinical trials could save $100 million in development costs per drug.15 In the case of medical devices, current capacity for technological innovation has outstripped the ability to assess performance in patients, resulting in prolonged delays between design and use. For very innovative and unproven technologies, the probability of an individual product's success is highly uncertain, and risks are perceived as extremely high. Whole fields may stagnate as a result of the failure of early products. The goal of critical path research is to develop new, publicly available scientific and technical tools -- including assays, standards, computer modeling techniques, biomarkers, and clinical trial endpoints -- that make the development process itself more efficient and effective and more likely to result in safe products that benefit patients. Such tools will make it easier to identify earlier in the process those products that do not hold promise, thus reducing time and resource investments, and facilitating the process for development of medical products that hold the most promise for patients.

 Scientific and Technical Dimensions Along the Critical Path

Whether working with devices, drugs, or biologicals -- medical product developers must negotiate three crucial scientific/technical dimensions on the critical path from scientific innovation to commercial product (see Table 1). These three dimensions are interdependent, and in none is success assured. The vast majority of development costs are attributable to these three dimensions.

 Table 1:  Three Dimensions of the Critical Path

 

DimensionDefinitionExamples of Activities
Assessing SafetyShow that product is adequately safe for each stage of development
  • Preclinical:  show that product is safe enough for early human testing
  • Eliminate products with safety problems early
  • Clinical:  show that product is safe enough for commercial distribution
Demonstrating Medical UtilityShow that the product benefits people
  • Preclinical:  Select appropriate design (devices) or candidate (drugs) with high probability of effectiveness 
  • Clinical:  show effectiveness in people
IndustrializationGo from lab concept or prototype to a manufacturable product
  • Design a high-quality product
    • Physical design
    • Characterization
    • Specifications
  • Develop mass production capacity
    • Manufacturing scale-up
    • Quality control

 

This table refers to scientific and technical dimensions.  Other business dimensions (e.g., obtaining capital, intellectual property considerations, marketing and distribution arrangements) are not within the scope of this table.

Developers must manage the interplay between each dimension from the earliest phases of development. For example, the first dimension -- ensuring product safety -- is crucial to consider when designing a drug molecule, choosing production cell lines or reference strains for biological production, or selecting biomaterials for an implanted medical device (Figure 6). The traditional tools used to assess product safety -- animal toxicology and outcomes from human studies -- have changed little over many decades and have largely not benefited from recent gains in scientific knowledge. The inability to better assess and predict product safety leads to failures during clinical development and, occasionally, after marketing.

The second dimension, demonstrating the medical utility of a new product -- showing that it will actually benefit people -- is the source of innumerable failures late in product development. Better tools are needed to identify successful products and eliminate impending failures more efficiently and earlier in the development process. This will protect subjects, improve return on R&D investment, and bring needed treatments to patients sooner.

A number of authors have raised the concern that the current drug discovery process, based as it is on in vitro screening techniques and animal models of (often) poorly understood clinical relevance, is fundamentally unable to identify candidates with a high probability of effectiveness.16 17The current scientific understanding of both physiology and pathophysiologic processes is of necessity reductionistic (e.g., is knowledge at the gene, gene expression or pathway level) and does not constitute knowledge at the level of the systems biology of the cell, organ, or whole organism, and certainly does not reach a systems understanding of the pathophysiology of particular diseases. Reaching a more systemic and dynamic understanding of human disease will require major additional scientific efforts as well as significant advances in bioinformatics. Nevertheless, progress in discovery will continue,18 and as candidates emerge, the best tools available should be used for their evaluation. This will require strengthening and rebuilding the relevant disciplines (e.g., physiology, pharmacology, clinical pharmacology) and working to identify ways to bridge between the laboratory and the whole organism and correlate early markers of safety and benefit with actual outcomes in patients.

In addition, it is likely that more interest will develop in earlier "proof-of-concept" trials that seek to confirm activity in humans before a commitment to full-scale development is made. The FDA is working to facilitate such studies.

The final dimension on the critical path can be described as the industrialization process -- turning a laboratory concept into a consistent and well-characterized medical product that can be mass produced. The challenges involved in successful industrialization are complex, though highly underrated in the scientific community. Problems in physical design, characterization, manufacturing scale-up and quality control routinely derail or delay development programs and keep needed treatments from patients. These problems are often rate-limiting for new technologies, which are frequently more complex than traditional products and lack standard assessment tools.

 A Better Product Development Toolkit Is Urgently Needed

It is clear to FDA scientists, who have a unique vantage point and experience base, that a better product development toolkit is urgently needed. The Agency oversees all U.S. human trials and development programs for investigational medical products. As part of its regulatory role, FDA works with the scientific community to set the clinical and technical standards used in development. During the clinical phases of product development, Agency scientists conduct ongoing reviews of product safety, efficacy, and quality data. At the marketing application stage, data submitted by medical product sponsors are evaluated against the established scientific standards. FDA scientists are in frequent communication with industry and academic scientists over development issues (Figure 7). Agency reviewers see the successes and associated best practices as well as the failures, slowdowns, barriers, and missed opportunities that occur during the course of product development. In addition, data on product testing, safety evaluation, and clinical trials are stored in the millions of pages of FDA files. FDA reviewers oversee the totality of the preapproval development process. Because of this perspective, FDA reviewers are in a unique position to help identify common themes and systematic weaknesses across similar products and can draw important lessons from what they see.

Few other groups of physicians and scientists are positioned to see so much of the broad picture. Of course, industry scientists encounter these problems in terms of their own product portfolios, but often lack cross-cutting information about an entire product area, or complete information about techniques that may be used in areas other than theirs. Academic programs focused on the medical product development process are rare and, at present, cannot be informed by FDA's broad experience with often confidential information. In fact, since the details of most failed programs cannot possibly be shared publicly or for applied research purposes, FDA holds the only broad, cross-cutting knowledge about how certain investigational products fail, why certain therapeutic areas remain underdeveloped, and when certain development hurdles persist despite advances in technology that could mitigate them. Indeed, these failures may trigger regulatory actions such as putting clinical holds on human trials, or turning down applications. In the course of such an action, FDA identifies problems and offers advice on how to overcome them. Advice given to product developers is based on FDA's experience with the totality of other applications and FDA's efforts to keep up with the latest science; it does not reflect specific proprietary information from individual applications. Despite these efforts, the ability of product developers and FDA scientists to overcome development challenges is often confounded by the limitations of current tools to address development challenges.

When the tools and concepts fall short, FDA works proactively with product developers and the scientific community to identify and resolve critical development problems and stimulate research, encouraging the development of solutions. The Agency often makes this information available to the public through guidance documents that synthesize current knowledge on approaches to development problems, or, as appropriate, through workshops, or peer reviewed publications (Figure 8). Guidance documents can also help ensure that FDA's safety and effectiveness standards in a particular area of product development are up to date.

Sponsors report that the availability of FDA guidance documents19 often substantially decreases uncertainties associated with product development. Our own research has confirmed this. For example, compared to device development lacking FDA guidance, medical devices developed in areas with extant FDA guidance documents are almost twice as likely to be approved after the initial review process and are approved in a third less time.20 FDA has undertaken efforts to develop such guidances in some of the most crucial public health issues.

There is currently an urgent need for additional public-private collaborative work on applying technologies such as genomics, proteomics, bioinformatics systems, and new imaging technologies to the science of medical product development. Properly applied, these new technologies could provide tools to detect safety problems early, identify patients likely to respond to therapy, and lead to new clinical endpoints. New medical technologies, including bioengineered tissues, cellular and gene therapies, nanotechnology applications, novel biomaterials, and individualized drug therapies, will all need new product development tools and standards, as discussed below, to be able to move from the laboratory to the market quickly and safely.

There is also an urgent need to improve the efficiency and effectiveness of the clinical trial process, including trial design, endpoints, and analyses. The NIH is addressing very important clinical research infrastructure problems in its Roadmap initiative, and FDA is collaborating in the Roadmap efforts. In addition, much more attention and creativity need to be applied to disease-specific trial design and endpoints intended to evaluate the effects of medical products.

 Tools for Assessing Safety

For effective development, safety issues should be detected as early as possible, and ways to distinguish potential from actual safety problems should be available. Unfortunately, in part because of limitations of current methods, safety problems are often uncovered only during clinical trials or, occasionally, after marketing. One pharmaceutical company estimates that clinical failures based on liver toxicity alone have cost them more than $2 billion in the last decade -- dollars that could potentially be directed toward successful new product development.21 Sometimes, early tests suggest the possibility of safety problems that never materialize, potentially eliminating candidates unnecessarily. Many of FDA's targeted efforts to date have involved defining more reliable methods for early prediction and detection of significant safety problems. The Agency seeks to prevent harm to patients during clinical development as well as potentially devastating setbacks to a new technology's progress and to public confidence.

Tools for safety assessments include product testing (e.g., for contamination), as well as in vitro and animal toxicology studies, and human exposure. Despite some efforts to develop better methods, most of the tools used for toxicology and human safety testing are decades old. Although traditional animal toxicology has a good track record for ensuring the safety of clinical trial volunteers, it is laborious, time-consuming, requires large quantities of product, and may fail to predict the specific safety problem that ultimately halts development. Clinical testing, even if extensive, often fails to detect important safety problems, either because they are uncommon or because the tested population was not representative of eventual recipients. Conversely, some models create worrisome signals that may, in fact, not be predictive of a human safety problem.

Many of FDA's recent targeted efforts have involved working with the scientific community to define more reliable methods to predict and detect significant safety problems. For example, in the past, failure to predict unfavorable human metabolism of candidate drugs has led to costly failures in the clinic as well as multiple drug market withdrawals. FDA recommendations on the use of human cell lines to characterize drug metabolic pathways provide a straightforward in vitro method for prediction of human metabolism, allowing developers to eliminate early on compounds with unfavorable metabolic profiles (e.g., drug-drug interaction potential). Failures in the clinic due to drug interaction problems are now far less likely.

In another effort, FDA developed and standardized methods for documenting clearance of retrovirus-like particles from tissue culture fluids. This effort successfully addressed potential safety concerns that surrounded the early use of monoclonal antibodies and paved the way for the development of many important medical treatments. Through its own laboratory efforts, FDA has continued to refine these methods, share them publicly, and reduce their cost.

Additional examples of FDA efforts are listed under Highlights.

Highlight: Tools for Assessing Safety

  1. The need to ensure the safety of biological products by preventing contamination has resulted in numerous Agency research programs and resulting animal models, test methods, and technical standards.
    • A reference standard for evaluating gene therapy vector contamination by retroviruses has been developed with FDA input and is being distributed by the American Type Tissue Collection (ATTC).
    • In the wake of concern over the safety of gene therapies for genetic diseases, FDA developed an animal model for assessing the safety of adenovirus vectors.
    • FDA developed several rodent toxicity models to assess the neurovirulence of live virus vaccines, an approach that has both reduced the use of primates for testing and sped the testing process.
    • With the potential resurgent need for smallpox vaccination, FDA scientists developed a new technique to detect the presence of contaminating virus in smallpox vaccine products. This technique can also be applied to characterization of othervaccine and cellular products.
  2. FDA collaborated with industry and scientific groups to develop the data that allowed international adoption of a transgenic mouse model for drug carcinogenicity testing. This assay takes less time, saves two thirds of the cost, and uses half as many animals as a traditional study.
  3. FDA has mined its databases to develop structure-activity relationship software to help identify molecular substructures with potentially negative toxicologic properties early in the development process.

 Towards a Better Safety Toolkit

There are currently significant needs, but also significant opportunities, for developing tools that can more reliably and more efficiently determine the safety of a new medical product.

Examples of tools that are urgently needed include better predictors of human immune responses to foreign antigens, methods to further enhance the safety of transplanted human tissues, new techniques for assessing drug liver toxicity, methods to identify gene therapy risks based on assessment of gene insertional and promotional events, and efficient protocols for qualifying biomaterials.

Opportunity: Proteomic and toxicogenomic approaches may ultimately provide sensitive and predictive safety assessment techniques; however, their application to safety assessment is in early stages and needs to be expanded.22 Targeted research aimed at specific toxicity problems should be undertaken.

Opportunity: As biomedical knowledge increases and bioinformatics capability likewise grows, there is hope that greater predictive power may be obtained from in silico (computer modeling) analyses such as predictive toxicology. Some believe that extensive use of in silico technologies could reduce the overall cost of drug development by as much as 50 percent.23

  • FDA's files constitute the world's largest repository of in vitro and animal results that are linked with actual human outcomes data. Further datamining efforts that effectively protect proprietary data could form the basis for useful predictive safety models.
  • Use of extant clinical data may help construct models to screen candidates early in the development process (e.g., for liver toxicity).

Opportunity: There is an urgent need to develop tools to accurately assess the risk of new drugs causing heart rhythm abnormalities. For instance, there are ongoing international efforts to develop, test, and validate nonclinical models that may be useful in predicting human risk. In addition, the clinical risks associated with a small degree of QTc interval prolongation need to be fully defined.

The above are only a few of the opportunities FDA reviewers and outside experts have identified.

 Getting to the Right Safety Standards

Because safety issues are a significant cause of delay and failure during development, some have advocated simply lowering safety standards. This is not a preferable solution. For ethical human testing, there is wide agreement that reasonable assurance of safety must be achieved before clinical trials begin. Patients, prescribers, payers, and the public share the expectation that marketed medical products will have a well-understood safety profile and a positive benefit/risk analysis. Today's problems arise from the inability to confidently predict safety performance in a timely and efficient manner. Current tools are not only cumbersome, they are also imprecise and thus leave considerable residual uncertainty. The degree of uncertainty inherent in current techniques can result in conservative standard setting. We need new tools that can eliminate problem products early and can better predict ultimate safety performance. Applied critical path research provides the real opportunity for improving our ability to identify safety issues early and manage the remaining risks appropriately.

 Tools for Demonstrating Medical Utility

Predicting and subsequently demonstrating medical utility (also called benefit or effectiveness) are some of the most difficult and important challenges in product development. Currently available animal models, used for evaluating potential therapies prior to human clinical trials, have limited predictive value in many disease states. Better predictive nonclinical screening methods are urgently needed. In many cases, developers must gamble on the results of the large-scale, expensive trials necessary to assess effectiveness in people. Such human trials are currently highly empirical, because most sources of variability in human responses are not understood and thus cannot be controlled for. It is clear to many in the field that new scientific advances have the potential to revolutionize clinical development. However, the path from scientific innovation to usable tool is not clear.

FDA has identified a number of opportunities for targeted efforts in the area of effectiveness (see next section) and, where feasible, has undertaken targeted action. For example, FDA scientists developed statistical methods to control reader variability in trials of imaging devices and made the analysis software publicly available. Use of this method allows developers to reduce the sample size of imaging device trials by as much as 60 percent.24 Similarly, FDA analysis of hypertension trials using automated blood pressure monitoring allowed for elimination of the placebo group in such trials.

Adopting a new biomarker or surrogate endpoint for effectiveness standards can drive rapid clinical development. For example, FDA adoption of CD4 cell counts and, subsequently, measures of viral load as surrogate markers for anti-HIV drug approvals allowed the rapid clinical workup and approval of life-saving antiviral drugs, with time from first human use to market as short as 3.5 years. FDA convened the data holders, conducted analyses in conjunction with industry and academia, and provided guidance on trial design. Similarly, FDA adoption of the eradication of H. pylori as a surrogate for duodenal ulcer healing greatly simplified the path of those therapies to the market. FDA often approves vaccines based on their meeting validated surrogate markers for achieving protective levels of immunity. This greatly simplifies effectiveness studies, thus reducing time and costs.

Highlights of other recent FDA efforts are provided on the following page. Although there are many examples of successful outcomes, similar efforts are needed in many other areas of product development to improve the process for getting safe and effective new treatments to patients.

Highlight: Answering the Challenge of Bioterrorism -- Evaluating Efficacy

With the increasing challenges of bioterrorism, there is both a need and an opportunity for animal models that are relevant and predictive of countermeasure effectiveness in humans, since effectiveness testing in humans is often not feasible. In some cases, approval can be granted on the basis of animal model findings. FDA and its partners can play a major role in both developing such models and helping define appropriate and efficient pathways for their use in product development. Such efficiency is critical both for proper stewardship of what are often limited or ethically sensitive animal resources, as well as for ensuring reliable threat preparedness in a timely manner.

  • FDA developed an immunocompromised mouse model for studying the efficacy of treatments for smallpox vaccine side effects.
  • FDA defined appropriate animal studies to evaluate the efficacy of next generation anthrax vaccines.
  • Working with government and academic scientists, FDA developed protocols for the efficient use of animal models to evaluate antimicrobial efficacy against bioterrorist threat agents.

Highlight: Trial Design for Digital Mammography -- Overcoming Clinical Trial Hurdles

Although the initial approval of digital mammography did not include this claim, it was believed that digital techniques would prove more accurate than the conventional screen film. A 40,000-patient study would be needed to evaluate this.

No company was able to do a 40,000-patient study. FDA proposed a trial in which four companies would each do a study of 10,000 patients, using a common protocol. The National Cancer Institute (NCI) was willing to conduct the study. The results from the four arms of the study could be pooled. The pooled trial will be able to test whether digital mammography is superior to conventional screen-film, and each firm will be able to use results from its own product. The trial costs have been shared among the companies and the NCI. The trial is completely enrolled and in the 1-year follow-up phase.

 Towards a Better Effectiveness Toolkit

We believe targeted efforts in a variety of areas could substantially improve the efficacy toolkit. These efforts, a few examples of which are listed here, can only be successful with the collaboration of industry, academia, and the patient and health care communities.

Opportunity: FDA actions and the subsequent passage of the “Best Pharmaceuticals for Children Act”25 have spurred a significant increase in the number of pediatric studies of pharmaceuticals. Although the results of each individual trial have been informative for the particular drug studied, a significant opportunity now exists for analysis of what has been collectively learned about the pharmacokinetics, pharmacodynamics, safety, and efficacy of drugs in children. Such an analysis could begin to build a knowledge base to better inform future pediatric studies.

Opportunity: "The appearance of new quantitative measuring technologies absolutely galvanizes new drug research."26 Additional biomarkers (quantitative measures of biological effects that provide informative links between mechanism of action and clinical effectiveness) and additional surrogate markers (quantitative measures that can predict effectiveness) are needed to guide product development. In some cases, datamining and analysis, with possibly a single additional clinical trial, may be all that is necessary to confirm the surrogacy of a particular marker. In other cases (e.g., the NIH's Osteoarthritis Initiative27), epidemiologic studies on disease natural history must be undertaken to provide data on markers of disease processes. For biomarkers that currently appear promising, specific projects need to be undertaken to:

  • Assemble existing data on the association of the marker with clinical outcomes
  • Assemble existing data on the performance of the marker during intervention trials compared to the performance of current outcome measures
  • Identify any data gaps or remaining uncertainties
  • Identify clinical trials under development in which the remaining questions could be addressed in a straightforward manner

As previously stated, strengthening and rebuilding the disciplines of physiology, pharmacology, and clinical pharmacology will be necessary to provide the capacity to develop and evaluate new biomarkers and bridge across animal and human studies.

Opportunity: Imaging technologies, such as molecular imaging tools in neuropsychiatric diseases or as measures of drug absorption and distribution, may provide powerful insights into the distribution, binding, and other biological effects of pharmaceuticals, but their predictive value needs further study and evaluation. New imaging technologies will ultimately contribute important biomarkers and surrogate endpoints, but how soon these new tools will be available for use will depend on the effort invested in developing them specifically for this purpose.

Opportunity: For many therapeutics, effectiveness criteria are best defined by the practitioners and patients who use the products. Much work needs to be done on clinical trial design and patient-driven outcome measures to ensure that endpoints in new therapeutic areas accurately reflect patient needs and values. Community (health professional and patient) consensus on appropriate outcome measures and therapeutic claims can lay a clear development path for new therapeutics, especially when there is international regulatory harmonization.

Opportunity: The concept of model-based drug development, in which pharmaco-statistical models of drug efficacy and safety are developed from preclinical and available clinical data, offers an important approach to improving drug development knowledge management and development decision making.28 Model-based drug development involves building mathematical and statistical characterizations of the time course of the disease and drug using available clinical data to design and validate the model. The relationship between drug dose, plasma concentration, biophase concentration (pharmacokinetics), and drug effect or side-effects (pharmacodynamics) is characterized, and relevant patient covariates are included in the model. Systematic application of this concept to drug development has the potential to significantly improve it. FDA scientists use, and are collaborating with others in the refinement of, quantitative clinical trial modeling using simulation software to improve trial design and to predict outcomes. It is likely that more powerful approaches can be built by completing, and then building on, specific predictive modules.

There are many important additional opportunities in the area of clinical trial design and analysis. More clinically relevant endpoints need to be developed for many diseases. Enrichment designs have the potential for providing much earlier assurance of drug activity. Bayesian approaches to analysis need to be further explored.

Opportunity: The emerging techniques of pharmacogenomics and proteomics show great promise for contributing biomarkers to target responders, monitor clinical response, and serve as biomarkers of drug effectiveness. However, much development work and standardization of the biological, statistical, and bioinformatics methods must occur before these techniques can be easily and widely used. Specific, targeted efforts could yield early results.

 Getting to the Right Effectiveness Standards

In an era of concerns about health care affordability, we need to make sure that new medical products are effective and provide accurate up-to-date information about using them so patients and doctors can make smart decisions about health care. As health care costs rise, patients, medical professionals, and health care purchasers are all demanding more value from the medical treatments they use. With more treatments in development than ever before, finding better ways to demonstrate their effectiveness for particular kinds of patients is essential for making sure that all Americans get the most value from their health care dollars.

 Tools for Characterization and Manufacturing

The industrialization challenges posed by the demands of physical product design, characterization, scale-up, and manufacturing are often little understood outside of FDA and the pharmaceutical and device manufacturing communities.29 Many product failures during development are ultimately attributable to problems relating to the transition from laboratory prototype to industrial product. It is crucial that technical standards (e.g., assays, procedures, or reference standards) and improved methods for design, characterization, and product manufacture are available to improve predictability in this area.

Developing interim standards is especially important for novel technologies and can help keep product development on track as a new field matures. Otherwise, innovators are put in the position of having to invent standards in addition to inventing new products. At the same time, interim standards must allow for flexibility, innovation, and change as new fields develop. This takes expertise, effort, and collaboration among industry, academia, and FDA.

For example, recombinant proteins and monoclonal antibodies have provided significant therapeutic advances over the last 15 years. During this time, FDA has issued multiple technical guidance documents on topics such as characterization of production cell lines, manufacturing and testing techniques, specifications, stability evaluation, and changes to manufacturing processes.30 Recent guidances address the use of transgenic animals or bioengineered plants as production methods for such products.

As new industrialization challenges are identified during the review process, Agency scientists routinely hold scientific workshops, conduct research, collaborate with academic and industrial scientists and synthesize the emerging data. Recently, when safety problems developed with gene therapy adenovirus vectors, the need for a better potency standard was recognized. FDA collaborated with industry and governmental partners to develop the currently available reference standard for characterization of adenovirus vectors. To stimulate the needed vaccine development efforts, FDA scientists recently developed a breakthrough synthetic technology for conjugate bacterial vaccines that increases yields three fold and also lowers costs. For additional examples, see Highlights below.

Highlight: Industrialization

In the area of medical devices, blood glucose monitors represent a critical technology for many of the 16,000,000 diabetics in the United States. Numerous new devices are being developed for blood glucose monitoring.

  • FDA helped develop a uniform testing protocol to evaluate glucose meter performance and compared the measurements to the hexokinase (HK) laboratory method incorporating reference materials developed by the National Institute of Standards and Technology.
  • It was determined that separate accuracy and precision goals should be defined for extreme ranges to keep pace with changing clinical demands for tighter glucose measurement.31

Highlight: Industrialization Standards

Together with CDC and industry, FDA was able to help make available difficult-to-obtain standards and samples needed for the successful rapid development and evaluation of West Nile Virus nucleic acid blood donor screening.

 Towards a Better Manufacturing Toolkit

Rapid, successful development of new medical technologies depends on the concomitant availability of adequate methods to characterize, standardize, control, and mass produce them. Applied research in these areas is required to provide the infrastructure necessary for translating laboratory prototypes into commercial products. There are a number of urgent needs in the industrialization area. FDA is actively working on guidance in many of these areas to the extent permitted by available resources.

Opportunity: Additional characterization procedures and standards for expanded stem cell and other cellular products, bioengineered tissues, and implanted drug-device combinations (e.g., drug-eluting stents) are urgently needed. For example, developing test standards for coronary stent compressibility will decrease the likelihood of failed designs and allow smaller clinical trial programs.

Opportunity: The pharmaceutical industry generally has been hesitant to introduce state-of-the-art science and technology into its manufacturing processes, in part due to concern about regulatory impact. This led to high in-process inventories, low factory utilization rates, significant product wastage, and compliance problems, driving up costs and decreasing productivity. FDA has led an initiative to stimulate the use of process analytical technologies -- automated sensors that monitor and control processes -- and other modern manufacturing technologies that can improve efficiency and increase flexibility while maintaining high-quality standards. Further research and data sharing are necessary to make these efficiencies a reality.

Opportunity: Scientists involved in reviewing medical devices at FDA report an urgent need for predictive software to model the human effects of design changes for rapidly evolving devices. We believe that such software may be attainable with a concentrated effort, by assembling currently available data and identifying existing data gaps.

 Getting to the Right Manufacturing Standards

Problems with scale-up and mass production can also slow development and escalate costs. Currently, FDA is involved in an extensive, multi-year effort to incorporate the most up-to-date science into its regulation of pharmaceutical manufacturing and to encourage industry to adopt innovative manufacturing technologies.32 Moreover, we are also looking critically at areas where FDA regulation may have slowed adoption of improvements.

The availability of efficient, science-based standards for product characterization and manufacturing creates a win-win for consumers, patients, and the industry.

 A Path Forward

Greater success along the critical path demands greater activity in a specific type of scientific research that is directed at modernizing the product development process. This critical path research -- highly pragmatic and targeted in its focus on issues such as standards, methods, clinical trial designs, and biomarkers -- is complementary to, and draws extensively from, advances in the underlying basic sciences and new technologies. Without a concerted effort to improve the critical path, it is likely that many important opportunities will be missed and frustration with the slow pace and poor yield of traditional development pathways will continue to escalate.

Dealing with product development problems is the day-to-day work not only of clinical research and product developers, but also of FDA review scientists. The Agency frequently attempts to resolve problems identified during the review process. Extensive experience in evaluating and working to overcome hundreds of product development challenges and roadblocks has enabled FDA to intervene in a targeted manner, helping to reduce or remove specific obstacles in areas critical to public health. However, there are a host of additional opportunities where more progress is both necessary and possible. Due to the scope of the existing problems in product development and the expected surge in products resulting from investments in translational research, we believe that critical path research and standards programs should be high priority to help ensure that scientific innovations can be translated efficiently into public health benefits. These additional efforts should be targeted towards removing specific identified obstacles in development. Although there are numerous public and private groups with expertise to help develop solutions, we believe that FDA is ideally positioned to bring together the stakeholders to identify and address the most significant problems. We believe that efforts targeted at significant challenges and roadblocks have yielded important returns, and can have even greater public health benefits in the near future.

 The Orphan Products Grant Program

FDA's Orphan Products grant program provides an instructive example of a successful targeted intervention. This program provides up to three years of very modest funding ($150,000-300,000 per annum) for clinical development costs of qualified products. Between 1989 and 2003, FDA approved 36 novel products (including 23 novel drugs) participating in this program. Thus orphan grant recipients have been an appreciable part of the 20 to 40 new drugs approved yearly during the last 14 years, despite the fact that industrial development of drugs for such limited uses is traditionally very hard to stimulate and only limited funding has been available.33 Recipients of orphan grants also benefit from advice and direction from FDA scientific reviewers on surmounting development obstacles. This program is widely viewed as a major success in assisting in development of treatments for rare diseases, at a very modest investment. FDA is conducting an internal review of how the successes of the Orphan Products development research might be applied to other kinds of critical path problems.

The Next Steps

The slowdown in new medical products reaching patients in recent years despite growing public and private investment in R&D and tremendous progress in the basic biomedical sciences illustrates that better biomedical ideas alone are not enough. We must also ensure the successful movement of those ideas along the critical path of development, ultimately delivering reliable, safe, and effective treatments to patients at affordable prices. We must achieve breakthroughs in the way we get these treatments to patients and make them practical and efficient to develop and produce. This is an essential step in achieving more timely, affordable, and predictable access to therapies based on the latest biomedical insights -- that so far are having little impact on patient care. If we do not work together to find fundamentally faster, more predictable, and less costly ways to turn good biomedical ideas into safe and effective treatments, the hoped-for benefits of the biomedical century may not come to pass, or may not be affordable.

Ensuring that the development pathway keeps pace with biomedicine is crucial to advancing the health of Americans. This must be a joint effort involving the academic research community, industry, and scientists at the FDA, and it must be launched soon to have a timely impact. In the months ahead:

  • FDA will lead in the development of a national Critical Path Opportunities List intended to bring concrete focus to the tasks that lie ahead
  • We will develop this list through extensive consultation with all public and private stakeholders.
  • In addition, FDA will make internal changes to intensify its ability to surface crucial issues and to support high-priority critical path research efforts.

Since FDA is involved in setting standards for the development of new medical products, we must take proactive steps to use the best science to guide the development process and ensure that development standards are rigorous, efficient, and achieve maximum public health benefit.

We look forward to working with the scientific and product development communities to take advantage of this unprecedented opportunity to improve the health of the public and its access to affordable, innovative treatments.

 

Figures 1-8

 

Figure 1:  10-Year Trends in Biomedical Research Spending

10-year Trends in Biomedical Research Spending

The figure shows 10-year trends in biomedical research spending as reflected by the NIH budget (Budget of the United States Government, appendix, FY 1993-2003) and by pharmaceutical companies' research and development (R&D) investment (PAREXEL's Pharmaceutical R&D Statistical Sourcebook 2002/2003).

 
Figure 2:  10-Year Trends in Major Drug and Biological Product Submissions to FDA

10-year trends in major drug and biological product submissions to FDA

 

The figure shows the number of submissions of new molecular entities (NMEs) - drugs with a novel chemical structure - and the number of biologics license application (BLA) submissions to FDA over a 10-year period.  Similar trends have been observed at regulatory agencies worldwide.

 
Figure 3:  Investment Escalation per Successful Compound

Investment Escalation per Successful Compound

SOURCE:  Windhover's In Vivo:  The Business & Medicine Report, Bain drug economics model, 2003.

The figure shows one estimate of the total investment required to "launch" (i.e., market) a successful drug in two time periods.  Most of the recent cost increases are within the "critical path" development phase, between discovery and launch.

The overall increase between 1995-2000 and 2000-2002 is estimated to be 55 percent.

 
Figure 4:  The Critical Path for Medical Product Development

The Critical Path for Medical Product Development

Figure 4 shows an idealized "critical path" that encompasses the drug, biological product, and medical device development processes.  At the far left, ideas coming out of basic scientific research enter into an evaluation process (prototype design or discovery).  In drug development the "discovery" process seeks to select or create a molecule with specific desired biological activities.  Medical device development is generally much more iterative, so that prototypes often build on existing technologies.

The critical path begins when candidate products are selected for development.  They then undergo a series of successively more rigorous evaluation steps as they move from left to right along the path.  A low percentage of candidates entering preclinical development survive to the market application stage.

 
Figure 5:  Research Support for Product Development

Research Support for Product Development

Figure 5 shows how different types of research support the product development process.  Basic research is directed towards fundamental understanding of biology and disease processes.  Basic research provides the foundation for product development as well as translational and critical path research.  Translational research is concerned with moving basic discoveries from concept into clinical evaluation and is often focused on specific disease entities or therapeutic concepts.  Critical path research is directed toward improving the product development process itself by establishing new evaluation tools.

The clinical phase of product development also depends on the clinical research infrastructure.  One of the objectives of NIH's "Roadmap Initiative" is strengthening this infrastructure.

 
Figure 6:  Working in Three Dimensions on the Critical Path

Working in three dimensions on the critical path

Figure 6 is a highly generalized description of activities that must be successfully completed at different points and in different dimensions along the critical path.  Many of these activities are highly complex - whole industries are devoted to supporting them.  Not all the described activities are performed for every product, and many activities have been omitted for the sake of simplicity.

 
Figure 7:  Industry - FDA Interactions During Drug Development
Industry - FDA Interactions During Drug Development

This figure depicts the extensive industry-FDA interactions that occur during product development, using the drug development process as a specific example.*  Developers often meet with the agency before submitting an investigational new drug application (IND) to discuss early development plans.  An IND must be filed and cleared by the FDA before human testing can commence in the United States.  During the clinical phase, there are ongoing submissions of new protocols and results of testing.  Developers often request additional meetings to get FDA agreement on the methods proposed for evaluation of safety or efficacy, and also on manufacturing issues.

*Note:  Clinical drug development is conventionally divided into 3 phases. This is not the case for medical device development.  This is why preceding figures look slightly different.

 
Figure 8:  Problem Identification and Resolution During the FDA Product Review Process

Problem identification and resolution during the FDA product review process

Figure 8 shows how FDA's review and oversight of clinical trials and marketing applications lead to cycle of problem identification and attempted resolution.  Recurring problems identified during review trigger efforts to develop scientific solutions to prevent such problems in future applications.  Multiple cycles of research and public input may be required.  "Public standards" include, for example, accepted laboratory test methods, animal efficacy models or safety test protocols, clinical trial designs or endpoints, and clinical monitoring methods.  Once publicly accepted, these tools may be used by all developers.  FDA often seeks international acceptance of such standard tools, thus reducing unnecessary animal or human testing worldwide.

 

Footnotes

 1. The term medical product includes drug and biological products as well as medical devices.

 2. See About FDA.

 3. For purposes of this document the terms novel or new refer to applications for medical products of a type that have never before been submitted to the Agency (i.e., new molecular entity - NME).

 4. See CDRH - Recent Device Approval.

 5. Tufts Center for the Study of Drug Development, Backgrounder: How New Drugs Move Through the Development and Approval Process, Boston: November 2001; and Gilbert J, P Henske, and A Singh, "Rebuilding Big Pharma's Business Model," InVivo, the Business & Medicine Report, Windhover Information, Vol. 21, No. 10, November 2003.

 6. Finkelstein R, T Miller, and R Baughman,"The Challenge of Translational Research--A Perspective from the NINDS," nature neuroscience supplement, Vol. 5, November 2002.

 7. See nihroadmap.nih.gov/overview.asp.

 8. See http://spores.nci.nih.gov/current/index.htm.

 9. See www.mdbio.org. exit icon

 10. Eggermont A and H Newell, "Translational Research in Clinical Trials: The Only Way Forward," European Journal of Cancer, Elsevier Science, 37 (2001). EORTC also set up in October 2002 the Translational Research Advisory Committee to support and provide expert advice on translational research projects conducted within EORTC.

 11. Rowett, L, " U.K. Initiative to Boost Translational Research," Journal of the National Cancer Institute, Vol. 94, No. 10, May 15, 2002.

 12. Baumann M, SM Bentzen, W Doerr, MC Joiner, M Saunders, et al., "The Translational Research Chain: Is It Delivering the Goods?, Int. J. Radiation Oncology Biol. Phys., Vol 49, No. 2, 2001, Elsevier Science.

 13. Gilbert J, P Henske, and A Singh, "Rebuilding Big Pharma's Business Model," In Vivo, the Business & Medicine Report, Windhover Information, Vol. 21, No. 10, November 2003.

 14. Lloyd I, "New Technologies, Products in Development, and Attrition Rates: R&D Revolution Still Around the Corner," in PARAXEL'S Pharmaceutical R&D Statistical Sourcebook 2002/2003.

 15. Boston Consulting Group," A Revolution in R&D: How Genomics and Genetics Will Affect Drug Development Costs and Times," in PAREXEL's Pharmaceutical R&D Statistical Sourcebook 2002/2003.

 16. Duyk J, "Attrition and Translation," Science, Vol. 302, October 24, 2003.

 17. Horrobin DF, "Modern Biomedical Research: An Internally Self-Consistent Universe with Little Contact with Medical Reality?," Nature Reviews Drug Discovery, Vol. 2, No. 2, February 2003.

 18. Glassman RH, and AY Sun, "Biotechnology: Identifying Advances from the Hype," Nature Reviews Drug Discovery, Vol. 3, No. 2, February 2004.

 19. The Agency publishes 50 to 75 draft and final guidances each year, including guidances resulting from involvement in the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH).

 20. FDA, “Improving Innovation in Medical Technology: Beyond 2002,” January 2003.

 21. Rotman, D, "Can Pfizer Deliver?" Technology Review, February 2004.

 22. Petricoin EM, V Rajapaske, E H Herman, A M Arekani, S Ross, et al., "Toxicoproteomics: Serum Proteomic Pattern Diagnostics for Early Detection of Drug Induced Cardiac Toxicities and Cardioprotection," Toxicologic Pathology, 32(Suppl. 1):1-9, 2004.

 23. PricewaterhouseCoopers, "Pharma 2005 Silicon Rally: The Race to e-R&D" Paraxel's Pharmaceutical R&D Statistical Sourcebook 2002/2003.

 24. See, for example, Wagner RF, SV Beiden, G Campbell, "An Approach to Multiple-Reader, Multiple-Case Receiver Operating Characteristic Analysis: Controversial – or Subtle?," Acad Radiol. 2003, Oct; 10(10):1176-7; Wagner RF, SV Beiden, "Independent Versus Sequential Reading in ROC Studies of Computer-Assist Modalities: Analysis of Components of Variance," Acad Radiol., 2003 Feb; 10(2):211-2.

 25. Public Law 107-109, Jan. 4, 2002.

 26. Niblack J, "Biomarkers and Surrogate Endpoints," GJ Downing, ed. Exceptional Medical Int. Congress Series, 1205, Elsevier, 2000.

 27. See http://www.niams.nih.gov/ne/oi/.

 28. Sheiner LB, "Learning VS Confirming in Clinical Drug Development," Clin. Pharmacol. Ther., 1997, 61:275-291.

 29. See, for example, the Washington Fax interview with John La Montagne, Deputy Director of the National Institute of Allergy and Infections Diseases, National Institutes of Health, June 9, 2003.

 30. See Agency guidances.

 31. Chen ET, JH Nichols, SH Duh, G Hortin, "Performance Evaluation of Blood Glucose Monitoring Devices," Diabetes Technology & Therapeutics, 5(5):749-768, 2003.

 32. FDA/CDER, "A Risk-Based Approach to Pharmaceutical Current Good Manufacturing Practices (cGMP) for the 21st Century" 

 33. For comparison, FDA approved a total of 21 novel drugs in 2003.