Independent Evaluation of FDA's First Cycle Review Performance – Retrospective Analysis Final Report Text
Printable version (702 KB PDF)
Prepared by Booz Allen Hamilton Inc.
Prescription Drug User Fee Act III – Initiatives & Evaluations
Contract No. 223-04-8100 Task No. T1
TABLE OF CONTENTS
FINDINGS AND RECOMMENDATIONS
Therapeutic Area, Medical Need and Novelty of Mechanism of Action
Advisory Committee Meetings
The Review Process
Impact of Pre-Submission Meetings
Issue Identification Timing and Resolution
Communication Style and Timing
GMP Inspection Process
Exhibit 1. Action Package Approval Rate
Exhibit 2. Overview of Analysis Process
Exhibit 3. Drivers and Hypotheses of Multi-Cycle Reviews
Exhibit 4. RPM Product Summary Comments
Exhibit 5. Approval Rate vs. Novelty and Indication
Exhibit 6. Approval Cycle Percentage by Application Type
Exhibit 7. Approval Rate vs. Drug Origin
Exhibit 8. Number of Approval Cycles as a Function of Advisory Committee Meetings
Exhibit 9. Percentage of Multi-Cycle Reviews by Sponsor Experience
Exhibit 10. Approval Rate vs. Sponsor Type and Origin
Exhibit 11. Key Deficiencies Cited in Action Letter of Multi-Cycle Applications by Category
Exhibit 12. Key Deficiencies Cited in Action Letter of Multi-Cycle Applications by Area
Exhibit 13. FDA Meeting/Communication Perceptions and Practices
Exhibit 14. Effect of End of Phase 2 Meetings on Approval Rate
Exhibit 15. Effect of Pre-NDA/BLA Meetings and Timing on Approval Rate
Exhibit 16. Pre-NDA/BLA Meetings and Issue Identification
Exhibit 17. Resolution of Issues Identified Pre-submission
Exhibit 18. Deficiency Timing in Multiple Cycle Applications
Exhibit 19. Sponsor Resolution of Deficiencies
Exhibit 20. Check-and-Follow Up Communication
Exhibit 21. FDA-Sponsor Interaction Opportunities
Exhibit 22. FDA Meeting Routine by Division
Exhibit 23. FDA-Sponsor Communications
Exhibit 24. Effect of Communication Style on First-Cycle Approval Rate
Exhibit 25. Multi-Cycle Product Reviews Marked with Ineffective FDA-Sponsor Interaction
Exhibit 26. Issue Resolution Timing – Multiple Cycle Applications
Exhibit 27. Issue Resolution Timing – Single Cycle Applications
Exhibit 28. Review Communication Summary
Exhibit 29. Post-Marketing Commitments Single vs. Multi-Cycle Reviews
Exhibit 30. Focus and Burden of Post-Marketing Commitments
Exhibit 31. Disposition of Significant First Action Deficiencies in Multi-Cycle Approvals
Exhibit 32. Submission Timing vs. Number of Submissions or First-Cycle Approval Rates
Exhibit 33. Submission Timing vs. Number or Type of Issues per Application
Exhibit 34. Foreign or Domestic CGMP Inspection vs. % of Single or Multiple Review Cycles.
Exhibit 35. Schematic of the CGMP Inspection Process and Improvement Opportunities
Exhibit 36. Summary Overview of Recommendations
This report summarizes a retrospective analysis of factors affecting the FDA first cycle review of new drugs (NDAs) and biologics (BLAs). Several factors appear to be significant contributors to a multi-cycle review versus a first-cycle approval. Application quality and communication (between the FDA and sponsors, and FDA internal) emerged as having significant influence. Variations in FDA review practices across divisions may also be a factor, although these are, in part, driven by medical need and the specifics of therapeutic areas. Another potential factor contributing to multi-cycle reviews is the significant delay or lack of response from sponsors to concerns highlighted by FDA reviewers. This report suggests measures that can be adopted by the FDA and sponsors to increase review efficiency and communication effectiveness, which may lead to higher first-cycle approval rates.
The study was comprised of all NME applications (77), submitted between FY 2002 and 2004 that had reached first action by December 1, 2004. The focus is on the management and procedures for FDA product reviews and is not intended to evaluate the merit of the underlying science or quality of discipline reviews. The primary sources of data are FDA-compiled product Action Packages which contain records of FDA internal and FDA-sponsor communications and review documents, as well as interviews with FDA review team members, division directors and members of the FDA senior leadership team. Gaining the perspectives of sponsors on the root causes for multi-cycle approvals was beyond the scope of this study.
A planned prospective study will include an opportunity to solicit input from sponsors.
Of the 77 submissions (14 BLAs, 63 NDAs), 36 (47%) received first-cycle approval, 18 (23%) were approved in multiple cycles and 22 (30%) were still pending at the time of analysis. Drivers of multi-cycle reviews were found to include product characteristics, sponsor characteristics, quality of the design and execution of the drug development program, variations in review processes, and development of post-marketing commitments.
Priority and Fast-Track products have higher first-cycle approval rates. Beyond the unmet
medical need however, increased regulatory and sponsor attention throughout the drug
development and review process may contribute to the timely identification and resolution of
The degree of sponsor experience with FDA regulations and procedures is generally of importance. Large US-based companies have the highest first-cycle approval rate, at approximately twice the rate of small biotechnology companies with no prior FDA approvals. The underlying drivers seem to be lack of personnel with US regulatory experience and suboptimal sponsor-internal regulatory processes. The FDA can actively aid these sponsors by dedicating resources to education programs emphasizing critical drug development/regulatory requirements, updating and streamlining the portfolio of guidances, and proactively directing sponsors to these guidances.
Drug development program
Most products that fail to receive first-cycle approval have key deficiencies in only one or two categories, with an even breakdown between the categories of safety, efficacy, and chemistry (includes manufacturing related issues). There is also no single dominant cause, with the basis for the deficiencies falling evenly across development program design, execution, and failure to meet endpoints.
FDA reviewer team members agree that early on-going dialog with sponsors is the most important factor in identifying issues and potentially providing an opportunity for timely resolution, ideally before first action is taken. All divisions interviewed routinely strive to start discussions with the sponsors before the submission. These efforts meet with mixed success: End-of-Phase 2 meetings appear to significantly contribute to first-cycle approval while Pre-NDA/BLA meetings had a lesser impact. In some instances, substantial deficiencies were not documented/identified until the review phase, potentially preventing first-cycle approval despite the possible availability of pertinent information at the time of Pre-BLA/NDA meetings. This finding may be attributed to the general focus of these meetings on application formatting rather than review of development results. When issues are identified, there is often insufficient time to adequately address these as submission timelines are generally not delayed. This may be due to sponsors' unwillingness to adopt FDA suggestions or a lack of clarity in FDA communications on the severity of the issues raised. There are also examples where sponsors are able to resolve issues via a different path than originally recommended by the FDA. These findings point to broad issues around coverage of problem areas prior to submission, ineffective communication between the FDA and sponsors, and unclear prioritization of issues and/or problem resolution requirements.
An approach to address this challenge is the development of an open and accountable communication system centered around issue resolution. This system may include a pre-submission check-list and follow-up responsibilities that will guide FDA-Sponsor discussions and ensure that these communications are better leveraged to achieve agreement on issue resolution. This system – termed in this report as check-and-follow up communication – will increase consistency and reduce the risk of overlooking key issues at pre-submission stages.
Application review under PDUFA
Broad variations were observed in the frequency and timing of communications throughout the review. However, there was no systemic difference with respect to these parameters between single vs. multi-cycle approvals. Nevertheless, effective communication and responsiveness to FDA inquiries marked first-cycle approvals while persisting disagreements over issue resolution were associated with approval delays. Additionally, there were instances of multi-cycle approvals where earlier FDA communication of major issues may have possibly led to resolution within the first-cycle.
The recently introduced Good Review Management Principles and Practices (GRMPs) guidance recommends specific timelines for NDA/BLA review procedures. These along with additional structured communications within the review team and with sponsors recommended in this report could ensure a more productive review. Further, early and open communication with the sponsors will allow sponsors to address/resolve issues in a timely manner, potentially within the first review cycle.
There is broad variability in the use of post-marketing commitments (PMCs) and a lack of guidelines for PMC development. Most approvals have post-marketing commitments, with no significant difference between single and multi-cycle approvals in the average number, focus, and burden of commitments. A number of products approved in multiple review cycles had certain deficiencies in the first action letter that remained unresolved in the second review and were included as post marketing commitments in the approval letter. A guide for post-marketing commitments will introduce transparency, facilitate discussions, help sponsors prioritize deficiencies and help guide sponsor's development plans, ultimately improving the quality of submissions.
The FDA receives between two and three times the number of submissions in the fourth quarter compared to any other quarter in the calendar year. These applications have the lowest rate of first-cycle approvals. High-level metrics show no difference in the quality of these applications compared to submissions in other quarters, suggesting potential FDA staff workload issues. A deeper analysis quantifying FDA workload is however, necessary to better establish the underlying drivers and identify improvement opportunities.
The manufacturing facility inspection process is often considered a potential bottleneck in meeting PDUFA clock goals and the cause of multi-cycle reviews. A cursory mapping of this process suggests that the current system does not offer sufficient flexibility to complete inspections early on to enable problem rectification within the review cycle. This is further exacerbated in instances where foreign inspections are required, and/or review times are compressed due to, for example, Priority status of applications. Earlier involvement of CDER Consumer Safety Officers (for example, at pre-NDA stages) is recommended to foster better planning and mitigate risks1. Although this practice will not necessarily, in all instances, enable a complete resolution of facility issues within the first-cycle, it will provide a better opportunity for the sponsor to input a genuine effort. Furthermore, a closer review of the current processes for inspection team notification and scheduling can potentially yield measures whereby the inspection process can be streamlined, reducing the overall inspection time, and ultimately time to market for new products.
A number of the suggested recommendations may have resource implications for the FDA and sponsors. Quantification of the specific resource needs was beyond the scope of this project. However, the expected improvements to the review process, increased rate of first cycle approvals, and over time, the implied reduction in duplicative efforts from multi-cycle reviews may off-set the additional resource needs. An increase in resources is expected for the initial implementation phase, during which the benefits of the improved process have not yet been realized.
In 1992, Congress passed the Prescription Drug User Fee Act (PDUFA) authorizing revenues from fees paid by the pharmaceutical industry. These revenues provide the Food and Drug Administration (FDA) with additional resources and allow the FDA to expedite and improve the review of human drug applications.
PDUFA is renewed every five years. Currently in its second renewal (PDUFA III), the FDA has committed to achieving specific performance goals to improve the effectiveness and efficiency of NDA and BLA reviews. Several of these goals are aimed at improving the portion of the review process that occurs between the initial submission of the application and subsequent FDA action (i.e., the first review cycle).
The PDUFA III goals specify that the FDA will retain an independent expert consultant to evaluate the review process improvement initiatives and the impact of the Good Review Management Principles (GRMP) initiative. The FDA has contracted Booz Allen to perform an independent program evaluation of the product review process. The primary goal of the overall evaluation is to determine the impact of the FDA's implementation of initiatives to enhance first-cycle review performance of New Molecular Entities (NMEs) during the five-year period of PDUFA III.
Under this task, the evaluation will consist of a retrospective analysis focused on the review processes that are conducted on NME NDAs and BLAs. This report highlights findings from Booz Allen's first task-order, a retrospective study of all NMEs submitted during PDUFA cohort fiscal years – 2002 to 2004 that have reached first action by December 1, 2004.
Action packages were analyzed relating to all NME NDAs and BLAs submitted during fiscal years 2002-2004 that reached first action by December 1, 2004 (77 applications in total). Action Packages are typically a compilation of product review documents (e.g., discipline review letters and review meeting minutes) and, in some instances, pre-submission documents (e.g., pre-NDA/BLA meeting minutes). These packages generally contain the critical information required for Office and/or Division Directors to formulate the action (Approval, Approvable, or Not Approvable2). Action packages were not, however, specifically developed for the purposes of this retrospective study and content gaps were encountered. Where feasible, input from FDA Regulatory Project Managers (RPMs) involved in the product review was solicited to fill in missing information. The study was not intended to evaluate the merit of the underlying science or quality of discipline reviews, but rather to investigate process issues that may drive multiple review cycles. Finally, this analysis is solely based on data originating from the FDA. A planned prospective study will also aim to capture sponsor perspectives and data.
Exhibit 1 depicts the breakdown of the action packages used in the study with respect to review outcomes. The key focus of the activity was to understand:
- Characteristics and best practices promoting first-cycle approval
- Drivers/lessons learned from multiple cycle approvals
- Overall improvement opportunities for the FDA and sponsors
A two-staged approach was followed, as depicted in Exhibit 2:
The first stage was comprised of generating hypotheses of potential multiple cycle review drivers and the appropriate metrics. To test these hypotheses, action packages were reviewed and information relevant to the metrics was captured in data collection instruments (DCIs). Word summaries of each product were also created reflecting important regulatory events and key drivers of multiple cycle reviews or, in the case of single cycle approvals, best practices. The DCIs and word summaries were used to drive qualitative and quantitative analysis of the potential drivers of multiple cycle reviews. The results of these analyses were used to synthesize emerging themes and recommendations in the final activity of this stage.
In the second stage, findings and recommendations were reviewed with FDA Regulatory Project Managers (RPMs) for validation. Additionally, divisional philosophies and best practices were captured and used to validate and expand on the themes and recommendations formulated in the first stage.
Exhibit 3 shows a sample set of metrics and hypotheses developed in conjunction with FDA reviewers3 :
|Characteristics||Key Hypothesis Areas|
|Drug/disease Characteristics||Review designation|
Mechanism of action (MoA)
|Sponsor Characteristics||Company size|
Country of origin
Experience with FDA
Experience in therapeutic area
|Review process||Distribution of major deficiencies|
Timing of interactions
Issues and resolution
Impact of cGMP inspections
These are the Sample Hypotheses:
- Submissions designated Priority are less likely to require more than one review cycle as compared to Standard
- Drug-Device combinations requiring input from multiple divisions are more likely to require a multiple review
- Products developed by experienced companies are less likely to require multiple review cycles
- Allowing sponsor to address issues in Phase 4 studies will decrease multiple review cycles
A typical action package may contain the following elements:
- Action Letter(s)
- Discipline Reviews (Chemistry, Medical, Labeling, CMC, and Consults)
- Correspondence from the FDA to Sponsor (Letters and Faxes)
- Internal FDA correspondence (Emails and Inspection results)
- Meeting notes
For each of the 77 products, data on common variables were recorded into DCIs for analysis across the broad array of products. These formed the basis of the quantitative analyses (see section on Findings and Recommendations). The captured information included:
- Clock and goal dates
- Review team members
- Requests for information and timing
- Nature of the issues raised
- Timelines for responding to communications and information requests
Publicly available data sources were used to supplement product and sponsor company background information including:
- Novelty of mechanism of action
- Sponsor profiles (e.g., previous experiences with FDA)
A sample of approximately one third (23 out of 77) of the Action Package Product Summaries was selected for validation by the respective RPMs, verifying the accuracy of Booz Allen analysis. Products were selected such as to minimize the impact on RPMs (i.e., RPMs involved with multiple products in the cohort were selected over RPMs with only one product assignment) and to represent a variety of review divisions. In general, RPMs agreed in all instances with the assessment of the critical issues and product review analyses. Changes were minor and included comments on the background information on sponsors, products and submissions (Exhibit 4).
The hypotheses that were tested were grouped under four key characteristics of multiple cycle reviews:
- Drug/Disease Characteristics
- Sponsor Characteristics
- Review Process Characteristics
- FDA Characteristics
A statistical analysis was not feasible due to the low number of applications in the cohorts. In some instances, the number of product applications meeting the test criteria was even further limited (for example, novel mechanism of action coupled with product origin: in-house vs. acquired technology), potentially impacting the ability to generalize conclusions.
Certain product designations such as the Drug Efficacy Study Implementation (DESI) program or 505 (b)(2) significantly alter filing requirements. The DESI program applies to drugs approved before 1962 solely on the basis of safety. Depending on product characteristics, the sponsor of a DESI application may be only responsible for demonstrating efficacy. Extraneous demands (e.g., for counterterrorism or to address drug shortages) led the FDA to initiate its own effort to collect and evaluate safety/efficacy data and issue Federal Register notices for applications under the 505(b)(2) designation. As a result, these applications are primarily focused on manufacturing. Both DESI and 505(b)(2) applications were excluded from most analyses as a result of their non-standard content.
The impact of drug/disease characteristics on first-cycle approval rate was categorized by therapeutic area, medical need, novelty of the mechanism of action, review designation and drug origin.
As a product class, novel drugs targeting acute, life threatening conditions have the highest rate of first-cycle approval (73%) compared to either criteria alone (40% for life-threatening and 64% for novel mechanism of action; see Exhibit 5). Non-novel products for non-life threatening conditions had the lowest first-cycle approval rate with 28%. These findings are not unexpected given the severity of the medical conditions addressed, the different levels of acceptable risk and the urgency for new therapies. However, other factors may also be contributing as interviews with FDA reviewers suggest that novel drugs for which limited and/or ineffective therapy choices are available receive greater attention from the FDA and sponsors. Extensive effort is placed on completing reviews of these drugs within six months, regardless of their priority status, and division directors proactively align resources to support expedited reviews. Conversely, for products for which alternative therapies are available, sponsors may forego potential drug development meetings and the division director involvement may come later in the review.
|Indication is not for life-threatening condition||Indication is for life-threatening condition|
|Does not have novel mechanism of action||28% single-cycle (n=7);|
72% multi-cycle (n=18)
|40% single-cycle (n=8);|
60% multi-cycle (n=12)
|Does have novel mechanism of action||64% single-cycle (n=7);|
36% multi-cycle (n=4)
|73% single-cycle (n=8);|
27% multi-cycle (n=3)
Note: not including DESI, 505(b) (2) drugs
Fast-Track/ Rolling and Priority review status are used to expedite the drug development and review processes of products addressing diseases with significant unmet medical needs. For products with these designations, the FDA may engage in more pre-submission communications with sponsors, and review applications in six months. Orphan drug and Fee-Waiver designations provide for financial incentives to small companies and those developing drugs for rare indications but do not impact the review process per se. The Fast-Track and Priority review programs seemed effective in driving single cycle approvals as 62% of drugs (16 of 26) with Priority status received first-cycle approval compared to only 34% for non-priority drugs (Exhibit 6a). Similarly, high first-cycle approval rates were observed for Fast-Track products. Orphan drug and Fee-Waiver designations however, did not lead to similar outcomes with only 33% and 12% first-cycle approval rates, respectively (Exhibit 6b). Notably, many Orphan designated products also merited a Fast-Track and/or Priority review designation. No difference was seen in the first-cycle approval rate between Orphan and Fast-Track versus Orphan without Fast-Track status. Of the Priority applications, six also had Orphan status with three of these applications achieving first-cycle approval (50%). Of the 19 remaining Priority applications, 12 (63%) achieved first-cycle approval. A lack of sponsor's regulatory experience may be a compounding factor for the Orphan and Fee-Waived application first-cycle approval rates, since most of the products with these designations were developed by small companies with previously no approved products (see section on Sponsor Characteristics).
|Priority review||62% single-cycle (n=16)|
36% multi-cycle (n=10)
|34% single-cycle (n=14)|
66% multi-cycle (n=27)
|Fast-track/rolling submission||58% single-cycle (n=7)|
42% multi-cycle (n=5)
|42% single-cycle (n=23)|
58% multi-cycle (n=32)
|Orphan status||33% single-cycle (n=3);|
all priority applications67% multi-cycle (n=6);
3 priority applications
|47% single-cycle (n=27);|
3 priority applications53% multi-cycle (n=31);
7 priority applications
|PDUFA fees waived||12% single-cycle (n=2)|
88% multi-cycle (n=14)
|55% single-cycle (n=28)|
45% multi-cycle (n=23)
Further supporting the significance of increased focus, greater sponsor diligence may be contributing to the higher success of externally sourced (in-licensed) products, compared to those originated in-house (Exhibit 7). This may arise from the increased scrutiny that products may be exposed to at the selection phase and/or closer attention paid by sponsors during drug development.
First-cycle approval rates were slightly lower for products for which input from Advisory Committees was solicited (Exhibit 8). This is consistent with the notion that such meetings are generally requested for products with significant unknowns or controversial issues.
Sponsor experience with the FDA approval processes appears to contribute to first-cycle approvals. The first-cycle approval rate for sponsors that had drugs approved previously by the FDA was 51% compared to 30% for sponsors with no prior approved drugs (Exhibit 9a). Experience within the specific therapeutic area however, did not seem to have an additional effect (Exhibit 9b).
Consistent with the importance of experience, larger and US-based sponsors are more likely to gain first-cycle approval (64% and 86% for US-based large pharmaceutical and biotechnology companies, respectively; see Exhibit 10). On the other hand, inexperienced drug developers (generally small biotechnology companies without prior US-approved products) had the lowest first-cycle approval rate (33%).
1st Cycle Approvals by Type of Sponsor Company
|1st Cycle Approval Rate|
|Non-drug company||0% (n=3)|
|Small Biotech||33% (n=18)|
|Large/Medium Pharma||345 (n=38|
|Large Biotech||88% (n=8)|
Does not Include DESI, 505(b)(2) products
Mid-Pharma include generic and specialty pharmas
Source: BAH Analysis
1st Cycle Approvals by Origin of Sponsor
|Origin of Sponsor||U.S.-Based Companies||Foreign Companies|
|Big Pharma||64% n=11||43% n=14|
|Mid-Pharma||40% n=5||12% n=8|
|Big Biotech||86% n=8||Na n=0|
|Small Biotech||33% n=15||33% n=3|
Does not Include DESI, 505(b)(2) products
Non-pharmas (3 products) not shown
Large/Big Pharma - Pharmaceutical Companies with Market Cap over $5B
Medium Pharma - Pharmaceutical Companies with Markets Cap under $5B, Generics Companies and Specialty Pharmas
Large/Big Biotech - Biotechnology Companies with Market Cap over $1B US
Small Biotech - Biotechnologies with Market Cap under $1B US, or for whom the product under discussion was first key product
According to FDA reviewers, unfamiliarity with FDA regulations and the drug application process is a key problem for inexperienced sponsors and results in poor quality submissions. In the case of foreign companies, language barriers as well as communication styles, which can be less formal in other countries, may also be an issue.
Sponsor-side improvement opportunities may involve complementing teams with experienced regulatory consultants or leveraging clinical research organizations (CROs) experienced with FDA processes for submissions. Additionally, inexperienced sponsors would likely benefit from improving communications. These improvements include engaging in early and open dialog employing FDA-preferred methods (e.g. appropriate forms and correct submission procedures), and developing processes to rapidly respond to FDA requests.
The FDA can facilitate these processes by targeting less experienced sponsors with workshops and updated and streamlined guidance portfolios, as well as improving the utility of the website which includes sections targeted to these sponsors. Implementing and maintaining these recommendations may require additional FDA resources. These resource needs could be offset, in the long-term, by reducing the incidence of multiple cycle reviews. An in-depth analysis into workload duplicities is necessary to quantify the cost vs. benefit (savings).
The impact of regulatory review processes and interactions between FDA and sponsors during the clinical development phase (i.e., pre-submission) and the review phase on the first-cycle approval rate were assessed. Variables included the timing and frequency of meetings, timing and effectiveness of communication of application issues, timing of manufacturing inspections, etc. Not all action packages contained comprehensive documentation of all pre-submission events. However, more significant milestone meetings (such as End of Phase 2) were generally included and these comprised the basis for the pre-submission analyses.
The majority of multi-cycle applications have significant deficiencies in only one or two key categories (Exhibit 11). A "significant" deficiency is defined as a product or application related issue that would prevent first-cycle approval if not adequately addressed. Of the 37 applications requiring multiple cycles, 20 were cited for a single significant deficiency in the safety, efficacy or CMC categories. Nine applications failed due to deficiencies in a combination of two of these categories and two for a combination of application format and either CMC or safety. The six remaining multiple cycle applications failed with significant deficiencies in more than two categories. The overall distribution of the issues was fairly evenly divided between safety and efficacy (20 and 17, respectively) with CMC issues trailing only slightly (12) and a relatively small number of submission format issues.
Applications that took multiple cycles to review had deficiencies that prevented first cycle approval. Below is a list of deficiency issues and the number of applications in each category:
|Issue Category||Number of Applications|
|Single Key Deficiency (54% of applications)|
|Two Key Deficiencies (30% of applications)|
|Safety + Efficacy||6|
|Safety + CMC||2|
|Safety + Format||1|
|CMC + Format||1|
|CMC + Efficacy||1|
|More than 2 Key Deficiencies (16% of applications)||6|
Total Issues By Category:
|Category||Number of issues|
The origin of these deficiencies, in turn, is expected to fall in the areas of design (e.g. of the trial or manufacturing process), execution (e.g. unacceptable clinical execution), or failure to meet study objectives (e.g. clinical endpoints) (Exhibit 12). Of the 61 significant deficiencies cited in 37 first action letters, 17 related to trial design, 15 to execution and 23 to endpoints. The remaining six application format deficiencies were related to inconsistent documentation or record keeping, inability to locate information or failure to translate from foreign languages into English.
|Trial Design||Execution||Endpoints||Application (Format)||Total|
Note: Not including DESI, 505(b)(2) drugs
Review team members generally consider open and frequent communication as having a high impact on the review process (Exhibit 13a). All divisions interviewed frequently engage in both End of Phase 2 (EOP2) and Pre-NDA/BLA meetings in an attempt to identify issues early, thereby maximizing the time and potential for problem resolution - ideally before the first review is completed (Exhibit 13b).
a. Perceived Impact of Common Review Factors
|Review Factor||Perceived Impact on Review*|
|RPM experience/knowledge in drug class or therapeutic area||Moderate|
|FDA internal communication||High|
*Based on interviews with 15 RPMs, 8 Reviewers and 7 Division Directors
b. Meeting Routine Pre-Submission Table
Pre-IND - Pre-IND meeting or interaction
EoP2 - End of Phase 2 meetings
Pre-BLA/NDA - Generally ~6 months pre-submission
EOP2 meetings have a positive impact on first-cycle approval rate. Of 46 products with EOP2 meetings, 52% received first-cycle approval, vs. only 29% for products that did not have such meetings (Exhibit 14a). However, there seems to be room for improvement: of the multiple-cycle applications that had an EOP2 meeting, 25% of these applications had the critical issue preventing first-cycle approval identified at this meeting, indicating a failure or an inability by the sponsor to resolve problems prior to submission (Exhibit 14b). Further analysis is required to establish whether there was also an opportunity to identify the remaining deficiencies of these multi-cycle applications at the EOP2 stage (36% were identified at the pre-NDA/BLA meeting and 41% during the review).
Pre-NDA/BLA meetings, while important, do not appear to have as beneficial an effect on first-cycle outcome as EOP2 meetings. Products with pre-NDA/BLA meetings had a first-cycle success rate of 47% compared with 33% for products without such meetings (Exhibit 15a). This finding is not affected however, by the timing relative to the submission (Exhibit 15b).
Consistent with the above finding, interviews revealed that the content of the pre-NDA/BLA meeting is generally regarded as administrative, with a focus on application format considerations. In fact, there are many examples in which analysis of communications in action packages indicates that pre-NDA/BLA meetings fail to uncover major issues that contributed to multi-cycle reviews (Exhibit 16). In many of the cases illustrated, the relevant information should have been available at the time of the meeting and the subject matter fell within the boundaries of topics that can be covered at such meetings.
|Product *||Major Issues Not Addressed at Pre-NDA/BLA Meeting||Data Available at Pre-NDA/BLA?||FDA Action That May Have Promoted Earlier Issue Identification|
|Product A||• Variations in clinical trial designs produced conflicting results|
• Unclear organization of safety data files resulted in discrepancies
• Errors and inconsistencies in AE coding
• Inadequate preclinical data
|Information likely available pre-NDA to identify the issue||• Preview of all pivotal clinical protocols and data|
• Further discuss the proposed formats at Pre-NDA/BLA meetings
• Review complete pharm/tox studies; request preclinical data
|Product B||• Unacceptable manufacturing facility||Unknown||• Access prior FDA inspection reports|
|Product C||• High tumor incidence in animal studies||information likely available pre-NDA to identify the issue||Review preclinical data and/or ask for appropriate data|
|Product D||• Missing sub-population information||Information not likely to have been available pre-NDA||• Level of detail beyond the scope of pre-NDA meetings|
|Product D||• Inappropriate primary endpoint|
• Unclear criteria for clinical positive assessment
• Missing information in the AE database
• Inadequate analysis of AE events
|Information likely available pre-NDA to identify the issue||• Agree on clinical trial design and endpoints|
• Confirm guidelines for AE capture and reporting
|Product E||• Manufacturing contamination||Unknown|
• Review prior FDA inspection reports
|Product F||• Missing stability data; inappropriate validation methods|
• Discrepancies in AE description
|information likely available pre-NDA to identify the issue|
|• Provide relevant guidance Pre-NDA/BLA meeting|
• Confirm guidelines for AE capture and reporting
(*): Multiple review cycles products with no major pre-BLA/NDA issue
Source: BAH Analysis
Even when major deficiencies are identified in pre-submission meetings, sponsors do not always address them prior to submission. Seventy-one percent of applications with key issues identified during the pre-submission phase had not resolved these issues by first action (Exhibit 17a). Issues around safety and efficacy saw the lowest rate of resolution by first action (Exhibit 17b) potentially reflective of the generally more difficult and time-consuming nature of these issues.
In many cases, sponsors were informed of key deficiencies well in advance of the submission date. As shown in Exhibit 18, despite this early communication, sponsors do not always resolve these issues prior to submission. There are conceivably a number of explanations that can lead to this outcome:
- A lack of clarity on the severity of issues communicated by the FDA
- Differences in opinion, and sponsors' belief that the issues can be resolved during the review
- Sponsor's unwillingness to comply with FDA requests, which in some instances would also require postponing submissions to allow for sufficient resolution time
- Sponsor's interest in receiving a comprehensive review of all elements of the applications to highlight any additional deficiencies and address these all after the first action
Under the first two scenarios, ensuring a common understanding of the severity, and agreeing on a plan forward can help postpone submissions until such time that issues are adequately addressed. In the latter two cases, multi-cycle reviews will be difficult to avoid if the applications are accepted for filing by the FDA. A more detailed analysis with input from the sponsors is necessary to establish the underlying drivers in each instance.
In some cases, sponsors and the FDA are able to reach alternative resolutions to important pre-submission issues and gain first-cycle approval. These for example, included working with the FDA to salvage trials that had to be supplemented after initiation, or modifying or unbundling indications to pursue subgroups of the initially targeted patient population (Exhibit 19). This finding points to the importance of early and open discussions on acceptable resolution paths.
|Product I||FDA advised sponsor that European Pharmacopoeia methods are not acceptable||Sponsor did not update methods; committed to transition to US methods post-approval|
|Product J||FDA proposed a new trial due to deviations from the accepted clinical protocol||FDA and sponsor agree on approach to salvage on-going trial|
|Product K||Concern that standard regimens not equivalent to existing therapies||Sponsor changes application from 1st to 2nd line regimen|
|Product L||Efficacy questions around specific populations; FDA requested sub-population analyses||Application unbundled based on indication, salvaging approval for sub-population with greatest efficacy|
(*): Single-cycle approved products with major pre-submission issue(s) not addressed by time of approval
A root cause of problems in issue identification and resolution can be traced directly to the effectiveness and timing of communication between sponsors and the FDA. Currently, sponsors are responsible for requesting pre-submission meetings to discuss specific issues. While the FDA can also raise questions at these meetings, most divisions do not have formal protocols in place to ensure all key areas are covered. Further, in divisions where protocols do exist, they are not always applied consistently. As a result, issues may not be prioritized and follow up is solely at the sponsor's discretion. Finally, there is no consistent standard for issue resolution across divisions, and adoption of FDA suggestions and requirements prior to submission varies broadly.
An open and accountable communication system centered around issue resolution may increase consistency and transparency in issue identification as well as resolution (Exhibit 20). This system – termed in this report as check-and-follow up communication – may include checklists generated by each division will guide discussions between the sponsor and the FDA and help track sponsor progress against key drug development issues and requirements. Formal follow-up mechanisms in the form of meeting minutes and teleconferences, with appropriate sign-off, will serve to clarify and align the FDA and the sponsor's understanding of the key issues. Sponsor-submitted plans of action proposing approaches to issues raised are reviewed by the FDA to gain agreement on necessary measures for resolution. Such a system will reduce the potential for key issues being overlooked or neglected, and reduce the risk of unforeseen complications arising late in the review process. All divisions interviewed agreed that creation of a checklist with sufficient customization to meet the needs of each therapeutic area is feasible.
This enhanced FDA-Sponsor interaction tool can be deployed at the earliest stages, beginning with the pre-IND phase (Exhibit 21). These meetings represent an early opportunity to ground the FDA and sponsors on the key issues by informing the FDA of the sponsor's strategy and development plan, and providing the opportunity for feedback where appropriate. Progress can be tracked and future plans and protocols developed at EOP2 meetings. A mid-Phase 3 meeting provides an opportunity to review progress against development plans and design course corrections, if necessary. Finally, in addition to the discussions of submission protocol and format, trial results and data quality can be assessed prior to filing at the pre-NDA/BLA meetings.
Full realization of the benefits of this communication system will require participation and commitment of both the FDA and sponsors to engage in open discussions and follow through by executing problem resolution plans in a timely manner before applications are submitted.
In light of resource constraints, the FDA may consider a phased implementation approach, initially focusing on developing the checklists. This may already yield sufficient improvements diminishing the urgency for implementing the feedback loop. A pilot program will yield a clearer understanding of the costs, resource requirements and benefits.
|Pre-IND||Discuss product strategy||Understand sponsor’s strategy and product development plan; provide feedback, if appropriate—"everybody on the same page"|
|Pre-IND||Early regulatory input||Rudimentary labeling discussions enable the FDA to provide input on appropriateness of studies|
|EOP2||Track progress||Discuss progress against development hurdles (e.g., checklist)|
|EOP2||Develop future plans||Phase 3 protocol development, approval criteria, follow-up with Special Protocol Assessment (SPA)|
|Mid-Phase III||Discuss challenges||Review data and discuss deviations from original plan; discuss implementation issues, and major protocol violations|
|Mid-Phase III||Refine studies||Course corrections, as necessary; track progress against development hurdles|
|Pre-NDA/BLA||Discuss data||Broad overview of trial results, assessment data quality and completeness|
|Pre-NDA/BLA||Submission criteria||Clarify format, discuss inspection status, gain FDA opinion on application "readiness"|
Broad variations exist for assessing overall progress during the review period. Of the possible formal meetings held during this period, only the sponsor presentation offers an opportunity for interaction between the FDA and sponsors prior to first action. However, few divisions routinely take advantage of this opportunity (Exhibit 22).
|Division||1st Team Mtg||Sponsor Pres.||Filing Meeting||Mid-Cycle||Post Action|
1st Team Mtg - Internal planning mtg. w/in 45 days of submission
Sponsor Presentation - Within 45 days of submission
Filing Mtg - Establish suitability of application for filing
Mid-Cycle - Mid-Cycle meeting with review team
Post Action - Discuss "lessons learned" or clarification of deficiencies with sponsor
Source: Division Interviews
The FDA and sponsors frequently engage in less formal communications, for example email or telephone requests for information. Analysis of action packages revealed that broad variation existed in both the frequency of such communications (Exhibit 23a) or their distribution throughout the review (Exhibit 23b). As can be observed however, there was no systemic difference with respect to these parameters and the number of review cycles required for approval. A slight increase in communications was seen towards the end of reviews for single cycle approval compared to multiple cycle applications. This increase is likely attributable to final resolution of minor issues and labeling discussions.
The tone of most FDA-sponsor communications is positive in nature and does not appear to be a significant driver of single vs. multiple cycle reviews. However, at its extremes, communication styles can impact the review outcome. Effective communication and responsiveness – characterized by early identification and communication of issues and timely responses to requests for information (typically within one to two weeks) – contribute to favorable first-cycle outcomes (Exhibit 24). Communications labeled "ineffective", on the other hand, are characterized by late communication of issues and lack of responsiveness by sponsors. In some cases, key issues were not conveyed to the sponsor prior to the action letter. Conversely, repeated requests for information from the FDA were necessary before sponsor responses were received. All products falling in this category failed to obtain approval in the first review cycle.
|"Effective"||Midway between "Effective" and "Ineffective"||"Ineffective"|
|87% (n=13) of applications with effective communication approved in one cycle13% (n=2) of applications with effective communication approved in one cycle||52% (n=23) of applications with somewhat effective communication approved in one cycle48% (n=21) of applications with somewhat effective communication reviewed in multiple cycles||All of 18 applications with ineffective communication reviewed in multiple cycles|
Note: Not all characteristics apply to each application with the category
Source: BAH Analysis
In some product reviews, disagreements and/or sustained misunderstandings prevented the FDA and sponsors from resolving outstanding issues and ultimately led to the need for additional review cycles before the product could be approved (Exhibit 25).
|Product M||Consistent misunderstanding of FDA requests may have prolonged review and contributed to a second cycle|
|Product N||Disagreements over schedule IV classification impacted outcome; formal dispute resolution was required|
|Product O||Consistent misunderstanding between FDA and sponsor impacted the ability to reach timely agreements|
|Product P||Sponsor insisted on not following FDA advice regarding trial design; data quality was marginal with frequent clarifications necessary|
In addition, there are examples where earlier communication of key issues within the review cycle may have led to resolution in time to gain first-cycle approval. Exhibit 26 depicts two cases where the relative short period of time required for resolution of the key issues preventing approval may have been readily accommodated within the first review cycle, had the issues been identified and communicated to the sponsor only three to four weeks earlier. The underlying assumption is that earlier identification would have been feasible, and that the FDA would have sufficient time within the first review cycle to review resubmissions:
Product B: An unacceptable manufacturing plant and missing packaging/stability data were the key issues cited in the first action letter. These were initially communicated to the sponsor shortly before the action date. The sponsor provided a compliant facility and missing data were submitted approximately one month after the action date
Product Q: CMC deficiencies represented 95% of issues listed in the first action letter. The sponsor corrected deficiencies and resubmitted the application approximately 20 days after the first action
Examples of successful issue resolution through effective sponsor-FDA interaction and responsiveness are seen in single cycle applications (Exhibit 27). The examples provided in Exhibit 26 and Exhibit 27 revolve around CMC deficiencies, suggesting that this discipline may benefit most from earlier communication.
Formalized review team communications recommended in the recently introduced GRMP guidance (e.g. Filing, Mid-Cycle meetings) are intended to enforce early engagement of review teams and increase the dialog with sponsors. Exhibit 28 lists opportunities whereby FDA review teams or FDA and sponsors may come together to facilitate the review process. Supplemented with the additional GRMP-recommended meetings and open dialog, the combined formal and informal meetings may promote more productive communications that span the breadth of the review:
- Internal planning meetings - most effective when held before the filing meeting -create an opportunity to develop review plans and set expectations early in the review process
- Sponsor presentations to the review team - currently rare - can serve to orient reviewers to the actual submission (as opposed to the pre-submission outline), and generate discussion around the product
- Internal meetings - e.g., at the Mid-Cycle stage - offer an opportunity to develop initial, holistic opinions on the emerging outcome and discuss open issues
- Ongoing, proactive dialog with sponsors will ensure that goals are communicated and expectations managed.
A Mid-Cycle meeting can further provide an opportunity to assess whether appropriate levels of resources are deployed to complete the review in a timely manner, and to what extent additional discussions with the sponsor are warranted. Where feasible, these meetings can trigger early labeling discussions which often require several iterations before being accepted by both parties.
The introduction of additional meetings and/or restructuring of existing meetings may have resource implications for the FDA. As previously mentioned, savings from reduced multi-cycle reviews may however, off-set increased resource demands. Additional resources may be necessary during the period of overlap of current reviews and review of future submissions using the new recommendations.
Post-marketing commitments (PMCs) provide a mechanism to bring drugs to market more quickly by resolving issues that are not critical for approval during the marketing phase of the product life-cycle. For the cohort products, 80% of single-cycle and 88% of multi-cycle applications were approved with PMCs (Exhibit 29a). The number of PMC requests per application varied broadly, ranging from 2 to 20, with a similar average number of commitments regardless of review cycles (5.4 and 4.4 commitments for single and multiple cycle approvals, respectively, Exhibit 29b). Further, the focus and burden of post-marketing commitments do not differ between first- and multi-cycle approvals, with the majority consisting of additional clinical studies to further evaluate very specific safety and/or efficacy questions ( Exhibit 30).
A closer inspection of multi-cycle applications revealed that only CMC deficiencies are generally resolved through sponsor compliance of FDA recommendations (Exhibit 31a). Of the critical, non-CMC related issues, approximately 50% are resolved by complying with FDA requests. The remaining issues are resolved through an agreement to perform PMCs or via an alternative path (Exhibit 31b), based on additional discussions between sponsors and the FDA.
In two instances of products receiving approval after the second review cycle, the key deficiency preventing first cycle approval could not be fully resolved, and disagreements persisted:
- Product U: Unknown consequences of a chemical element accumulation, prompted the FDA to request additional studies. New data did not provide adequate resolution of the safety issue. Approval after second review required a PMC to assess the effects of long-term the chemical element accumulation
- Product V: Concern over the design of a safety study prevented first cycle approval. Interpretation of new study data remained inconclusive and depended on how the data was analyzed. The Office Director after further review and analysis approved the product with no Phase 4 commitment request related to the safety issue.
Interviews confirmed that divisions generally do not have consensus on PMC policy before communicating with sponsors. This has resulted in inconsistent usage between divisions or within divisions for different products, and some divisions largely avoid PMCs altogether due to difficulties in enforcement. Promoting earlier discussions between the FDA and sponsors and providing clearer guidelines on alternative acceptable pathways for addressing deficiencies will allow sponsors to focus efforts on the key requirements for approval, while shifting less critical issues to the post-market phase. This may reduce the time to market; in some cases through approval within the first review cycle.
Guidance for the use of PMCs is necessary to provide transparency and facilitate negotiations with sponsors. An understanding of division PMC practices/philosophies will help formulate PMC policies. Sponsors will benefit from the ability to prioritize deficiencies based on a clear understanding of issues that can be addressed through to PMCs versus ones that have to be resolved prior to approval. An approach to developing such guidances should involve establishing standardized principles across divisions and incorporating best practices for monitoring and revising labels as results from these studies emerge. Customization to the individual therapeutic areas and disciplines is necessary to ensure the effectiveness of these guidances. Interviewees agreed that creation of such guidances with a meaningful level of customization would be feasible.
Between two and three times as many applications are submitted in the fourth quarter of each calendar year than any other quarter (Exhibit 32a), with Q4 applications having a 26% 1st-cycle approval rate versus 64% for Q1-Q3 submissions (Exhibit 32b).
FDA interviewees attributed this finding to their perception that Q4 applications are often of lower quality, requiring greater review effort and failing to meet approval criteria. However, this could not be verified in this study, as Q4 submissions had similar numbers of issues compared to other quarter submissions when measured by the total number of issues communicated or the issue category (i.e. safety, efficiency, format. see Exhibit 33). This points to potential FDA staff workload issues, with all PDUFA goal dates coinciding around a similar timeframe. Furthermore, workload issues may be compounded by the coincidence of the end of review cycles with the generally lower staffing during the summer months. Further analysis is necessary to understand the nature of the application issues for better comparison of application quality and workload.
Clinical protocol and Current Good Manufacturing Practice (CGMP) compliance are integral to the review process and action. For an efficient review, FDA reviewers stressed the importance of effective internal communication with divisions overseeing manufacturing compliance. Interviewees cited that delays in CGMP inspections can slow the review process and/or result in multi-cycle reviews. In the cohort analyzed, 10 of 18 (56%) multi-cycle applications that were approved in two or more cycles had inspection deficiencies listed in the first-cycle action letter. Manufacturing deficiencies uncovered late in the review cycle may not allow sponsors sufficient time to correct issues before the goal date. This concern was particularly pronounced for applications requiring inspections at foreign locations which, due to increased administrative requirements as well as field inspector resource constraints, generally have longer lead times.
A correlation between foreign inspection and multi-cycle approval was not reflected in the analysis, as shown in Exhibit 34. Applications requiring foreign inspections actually had a slightly higher first-cycle approval rate as compared to applications requiring only domestic inspections.
Nevertheless, the long lead times for the planning and execution of site inspections (up to four months, with additional vulnerabilities for foreign inspections; see Exhibit 35 for overview of the manufacturing inspection process and representative timelines) can place single cycle approvals at risk, especially for applications with Priority status which have compressed review times. Applications that change status (e.g., from Standard to Priority), or for which additional inspection sites are identified late in the review, are also at added risk.
Earlier involvement of Consumer Safety Officers (before or during pre-NDA/BLA meetings) and efforts to streamline inspection planning and execution will mitigate this risk by increasing the sponsor's ability to resolve issues prior to the end of the first-cycle. Division interviews have suggested that inspection officers attending pre-NDA/BLA meetings gain earlier insight that aids the inspection process. Many divisions also encourage early submission of the CMC section of applications. A review of inspection team notification and scheduling can yield insights into ways whereby the planning can be streamlined and long lead times reduced.
Exhibit 36 provides a summary of suggested improvement opportunities identified in the retrospective analysis. Emphasis on implementation should, in particular, be placed on less-experienced sponsors who seem to be at greatest risk for multiple cycle review. Earlier and more effective communication with sponsors, enhanced by a check-and-follow up approach, will maximize the potential to identify and communicate issues and develop a resolution plan in a timely manner. Developing guidelines for, and increasing rigor in, the administration of post-marketing commitments may further increase the effectiveness of PMCs, providing patients earlier access to medicines while enabling select open issues to be effectively and reliably assessed after approval. Some of the proposed measures are part of the GRMP guidelines recently published by the FDA. A planned prospective study will attempt to capture the extent to which these guidelines have been implemented, and the costs and benefits that have been realized.
This exhibit has a time line runs from the Pre-IND stage through to the second cycle of review, or Phase 4 activities.
|Stage of Review||Recommendations|
|Pre-IND: Phase 1 and Phase 2||1. Pre-IND-Product strategy discussion with sponsor|
|EOP2 Meeting||2. EOP2 Meeting—Phase 3 planning, sponsor follow-ups with SPAs|
|Phase 3||3. Mid-Phase 3—Discuss preliminary results|
|Pre-NDA/BLA||4. Involve OC/DMPQ at pre-NDA/BLA|
|Submission of application|
|Application review||5. Early review cycle – sponsor presentation and internal planning meeting|
|Mid-Cycle Meeting||6. Mid-Cycle Meeting—Discipline presentations|
|Application review||7. A Division guide for post-marketing commitment development|
|First cycle ends|
|Phase 4 or 2nd Cycle|
|Pre-IND stage to|
Submission of application
|8. Use a checklist to support FDA-sponsor discussions|
|Submission of application to End of first cycle||9. A “check and follow-up” system for issue resolution|
|All stages from Pre-IND to end of first cycle review||10. Inexperienced/Foreign Sponsors hire appropriate outside expertise|
11. Guidance and website management
Note: Some divisions already have some of these concepts in place
1. CBER has incorporated this concept into its BLA review process; CBER Consumer Safety Officers participate in pre-BLA meetings. [return to text]
2. The Center of Biologics Evaluation and Research (CBER) uses correlated terms of Approved, Complete Response, and Not Approved. [return to text]
3. Exhibit 3 only displays four sample hypotheses taken from a comprehensive list of 62. [return to text]