For Industry
Long Description Independent Evaluation of FDA's First Cycle Review Performance – Retrospective Analysis Final Report
Exhibit 1. Action Package Approval Rate
Action packages for 77 products were studied (14 BLAs and 63 NDAs). Thirty-six (36) action packages, or 47%, were for products approved on the first cycle of review (7 BLAs and 29 NDAs).
Question: What best practices led to first cycle approvals?
Forty-one (41) action packages, or 53%, were for products having multi-cycle reviews.
Of those products having multi-cycle reviews, 18 (5 BLAs and 13 NDAs) or 23%, were approved after additional reviews when the study was started.
Question: Could anything have been done to avoid multi-cycle review?
Twenty (20) products (2 BLAs and 18 NDAs) or 25%,were deemed approvable after additional reviews at the time of the study. Three (3) NDAs were deemed NA after additional reviews at the time of the study.
Exhibit 2. Overview of Analysis Process
Stage 1-For the study, the hypothesis and metric generation was done first, followed by review of the action package. From the review the team created a word summary, which was analyzed qualitatively, and also completed a data collection instrument, which was analyzed quantitatively. From these two sources came the emerging themes and recommendations.
Stage 2-After the development of the emerging themes and recommendations came the validation of the themes and recommendations, and the division-level interviews on philosophies and best practices. The themes and recommendations, and the results of the interviews, were combined to generate the improvement opportunities.
Exhibit 4. RPM Product Summary Comments
RPM (Regulatory Project Manager) Product Summary Comments (Review of product summaries of 23 action packages, which was approximately one-third of the cohort, were reviewed with regulatory project managers for accuracy)
Of the comments by RPMs, 46% were editorial in nature, 29% elaborated on the nature of a critical issue, 13% suggested an alternative critical issue, 8% made changes to the narrative timeline, and 4% had additional meeting minutes to contribute.
Exhibit 7. Approval Rate vs. Drug Origin
For self-originated drugs (those originated in-house) 42% were approved in a single cycle and 58% in multi-cycles. For in-licensed drugs (those originated externally) 65% were approved in a single cycle and 35% in multiple cycles.
Exhibit 8. Number of Approval Cycles as a Function of Advisory Committee Meetings
For reviews having Advisory Committee meetings, 31% (n=4) were approved in a single cycle and 69% (n=9) in multiple cycles. For reviews not having Advisory Committee meetings, 46%(n=25) were approved in a single cycle and 54% (n=29) in multiple cycles.
Exhibit 9. Percentage of Multi-Cycle Reviews by Sponsor Experience
a. The first cycle approval rate for sponsors that had drugs approved previously by the FDA was 51% of 47 submitted compared to 30% of 20 submitted for sponsors with no prior approved drugs.
b. The first cycle approval rate for sponsors that had drugs approved previously by the FDA was 53% of the 38 products in the same therapeutic area, compared to 44% of the 9 products in same therapeutic area. Note: Not including DESI, 505(b)(2) drugs
Exhibit 14. Effect of End of Phase 2 Meetings on Approval Rate
a. EOP2 Meeting--
Of the 46 application reviews that had EOP2 meetings, 52% were reviewed in one cycle and 48% in multiple cycles. Of those that did not have a EOP2 meeting, 29% were reviewed in one cycle and 71% in multiple cycles.
b. Timing of Issue ID for Multi Cycle Products with EOP2 Meeting--
Of the 22 applications that had EOP2 meetings and were reviewed in multiple cycles, 23% had the key issue identified at the EOP2 meeting, 36% at the Pre-NDA meeting and 41% during the review.
Exhibit 15. Effect of Pre-NDA/BLA Meetings and Timing on Approval Rate
a. Pre-NDA/BLA Meeting --Of the 58 application reviews that had Pre-NDA/BLA meetings, 47% were reviewed in one cycle and 53% in multiple cycles. Of those that did not have a Pre-NDA/BLA meeting, 33% were reviewed in one cycle and 67% in multiple cycles.
b. Timing of Pre-NDA/BLA Meeting--Of the applications that had Pre-NDA/BLA meetings, 24 had the Pre-NDA/BLA meeting less than 6 months before application submission. Of those, 46% were reviewed in one cycle and 54% in multiple cycles. Of the 34 that had a Pre-NDA/BLA meeting more than 6 months before submission, 58% were reviewed in one cycle and 41% in multiple cycles.
For those applications that had Pre-NDA/BLA meetings, the findings were the same relative to review in one cycle or multiple cycles for applications with or without an EOP2 meeting.
Exhibit 17. Resolution of Issues Identified Pre-submission
a. Percent of Applications with Significant Pre-Submission Issues Resolved by First Action
Of those applications with significant Pre-submission issues 29% (7) were resolved by first action, and 71% (17) were not resolved.
b. Percent of Applications with Significant Issues Identified Pre-Submission and Resolved by First Action
Of those 7 applications with significant pre-submission issues resolved by first action, only about 20% involved safety or efficacy issues, while three quarters involved trial execution or trial design.
Pre-submission meetings identified major issues in 24 products—in some cases, more than one major issue was identified; 17 drugs required more than 1 review cycle.
Exhibit 18. Deficiency Timing in Multiple Cycle Applications
Three timelines show the disposition of Products G, A, and H.
Review of Product G:,
- FDA issued a manufacturing warning letter before the sponsor submitted the Product G application for review
- About 600 days after the warning letter, FDA issued an Action letter of Approvable
- In the 500 days after the Approvable action, the sponsor resubmitted the application and at 500 days, received another Approvable letter
- Soon after the second Approvable letter, the sponsor resubmitted the application, and corrected the deficiencies
- FDA issued a third action letter of Approval soon after the deficiencies were corrected
Review of Product A:
- FDA communicated with the sponsor on multiple deficiencies well before the application was submitted
- About 1400 days after FDA told the sponsor about the multiple deficiencies, FDA issued a Not Approvable letter
- Approximately 600 days after that letter, the sponsor resubmitted the application, correcting the multiple deficiencies
- Soon after that resubmission, the product was approved
Review of Product H:
- FDA communicated with the sponsor on multiple deficiencies well before the application was submitted
- About 1500 days after FDA told the sponsor about the multiple deficiencies, FDA issued an Approvable letter
- Approximately 400 days after that letter, the sponsor resubmitted the application, correcting the multiple deficiencies
- Soon after that resubmission, the product was approved
Exhibit 20. Check-and-Follow Up Communication
Failure of Early ID
Root Cause:
- Sponsors drive pre-submission discussions
- Meeting protocols vary across teams
Leads to:
Unclear Problem Resolution Plan
Root Cause:
- No formal process for developing resolution plans
- Sponsors choose to follow-up or not
Leads to:
Lack of Priority
Root Cause:
- Differing views of importance for FDA and sponsor; often not communicated or unclear
Leads to:
Variability in Standards
Root Cause:
- Unclear criteria and non-uniform enforcement of requirements (not always driven by medical need)
quot;Check and Follow Upquot; Communication
Step 0—Development plan quot;Checklistquot;
- Guides pre-submission meetings
- Cover all major areas
- Track progress
- Examples of items on quot;Checklistquot; are endpoints, length, size
Step 1--Sponsor-FDA discussion
Step 2—Meeting minutes: sign-off on action items
Step 3—Follow-up; issue clarification and alignment
Step 4—Sponsor's plan of action
Step 5—Evaluation of plan by FDA
Step 6—back to Step 1 if necessary
Exhibit 23. FDA-Sponsor Communications
a. Total Number of Communications
Single-cycle applications:
Average number of communications between FDA and sponsor was 25.
| Application designation— first, second, third |
Number of Communications Between FDA and Sponsor | NDA/ BLA |
|---|---|---|
| 1 | 1 | NDA |
| 2 | 4 | NDA |
| 3 | 5 | NDA |
| 4 | 6 | NDA |
| 5 | 7 | NDA |
| 6 | 7 | NDA |
| 7 | 7 | NDA |
| 8 | 8 | NDA |
| 9 | 12 | NDA |
| 10 | 13 | BLA |
| 11 | 15 | NDA |
| 12 | 15 | NDA |
| 13 | 16 | NDA |
| 14 | 18 | NDA |
| 15 | 18 | NDA |
| 16 | 19 | NDA |
| 17 | 24 | NDA |
| 18 | 26 | NDA |
| 19 | 28 | NDA |
| 20 | 30 | BLA |
| 21 | 33 | BLA |
| 22 | 35 | NDA |
| 23 | 38 | NDA |
| 24 | 38 | BLA |
| 25 | 41 | BLA |
| 26 | 41 | NDA |
| 27 | 45 | NDA |
| 28 | 52 | BLA |
| 29 | 62 | NDA |
| 30 | 92 | BLA |
Multi-cycle applications:
Average number of communications between FDA and sponsor was 21.
| Application designation— first, second, third |
Number of Communications Between FDA and Sponsor | NDA/ BLA |
|---|---|---|
| 1 | 1 | BLA |
| 2 | 4 | NDA |
| 3 | 6 | NDA |
| 4 | 7 | NDA |
| 5 | 7 | BLA |
| 6 | 8 | NDA |
| 7 | 8 | NDA |
| 8 | 8 | BLA |
| 9 | 10 | NDA |
| 10 | 10 | BLA |
| 11 | 12 | NDA |
| 12 | 12 | NDA |
| 13 | 13 | NDA |
| 14 | 13 | NDA |
| 15 | 14 | NDA |
| 16 | 14 | NDA |
| 17 | 15 | NDA |
| 18 | 15 | NDA |
| 19 | 16 | NDA |
| 20 | 17 | NDA |
| 21 | 18 | NDA |
| 22 | 18 | NDA |
| 23 | 20 | BLA |
| 24 | 21 | NDA |
| 25 | 22 | BLA |
| 26 | 23 | NDA |
| 27 | 25 | NDA |
| 28 | 27 | NDA |
| 29 | 30 | NDA |
| 30 | 30 | NDA |
| 31 | 31 | NDA |
| 32 | 38 | NDA |
| 33 | 40 | NDA |
| 34 | 42 | NDA |
| 35 | 43 | NDA |
| 36 | 56 | NDA |
| 37 | 81 | NDA |
Excluding DESI, 505(b)(2) drugs
Packages with no communications omitted from graphs and average calculations.
Source: BAH analysis
b. Distribution Throughout Review Cycle (per application)
| Elapsed Time in Review Cycle (Shown as fraction of total review time) | Communications per Application— Single-cycle applications |
Communications per Application— Multiple-cycle applications |
|---|---|---|
| 0 | ||
| 0.2 | 1.3 | 1.4 |
| 0.4 | 2.9 | 3 |
| 0.6 | 3.2 | 4.3 |
| 0.8 | 4.5 | 5 |
| 1 | 11.2 | 6.6 |
Increase for single-cycle applications in the final 20 percent of the review cycle likely due to finalizing open issues (e.g., labeling)
Note: Similar findings observed for analysis by issues sub-type (e.g., CMC)
Exhibit 26. Issue Resolution Timing - Multiple Cycle Applications
For Product B: Manufacturing facility issues were the key ones cited in the first action letter, which was issued 302 days after submission. These issues were initially communicated to the sponsor 30 days before the action date. The sponsor resubmitted the application and resolved the issues 40 days after the first action date. The product was approved on Day 380, representing a delay of approximately 3 months.
For Product Q: FDA communicated to the Sponsor cGMP issues approximately 150 days before the date of the first action letter, which issued on Day 300. The sponsor corrected deficiencies and resubmitted the application approximately 20 days after the first action letter. The product was approved on Day 503, representing a delay of about 7 months.
Exhibit 27. Issue Resolution Timing - Single Cycle Applications
For Product R: Initial communication concerning deficiencies and additional clinical endpoint study needed occurred about 80 days after submission. These issues were resolved approximately 80 days later and the product was approved on Day 183.
For Product S: Initial communication concerning cGMP and Phase 4 immunogenicity issues occurred about 170 days after submission. These issues were resolved approximately 100 days later and the product was approved on Day 277.
For Product T: Initial communication concerning facility deficiencies and carcinogenicity issues occurred shortly after submission. These issues were resolved approximately 280 days before the product's first action date and the product was approved on Day 302.
Exhibit 28. Review Communication Summary
After Submission, the internal planning meeting occurs. The objectives of this meeting are to:
- Generate interest: Nature of submission, anticipated major hurdles, establish resource commitment
- Develop a Review Plan: Approach and roles, assign ownership; schedule all subsequent mtgs/deadlines
- Set Expectations: Workloads and priorities; set meeting and progress expectation
Between Submission and Filing, is the Sponsor presentation (which is most beneficial when conducted before the Filing meeting). The objectives of this meeting are:
- Orientation: Overview of the actual submission content
- Generate Discussion: Intellectual discussion of the product and application
Between Filing and First Action, is the mid-Cycle Meeting. The objectives of this meeting are:
- Maintain Momentum: Opinion on emerging action outcome
- Stay Apprised: Discipline presentations, exchange critical info, communicate challenges
- Communicate to Sponsor: Manage expectations; dialog on resolution approach
- Plan for the Finish: Process issues and mitigation plans, reinforce deadlines, start labeling discussions.
From Submission to Filing to First Action there is ongoing FDA-Sponsor Communication
- Inform Early: Communicate issues at earliest feasible time (triage as appropriate)
- Set Goals: Share workload and priorities with sponsor
- Manage expectations: Reach consensus on meeting and progress expectations
Note: internal planning meetings and sponsor presentations are most beneficial when conducted before the Filing Meeting
Exhibit 29. Post-Marketing Commitments Single vs. Multi-Cycle Reviews
a. Percent of approved drugs with post-marketing commitments by cycle
For drugs approved on the first cycle, 80 percent had post-marketing commitments (n=24) and 20 percent (n=6) did not have them. For drugs approved in multiple cycles, 88 percent had post-marketing commitments (n=15) and 12 percent (n=2) did not have them.
(Note: Did not include DESI, 505(b)(2) drugs)
b. Approvals with post-marketing commitments by number of commitments
For drugs approved on the first cycle, 20% had no post-marketing commitments (n=6), 27% had 1 to 4 commitments (n=8), 33% had 5 to 8 commitments (n=10), and 20% had 9 or more commitments (n=6). For drugs approved in multiple cycles, 12% had no post-marketing commitments (n=2), 47% had 1 to 4 commitments (n=8), 35% had 5 to 8 commitments (n=6), and 16% had 9 or more (n=1). Average number of commitments per approval was 5.4 for first cycle approvals and 4.4 for multi-cycle approvals.
Exhibit 30. Focus and Burden of Post-Marketing Commitments
a. Commitments by Area of Focus
| Area of Focus (Studies to demonstrate safety and efficacy are counted as two areas of focus) |
First Cycle Approvals: 30 products, 167 PMCs * |
Multi-Cycle Approvals: 17 products, 68 PMCs * |
|---|---|---|
| Safety | 45, 26% | 17, 25% |
| CMC | 20, 12% | 14, 21% |
| Pharm-Tox (non-clin.) | 8,5% | 12, 18% |
| Drug-Drug Interaction | 10, 6% | 1, 1% |
| Efficacy | 41, 25% | 5, 7% |
| Patient sub-pop. | 16, 10% | 10, 15% |
| ADME (clinical) | 24, 14% | 7, 10% |
| Education/Labeling | 3, 2% | 2, 3% |
(1) Includes post-marketing commitments from 39 submissions
(2) Does not Include DESI, 505(b)(2) products
(*): Studies to demonstrate safety and efficacy are counted as two areas of focus
b. Type of Post Marketing Commitment
| Type of Post Marketing Commitment | First Cycle Approvals: 30 products, 162 PMCs |
Multi-Cycle Approvals: 17 products, 75 PMCs |
|---|---|---|
| New Clinical Study | 79, 49% | 38, 50% |
| Non-Clinical Study | 19, 12% | 4, 5% |
| Continue/Extend Study | 54, 33% | 23, 31% |
| Submit Report Data | 10, 6% | 2, 3% |
| Unknown | 0 | 8, 11% |
Exhibit 31. Disposition of Significant First Action Deficiencies in Multi-Cycle Approvals
| Disposition of deficiencies | a. CMC | b. Non-CMC |
|---|---|---|
| Full compliance with FDA requests | 9, 100% | 8, 49% |
| New agreement for issue resolution | 4, 25% | |
| Became post-marketing commitment upon approval | 2, 13% | |
| Unknown | 2, 13% |
Analysis includes 18 multi-cycle approved products, some had >1 major issue.
Source: BAH Analysis
Exhibit 32. Submission Timing vs. Number of Submissions or First-Cycle Approval Rates
a. Number of Submissions by Quarter (aggregate 2001-2004*)—1st Quarter,11; 2nd Quarter, 12;
3rd Quarter, 19; 4th Quarter, 35.
Similar findings observed for individual years.
b. Approval Rate by Quarter Submitted—1st Quarter, 0.64; 2nd Quarter, 0.58; 3rd Quarter, 0.68;
4th Quarter, 0.26. Average: 47%
Findings do not change based on application priority status.
Exhibit 33. Submission Timing vs. Number or Type of Issues per Application
Total Number of Issues per Application:
1st Quarter, 41; 2nd Quarter, 21; 3rd Quarter, 41; 4th Quarter, 36. Average: 36.
Key issues per Application by Type:
Formatting issues--1st Quarter, 5.2, 2nd Quarter, 1.9; 3rd Quarter, 10; 4th Quarter, 5.5. Average: 5.7
Safety issues—1st Quarter, 11; 2nd Quarter, 6; 3rd Quarter, 13; 4th Quarter, 9. Average: 10.0
Efficacy issues—1st Quarter, 8.1, 2nd Quarter, 3.1, 3rd Quarter, 4.7; 4th Quarter, 6. Average: 5.5
Sources: BAH Analysis; Division Interviews
Exhibit 34. Foreign or Domestic CGMP Inspection vs. % of Single or Multiple Review Cycles
For applications needing a foreign inspection, 50% of them were first cycle approvals and 50% were multiple cycle approvals (n=56). For applications needing a domestic inspection, 38% of them were single-cycle reviews and 62% were multiple-cycle reviews (n=21).
Exhibit 35. Schematic of the CGMP Inspection Process and Improvement Opportunities
CGMP Inspection process is a timeline starting with NDA submission, followed 2-6 weeks later by the Filing meeting, followed 1-2 days later by Determination of Inspection necessity, which requires the notification of the Office of Compliance and the Field office. Field planning takes about 1-3 months (longer for foreign sites (permits)) before the inspection occurs. That takes about 10 days, followed 2-3 weeks later by the documentation of the report on results. Review by the Office of Compliance takes about 4 days followed by their recommendation. The inspection process takes 2-4 months to complete. Improvements in this process can be made by early involvement at the pre-submission or Filing meeting stage, and by streamlined planning and coordination of the inspection.
