U.S. flag An official website of the United States government
  1. Home
  2. For Industry
  3. FDA User Fee Programs
  4. Prescription Drug User Fee Amendments
  5. CMA Final Report Part III
  1. Prescription Drug User Fee Amendments

CMA Final Report Part III

 

 4.2.4                Early Review Analysis

Although there seems to be a positive trend in first-cycle approvals for Pilot 1 products versus the comparison cohort (Exhibit 4–2), this may not be solely due to the review of early submitted portions of an application and subsequent feedback.  Some other factors may come to play that contribute to the first-cycle approval include:

  • Timing of RU submissions relative to the complete application
  • Number of major application deficiencies and identification timeframe
  • Product indication/medical need
  • Product efficacy/safety profile.

A review of Pilot 1 product outcomes suggests that other factors beyond early review may have contributed to favorable first-cycle outcomes, irrespective of Pilot participation.  The early RU review for Product C for example, had no critical deficiencies that could have led to a delay in approval (Exhibit 4–10).  In the case of Product G, the majority of deficiencies were identified with the complete submission and not in the early submitted RUs (Exhibit 4–10).  Of the products analyzed, it appears that only Product E benefited significantly from early CMC RU review, which increased the time available for the sponsor to address the issues within the first-cycle (Exhibit 4–10).

Exhibit 4–10.  Factors Beyond the Pilot 1 Process Effecting First-cycle Review Outcome

ProductPilot 1 Benefit?Comment
B

no

•  CMC delayed and filed as final RU - several issues were identified requiring an amendment, and triggering a 3-month goal extension
C

no

•  Only minor issues were discussed during the review; did not provide a basis for an approval delay
E

yes

•  One RU, CMC, was submitted four months before the complete submission; early submission allowed CMC issues to be addressed easily
G

no

•  Major issue in CMC discipline which was the RU submitted - there was no added benefit from the pilot
H

no

•  Product addresses high unmet need; would have received high attention regardless of filing/review designation
I

no

•  CMC issues elevated a little sooner due to early RU; timing of other issues, e.g., immunogenicity, was similar to a regularly filed priority review
J

no

•  Product addresses high unmet need; would have received high attention regardless of filing/review designation

The outcome of three Pilot 1 products that did not receive first-cycle approval is likely to also be independent of the review process, since all three failed to adequately demonstrate efficacy: a deficiency that typically requires significantly longer resolution time than afforded by early feedback under Pilot 1 (Exhibit 4–11).

 

Exhibit 4–11. Multi-cycle Pilot 1 Products Would Have Had The Same Outcome Regardless of Pilot Status

ProductPilot 1 Benefit?Comment

A

no

Failed to meet non-inferiority criterion for primary efficacy; new study required

D

no

Failed to demonstrate efficacy due to an unacceptable study population; not approvable

F

no

Failed to meet clinically meaningful endpoints; new study required

 

In comparison, several of the Fast-Track/Rolling Review products that received early review and feedback prior to the complete submission, as identified by a FDA request for information, also received 1st-cycle approval.  Similar to the Pilot products however, additional factors may have influenced the outcome, such as the unmet medical need (Exhibit 4–12).

 

Exhibit 4–12. Some Fast-Track/Rolling Products Received Early Review and Feedback

Exhibit 4-12. Some Fast-Track/Rolling Products Received Early Review and FeedbackD
 

 

In general however, it is not clear whether an early review of Rolling Submissions occurs consistently.  Using FDA's request for information (RFI) as an indicator, 3 of 8 Fast-Track applications that submitted rolling portions of the application received an early review (Exhibit 4–12).  For the other 5 applications, it is difficult to ascertain whether or not a review began before the complete submission since the first RFI was not communicated until after the complete submission (Exhibit 4–13).

As with the Pilot products, the comparison cohort shows that the impact of early review on first-cycle approvals is still unclear.  Of the 4 comparison cohort products that received first cycle approval, only 2 received early review.  This again demonstrates that factors in addition to or independent of early review and subsequent feedback contribute to a favorable first-cycle outcome.

 

Exhibit 4–13. Fast-Track/Rolling Products With Limited Evidence of Early Review

 Exhibit 4-13. Fast-Track/Rolling Products With Limited Evidence of Early ReviewD
 

 

 4.3           Incremental Pilot 1 Effort for Sponsors and FDA

This section analyzes the incremental effort to implement Pilot 1 for sponsors and FDA.  The incremental effort is related to activities specifically conducted to prepare and review products under the Pilot program above what would typically be expended for a Fast-Track/Rolling review product  The assessment of these additional activities was the basis for estimating the incremental effort related directly to the Pilot.

 4.3.1                 Sponsor Effort

Sponsors that participated in the Pilot program noted that the majority of activities related to application preparation and submission would have been performed irrespective of the submission route.  Only few additional activities were reported with minimal incremental effort (Exhibit 4–14).  These were comprised of:

  • Internal meetings to decide whether to participate in the Pilot
  • Preparation of the Pilot enrollment application
  • RU submission schedule negotiation with the FDA

 

Exhibit 4–14.  Major Fast-Track Activities Versus Additional Pilot-specific Activities Reported by Participating Sponsors

Major Sponsor Fast Track Activities

Pre-submission

Post-submission

Request for Fast-Track Designation

Respond to FDA RFIs

EOP1 Meeting

Submit Amendments (if applicable)

EOP2 Meeting

Receive Action Letter (at 6-10 months)

SPA

 

Pre-NDA/BLA Meeting

 

Prepare Application Sections

 

Submit Complete Sections

 

Additional Sponsor Pilot-Specific Activities

Pilot-specific Activities

Incremental Effort

(# Labor Hours)

Comment
Pilot Application Preparation

Sponsors reported the Pilot 1 application effort was minimal
Internal Meetings

Sponsors reported 1-3 additional meetings
Meetings with FDA

N/A

Sponsors reported no Pilot-specific meetings with FDA

Furthermore, many sponsors mentioned that the Pilot helped with internal workload distribution, a benefit especially highlighted by smaller companies with limited resources dedicated to application preparation and interfacing with the FDA review team.  Specifically, sponsors noted some pre-submission and post-submission workload benefits resulting from the Pilot:

  • Pre-submission: ability to prepare each unit separately helped focus on one area at a time, reduced complexity, reduced the need to coordinate multiple resources at the same time
  • Post-submission: more efficient resource management through earlier re-deployment once individual discipline reviews were completed
  • Focused attention on resolving discipline-specific issues raised in the discipline review letter

Overall, the sponsors perception of Pilot 1 was that the effort involved was not burdensome and there was no downside to participating in the program.

 4.3.2                 FDA Effort

Similar to the effort analysis conducted for sponsors, the Pilot effort for FDA was measured in terms of additional activities that FDA reviewers needed to perform beyond what is typically necessary for the Fast-Track/Rolling reviews.  The activities included start-up efforts, additional filing meetings, Pilot admittance/RU schedule negotiations, acknowledgement/discipline review letter writing and review, additional internal FDA meetings, and RU re-work.  The additional Pilot-specific activities for FDA are described below and are also summarized in Exhibit 4–15.    

Start-Up Effort
As this was a new program, each Division receiving a Pilot 1 product incurred start-up efforts associated with establishing how to approach the reviews, for example, by creating appropriate letter templates and/or ensuring the review team were aware of the requirements of the Pilot for review purposes.  By design, the CMA guidance did not specify Pilot implementation, therefore, each Division addressed this separately.  Divisions that had multiple Pilot 1 products noted that this effort was mostly incurred with the first Pilot submission.  Subsequent Pilot products accepted into the same review Division required minimal additional startup effort.

Additional Filing Meetings
In a typical Fast-Track/Rolling review, FDA review Divisions will hold one filing meeting after the complete submission. Under the Pilot, two of the Divisions interviewed held a filing meeting for each RU in addition to a filing meeting after the complete submission.  Other Divisions continued to hold one filing meeting upon receipt of the complete submission; however, a determination whether each RU is substantially complete and meets the conditions of the Agency agreement within 60 days must still occur.

Pilot Admittance/RU Schedule Negotiations
FDA incurred costs for reviewing each request for Pilot participation in order to determine eligibility.  For products that were ultimately accepted, some review team members reported minimal negotiation time with sponsors regarding the most appropriate RU schedule. 

Acknowledgement/Discipline Review Letters
The FDA incurred additional costs for letters acknowledging the receipt of RUs or reporting the results of discipline reviews.  These included direct costs associated with developing the communications as well as multiple levels of review. 

Internal FDA Meetings
Aside from start-up meetings to ensure that the review team was aware of the review process under Pilot 1, there were not many additional Pilot-specific meetings reported.  However, there were a few instances where a typical internal meeting that may only happen once during the course of a non-Pilot review, happened two or more times due to the review of multiple RUs plus the complete submission under the Pilot.

 RU Re-work
A concern voiced by FDA reviewers was the potential for duplicative review efforts-initially upon receipt of the RU, and again during review of later sections that required a reference to prior RUs.  Alternatively, certain deficiencies identified upon early RU review be resolved by the sponsor in time for re-submission during the first action.  In practice however, there was only one product where some incremental RU rework was required.  Most reviewers noted that RU reviews did not differ in terms of level of effort or approach compared to any product review with a 6-month clock.  In particular, it was noted that RUs were typically not reviewed more than once.  Two of the Pilot products had amendments to RUs that triggered clock extensions, but these amendments likely would have occurred regardless of Pilot status.  Therefore, the re-review work on the amendments was not viewed as a Pilot-specific incremental effort.

 

Exhibit 4–15.  Major FDA Fast-Track Review Activities Versus Additional Incremental  Pilot 1-specific Activities

Major Sponsor Fast Track Activities

Pre-submissionPost-submission

Grant Fast-Track Designation Request

Filing Meeting

EOP1 Meeting

Discipline Reviews

EOP2 Meeting

Mid-cycle review Meeting

SPA

Advisory Committee Meeting (if applicable)

Pre-NDA/BLA Meeting

Discipline Review Complete/Leadership Signoff

 

Labeling (if applicacble)

 

Action Letter

Additional Sponsor Pilot-Specific Activities

Pilot ActivitiesRecurring Activities"Startup" Activities
Additional Filing meeting

Some divisions have filing meeting for each RU

Additional meetings to clarify Pilot process for the review team
Pilot Admittance/RU Schedule neg.

Additional meetings to determine eligibility of applications and to negotiate RU schedule

Establishing which meetings (e.g., Filing) were necessary by RU
Ack/Discipline Letters

Sending an acknowledgement and discipline review letter for each RU

Pilot acceptance decision-making process
Internal FDA meetings

Typical internal meetings may happen for each RU instead of only once in standard review

Generating letter templates
RU Re-work (RU were typically only reviewed once per cycle except amendments)

Revisiting a discipline review in case of long lags between submissions (1 instance for 11 applications)

 

 

While the incremental Pilot-specific activities did not occur for every Pilot product, in order to estimate the potential incremental effort for FDA, the analysis assumed a scenario in which the all the incremental activities would take place (i.e., worst-case/conservative scenario). To quantify the incremental effort, FDA data sources such as interviews, time tracking data, and the document storage system were used to capture those involved in the various Pilot activities.  For each activity, the effort for discipline reviewers, regulatory project managers, team leaders, Division directors/deputy, office directors/deputy was estimated based on, for example, meeting preparation time, actual meeting time, and post-meeting efforts such as writing/reviewing meeting minutes for each review team member.  

Based on this analysis, the total incremental effort for the Pilot was estimated to be between 190-360 direct labor hours per application.  This effort is above what would typically be expended for a non-Pilot, Fast-Track/Rolling reviewed product in those Divisions included in the evaluation (i.e., experienced the Pilot 1 process).  The breakdown by Pilot-specific activity is presented in Exhibit 4–16.  The effort is displayed as a range, where the lower bars on the chart represent the low end range and the upper bars the high end range of effort as reported by review team members involved in the various activities.

 

Exhibit 4–16.  Incremental Effort for FDA of Pilot-Specific Activities per Application (1 cycle)

Baseline effort= Non-pilot Fast-Track/rolling submissions within divisions with pilot products

FDA ActivitiesAdditional Filing MeetingsPilot Application/ NegotiationAcknowledgement/ DR LettersAdditional Internal MeetingsStart Up EffortRU Re-workTotal Range
Lower Limit Effort44hrs42hrs35hrs24hrs33hrs12hrs190hrs
Upper limit Effort90hrs59hrs72hrs58hrs69hrs0360hrs

The effort is not distributed evenly among the review team members (Exhibit 4–17).  Most of the incremental effort impacts the Regulatory Project Managers (RPMs), in charge of coordinating key review activities including: sponsor communications, scheduling reviewers, FDA-sponsor meetings, internal product-specific meetings and document reviews, and tracking PDUFA deadlines.

 

Exhibit 4–17. Incremental Effort of Pilot-Specific Activities by Function (1 cycle)

Baseline effort= Non-pilot Fast-Track/rolling submissions within divisions with pilot products

FDA RolesRPMReviewersTeam LeadersDivision Director/DeputyOffice Director/DeputyTotal Range
Lower Limit Effort53hrs66hrs45hrs23hrs3hrs190hrs
Upper limit Effort104hrs158hrs72hrs00360

 

For the Divisions that participated in Pilot 1, Exhibit 4–18 shows the incremental effort for these Divisions to perform a Pilot 1 style review on all of their Fast-Track product applications.  The estimated incremental effort increase is between 1,520-2,880 direct labor hours (based on the average number of Fast-Track applications submitted to those Divisions between FY 2003-2005).    

 

Exhibit 4–18. Extrapolation of the Incremental Pilot Effort to the Average Number of Fast-Track Products Submitted to Divisions with Pilot 1 Experience

  4-18. Extrapolation of the Incremental Pilot Effort to the Average Number of fast-Track Products Submitted to Divisions with Pilot 1 ExperienceD
 

The estimated incremental Pilot 1 effort may not be evenly distributed among the Divisions with Pilot experience based on historical Fast-Track submissions.  For example, between FY2003-FY2005, the Anti-infective and Ophthalmology and Drug Oncology Divisions received approximately 56% of all Fast-Track applications submitted in those fiscal years (Exhibit 4–19).  Based on interviews with reviewers in those Divisions, to accommodate for this influx of Fast-Track applications, they strive to perform a six month review on all products considered a high priority, irrespective of the review designation.  Because many of the procedures for early feedback and accelerated review have already been integrated, these Divisions may incur less Pilot 1 implementation effort (in the lower part of the range; Exhibit 3-18) compared to other Divisions.  The estimated incremental increase may not be transferable to Divisions that did not have a Pilot 1 product or who do not have significant experience reviewing Fast Track applications.  To understand the broader potential impact of expanding Pilot 1, FDA may consider prospectively monitoring the Pilot 2 products still in drug development and give sponsors the option to submit their applications as Pilot 1 products (if an application is submitted for those products).  This would allow for further evaluation of the Pilot 1 effort, including the evaluation of 2 additional Divisions that were not included in this Pilot 1 evaluation.

 

Exhibit 4–19.  Three Year Total Fast-Track Submissions by Divisions (FY2003-2005)

Exhibit 4.19. Three Year Total Fast-Track Submission by Divisions (FY2003-2005)D
 

There is a possibility that the number of Fast-Track requests will increase if sponsors value the commitment of early review.  This would obviously increase the estimated incremental effort shown in Exhibit 4–18. 

If this increase were to occur, a major FDA concern is that important work without a PDUFA goal deadline may be shifted to a later date.  Consequently, delaying such activities may have a negative impact on public health issues and other FDA responsibilities.

On balance, this workload prioritization challenge is a broader concern for the FDA, one that exists regardless of this Pilot program and especially with product reviews with a 6-month clock.  The interviewed FDA Pilot participants did not report challenges completing other critical work specifically because of the Pilot.  To caveat this finding, interviewees were primarily asked to compare the Pilot effort to their experiences with other Fast-Track, priority reviewed products and were not specifically asked about the shifting of non-PDUFA work.  This is not to say, however, that high workload demands were not a concern, they were just not directly attributed to the implementation of the Pilot.

While this evaluation cannot validate or rule out the impact of non-PDUFA work being displaced, it is likely a broader concern for all products receiving a priority review.  Therefore, additional evaluation may be warranted to determine the impact of the displacement of non-PDUFA work for products receiving a priority review.

 

Next

Back to Top