Executive Summary: CMA Pilot 1 Evaluation and Pilot 2 Preliminary Evaluation Studies -- Final Report
This report summarizes the results of an evaluation of the Food and Drug Administration's (FDA) Continuous Marketing Application (CMA) Pilots 1 & 2. The Pilots were part of the Prescription Drug User Fee Act (PDUFA) reauthorized in 2002, also known as PDUFA III.
The evaluation included interviews of FDA and sponsors who participated in the Pilots, and also included interviews with several sponsors who had eligible products, but opted not to participate. The group of non-pilot, eligible products formed the basis for a comparison cohort (i.e., Fast-Track/Rolling Review products submitted between 2002-2004) against which the Pilots were measured to determine any differences. In addition to interviews, the evaluation incorporated available FDA data sources such as product action packages, FDA's time tracking system, PDUFA goal date tracking system, and document filing system. All collected data were then used to evaluate the impact, benefit and effort of the Pilots.
CMA Pilot 1
CMA Pilot 1 allowed eligible sponsors submitting a New Drug Application (NDA) or Biologics License Agreement (BLA) to submit Reviewable Units (RUs)-early submissions of complete sections of the application. FDA committed to review the RUs within 6 months and provide the sponsor with a discipline review letter upon completion of the review.
Pilot 1 products were compared to similar non-pilot products to determine any differences in program outcomes, including first cycle review rates, application quality (i.e., based on the number of issues reported during the review and the number of amendments requiring extensions), and the number of FDA/sponsor communications. Other factors were examined including the impact of RU timing and order, RU interdependencies, the impact of early review, and sponsor's motivations for participating in the Pilot. Also, the evaluation analyzed the additional incremental effort of the Pilot for sponsors and FDA over any other non-Pilot, Fast Track/Rolling reviewed product.
A brief summary of the Pilot 1 program outcome findings are as follows:
Sponsor Motivations: FDA's commitment to a 6-month review of the reviewable unit submissions was a primary motivator for eligible sponsors to participate in the Pilot.
1st-cycle Approval Rate: Pilot 1 products had a favorable 1st-cycle approval rate (70%). The Pilot product approval rate, however, was not significantly different from the historical approval rate for priority-reviewed products between 2002-2004 (i.e., product receiving a 6-month review).
Application quality: Two metrics identified as proxy indicators of application quality for purposes of this evaluation were the number of issues identified by FDA in the application and the number of amendments requiring extensions on the PDUFA goal date. The results of these metrics for the Pilot products and the comparison cohort were similar-indicating that the Pilot did not appear to have any impact on application quality.
FDA/sponsor communications: There was no significant difference between the number of FDA/sponsor communications for the Pilot 1 products and for the comparison cohort products. .
RU Interdependence: RU Interdependence was evaluated to determine if segmented early RU submissions impacted the review. FDA and sponsors indicated that Chemistry, Manufacturing and Control (CMC) is considered the most independent RU, and therefore the one that would potentially benefit the most from early submission. On the other hand, development of the Clinical section is often the rate-determining step, and thus typically submitted last with the complete application submission.
RU Timing: Few sponsors can have a complete RU submission 12 months prior to the complete submission. For CMC, most sponsors considered 3 to 6 months prior to the complete submission the more feasible RU submission timeframe. These limitations on early submission prevented maximizing the opportunity for early review and subsequent feedback in the Pilot.
Early Review: One Pilot 1 product benefited from early review. Other Pilot products with 1st-cycle approvals would have likely had favorable actions regardless of early review because it seemed that the unmet medical need nature of the products had influence on the risk/benefit analysis and the high level of attention the application received. Also, for some of the products, a major issue may have been identified in the sections submitted with the complete application, therefore, although many of those issues were addressed within the first cycle, there was no benefit of early review or feedback.
Effort: The evaluation also included an analysis of the additional effort for sponsors and FDA to participate in the Pilot. Sponsors indicated that the incremental effort for Pilot 1 was minimal. Aside from completing and submitting the Pilot application, they reported that the activities they conducted were similar to any application submission. In addition, smaller-sized sponsors indicated that the Pilot helped distribute their workload. On the other hand, FDA incurred the majority of the effort in launching the Pilot at the Division level. Most of the FDA participants reported that incremental on-going efforts of the Pilot were not overly burdensome, with Regulatory Project Managers incurring the highest effort increase compared to others on the review team. Based on the effort data collected, FDA's incremental effort for Pilot 1 was estimated to be between 190-360 direct labor hours per application. This incremental effort was above the effort that would have been expended for a non-pilot, Fast-Track, priority review in the Divisions that experienced a Pilot 1 review.
The result of the evaluation was that there is no conclusive finding that indicates whether the Pilot 1 program should continue or be terminated. This may be due to several influencing factors such as the small sample size of the Pilot and the comparison cohort and the high unmet medical need nature of many of the products. The key findings of the evaluation showed:
- Pilot 1 offers some positive aspects:
- Sponsors valued FDA's 6-month RU review commitment
- Helped distribute sponsors' workload
- Additional time to address issues for early submitted RUs is a review process benefit
- FDA Pilot-participants were not overly burdened by the Pilot
- Both the Pilot 1 products and the comparison cohort products showed:
- A strong first-cycle approval rate
- Similar level of application quality
- Similar levels of communication.
Given that this evaluation focused on the comparison of the Pilot 1 program to the Fast Track/Rolling Review program, many Pilot participants offered their perceptions of the Fast Track program. Industry strongly valued the subtle differences that Pilot 1 offered over the Fast Track/Rolling Review program where FDA remained neutral.
While this analysis uncovered no resounding reason to continue the Pilot as a separate program, there may be merit to integrating some positive attributes and lessons-learned from the Pilot 1 program into the existing Fast-Track/Rolling Review structure. For example, some challenges with the current Fast Track/Rolling Review program are:
- Rolling submission requirements are not specifically defined; therefore, sponsors do not have to submit complete rolling submission sections to FDA
- FDA may or may not review a rolling submission prior to the complete application submission, depending on workload demands.
For these particular challenges, the Pilot 1 structure offers potential improvements over the Fast Track/Rolling Review program that include:
- Requiring a well-defined, complete RU submission
- Committing the FDA to a 6-month review of the early submitted RU.
If the Fast Track/Rolling Review program were modified with these Pilot 1 attributes, this would allow the FDA to plan better for reviews because they can expect a complete section for early review; early review would be conducted consistently across FDA divisions for early submissions; and issues would be identified earlier, and in some cases, may lead to resolution prior to the first action date, or may help reduce the time between cycles if sponsors can begin addressing deficiencies earlier.
Some considerations before deciding to make any modifications may include:
- With the intent of further understanding the impact of and improving features of the Pilot 1 program, the FDA should consider prospectively monitoring the current Pilot 2 products, especially those in FDA Divisions that did not have a Pilot 1 product. Also, ensure there is a mechanism to actively track metrics such as submission quality, review outcome, unforeseen additional FDA effort, and possible displacement of non-PDUFA work.
- Conduct a detailed evaluation of the Fast Track program to determine if it merits a change.
Additionally, if modifications are implemented, other considerations may include:
- If RUs are submitted close (e.g., 1 or 2 months) to the complete application submission, build in flexibility to allow the review to be conducted under the complete submission PDUFA clock rather than a separate 6 month RU clock
- Consider requiring early submissions to be electronic.
Further, if the FDA decides to implement any modifications, additional resources would be required since the Agency would incur most of the additional workload burden. If implemented, the FDA may incur, in addition to the incremental costs described in this report, additional costs during the transition phase as this program is rolled out more broadly to the Divisions which in parallel need to complete reviews of other applications currently under review. It is imperative that the FDA receive additional resources commensurate with the effort incurred to transition to and maintain the new process, in order to ensure that review Divisions are not overburdened.
CMA Pilot 2
CMA Pilot 2 allowed eligible sponsors to establish an agreement with FDA to have scientific exchanges (e.g., meetings, protocol reviews, document reviews) throughout the product development process. This evaluation for Pilot 2 is preliminary. Most of the products are still in the middle of product development, so it is too early in the process to analyze the impact of the program outcome. Therefore, the evaluation focused on interim FDA and sponsor perceptions of the Pilot through interviews and FDA's data systems mentioned above.
Findings showed that there were two different Pilot 2 approaches sponsors used to schedule exchanges with FDA-one approach established an estimated schedule in advance (the Fixed Schedule) and the other focused on when FDA could provide feedback based on the type of interaction (the Trigger Method), scheduling interactions as needed. Due to the uncertainty of product development, the schedules established early using the Fixed Schedule approach often slipped. The Trigger Method was received more positively from both FDA and the sponsors. Sponsors received guaranteed feedback that they valued the most from the Pilot, and the FDA did not feel committed to a blanket agreement to grant all interactions before their need was determined. Also, the negotiation process between the sponsor and FDA for the Trigger Method approach was much less burdensome than the process for agreeing to a fixed schedule of meetings.
In terms of effort, following the Fixed Schedule method and actually conducting all planned meetings, could result in a doubling of the number of communications for a Pilot 2 product over the Fast-Track program. The estimated incremental effort is over 500 hours for a Pilot 2 product over a Fast-Track product, based on a Fixed Schedule approach.
Although it is too early in the Pilot 2 program to determine the value of its impact, there were early observations/takeaways to take into account:
- Guidance should be provided to sponsors on using the Trigger method (if there is any extension or adoption of the Pilot 2 program in the future)
- Communication approaches vary across FDA Divisions and among sponsors; FDA wanted sponsors to communicate problems upfront; and sponsors wanted more non-binding open dialogue from FDA
- Since most of the Pilot 2 products are mid-development, FDA should continue to monitor these products prospectively through application review (where applicable).