FDA – Industry MDUFA III Reauthorization Meeting
February 9, 2011, 1:00 – 5:00 pm
FDA Switzer Building, Washington, DC
To discuss FDA’s root cause analysis of Original PMAs and Panel Track Supplements received in fiscal years 2008 – 2010 and which missed the Tier 1 review goal of 180 days under MDUFA II.
|Malcolm Bertoni||Office of the Commissioner (OC)|
|Ashley Boam||Center for Devices and Radiological Health (CDRH)|
|Nathan Brown||Office of Chief Counsel (OCC)|
|Kate Cook||Center for Biologics Evaluation and Research (CBER)|
|William Hubbard||FDA Consultant|
|Don St. Pierre||CDRH|
|Ruth Watson||Office of Legislation (OL)|
|David Fisher||Medical Imaging Technology Alliance (MITA)|
|John Ford||Abbott Laboratories|
|Donald Horton||Laboratory Corporation of America Holdings|
|Mark Leahey||Medical Device Manufacturers Association (MDMA)|
|Joseph Levitt||Hogan Lovells US LLP|
|David Mongillo||American Clinical Laboratories Association (ACLA)|
|James Ruger||Quest Diagnostics|
|Patricia Shrader||Becton Dickinson|
Meeting Start Time: 1:00 pm
FDA Presentation of Original PMA and Panel Track Supplement Analyses and Group Discussion
FDA analyzed Original PMAs and Panel Track Supplements received by CDRH under MDUFA II to determine factors contributing to review times in excess of the 180 day Tier 1 goal. FDA examined the receipt cohort from fiscal years 2008 - 2010 and identified an analysis cohort that included 48 submissions that met the Tier 1 goal and 37 submissions that missed the Tier 1 goal.
FDA grouped the major reasons why reviews exceed the 180 day Tier 1 goal into the following three main factors: factors related to FDA, factors related to Industry or related to both Industry and FDA, and the impact of Advisory Panel review.
FDA identified as a major contributing factor staff turnover. FDA evaluated the analysis cohort described above to determine if turnover in a key member of the review team (i.e., when the lead reviewer, medical officer, or branch chief changes during the PMA review period) correlates with extended review times. The cohort that missed the Tier 1 goal exhibited only slightly higher turnover of lead reviewers when compared to the cohort that met the Tier 1 goal. However, the cohort that missed the Tier 1 goal exhibited significantly higher turnover of medical officers and branch chiefs when compared to the cohort that met the Tier 1 goal. FDA also analyzed whether turnover of multiple members of the review team correlates with extended review times. Results show that turnover of a single key member of the review team did not have a substantial impact; however, when two key members turn over and one of them is a medical officer, the incidence of missing the Tier 1 goal is much higher. In the cohort of PMAs that missed the goal, in almost all cases where there were two review staff who changed, the medical officer was one of the reviewers who changed. In the cohort of PMAs that met the goal, where there were two review staff who changed, the medical officer changed in less than half of the cases. In the rare cases where there is turnover in the lead reviewer, medical officer, and branch chief, the Tier 1 goal has not been met. FDA concluded that turnover of the medical officer, and in some cases the branch chief, has a substantial impact on their ability to meet the Tier 1 goal.
Industry asked a number of questions relating to the analysis and how review teams are structured. Industry requested that a similar analysis be conducted on Original PMAs and Panel Track supplements when there is no turnover.
FDA also presented comparative attrition rates for various Centers in 2010. CDRH attrition was 8.6%; CDER attrition was 4.7%; CBER attrition was 4.8%. Industry sought clarification regarding attrition versus turnover, which FDA discussed. Industry requested attrition rates for ODE and OIVD over the past ten years. In ODE, annual attrition rates ranged from 6% - 10%. In OIVD, annual attrition rates ranged from 6% - 11% except for fiscal year 2006 in which it was only 2%.
FDA highlighted examples of other responsibilities reviewers of Original PMAs and Panel Track Supplements may have to address concurrently. The list includes: Original IDEs, IDE supplements, PMAs, PMA supplements, PMA reports, 510(k)s, PreIDEs and preIDE meetings/teleconferences, developing guidance documents, participating in standards development, 513g submissions, Congressional inquiries, training, consulting reviews, and other (e.g. standards, guidances, etc.). Of note, in some cases lead reviewers have been responsible for multiple Original PMAs at the same time. The significance of this list is the amount of other types of work a reviewer needs to focus on during the course of review of an Original PMA or Panel Track supplement and therefore, the limited amount of time left for the review of the Original PMA or Panel Track supplement. Industry noted that these responsibilities are within the job description of review staff and are therefore not surprising. Industry asked if time devoted to each item on the list is tracked. FDA explained that CDRH tracks activities in their Center Time Reporting System, in which time is tracked per activity for a 2 week period, four times per year. CBER has a similar system. FDA does not currently track 100% of the time; the quarterly 2-week sample is consistent with generally accepted accounting methods. Additionally, the system is not used to monitor performance of an individual reviewer. FDA uses a separate tracking system to monitor the number of submissions an individual reviewer has under review at a given time and whether associated goals are met.
Another factor identified by FDA was timeliness in issuing a Major Deficiency (MAJR) letter. Seventy-three percent of PMAs that missed the Tier 1 goal did not have a first action by day 100 following filing of the PMA. Fifty-five percent of PMAs that met the Tier 1 goal did have a first action by day 100. FDA concluded that generally when a major deficiency letter is sent earlier in the process, the chance of meeting the Tier 1 goal is higher. Industry asked how many PMAs are reviewed without issuance of a MAJR. FDA indicated that there are very few, but that occasionally a PMA is reviewed without issuance of a MAJR.
FDA analyzed the same cohort of PMAs for potential issues with clinical data submitted by Industry. Four submissions contained trial design issues due to the Sponsor’s failure to follow “future concerns” noted during the IDE process (“future concerns” have been identified when FDA decides that a clinical trial can proceed safely, yet FDA has concerns about whether the proposed study will support approval of the PMA, such as when a trial does not appear to be designed to produce sufficient evidence of effectiveness for the intended use).
Industry questioned whether this category of “future concerns” includes cases where FDA agrees at the time of IDE that a protocol would support a marketing application and then asks for new or additional data at the time of PMA review. FDA responded that such cases were not identified in the root cause analysis. Further, this is a practice FDA tries to avoid. FDA asked Industry for specific examples of this concern, which FDA would be willing to analyze.
In the same analysis, FDA found that ten submissions had clinical trial execution issues. Industry asked how this category differs from failure to follow the trial design that was agreed upon in the approved protocol. FDA explained that the latter involves cases where sponsors present only partially completed trials (i.e., data collection is ongoing and will be available later), while trial execution includes cases where the dataset submitted in the PMA lacked a substantial amount of patient follow-up outlined in the protocol (i.e., data collection was missed). FDA described one de-identified example. Industry asked why this specific example PMA was filed given the substantial amount of missing data. FDA explained that there had been a policy shift to file all PMAs and outline all such concerns in a major deficiency letter. In the past year, FDA has returned to the agency’s previous policy of refusing to file such submissions. Industry commented that trial execution issues should no longer arise given this new policy. FDA explained that there are many submissions which present complicated filing decisions. In these cases, FDA tends to file the submission and then attempts to work with the Sponsor to develop an analysis of existing data that could support an approval decision.
FDA also evaluated issues with clinical trial results. In this category, 19 submissions required new analyses and six submissions did not contain data to support the proposed indications. The submissions identified as requiring new analyses were not due to FDA identifying a more rigorous standard to be met than was agreed upon in the IDE protocol nor due to issues of insufficient statistical power, but rather due to prospectively collected data not demonstrating expected outcomes, prospectively collected data not supporting the proposed indications, the Sponsor’s decision to change the proposed indications, and/or the need for sensitivity analyses to evaluate potential differences in trial outcomes between investigational sites. Industry asked if any cases where significant new analyses were needed related to a change in the medical officer or statistician. FDA replied that none of the 19 submissions reported reflect this case, but rather cases where the Sponsor did not follow what they agreed to in the protocol, or the results of the trial led to the need for new analyses (such as an analysis of a subset of the population).
FDA also evaluated the use of the interactive review process in PMAs which missed the Tier 1 goal. Industry asked if FDA evaluated the number of times extensive interactive review resulted in a positive outcome. FDA did not conduct such an analysis as the purpose here was to look at the root cause of missing the Tier 1 goal. Seven submissions that missed the Tier 1 goal were found to have “extensive” use of interactive review. Of these seven submissions, two met the Tier 2 goal, two missed the Tier 2 goal, and three remain pending. Industry asked how FDA differentiates between different types of interactive review. FDA used examples to demonstrate what is meant by “extensive” interactive review. Based on this analysis, FDA concluded that when reviewers utilize the interactive review process more broadly than the program was intended (as outlined in FDA guidance), it may result in excessive iterations of information requests while “on the FDA clock,” which can lead to the PMA missing the Tier 1 goal. Industry indicated that FDA had the ability to send a letter and stop the clock in these cases rather than continue with the interactive review process. FDA explained that it tries to avoid stopping the clock a second time and issuing a second major deficiency letter. In cases where a major deficiency letter has already been issued and concerns remain upon review of the response, FDA most often does utilize the interactive review process to obtain sufficient information to support approval. Industry questioned what is being asked for during interactive review following a major deficiency letter. Specifically, Industry questioned why so many concerns arise later if the major deficiency letter incorporated an exhaustive list of FDA concerns. FDA indicated shared frustration with the need to make multiple requests to obtain the information necessary for a complete review.
Industry expressed concern about the broader implications of FDA’s analysis. FDA clarified that the analysis was focused only on how the interactive review process had contributed to PMA reviews exceeding 180 FDA days within the analysis cohort. Both sides agreed that interactive review is important; it works well in many cases; there is room to improve its use; and that issues associated with its current use relate to both FDA and Industry. Given the interest both sides expressed in improving how interactive review works across the program, this topic will be addressed in future discussions.
To conclude this analysis, FDA indicated that 26 of the 36 PMAs missing the Tier 1 goal had significant clinical trial issues as described above (i.e., trial design, trial execution, trial results). Of these 26 submissions, 14 have had a MDUFA decision and 12 are pending but past day 180. Of the 14 submissions with MDUFA decisions, ten were Not Approvable, one was Withdrawn, and three were Approved/Approvable pending a successful GMP inspection. Industry questioned the historical trends for the rate of Not Approvable letters. FDA noted these data were provided in FDA’s response to Industry’s data request and that the rate is relatively consistent at approximately 20% per year. FDA also indicated they are spending a significant amount of resources on submissions with problematic datasets. In these cases, the reviewer’s top priority is to reach a positive decision, even if that requires missing the Tier 1 goal. From a public health perspective, FDA believes it is appropriate to take the time to do this rather than meet a Tier 1 goal by sending a Not Approvable letter
Another root cause for missing the Tier 1 goal identified by FDA is the use of Advisory Panel review. When a PMA goes to an Advisory Panel for review, the following additional work must be completed often by lead reviewer, branch chief, and/or medical officer: publish Federal Register notice (Designated Federal Official (DFO), lead reviewer), develop panel roster (DFO, lead reviewer, branch chief), write FDA executive summary (lead reviewer, medical officer, statistician, epidemiologist), develop FDA slide presentation (lead reviewer, medical officer, statistician, epidemiologist), review sponsor’s summary and slide presentation (lead reviewer, medical officer, statistician, epidemiologist). In addition, certain processes for identifying and addressing potential conflicts of interest involve other offices at FDA, which extends the time required to confirm the panel roster. As a result of all these required steps, advisory panel review almost always causes a PMA to miss the Tier 1 goal. Industry noted that the Tier 2 goal was established to include all PMAs that require panel review, which is approximately 20%. Industry therefore requested data on whether Advisory Panel review resulted in missing the Tier 2 goal, which they believed to be a more relevant analysis. Of the 13 submissions that went to panel and missed the Tier 1 goal, five also missed the Tier 2 goal, seven met the Tier 2 goal, and one has not yet reached the Tier 2 goal. FDA believes these data point to Advisory Panel review as a contributing factor for missing goals. Industry also questioned whether Advisory Panel review results in a better or worse chance of PMA approval, irrespective of time on the clock.
In conclusion, FDA believes that the following factors contribute to missing the Tier 1 goal for PMAs and Panel-Track PMA Supplements: reviewer turnover (especially changing medical officers), clinical trial execution issues, problematic clinical data analyses, Advisory Panel review, and excessive use of the interactive review process on certain applications.
All agreed to meet from 11:00am – 3:00pm on February 23rd to discuss financial baseline data and from 10:00am – 4:30pm on March 7 th to discuss topics associated with the 510(k) review program.
Meeting End Time: 4:45 pm