RADIOLOGICAL DEVICES PANEL MEETING
December 10, 2002
Gaithersburg Holiday Inn
RADIOLOGICAL DEVICES PANEL MEETING
December 10, 2002
Minesh P. Mehta, M.D. Chair
Harry K. Genant, M.D. Voting Member
Geoffrey S. Ibbott, Ph.D. Voting Member
Emily F. Conant, M.D. Temporary Voting Member
Regina J. Hooley, M.D. Temporary Voting Member
Marilyn R. Peters, M.N., M.P.H. Consumer Representative (non-voting)
Ernest L. Stern Industry Representative (non-voting)
Robert J. Doyle Panel Executive Secretary
Nancy Brogdon Division Director, DRARD
Stanley Stern, Ph.D. Health Physicist, CDRH
John Monahan Biologist, DRARD
William Sacks, Ph.D., M.D. Medical Officer, DRARD
Harry Bushar, Ph.D. Statistician, CDRH
Robert A. Phillips, Ph.D. Branch Chief, DRARD
CALL TO ORDER
Panel Chair Minesh P. Mehta, M.D., called the meeting to order at 8:32 a.m. He noted that the voting members present constituted a quorum and asked the panel members to introduce themselves.
Robert A. Phillips, Ph.D., Branch Chief, Radiology Branch, ODE, CDRH, updated the panel on the devices FDA had approved since the panel’s March 2001 meeting. FDA approved the Sirtex SirSpheres brachytherapy product; the Deus Technologies RapidScreen computer-aided detector (CAD) device for chest radiographs; the Diagnostic Medical Systems bone sonometer; the Fischer Imaging SenoScan (a full-field digital mammography [FFDM] device); the Hologic Lorad Digital Breast Imager FFDM device; the CADx Medical Systems SecondLook CAD system for mammograms; and the Intelligent Systems Software [now iCAD] MammoReader CAD system for mammograms. In addition, FDA approved PMA supplements for soft-copy imaging for digital mammography systems.
Panel Executive Secretary Robert J. Doyle read the appointment to temporary voting status for panel consultants Emily F. Conant, M.D., and Regina J. Hooley, M.D. He then read the conflict of interest statement: Full waivers had been granted to Regina J. Hooley, M.D., Geoffrey S. Ibbott, Ph.D., and Prabhakar Tripuraneni, M.D., for their interests in firms that could be affected by the panel’s recommendations. Mr. Doyle noted that the Agency had taken into consideration other matters concerning Drs. Hooley, Ibbott, and Tripuraneni, all of whom reported interests in firms at issue that were not related to the day’s agenda. Finally, Mr. Doyle noted that FDA encourages comments from the public concerning guidance documents.
Stanley Stern, Ph.D., Health Physicist, CDRH, presented information on the development of amendments to the U.S. radiation safety standard for diagnostic x-ray computed tomography (CT). [He noted that proposed amendments to the Performance Standards for Ionizing Radiation Emitting Products were published in the Federal Register on December 10, 2002.] Current regulations covering CT have not kept pace with technological developments. Public health concerns include the total CT dose to the population as well as high radiation doses associated with CT exams of children and small adults and asymptomatic self-referrals for CT screening (whose radiation doses are among the highest of all modalities). In addition, CT fluoroscopy, which visually guides interventional procedures and has applications in a broad range of specialties, carries the additional risk of radiation-induced skin injury.
Dr. Stern described the current standards for CT equipment radiation safety performance, which are 20 years old and predate electron beam and multislice devices. The current regulations do not fully address these modalities. Consequently, FDA is considering amendments that involve dose-index standardization, automatic exposure control, and field-size limitations. Dose-index standardization would require each new CT system to display and/or record values of standardized dose indices for each patient’s exam. This requirement could reduce patient CT dose by an average of about 15 percent. Automatic exposure control would automatically reduce the dose as the x-ray beam scanned a thinner part of the anatomy; this requirement would reduce patient CT dose by about 30 percent.
FDA is concerned that a number of multislice models use radiation inefficiently. Although the amount of radiation applied to construct one image or a set of images is the same for each configuration, the radiation distribution is much wider than for single-slice system, and radiation falls beyond the detectors and is not used for image construction. FDA is therefore considering regulations to limit the x-ray field to a size no larger than that needed to construct multislice images. This requirement could reduce the patient CT dose by about 30 percent. If all CT equipment were to include the proposed features, 8,700 radiation-induced cancer mortalities per year could be avoided; the projected collective dose savings are 193,000 person-Sieverts per year. FDA has developed a framework for analysis that will lead to a concept paper. The Agency expects industry professional groups and States to contribute to the regulatory development process and expects to update the panel in June 2003.
No comments were made.
Panel Chair Minesh P. Mehta, M.D., stated that the purpose of the meeting is to discuss, make recommendations on, and vote on a PMA (P010035) for a device that produces a computerized thermal image of the breast of women recommended for biopsy.
John Brenna, president and CEO, Computerized Thermal Imaging (CTI), described the CTI BCS2100 device as a noninvasive adjunct device designed to work with mammography to eliminate the need for biopsy of benign masses. The device uses infrared (IR) technology to provide physiological information that supplements the anatomical information provided by mammography. The device is painless and noninvasive. CTI plans to market the device exclusively to MQSA-certified facilities under the control of board-certified radiologists.
Yuri Parisky, M.D., professor of radiology, University of Southern California School of Medicine and principal investigator at USC/Norris and USC/LAC, provided background information on the development of breast imaging devices. The introduction of diagnostic mammography did not add significant specificity or sensitivity and did not reduce biopsy rates. Ultrasound led to a drop in biopsies. Other modalities have been introduced; all have high sensitivity but low specificity. Every year, approximately 1.3 million women have biopsies, of which 80 percent are benign. Breast biopsy procedures cause discomfort, carry risks of complications, and create physical and emotional stress. The sponsor’s device has good sensitivity and specificity that can permit the avoidance of some biopsies.
Kevin Hughes, M.D., FACS, surgical director of breast screening, Massachusetts General Hospital/Harvard Medical School, and principal investigator, Lahey Clinic, Boston, provided a clinical perspective on how the CTI device will be used. Patients who are identified through mammography as having suspicious growths will first go to IR imaging rather than biopsy. If the IR image is negative (i.e., indicates that the growth is likely to be benign), the patient can go to follow up instead of biopsy; however, the patient will still go to biopsy if other clinical information warrants the procedure. If the IR image is positive the patient goes on to biopsy. The IR image is not visually interpreted; the computer determines whether an image represents a suspicious lesion in a region chosen by the radiologist.
Lynn Satterthwaite, vice president of engineering, CTI, provided details on the mechanism of the CTI device. The body emits IR radiation as a result of heat produced by physiologic processes. The device passively senses IR radiation and records it as temperature. Mr. Satterthwaite described the data acquisition system, including the camera, which detects differences in temperature of 0.1°C between any two pixels. Six mirrors ensure that the entire breast is imaged; the mirrors do not contact the breast, therefore, they do not interfere with temperature measurements. Thirty seconds into the imaging process, the computer turns on a cooling system that continues through the remainder of the 3-minute imaging session. The cooling challenge, which is computer controlled, elicits a physiological response and enhances the temperature contrast between benign and malignant tissue. Once the IR data are processed, two composite images are generated.
The second part of the process is evaluation of the data. The technologist outlines the margins of the breast on the composite image; the physician confirms, or changes the outline if necessary. The physician then determines a region of interest (ROI) and marks it on the image. The device analyzes the ROI as well as an expanded area around it comprising about 1/12 of the total breast area; it then scans 8.3 million temperature data points per breast. An index of suspicion (IOS) calculation is then generated; if the IOS is below the threshold of 20.59, it is considered a negative result, and the patient is referred to short-term follow up rather than biopsy. In summary, the BCS 2100 performs computerized analysis to differentiate between malignant and benign tissue. The device comprises noninvasive, safe components and is designed to be an adjunctive medical device.
Karleen Callahan, Ph.D., director of clinical research, CTI, described the clinical study. The objective was to determine whether the CTI BCS 2100, when used in conjunction with mammography, increases the ability of physicians to differentiate benign from malignant breast abnormalities. The hypothesis was that the device could differentiate benign from malignant breast lesions on the basis of the relatively lower strength of the IR signal in benign tissue, thereby reducing the number of benign biopsies.
The study was a blinded investigation that began at one site with 600 subjects and was later expanded to five sites in order to obtain enough malignancies for analysis purposes. Effectiveness was determined on the basis of area-under-the-curve (AUC) analyses as well as sensitivity, specificity, and subpopulation analyses. Subjects were identified and enrolled, had IR imaging, and then proceeded to biopsy. The pathology was sent to an independent research organization. After a panel of independent evaluators completed the trial read of the IR image (i.e., determined the ROI), the data were unblinded and sent to CTI for analysis along with the mammogram data and patient information on case report forms.
The evaluator panel consisted of seven currently practicing mammographers; each image was randomly assigned to three evaluators, who were blinded to all lesion pathology results. The clinical research organization Quintiles monitored the investigative sites, held the blinded pathology results, and was provided with a locked database of IR testing outcomes prior to unblinding the pathology data.
Included subjects were those with lesions recommended for surgical or core biopsy on the basis of mammography or clinical findings. Exclusion criteria consisted of previous breast surgery, radiation exposure in the breast of interest, breast implants or breast reduction, weight of greater than 300 lbs, pregnancy, and previous cancer in the breast of interest. Two mild adverse events were reported that were related to the device, both of which involved patient discomfort from lying on the bed. Two other adverse events were found not to be related to the device.
The sponsor amended the protocol several times over the course of the study. In November 1998, the evaluation method of the mammographic and IR image was changed. In June 1999, the sponsor reduced the breast surgery exclusion period to 1 year. Amendment 5 involved unvaulting of additional data (69 subjects with 78 lesions) that had been collected under the same protocol but not yet analyzed. This was done in response to deficiencies noted by FDA.
Steve Rust, Ph.D., senior research leader, Battelle Statistics and Data Analysis Systems, provided details on the sponsor’s statistical analysis, focusing on the confirmatory study. The study involved 275 subjects who were in the pipeline at the time the dataset was “frozen” and used the same evaluation process as the initial study. A target population of masses was selected after the pathology for the original study subjects was unblinded; the target was based on results from prospectively planned subset analyses by lesion type. Subset analysis could have resulted in seven possible target populations: masses; calcifications; distortions; masses and calcifications; masses and distortions; calcifications and distortions; and masses, calcifications, and distortions.
Dr. Rust stated that two options for reaching valid statistical conclusions for the mass subset were possible: (1) Perform a new study and draw conclusions from only the new study data, or (2) apply a correction that validates the statistical conclusions drawn from the original study data. Either approach is correct. CTI thus conducted the confirmatory study using a subset of 78 masses, added the data to that of the original study population, and applied a Bonferroni correction to the results for the combined data. He stated that the conclusions are valid for any of the seven target populations.
Dr. Parisky summarized the sensitivity and specificity data. The 769 original subjects had a total of 875 lesions, 688 of which were benign; of those, 96 were correctly assigned a negative IR result (resulting in 14 percent specificity). Of 187 malignant lesions, 180 lesions were correctly assigned a positive IR result, and 7 were falsely assigned a negative IR result. Of the seven false negatives, all were microcalcifications. Pathology results found four ductal carcinomas in situ (DCIS), two DCIS with focal microinvasion; and 1 intraductal and infiltrating ductal carcinoma were found. No invasive malignancies were described as false negatives. The confirmatory study of 78 masses included 63 benign masses; of those, 16 were correctly assigned a negative IR result, resulting in 25 percent specificity. Of 15 malignant masses, 14 were correctly assigned a positive IR result, and 1 was assigned a false negative IR result, resulting in 93 percent sensitivity. The combined data (confirmatory study plus the masses from the original study; N = 490) resulted in 19 percent specificity and 99 percent sensitivity.
Dr. Rust then provided data on health care costs and benefit. He estimated that the device could save $94 million to $183 million annually and prevent 89,329 biopsies.
Finally, Mr. Satterthwaite read the proposed indication for use and described aspects of the device’s regulatory history. He stated that FDA approved combining the confirmatory study results with those from the original study and that the original study protocol was developed with advice from FDA. Results of the clinical study found 99 percent sensitivity and 19 percent specificity; the device is therefore safe and effective.
Panel members asked many questions about the study methodology as well as the device. Their methodological concerns included the lack of ultrasound data, the definition of mass, variability among readers and the reproducibility of results, the rationale for stopping enrollment before the target of 3,000 patients was reached, the statistical validity of pooling reader results, and the exclusion criteria. Sponsor representatives provided clarification in response to the panel’s questions. Device-related concerns included calibration of the device and procedures for quality assurance. Mr. Satterthwaite noted that the device is calibrated before each imaging session. Finally, panel members observed that the cost–benefit data seemed a bit high, but Dr. Rust said that the numbers were based on surgical as well as needle biopsy costs.
John Monahan, biologist and lead FDA reviewer, noted that the PMA was a modular submission. The first module, which contained preliminary information, was received in 1999. Modules 2 through 4 provided data on the device’s software, manufacturing, and engineering.
William Sacks, Ph.D., M.D., medical officer, CDRH, described the sponsor’s clinical study. He noted that the device was a new type of thermographic device intended for use as an adjunct to mammography. It renders a value on the basis of an IOS score. In its intended use the device can only decrease the number of biopsies (i.e., increase specificity); it cannot increase detection of cancers (i.e., increase sensitivity).
Harry Bushar, Ph.D., statistician, CDRH, reviewed the clinical study protocol and the sponsor’s statistical analysis. He pointed out that diagnostic mammography, not just clinical examination, is required for use of the device. The sponsor’s AUC analysis loses statistical significance when mammography level of suspicion (LOS), in terms of the American College of Radiology Breast Imaging Reporting and Data System (BIRADS) score, is expanded by just two additional intermediate categories, after excluding calcifications alone. Further, the sponsor’s initial rejection of ROC, followed by rejection of calcification-alone sensitivity, indicates exploration (of the database). The sponsor's attempted Bonferroni adjustment to correct for this exploration, by widening sensitivity and specificity confidence interval estimates, is not statistically acceptable. Therefore, the submission requires the results of a new study.
Dr. Sacks followed and stated that safety has two aspects: adverse events and accuracy of output. Only four minor adverse events were reported. Safety is more closely related to sensitivity (i.e., false negatives though also false positives); effectiveness is more closely related to specificity. The sponsor provided four clinical submissions: the PMA and three amendments (Amendments 4, 5, and 7) which were provided in response to FDA deficiency letters. The FDA reviewed each submission and raised numerous questions concerning the adequacy of the data. He noted that, based on the data, if the device were used on all 585,000 women with mammographic masses each year, 5 to 7 percent of 1.3 million biopsies would be obviated; however, 1 to 6 percent of the malignant masses and 0.5 to 3 percent of all breast cancers might be delayed in diagnosis.
Nancy Brogdon, Director of DRARD, noted that Marilyn R. Peters, M.N., M.P.H., consumer representative, and Alicia Y. Toledano, Sc.D., are completing their terms on the panel at the end of January, 2003. She thanked them for their service.
Panel members were concerned about the study methodology, including disagreement among readers, handling of unreadable images, missing readings, and exclusion criteria. Sponsor representatives explained that the study attempted to get three reader evaluations for each lesion. In those cases, all three results were used in the analysis. If two evaluations were obtained, they each were weighted 50 percent, and the analysis statistically accounted for the missing reading. Any lesions that did not show up on a mammogram could not be used. Panel members noted that women who did not have biopsies were excluded, as were those who had a biopsy that found fluid-filled cysts. The exclusion criteria may have opened up the results of the trial to significant bias. They also noted the lack of data on inter- and intrareader reliability, effects of breast size, and the rationale supporting the cooling challenge.
Panel members expressed concern about the primary efficacy findings and the robustness of the data. The primary efficacy endpoints did not reach significance or demonstrate a trend. They only reached significance in post hoc analysis. It is important to know the reproducibility from one reader to the next. Is a trained panel of readers necessary, or can a single reader assess the image? Dr. Rust noted that the sponsor allowed itself to be statistically penalized for reader-to-reader variability. Dr. Genant responded that because of the way the sponsor handled the issue, it is not clear that three trained readers are any more reliable than a single reader.
Many panel members noted that the sponsor did not collect ultrasound data, even though in clinical practice, almost all masses will undergo ultrasound to help determine whether biopsy is needed. It is unclear whether the device will be useful in masses with BIRADS scores of 3 or 4 on ultrasound as opposed to BIRADS 5. Also, many lesions with BIRADS scores of 0, 1, 2, or 3 were included in the biopsy group. Dr. Parisky said that in those cases, the patient wanted biopsy, but Dr. Conant noted that even so, the numbers were still high.
The panel expressed doubts about the feasibility of using the device in clinical practice. They repeatedly raised concerns about patient flow and suggested that the ability to demonstrate clinical usefulness was negated by the use of the readers in the clinical study. Dr. Hughes stated that he uses the device successfully in his clinic. Mr. Satterthwaite noted that the device has been improved; the cooling challenge is now automated, for example.
Dr. Toledano asked whether the sponsor could reanalyze the existing data in an exploratory manner, as though masses had been separated by type from the start. Dr. Callahan said that it was possible; the data are in case report form. No data are available on why any biopsy was not performed, though.
Panel members asked for clarification on how the sponsor determined the threshold for positive IR findings (i.e., the IOS score). Dr. Parisky said that the threshold was determined prior to unblinding in the original study, so the sponsor kept that threshold. The objective was to try to achieve 99.3 percent sensitivity with 75 percent confidence. It was never intended to be a hypothesis for performance evaluation.
1(a) The data in Amendment 4 were selected retrospectively from the original PMA dataset, albeit based on lesion type analyses that were prospectively planned for in the clinical trial protocol. Are the data from Amendment 4 applicable for the assessment and determination of effectiveness of the BCS 2100?
The panel agreed that the data are not valid in and of themselves for documenting effectiveness. They are encouraging, but they represent a post hoc analysis. The sponsor prospectively planned to analyze performance by lesion type but had no plan to exclude certain lesion types from the indications. The current indications are based on masses. The data and analysis for masses cannot come from the data indicating that the sponsor should limit the indications to masses.
1(b) The additional data in Amendment 5 consist of 78 masses. Are these additional data by themselves sufficient for the assessment and determination of effectiveness of the BCS 2100?
The panel concurred that the additional data are insufficient; they do not include enough patients.
1(c) When combined, Amendment 4 provides 84 percent (412) of the masses and Amendment 5 provides 16 percent (78). What is the validity of combining these data to assess and determine effectiveness of the BCS 2100?
The panel concurred that combining the data is not a valid approach to analysis.
2. Please discuss the same questions for safety.
2(a) The data in Amendment 4 were selected retrospectively from the original PMA dataset, albeit based on lesion type analyses that were prospectively planned for in the clinical trial protocol. Are the data from Amendment 4 applicable for the assessment and determination of safety of the BCS 2100?
The panel concurred that use of the device is physically safe, although the possibility of false negatives raises a different type of safety concern. The panel felt that it sufficiently addressed questions 2(b) and (c) in its answers to the previous questions. The data and the analysis methods are not sufficient to determine safety and effectiveness.
3. Please discuss whether safety and effectiveness has been established. As part of this, please discuss the risk/benefit trade-off whereby a false negative results in a 6-month delay of cancer diagnosis and a true negative obviates biopsies that would otherwise turn out benign.
The panel concurred that the risk/benefit tradeoff was high. A false negative could result in a 6-month delay in cancer diagnosis, but not enough is known about the natural history of DCIS to establish whether 6 months is the appropriate follow up interval. Patients with false positives are subjected to additional tests, emotional turmoil, and expense. For DCIS, delayed diagnosis might not be of too much concern, but it would be a problem if the cancer were infiltrating.
4. Is the proposed labeling adequate to ensure safe and effective use of the BCS 2100? Please include in your discussion the following specific items (a) and (b):
4(a) Given that only 2 of 105 cancers were smaller than 5 mm, should the labeling specify a lower size limit for an eligible mammographic mass? If so, what size limit?
The panel concurred that the labeling should specify a lower size limit, but the data are insufficient to determine that limit.
4(b) Should the labeling address lesion depth? If so, in what way?
The panel agreed that data were insufficient to answer the question.
5. Should the labeling be revised to address any potential psychological impact of a positive mammogram, followed by a positive BCS 2100 result, on a woman who does not, in fact, have cancer (i.e., false positive)?
The panel said that clinicians do not always take time to reassure patients that a positive finding does not mean that the patient has cancer, but it is unclear how labeling could help that situation.
6(a) Do the above, or any other, issues require resolution before approval of the PMA?
The panel felt that several important issues need to be addressed before approval. The existing data may not be able to resolve those issues, which include reproducibility of results and the stability of the instrument. Equilibration is a concern because not every clinic will maintain the proper temperature for a successful cooling challenge. Establishment of the IOS threshold is also an issue—several panel members felt that it should be determined based on masses, not a wide variety of lesions. Effectiveness needs to be established in a target population that is directly relevant to the proposed indications for use, taking into account the clinical flow of patients. Several panel members stated that it might be possible to resolve the issues without additional clinical trial data.
6(b) Do the above, or any other, issues suggest the need for a post-market study?
The panel concurred that if the device were approved, then a postmarket clinical trial would be needed.
No comments were made.
Dr. Parisky reminded the panel that the ideal threshold for false negatives cannot be met by any existing modalities, including PET scan, MRI, and ultrasound has high rate. The panel has set a standard that will only allow existing technologies to enter the marketplace. Mr. Brenna noted that the sponsor worked closely with FDA during the clinical study. He said that the product is noninvasive, adjunctive, and safe and shows high promise with sensitivity.
Mr. Doyle read the voting instructions. A motion was made to recommend that the PMA was “Approvable with Conditions.” This motion failed on a 3-4 vote. Another motion was then made to recommend “Not Approvable.” The panel voted in favor of this motion on a 4-3 vote with the panel chair casting a tiebreaking vote in each case. When asked to state the reasons for their votes, panel members voting for “Not Approvable” stated that although the device is safe, its clinical effectiveness was not demonstrated by the study. Members expressed concerns about quality assurance issues and the sponsor’s clinical trial design. They felt that a prospective study that better defined masses and included ultrasound data would be helpful. They also felt that the relevant population of interest should be women with masses seen on mammography or found by clinical exam. In addition, the sponsor’s study did not provide data on inter- and intrareader variability.
Panel members voting against “Not Approvable” stated that the device is noninvasive and has utility. They felt that as the data are fine-tuned and the company gets more experience, use of the device would be more compelling. They also felt that the device could help spare tens of thousands of women the psychological trauma of biopsy. In addition, postmarket research could provide the information many panel members felt was lacking in the sponsor’s data.
Dr. Mehta thanked the participants and adjourned the open session at 3:35 p.m.
I certify that I attended this open session of the Radiological Devices Advisory Panel on December 10, 2002, and that these minutes accurately reflect what transpired.
Robert J. Doyle
I approve the minutes of the December 10, 2002,
open session as recorded in this summary.
Minesh P. Mehta, M.D.
Summary prepared by
Caroline G. Polk
Polk Editorial Services
1112 Lamont St., NW
Washington, DC 20010
Edited by FDA