NATIONAL MAMMOGRAPHY QUALITY ASSURANCE
September 28, 2000
Gaithersburg Holiday Inn
2 Montgomery Village Avenue
Gaithersburg, Maryland 20877
National Mammography Quality Assurance Advisory Committee
September 28, 2000
Barbara Monsees, M.D.
Charles Finder, M.D.
Kambiz Dowlat, M.D.
Nancy J. Ellingson, R.T.
Patricia Hawkins, M.P.H.
Debra M. Ikeda, M.D.
Amy F. Lee, M.D.
Ellen B. Mendelson, M.D.
Michael H. Mobley, M.P.A.
Robert Nishikawa, Ph.D.
Robert J. Pizzutiello, Jr., M.S.E.E.
Donald C. Young. M.D.
SEPTEMBER 28, 2000: OPEN COMMITTEE DISCUSSION
Committee Chair Barbara Monsees, M.D., called the meeting to order at 9:04 a.m. Executive Secretary Charles Finder, M.D., welcomed participants and read the conflict of interest statement, announcing full waivers for 12 of the members because of their professional affiliations or financial involvement with organizations that could be affected by the committee’s deliberations. In discussions of equipment standards, committee members Kambiz Dowlat, M.D.; Robert Nishikawa, Ph.D.; Debra Ikeda, M.D.; and Robert Pizzutiello, Jr., were limited to issues of mammography technology and to descriptions of their observations and experience with mammography products; they also had to refrain from voting on specific equipment standards. Mr. Pizzutiello had to refrain from discussions involving 2002 criteria in evaluation of personnel competency. Dr. Finder noted that some committee members had received compensation for lectures in their areas of specialization, but only general fees for professional expertise.
APPROVAL OF ALTERNATIVE STANDARDS REQUEST
Dr. Finder stated that the FDA had not approved any alternative standards since the committee’s January meeting.
OPEN PUBLIC HEARING
Judy Destouet, M.D., a radiologist in private practice in Baltimore, MD who is assuming the leadership of the American College of Radiology (ACR) Mammography Accreditation Program, addressed the committee concerning personnel competency under the MQSA. She noted that high-quality mammography depends on several interrelated factors, including equipment, quality assurance processes, and adequacy of procedures followed in conducting or interpreting mammograms. A person may perform poorly at one facility and well at another site for reasons other than individual competency. Dr. Destouet also expressed concern with the use of medical outcome audit data to evaluate facilities, noting that the data are unreliable at low numbers and that patient demographics can skew statistics. She pointed out that audit data must remain confidential; otherwise, facilities and radiologists might become unwilling to accept challenging cases. In addition, no benchmarks exist for audit data, making them unhelpful as an evaluation tool. Finally, these problems, along with increased costs associated with competency requirements, could ultimately limit women’s access to mammography.
Dr. Finder then read into the record a letter from committee member Peter Dempsey, M.D. Dr. Dempsey wrote that the American Board of Radiology (ABR) was best equipped to handle physician accreditation and that the FDA should not become a certifying body. He suggested convening a meeting of the FDA, the ABR, and the American College of Radiology (ACR) to avoid duplication of effort in achieving the goal of fair, objective measurement of physicians’ skill in mammography.
Dr. Finder then read a letter from Carl D’Orsi, M.D., F.A.C.R. Dr. D’Orsi pointed out that many safeguards already address competency in interpreting mammograms, including board certification requirements and FDA regulations governing physicians and equipment. He noted the difficulty of developing competency tests and problems with relying on medical audit data. Dr. D’Orsi urged the FDA to require that States show a connection between interpretive improvement and additional State regulations.
FDA OVERSIGHT OF MQSA INSPECTORS AND INSPECTIONS
Angela Clingerman, Inspection Support Branch, Division of Mammography Quality and Radiation Programs, FDA, presented an overview of the MQSA Inspector Program. She described the inspector certification requirements, the goals and operation of the MQSA Inspector Quality Assurance (QA) Program, and the mechanisms for providing the FDA with feedback on inspector performance. She noted that the primary goal of the QA program is to provide support for inspectors, which is accomplished through the MQSA Inspector Helpdesk, the Policy Guidance Help System, the
mammography Web site, e-mail, and MQSA Auditors/Mentors.
OPEN COMMITTEE DISCUSSION
Dr. Monsees opened the floor to discussion. Nancy Ellingson, R.T., noted a problem with some inspectors not accepting certain approved continuing education credits for technologists. Dr. Monsees asked whether the facilities understood what their recourse was in those circumstances and added that she would like to see a feedback process following every inspection. Michael Mobley, M.P.A., asked whether the FDA maintains statistics on inspector activity and what the data show. In response, Helen Barr, M.D., deputy director of the Division of Mammography Quality and Radiation Programs, noted that the FDA had previously conducted a survey of facility satisfaction and had just hired a contractor to do the survey again. Ms. Clingerman said that her division was working on a spreadsheet describing inspector findings by region and that it should be finished by January 2001.
The committee discussed the benefits of having facilities complete surveys following inspections, expressing general agreement that a survey was a good idea. Members noted that surveys should be anonymous and that inspectors should receive their individual statistics along with a range within which they should fall. Those ranges should be developed carefully because of regional variation in results. Dr. Monsees asked Dr. Finder whether the FDA would be able to give such feedback to States and individual inspectors; Dr. Finder said that it would depend on what data were received. He said that all complaints about individual inspectors are investigated and the results given to the inspectors and the complaining facility. Several members expressed concern that facilities were unaware of complaint procedures.
Mr. Pizzutiello expressed concern about Level 1 violations for problems with scoring phantom images; inspectors are not being trained by medical physicists (MPs), and the inspection program could use improvement in that area. In response to Dr. Monsees’ request for suggestions on how to remedy the problem, Mr. Pizzutiello suggested senior expert review of Level 1 phantom image failures, although he acknowledged that doing so would require someone at the FDA level with expert, in-depth training. Mr. Mobley said that such a program would need to be seen as a support activity for inspectors. He said that in Tennessee, inspectors are notified of the results for the previous quarter, including rates of noncompliance. He pointed out the need for understanding the variation in inspection findings and the need for a system within FDA to evaluate and provide the data to inspectors. Dr. Monsees said that it sounded like Mr. Mobley was describing a process similar to the benchmarking done with ABR examiners. She suggested that the percentage of citations overturned on appeal was an important component of benchmarking that should be tracked.
Kathy Franke, chief of the FDA’s Inspection Support Branch, noted that the division is developing an electronic means of capturing the information that the FDA collects on each inspector’s performance. Once the system is developed, the agency will be able to produce spreadsheets and electronic means of communicating with States and inspectors. She described her division’s efforts to ensure high-quality inspectors and added that the division offers continuing education on phantom image scoring skills. Her colleague, Stephanie Belella, training coordinator, noted that all staff who have taught the phantom image scoring lecture have attended the ACR’s course in phantom image scoring.
Walid Mourad, M.D., FDA Inspection Support Branch, added that inspectors score two phantom images, not one. If a Level 1 citation for phantom image scoring is issued, inspectors are told to not finalize it right away; most States review the score with several inspectors. Mr. Pizzutiello reiterated his concern over the expertise of the people doing the end-of-line review. Patricia Hawkins, M.P.H., urged that the FDA proceed with caution in this matter to avoid an environment in which the industry intimidates the inspection process.
REVIEW OF SUMMARY MINUTES
The summary minutes of the January 2000 meeting were reviewed. No comments or changes were made. The minutes were unanimously approved. Dr. Finder noted that future meetings would likely shift to a spring/fall schedule starting in spring 2001 to avoid difficulties with inclement weather. He then presented awards for service to outgoing committee members Monsees, Mendelson, Hawkins, Mobley, and Pizzutiello.
DISCUSSION OF PROPOSED MQSA GUIDANCE UNDER THE FINAL REGULATIONS
NMQAAC Discussion Questions and Answers, September 28, 2000
Dr. Finder gave a brief history of the proposed MQSA guidance and described the notice-and-comment process. He emphasized that the panel was to discuss the guidance, not the regulations, and explained the differences between regulatory and guidance language.
Certification. Dr. Monsees noted that page 1, lines 20–26, of the document needed editing and that the text should specifically indicate that a lead interpreting or auditing physician is required. Lines 28–35 on the same page also needed editing.
Personnel–General. Dr. Monsees stated that the reference to "students" on page 2, line 19, was incorrect and that the sentence needed editing.
Radiologic Technologist. Ms. Ellingson requested clarification of the (M) designation in connection with the ARRT certificate (page 3, line 21). She pointed out that the ARRT has changed its certification requirements to include the performance of 100 exams. Dr. Finder didn’t think that this required modification of the answer but would look into whether additional clarification was necessary.
Medical Physicist. The committee members had no comments on this section.
Equipment. Concerning page 5, line 35, Mr. Pizzutiello said that mammography units meeting the requirements for 2–6 cm measurements rarely also meet the requirements for measurements larger than 6 cm. He urged that the language extending the requirements beyond the 6 cm range be eliminated. Dr. Finder noted that the guidance states that if a unit cannot meet the requirements beyond the 6 cm range, a technique chart should be developed. Dr. Nishikawa said that he thought the sentence was fine as is, and the committee concurred.
Dr. Nishikawa noted that the language on page 6, lines 1–3, needed to clarify under what circumstances the AEC test was being performed. Dr. Finder agreed to look at revising the paragraph to try to make it clearer.
Medical Records. On page 8, line 40, Dr. Nishikawa suggested adding "or soft copy, if requested." Dr. Finder stated that such language is not included because at the current time, FDA has not approved soft copy interpretation for mammography.
Quality Control (QC) Tests–General. Concerning the table of required QC tests appearing on page 10, Mr. Pizzutiello suggested adding "in contact mode" to the end of the phrase in column 3, row 5 ("Dose"). Dr. Monsees concurred. Concerning the category "darkroom fog," Dr. Nishikawa stated that he did not see the point of testing every type of film. Mr. Pizzutiello stated that different films may have different sensitivities and that it would be important to test them all. Dr. Nishikawa was satisfied with Mr. Pizzutiello’s response and added that under kVp and thickness tracking, each screen-film (S-F) combination should be tested because the results could vary with different thicknesses. Mr. Pizzutiello agreed, and there was no objection from other committee members. The committee discussed the purpose of testing automatic exposure control (AEC); Dr. Mourad indicated that he saw no need to test for different S-F combinations in checking AEC performance, and the committee had no further comments.
Concerning page 13, line 18, Dr. Monsees asked whether it made sense to focus on average exposure as opposed to maximum exposure time. The committee discussed the wording; Mr. Mobley said that he read the guidance as meaning the time it takes for the technologist to set up the equipment and initiate exposure. Dr. Monsees suggested changing the wording to "up to the time it takes to fully expose the patient, up to the maximum exposure time." The committee concurred.
Mr. Pizzutiello referred the committee to page 14, line 31, and asked why equipment evaluation and annual physics surveys had different requirements. Dr. Finder said that in the regulations, equipment evaluation includes the same tests as the physics survey as well as other tests. Mr. Pizzutiello noted the distinction between good professional practice and regulatory requirements—he was concerned that the regulations were too weak as written and would lead to equipment being tested once during installation and never again. Dr. Finder noted that the issue had been raised during the regulatory process and that the committee could not require something in the guidance that is not required in the regulations. Dr. Monsees suggested that the guidance could indicate that during the annual physics survey, testing of all equipment configurations is not required but is recommended, and the committee concurred.
Medical Physicist’s Annual Survey. On page 16, lines 39–40, Mr. Pizzutiello suggested deleting "that could significantly increase patient dose" because the term "significantly" is ambiguous. Mr. Pizzutiello stressed that it was important to check the dose if clinical technique factors change. On lines 38–39 on the same page, Dr. Monsees suggested inserting "routine" before the word "change." The committee concurred.
Referring to the table on page 19, the rows labeled "Installation" and "Reassembly," Dr. Monsees suggested changing column three to state that the dose should be measured. Dr. Finder said that if the change were made, it would have to be included for other issues and that the table would get too big.
Mr. Pizzutiello said that he supported the concept of medical physicist (MP) oversight of equipment adjustments and asked whether a facility, pursuant to consultation with an MP, was required to do what the MP recommends (page 18, line 1). He suggested that the guidance include language saying that FDA strongly recommends that facilities follow the recommendations of MPs. Dr. Finder suggested that the wording could be changed but stressed that if an MP recommends an action not required by the regulations, the facility is not required to take that action.
Referring to page 18, lines 18–20, Mr. Pizzutiello asked whether a service engineer could verify his or her own repair. Dr. Finder said that verification did not have to be performed by a different person and that the guidance could be changed to clarify that fact. The committee discussed verification testing and the role of MPs and agreed that clarification of what constitutes a valid verification test was needed in the guidance. Ms. Ellison asked whether documentation of consultation with MPs was required. Dr. Finder responded that it was needed only if the regulations required the consultation.
Referring to "Film type change" in the table on page 19, Mr. Pizzutiello suggested changing "MP involvement optional" to "MP oversight." He noted that the MP should be involved whenever an adjustment could cause a change in dose and suggested that the categories "Chemistry type change leading to establishment of new operating levels" and "Replenishment adjustment leading to establishment of new operating levels" also should require MP oversight. In addition, the category "high voltage generator adjustment" was not significantly different from kVp internal adjustment and should be deleted. The committee concurred.
Mr. Mobley added that in-person MP evaluation might be appropriate for the category of "collimator adjustment" because of difficulties he had seen with service personnel being able to do it adequately. Mr. Pizzutiello disagreed, partly because collimator maladjustment is fairly easy to detect and partly because the patient suffers no consequences if the adjustment is off a little. The committee agreed that MP oversight was appropriate for collimator adjustment.
Referring to page 19, lines 5–6, Donald Young, M.D., asked whether a method of cleaning the equipment needed to be specified. Dr. Monsees said that the issue had been raised numerous times with the committee and that it was usually agreed that the facility should clean the equipment in accordance with the manufacturer’s recommendations. Dr. Finder said that the guidance should be sufficient because it asked facilities to establish standard operating procedures for disinfecting after contact with potentially infectious materials.
Mammography Medical Outcomes Audit. Mr. Mobley suggested that audit data could be useful in the evaluation process and acknowledged that it was a sensitive issue. Dr. Monsees noted that the issue would be covered in the afternoon session of the committee in the discussions of personnel competency.
Dr. Monsees asked whether the ACR’s QC manual had a table that was consistent with what was in the guidance document. Priscilla Butler, M.S. of the ACR said that the ACR table summarizes MQSA evaluation requirements and is a good supplement to the information in the manual. Dr. Monsees expressed concern that a disparity could exist between that manual and the table in the guidance document; Ms. Butler indicated that she did not believe that to be the case.
MQSA Final Regulations Modifications to Policy Guidance Help System #1
Dr. Finder gave a brief history of the policy guidance help system and described how the document before the committee would be used.
Referring to page 17, paragraph 1, Mr. Pizzutiello suggested inserting the phrase "due to complexities associated with reestablishing operating levels, MP oversight should accompany changes in operating levels." Dr. Monsees agreed that because the word "oversight" was used in the table in the guidance document, it should be used in the help system text.
Referring to the answer to question 11 on page 10, Dr. Nishikawa asked for clarification of the phrase "some credits." Dr. Finder said that no specific number of credits is required for MPs in the regulations. Dr. Nishikawa urged that the regulations be changed at some point to incorporate changes occurring as a result of digital technology. Kish Chakrabarti, Ph.D., of the FDA’s Accreditation and Certification Branch, offered further clarification on the language in the regulations.
Referring to pages 16 and 17, Mr. Mobley asked why the FDA had to wait for verification from state accreditation bodies before terminating certification. Dr. Finder explained that it had to do with the way databases were linked. Referring to page 17, Mr. Mobley asked whether there was any way to make the phantom image recommendations stronger. Dr. Finder said that he would look into it.
FDA’S ROLE IN EVALUATING PERSONNEL COMPETENCY
Dr. Finder presented background information on the FDA’s role in evaluating personnel competency and described the difficulties with using medical audit data as an evaluation tool. He gave two examples of problem situations and asked for the committee’s input on whether it is appropriate for FDA or the States to implement specific actions regarding personnel competency outside of the current facility-based program. Dr. Monsees reminded the committee of the letters from Drs. Dempsey and D’Orsi, which had been read into the record earlier.
The committee discussed the limitations of audit data, the need to keep audit data confidential, and the rarity of the problems that Dr. Finder described in his two examples. Members pointed out the difficulty with comparability of audit data across sites and the consequences of increased regulations on access to mammography. Dr. Mendelson noted that board certification is a thorough process and that no precedent exists in Federal policy for licensing physicians. Dr. Monsees expressed concern about developing a new infrastructure and said that existing organizations were sufficient. Dr. Nishikawa suggested that although MQSA should not be used to initiate competency requirements, someone should be trying to figure out how to measure competency.
Drs. Monsees and Nishikawa asked whether it made sense for the guidance to suggest that facilities compare their audit data or to recommend that facilities collect other data for the purposes of comparison with other facilities. Dr. Monsees noted that auditing physicians are responsible for reviewing the data and reporting to the facility and interpreting physicians. Ms. Hawkins noted that one can pass a competency exam but still have sloppy technique or a lack of ethics that leads to poor work quality. She stressed that facilities must be held responsible for the people who work there.
Amy Lee, M.D., reiterated some of the committee’s earlier concerns with audit data and the effects of regulation and asked whether a mechanism to increase competency existed within certifying bodies such as the ABR and ACR. Dr. Mendelson said that the ABR has developed subspecialty certification for some areas but mammography is not one of them and ABR is not adding subspecialties at this time. Dr. Lee reiterated her belief that existing mechanisms should be used to enforce competency and said that Dr. Finder’s examples were discovered through the existing system of checks and balances.
The committee discussed some of the areas that can cause problems with mammography quality: The main issues are positioning, compression, and absence of the posterior breast on the film as well as poor interpretation. Dr. Monsees pointed out that the lead physician in a facility ultimately is responsible for quality assurance and that the guidance document needs to say that if a facility contracts with a physicians’ group to read the mammograms, someone in that group must give quality assurance feedback to the technologists.
Dr. Young noted that the public expects mammography to be done properly and that trial lawyers serve as watchdogs over that expectation. The committee discussed the impact of trial lawyers on their profession.
Ms. Hawkins raised the issue of how to deal with personnel who practice at multiple facilities and suggested that all facilities at which a person works should be notified if a problem with that person’s work occurs at any one site. Dr. Monsees noted that her organization has six units under one facility number, under one roof, as well as a van with a separate facility number. The regulations require her to keep separate audit data on the van, even though it uses the exact same personnel, and that it did not make sense to do so.
Dr. Finder noted that some States are using audit data to take action and that a facility may decide not to collect data if it knows that the data will be used against it. He asked the committee members whether they had any ideas or suggestions about whether it was appropriate for States to use audit data to start investigating facilities.
Dr. Ikeda asked under what legal authority the States could collect audit data. Dr. Finder said that sometimes States have data because they are part of the Centers for Disease Control (CDC) program or, possibly, because of a State’s own legal or regulatory requirements. He added that the FDA does not collect data and pointed out that under MQSA, individual facilities collect data, not any one national organization. Dr. Ikeda expressed concern over mandatory reporting of audit data. Dr. Barr said that the problem is that States are using audit data to shut down mammography facilities; the data from facilities participating in the CDC program have many potential uses, and many of those facilities serve underserved populations. She said that proactive action is needed and suggested that perhaps the ABR could add a mammography certificate to its exam or that State medical boards could get involved. Ms. Hawkins asked if the quality of mammography among the CDC grantees was lower than for private patients. Dr. Barr did not believe that to be the case.
Dr. Finder reiterated the FDA’s request that the committee provide recommendations on personnel competency. He acknowledged that the issue is complicated and asked for suggestions about what, if anything, the FDA should do about States that come to FDA and say that they have a problem. Dr. Monsees suggested that the FDA could contact the State board if problems were found and the facility didn’t take action. She asked the committee if it had any other suggestions, and none were offered.
Dr. Finder raised the issue of evaluation of image quality versus evaluation of image interpretation. He pointed out that the latter type of evaluation is more resource intensive.
Richard Lippert, owner of a private company that monitors 150 private mammography facilities, said that the FDA had embraced the desirable goals developed by the Agency for Health Care Policy and Research and that the FDA should put them into a guidance document. Inspectors should focus on facilities’ continuous quality improvement mechanisms.
In response to Ms. Hawkins’ earlier question, Herschel Lawson, M.D. medical advisor to the CDC’s National Breast Cancer and Cervical Cancer Early Detection program, described some of the ways in which the CDC ensures that the radiology facilities in its program are the best available for patients in the program. Most of the data collected by the participating programs and sent to CDC for review are collected and maintained by the States. CDC requires the States to follow a data quality indicator guide for not only the data but also the outcomes and procedures. Technical assistance with the audits is provided to all 69 programs.
USE OF SMALL-FIELD DIGITAL IMAGE RECEPTORS
Dr. Chakrabarti presented an overview of the use of small-field digital image receptors (SFDIRs). He said that SFDIRs are being used in many stereotactic mammographic units and although they cannot be used for screening because of their small size, they could produce digital spot compression images for diagnostic mammography. He said that the FDA wanted the committee’s input on how the following MQSA issues apply to SFDIRs: 1) the accreditation process, 2) equipment evaluations and annual physics surveys, and 3) the inspection process.
Mr. Pizzutiello said that he saw no need for a different accreditation process and that SFDIRs could be treated as another image receptor. In annual physics surveys and equipment evaluations, the MP should evaluate the SFDIR. He suggested that changing the table on page 10 to include the general term "image receptor" for tests that are appropriate for evaluating image quality and dose would allow the guidance document to apply to SFDIRs. He added that only two tests should be required: phantom image and dose. The FDA should recommend that facilities follows manufacturers’ recommendations for routine quality control testing procedures. In response to a question from Dr. Finder, he added that system artifact tests also should be done.
Dr. Monsees said that SFDIRs used only for intervention procedures are not governed by MQSA. Dr. Nishikawa asked whether focal spot size and kVp should be checked annually, but Mr. Pizzutiello indicated that was unnecessary. Dr. Finder noted that currently focal spot size or system resolution could be evaluated. Mr. Pizzutiello said that looking at system resolution was inappropriate because no benchmarks exist. Dr. Nishikawa maintained that such data were useful; comparing data from year to year can indicate whether a system is degrading. Mr. Pizzutiello added that it would be useful for inspectors to check to see whether certain tests actually had been done.
Dr. Monsees again noted that only facilities using SFDIRs for diagnostic work, not intervention, were subject to FDA inspection and that the guidance should perhaps stipulate that facilities must state which purposes they were using the equipment for. Mr. Pizzutiello suggested that the FDA guidance refer to the tests described in the ACR Stereotactic Quality Control Manual to assist MPs who need information on how to do the tests.
FULL-FIELD DIGITAL MAMMOGRAPHY (FFDM) CERTIFICATION
Dr. Barr presented an update on the status of full-field digital mammography certification, noting that the system is working well. Dr. Monsees asked how many facilities had been certified, but Dr. Finder said that the information was not available. Dr. Barr said that GE maintains a Web site listing the locations of digital facilities, although the list does not reflect which ones have digital certification. Priscilla Butler of ACR described the progress being made in the development of their FFDM module to their accreditation program. ACR began alpha testing of technical parameters for the FFDM module in the Spring of 2000. The committee had no questions.
STATES AS CERTIFICATION AGENCIES: UPDATE
Kaye Chesemore, M.B.A., presented an update on the States as Certifiers (SAC) program. She provided background information and described some of the issues brought to the FDA’s attention after the proposed regulations were published in the Federal Register. She said that the SAC has been a successful program and that the States and FDA had worked cooperatively.
INSPECTION DEMONSTRATION PROGRAM: UPDATE
Dr. Barr presented an update on the MQRSA Inspection Demonstration Program. She noted that, to date, 34 States had responded to the invitation to participate and that 11 States had agreed to participate. Mr. Pizzutiello expressed concern over the small sample size and the likely outcome of inconclusive data. Dr. Barr replied that the inclusion criteria had been expanded. Mr. Pizzutiello noted that States had a financial disincentive to participate and that if States that were able would agree to contribute more than 5 percent of their facilities, the financial burden on the other States would be less. Dr. Barr stated that the geographic distribution of facilities would then be skewed. The committee discussed possible reasons for States’ not participating in the demonstration project.
Dr. Monsees thanked the Committee and the audience and adjourned the meeting at 3:25 p.m.
I certify that I attended the National Mammography Quality Assurance Advisory Committee Meeting on September 28, 2000 and that this summary accurately reflects what transpired.
Charles Finder, M.D.
Committee Executive Secretary
I approve the minutes of this meeting as recorded in this summary.
Barbara Monsees, M.D.
Summary minutes prepared by
1112 Lamont St., NW
Washington, DC 20010