MQSA 2004 Report to Congress - Full Report
PERFORMANCE EVALUATION OF ACCREDITATION BODIES UNDER THE MAMMOGRAPHY QUALITY STANDARDS ACT OF 1992 as amended by the MAMMOGRAPHY QUALITY STANDARDS REAUTHORIZATION ACT OF 1998
January 1, 2004 through December 31, 2004
A Report to Congress
The Mammography Quality Standards Act (MQSA) of 1992 (P.L. 102-539), as amended by the Mammography Quality Standards Reauthorization Act (MQSRA) of 2004 (P.L. 108-365), establishes standards for high quality mammography and requires all facilities to be accredited by a Food and Drug Administration (FDA) approved accreditation body (AB) in order for them to demonstrate that they meet these standards. FDA may approve either private nonprofit organizations or state agencies to serve as ABs. The MQSRA also requires the FDA to submit an annual performance evaluation of the approved ABs to the Senate Committee on Health, Education, Labor and Pensions and the House Committee on Energy and Commerce under 42 U.S.C. 263b(e)(6). This report covers the performance of the ABs under the MQSA from January 1 through December 31, 2004.
Status of Accreditation Body Approvals
Currently, there are four ABs: the American College of Radiology (ACR), a private nonprofit organization, and the state ABs of Arkansas (SAR), Iowa (SIA), and Texas (STX). The FDA approved each of these ABs under the final MQSA regulations. Since each AB’s approval expires on April 28, 2006, they will begin the renewal process in the fall of 2005.
FDA approved the State of California (SCA) under the interim MQSA regulations in 1994. Then, in 1998, the SCA applied for AB status under the final regulations. Despite the collaborative efforts of FDA and the SCA, the State was unable to develop its MQSA accreditation program to achieve approval under the final regulations. Therefore, on May 5, 2004, the SCA withdrew its application to become an FDA-approved AB under the MQSA final regulations. Through its withdrawal, the SCA relinquished its authority and responsibilities under the MQSA. Section 900.13(b)(1) and (2) (under 21 CFR Part 900) allows certificates of facilities previously accredited by a withdrawn AB to remain in effect for up to one year from the date of the withdrawal of approval, unless FDA determines that there are public health issues or that the AB fraudulently accredited any of its facilities. As a result of the SCA’s withdrawal, all SCA-accredited facilities were required to apply for accreditation from the ACR no later than May 5, 2005. On May 7, 2004, and May 13, 2004, the SCA and FDA, respectively, notified all SCA-accredited facilities (465) of the State’s withdrawal and the MQSA requirement that these facilities obtain accreditation from the ACR within one-year. As of May 5, 2005, all SCA facilities had transitioned to the ACR.
MQSA requires that each AB develop (or adopt by reference) standards that are substantially the same as the quality standards established by FDA under subsection (f) of the Act to assure the safety and accuracy of mammography. Regarding state laws, nothing in the Act limits the authority of any state to enact and enforce laws about matters covered by the Act that are at least as stringent as the Act or the standards promulgated under the Act.
All ABs have either adopted the final MQSA standards by reference, or have developed standards that are substantially the same as the quality standards established by FDA. Each AB incorporated the standards into its own accreditation processes.
FDA evaluates its ABs through: (1) examination of responses to questionnaires developed by FDA addressing performance indicators, (2) analysis of quantitative accreditation and inspection information, (3) review of selected files (including clinical and phantom images), (4) interviews with AB staff and management to answer questions or clarify issues, and (5) on-site visits. FDA uses the following eight performance indicators (as outlined in the final MQSA regulations) to assess performance: administrative resources, data management, reporting and record keeping processes, accreditation review and decision-making processes, AB on-site visits to facilities, random clinical image reviews, additional mammography reviews, and accreditation revocations and suspensions.
FDA staff analyzes unit accreditation pass and fail data along with data that describe the reasons for each AB failure decision. Significant differences in pass and fail rates or reasons for accreditation denial among ABs could, for example, indicate that one AB is interpreting the significance of a particular quality standard more or less strictly than another.
To complement the information submitted by the ABs, FDA analyzes information from its Mammography Program Reporting and Information System (MPRIS) database of annual facility inspections. Accredited facility performance during inspections is measured by average phantom image scores, average radiation dose values, and average processor speeds. Collectively, these measures reflect the overall functioning of all components of the mammography system.
Annually, the ABs respond to questionnaires developed by FDA that address specific performance indicators. FDA uses this information to assist in its analysis of each ABs performance for the preceding calendar year. Because the SCA withdrew its application for status as an AB in 2004 and relinquished its authority and responsibilities under the MQSA, it did not collect and thus did not submit any data pertaining to the performance indicators. Therefore, this report covers the performance of the ACR, the SAR, the SIA, and the STX ABs from January 1, 2004, through December 31, 2004.
(1) Administrative Resources and Funding
AB staffs generally include management, mammography radiologic technologists, MQSA inspectors, health physicists, information technology program application specialists, and administrative assistants. In 2004, all ABs continued to maintain adequate funding for their respective programs.
(2) Data Management (Process/Errors)
All ABs provide FDA with electronic transmissions of accreditation data in a secure and appropriately maintained manner. The percentage of data management errors either remained the same or increased only slightly (by 1 percent or less) of those noted in the previous year. FDA continues to work individually with ABs to (a) further minimize the number of data errors, (b) emphasize the importance of routinely performing quality assurance and quality control practices to correct errors before transmitting the data, and (c) provide reports that outline errors and the frequency with which they occur.
(3) Reporting and Recordkeeping
FDA’s review of the ABs’ reporting and recordkeeping practices includes examining procedures for handling serious consumer complaints and appeals for accreditation decisions.
(a) Serious Consumer Complaints
The MQSA requires ABs to develop and administer a consumer complaint mechanism whereby all facilities that an AB accredits must file serious unresolved complaints with their AB. By regulation, each AB must submit to the agency an annual report summarizing all serious complaints received during the previous calendar year, their resolution status, and any actions taken in response to them.
All ABs have an appropriate serious consumer complaint mechanism in place. Each AB submitted its serious consumer complaint report to FDA for the year 2004, indicating that the ABs follow acceptable procedures when resolving these complaints.
Each AB must have an appeals process for facilities to contest an AB’s adverse accreditation decision. In CY 2004, only two of the ABs received appeals to their accreditation decisions. The ACR received three appeals. It upheld the original adverse decision for two appeals and overturned the third. The SAR received one appeal in which it upheld its original adverse decision.
(4) Accreditation Review and Decision-Making Processes
Review of the ABs’ accreditation and decision-making processes includes evaluating procedures for clinical image review, phantom image review, and mammography equipment evaluation and medical physicist annual survey review.
(a) Clinical Image Review
As part of the accreditation process, mammography facilities must submit clinical images to their ABs for review. To evaluate the ABs’ performance in the clinical image review area, FDA’s MQSA qualified interpreting physicians (IPs) annually review clinical images from a sample of facilities that submit cases to the ABs for clinical image review. Generally, two FDA IPs independently conduct clinical image reviews for each facility in the sample for each of the ABs that perform clinical image review, evaluating each examination on the eight attributes listed in the final regulations.
The STX has a contract with the ACR to conduct its clinical image reviews. The remaining three ABs have their own clinical image reviewers to evaluate their facilities’ clinical images. A summary of FDA clinical image reviews follows.
American College of Radiology AB
FDA performed its evaluation of the ACR’s clinical image review process on October 13, 2004. FDA found that there was good agreement between FDA IPs and the ACR clinical image reviewers at the attribute evaluation level. In reviewing the exams and summary evaluation forms, FDA reviewers agreed with the final overall assessments (pass and fail) in all the cases. In three cases where there were disagreements between the ACR reviews on a pass/fail decision, FDA reviewers believed the cases were borderline situations. FDA determined that this spot review of cases indicates that the quality of clinical image review by the ACR remains high and has not deviated from past performance.
State of Arkansas AB
FDA performed its evaluation of the SAR’s clinical image review process on October 12 and 15, 2004. FDA’s IPs indicated that the quality of clinical image review performed by the SAR remains high and has not deviated from past performance. FDA commended the SAR for specifically asking its reviewers to state (in cases of failure) whether the images were of diagnostic quality and if an additional mammography review (AMR) should be considered. However, FDA’s IPs recommended that the SAR’s AB clinical image reviewers should provide more feedback to its facilities on ways to improve image quality even in “pass” cases.
State of Iowa AB
On September 30, 2004, one of FDA’s IPs performed an evaluation of the SIA’s clinical image review process. The FDA IP found consistent agreement among the SIA reviewers and agreed with the SIA reviewers’ final overall assessments (pass/fail) in all the cases reviewed. The review indicated that the quality of clinical image review performed by the SIA AB remains high and has not deviated from past performance.
Summary of Audits and Training of Clinical Image Reviewers by ABs
An audit of clinical image reviewers ensures uniformity, identifies any potential problems, and provides all individual clinical image reviewers with the necessary data to compare his/her results to the rest of the review group. Audit results are used to enhance reviewer training by emphasizing any performance issues. In 2004, the ACR (and the STX via the ACR contract), the SAR, and the SIA conducted audits of their clinical image reviewers to collect statistics on reviewer agreement and nonagreement rates.
Clinical image review quality control activities that promote consistency among the various clinical image reviewers exist at the ACR (and the STX via the ACR contract), the SAR, and the SIA. Each of these ABs conducts training sessions at which clinical image reviewers evaluate clinical images and discuss findings, including the application of AB clinical image review evaluation criteria.
(b) Phantom Image Review
As part of the accreditation process, mammography facilities must submit phantom images to their ABs for review. To evaluate the ABs’ performance in the phantom image review area, FDA’s MQSA expert staff annually reviews phantom images from facilities that submit cases to the ABs for phantom image review. Two FDA staff, working independently, review 10 randomly selected phantom images from each of the ABs that perform phantom image review. FDA evaluates all test objects (fibers, specks, masses) on these images as part of the review. Scores for these test objects should fall within the acceptable limit of plus or minus 0.5.
The STX has a contract with the ACR to conduct its phantom image reviews. The remaining three ABs have their own phantom image reviewers to evaluate their facilities’ phantom images. A summary of the phantom image reviews follows.
American College of Radiology AB
FDA reviewed the ACR’s phantom images in October 2004. Most of the test object scores of FDA reviewers were within the generally accepted range of the scores of the ACR reviewers. For three of the phantom images, the ACR reviewers’ fiber scores diverged by more than 0.5 from the FDA reviewers’ scores. FDA reviewers believed these cases were borderline situations that did not raise quality issues. In only one case did the ACR reviewers differ on a score by more than 0.5.
FDA determined that this spot review of the phantom images indicates that the quality of phantom image review by the ACR remains good and has not deviated from past performance.
State of Arkansas AB
FDA reviewed the SAR’s phantom images in November 2004. FDA compared its scores with the scores of the SAR reviewers. FDA reviewers indicated that the quality of phantom image review performed by the SAR remains good and has not deviated from past performance.
State of Iowa AB
FDA reviewed the SIA’s phantom images in November 2004. Most of the test object scores of FDA reviewers were within the generally accepted range of the scores of the SIA reviewers. There were three phantom images where the SIA reviewers’ scores diverged by more than 0.5 from the FDA reviewers’ scores. FDA believes these scores are acceptable since the SIA reviewers were more stringent in their scoring. The quality of phantom image review performed by the SIA remains good and has not deviated from past performance.
Summary of Audits and Training of Phantom Image Reviewers by ABs
An audit of phantom image reviewers ensures uniformity, identifies any potential problems, and provides all individual phantom image reviewers with the necessary data to compare his/her results to the rest of the review group. Audit results are used to enhance reviewer training by emphasizing any performance issues. In 2004, the ACR (and the STX via the ACR contract), the SAR, and the SIA conducted audits of their phantom image reviewers to collect statistics on reviewer agreement and nonagreement rates.
Phantom image review quality control activities that promote consistency among the various phantom image reviewers exist at the ACR (and the STX via the ACR contract), the SAR, and the SIA. Each of these ABs conducts training sessions at which phantom image reviewers evaluate phantom images and discuss findings, including the application of AB phantom image review evaluation criteria.
(c) Mammography Equipment Evaluation (MEE) and Medical Physicist Survey Report Reviews
The final regulations state that ABs shall require every facility applying for accreditation to submit an MEE with its initial accreditation application and, prior to accreditation, to submit a medical physicist survey on each mammography unit at the facility (21 CFR 900.4(e)(i)). All of the ABs have policies and procedures established for the review of both the MEE and the medical physicist survey report.
(5) AB Onsite Visits to Facilities
The final MQSA regulations (21 CFR 900.4(f)(1)(i)) require that each AB annually conduct on-site visits to at least five percent of the facilities the body accredits to monitor and assess facility compliance with the standards established by the body for accreditation. However, a minimum of 5 facilities shall be visited, and visits to no more than 50 facilities are required. During such visits, the AB is required to evaluate eight core elements: (a) assessment of quality assurance activities; (b) review of mammography reporting procedures; (c) clinical image review; (d) review of medical audit system; (e) verification of personnel duties; (f) equipment verification; (g) verification of consumer complaint mechanism; and (h) other identified concerns.
At least 50 percent of the facilities visited shall be selected randomly and the other facilities visited shall be selected based on problems identified through state or FDA inspections, serious complaints received from consumers or others, a previous history of noncompliance, or other information in the possession of the AB, the MQSA inspectors, or FDA (i.e., visits for cause).
American College of Radiology AB
The ACR conducted 50 on-site visits (46 random, four for cause) in CY 2004, thus meeting the minimum of 50 on-site visits required by regulation.
State of Arkansas AB
The SAR conducted six on-site visits (all random) in CY 2004, thus exceeding the minimum of five on-site visits required by regulation.
State of Iowa AB
The SIA conducted 39 on-site visits (38 random, one for cause) in CY 2004, thus exceeding the minimum of 7 on-site visits required by regulation.
The STX conducted 11 on-site visits (7 random, 4 for cause) in CY 2004, thus exceeding the minimum of 8 on-site visits required by regulation.
(6) Random Clinical Image Review
The final MQSA regulations (21 CFR 900.4(f)(2)(i)) require that each AB annually conduct random clinical image reviews (RCIRs) of at least three percent of the facilities the body accredits to monitor and assess facility compliance with the standards established by the body for accreditation.
American College of Radiology AB
During CY 2004, the ACR conducted 249 RCIRs, thereby exceeding the 246 required by regulation.
State of Arkansas AB
The SAR conducted 10 RCIRs in CY 2004, thus exceeding the minimum of the 2 required by regulation.
State of Iowa AB
The SIA conducted 38 RCIRs in CY 2004, thus exceeding the minimum of the 4 required by regulation.
State of Texas AB
The STX conducted seven RCIRs in CY 2004, thus exceeding the minimum of the five required by regulation.
(7) Additional Mammography Review
If FDA has reason to believe that mammography quality at a facility has been compromised and may present a serious risk to human health, the facility must provide clinical images and other relevant information, as specified by FDA, for review by its AB (21 CFR 900.12(j)). This additional mammography review (AMR) helps the agency to determine whether there is a need to notify affected patients, their physicians, or the public that the quality of mammograms may have been compromised. The request for an AMR may also be initiated by an AB or a State Certifying Agency (SAC). When an AB initiates an AMR, FDA encourages it to discuss the case with the agency prior to performing the AMR.
The following chart summarizes the number of AMRs conducted by each AB during CY 2004:
|AB||Number of AMRs Conducted or Initiated*||Number With Deficiency or Serious Risk||Number That Completed Corrective Action and/or Notification|
*Note: The STX has a contract with the ACR to conduct its clinical image reviews during an AMR. The remaining three ABs have their own clinical image reviewers to evaluate their facilities’ clinical images.
(8) Accreditation Revocation and Suspension
The MQSA final regulations (21 CFR 900.3(b)(3)(iii)(I)) require that each AB have policies and procedures for suspending or revoking a facility’s accreditation. If a facility cannot correct deficiencies to ensure compliance with the standards or if a facility is unwilling to take corrective actions, the AB shall immediately notify FDA, and shall suspend or revoke the facility’s accreditation.
State of Arkansas AB and State of Iowa AB
Neither the SAR nor the SIA revoked or suspended any facility’s accreditation in 2004.
American College of Radiology AB
The ACR revoked the accreditation of two facilities during 2004. After the ACR performed an AMR on each facility, it issued each a letter of revocation when its clinical image reviewers determined the facilities’ practices posed a serious risk to human health. Subsequently, under Section 900.13(a) of the final regulations, FDA determined that the certificates at both facilities were no longer in effect. Both facilities completed corrective action that included patient notifications and were subsequently reinstated by the ACR and FDA.
State of Texas AB
The STX suspended the accreditation of one facility during 2004. After the STX performed an AMR on the facility and its reviewers determined the facility’s practice posed a serious risk to human health, it issued the facility a letter of suspension. The facility completed its corrective action and subsequently the STX lifted the facility’s suspension. There was no FDA action required in this case.
(9) Quantitative Accreditation and Inspection Information
As additional performance indicators, FDA analyzed quantitative accreditation and inspection information related to (a) unit accreditation pass/fail data, (b) reasons for denial of accreditation, and (c) accredited facility performance during inspections. Note: There are a relatively small number of state-accredited facilities compared to the ACR-accredited facilities. Therefore, small variations in state-accredited facility performance may lead to differences across accreditation bodies that do not reflect actual differences in accreditation body performance.
(a) Unit Accreditation Pass/Fail Data Sorted by AB
|Number of Units||ACR||SAR||SIA||STX|
|Passed Accreditation||4,597 (99.6%)||46 (100%)||59 (100%)||71 (98.6%)|
|Failed Accreditation*||14 (0.4%)||0||0||1 (1.4%)|
*Units that were still denied accreditation as of December 31, 2004.
At the conclusion of the reporting period, the accreditation pass rate of mammography units among the accreditation bodies ranged from 98.6 - 100 percent. The rates for units that failed accreditation increased slightly from those in the last reporting period. The unit fail rate usually reflects the facility’s second and third attempts at unit accreditation. The majority of facilities whose unit receives a fail in the first attempt at accreditation initiate corrective action and subsequently passes.
(b) Reasons for Mammography Unit Denial
In 2004, clinical image review failure was the major reason for denial of unit accreditation. Phantom image review failure and failure to submit the required materials were the other reasons for mammography units being denied accreditation. Most of the facilities that receive a denial in the accreditation process complete rigorous corrective action plans under the ABs’ reinstatement protocols and eventually successfully achieve the levels of quality needed for accreditation.
(c) Facility Performance During Inspections Sorted by AB
In CY 2004, over half (68.8 percent) of the accredited mammography facilities had no MQSA violations while only 2.1 percent of the facilities had a violation characterized as “most serious.” FDA actively works with these facilities on corrective measures, or takes regulatory measures if a facility cannot improve its performance.
ACR SAR SIA STX Average Phantom
12.3 12.5 11.3 12.9 Average Dose (in millirads) 177.7 174.9 156.4 178 Average Processor Speed 106 113.8 100.4 111.3
*The maximum possible phantom image score is 16. Four fibers, three masses, and three speck groups must be visible on the image for a minimum passing score.
There were no significant differences in average phantom image scores among the facilities accredited by the four ABs. In general, average phantom image scores increased slightly from those reported in the 2003 Report.
In general, the average doses decreased slightly from those reported in the 2003 report and remain well below the dose limit of 300 millirads mandated by the MQSA final regulations. This dose limit has the advantage of permitting flexibility for the optimization of technique factors used during examinations to achieve improved image quality.
The average processing speeds among the facilities of all the ABs remained in the range to produce satisfactory clinical images and decreased slightly from those speeds reported in the 2003 Report. The evaluation of the mammography facility’s film processing speed is an important quality assurance measure. The speed of film processing impacts directly not only on the resulting image quality of the mammogram, but can also impact on the dose administered to the patient. If a mammography facility is processing film in accordance with the film manufacturer’s recommendations, then the processing speed should be close to 100 (80 – 120 is considered normal processing speed). If the processing speed falls significantly, then the clinical image is not completely developed, appears too light, and the quality of the mammographic image can be significantly compromised. Moreover, the facility may not realize its film processor is the source of the problem and may compensate by increasing the dose administered to the patient.
Status of the Action Items From the 2003 Report to Congress
In all instances, the ABs successfully completed their CY 2003 action items.
FDA’s AB oversight program promotes collaboration and cooperation. Therefore, each AB, in concert with FDA, addresses any action items that may arise during the year. FDA and the ABs, working in partnership with the certified mammography facilities in the United States as well as the states participating in inspection and other MQSA activities, are ensuring quality mammography across the Nation.