Medical Audit and Outcome Analysis - Are your Patients Getting the Full Picture?
by W. G. Mourad, Ph.D.
Information Management Branch
Division of Mammography Quality and Radiation Programs
The "interpretation" or "reading" of a mammogram is the final, and most critical, step in detecting abnormalities in the mammogram. If all the imaging chain components were optimized to the point of producing the best quality mammogram, this mammogram would remain only a technical curiosity until an interpreting physician (IP) took the final step, e.g., the detection and interpretation of the diagnostic information contained in it. Needless to say, if such a mammogram contained abnormal findings that the IP didn’t see or interpret correctly, it could have serious consequences.
For this reason, MQSA regulations allow only those physicians who have met specific baseline initial and continuing qualification requirements to qualify as IPs and interpret mammograms independently.
As a quality assurance program aimed at helping IPs receive feedback on the outcome of their interpretations, MQSA final regulations require each facility to “establish a system to collect and review outcome data for all the mammograms performed, including followup on the disposition of all positive mammograms and correlation of pathology results with the interpreting physician’s mammography reports.”
Furthermore, the regulations require a facility to analyze the outcome data individually for each IP and collectively for all IPs at the facility. In addition, a facility is required to promptly initiate follow-up on surgical and/or pathology results regarding “any cases of breast cancer among women imaged at the facility that subsequently become known to the facility.” For such patients, the facility is required to review the patient mammograms taken prior to the diagnosis of a malignancy. Clearly, the ultimate goal of these required reviews, analyses, and correlations is to improve the interpretive accuracy for all IPs at the facility.
The regulations allow a facility considerable latitude in establishing an adequate medical audit system that meets the needs of its mammography practice, and only outline the basic procedural requirements for such a system in broad terms. As stated below, a facility must:
- Begin its first audit analysis no later than 12 months after the date of its certification, and complete its analysis within an additional 12 months after that. Subsequently, it must conduct the audit analysis at least once every 12 months.
- Designate at least one of its IPs as the “audit” or “reviewing” IP who is responsible for:
• Reviewing the medical outcomes audit data once every 12 months
• Recording the dates of the audit periods
• Analyzing and documenting the individual and aggregate results, and
• Notifying all the facility’s IPs of their results and their facility’s aggregate results.
Inspection Observations Regarding Medical Audit and Outcome Analysis
The FDA issues a Level 2 (L2) citation to a facility for any of the following observations:
- Not ALL positive mammograms are entered in system
- Biopsy results are not present (or no attempt is made to get them)
- No audit (reviewing) interpreting physician has been designated
Inspection results from 10/1/1999 to 9/30/2003 (FY00 through FY03) regarding the above observations are shown in Figure 1 below. When reviewing these results, it is important to remember that the number of inspections has decreased from 9442 in FY00 to 8686 in FY03.
The trend of citations in all three categories is improving However, while none of the three citations had a rate much greater than 1% in any of the last 4 years, and in the last year, the aggregate of all three was not much more than 1%, this still is a serious problem for the violative facilities.
A facility can eliminate these citations by designating an auditing physician who is responsible for overseeing the medical audit system and assuring that the facility
- include all mammography examinations with positive assessments (those interpreted as suspicious or highly suggestive of malignancy) into their tracking system; and
- perform follow-up on these patients by working closely with referring physicians and/or pathology services to try and obtain final biopsy results and enter such results into its tracking system.
Regarding the review, analysis, and followup of the audit data and results, the regulations give a facility two years from its certification date or April 28, 1999 (the date the final regulations went into effect), whichever is later, before any of the next 3 observations would result in a Level 2 citation. FDA granted the two-year grace period to allow a newly established facility to set up a fully operational audit system before issuing any related citation. As a result, the first time any of the following observations could have resulted in citations was 4/28/2001:
- Analysis not done annually
- Analysis not done separately for each individual
- Analysis not done for the facility as a whole
Inspection results from 4/28/2001 to 9/30/2003 (part of FY01 through FY03) regarding the above observations are shown in Figure 2 below. (Note that the data for FY01 represents only 5 months of inspections):
The number of facilities cited in each of these three categories is approximately twice as high compared to the initial three categories. The encouraging news is that the number of citations has decreased by about 50% from FY02 to FY03.
Again, a facility can avoid such citations by ensuring that its auditing physicians perform the analyses annually, for each IP individually, and for the aggregate facility as a whole, as required in the regulations.
For additional guidance about how to meet FDA requirements regarding medical audits, please refer to FDA’s Policy Guidance Help System (PGHS) under, "Medical Outcomes Audit."
Figure Captions (508 compliance)
Figure 1. Figure 1 is bar graph representation of the number of facilities cited per fiscal year starting with 2000 and ending with 2003, for each of the following observations respectively:
1. Not all positives entered
2. No biopsy results
3. No designated auditing physician.
Figure 2. Figure 2 is bar graph representation of the number of facilities cited per fiscal year starting with 2000 and ending with 2003 (note that the data for FY01 represents only 5 months of inspections), for each of the following observations respectively:
1. Analysis not annual
2. No individual analysis
3. No analysis for whole facility