• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services

About FDA

  • Print
  • Share
  • E-mail

FY 2001 Medical Imaging Evaluation

A wide variety of new digital imaging and display devices is under development by academia and industry, with a broad range of performance characteristics.  The Center thus requires new/improved guidance for the evaluation of such devices.  To this end, OST scientists are developing evaluation methodologies for diagnostic medical imaging systems such as mammography and fluoroscopy, computed tomography, nuclear medicine, diagnostic ultrasound, and magnetic resonance imaging, as well as novel soft-copy display devices for viewing medical images. 

OST scientists are engaged in the development of appropriate methods for the evaluation of medical imaging system performance and dose. Investigations take the form of theoretical analysis, numerical simulation of the entire imaging chain, and experimental validation.  In some instances improved/optimized system designs are validated through actual system construction and clinical evaluation. Measurement and analysis procedures are also being developed to evaluate the performance of new soft-copy display devices that can have dramatically different light-emitting structures and associated performance characteristics whose impact on the image interpretation process is currently unknown. OST scientists provide reliable, quantitative laboratory measurements of imaging system characteristics to the imaging research community. OST scientists are also elucidating the fundamental mechanisms underlying the interaction between the image-forming radiation and the anatomy being imaged.

These investigations inform the Center's regulatory decision-making on new digital image devices. The expertise developed through this program is being applied to the review of PMAs for ultrasound bone sonometers and new digital radiographic imaging systems. This program contributes to the development of premarket guidance documents including "Information for Manufacturers Seeking Marketing Clearance of Digital Mammography System" and a guidance document on preclinical requirements for bone densitometers. OST scientists are applying their expertise to the development of a CDRH web site on CT, the development of amendments to the diagnostic x-ray performance standard, the development of an advisory pertaining to pediatric CT exposures, and the joint planning of a consensus development conference on CT with NIH. Improved knowledge of the fundamental imaging mechanisms will lead to an understanding of the sources of variability in imaging data. Having that, inter-machine and inter-institute measurements can be corrected, leading to absolute, quantitative measures which can then be codified through a measurement standardization process. The x-ray spectral measurements program provides a source of otherwise unavailable data to the entire mammography research community for use in developing new equipment performance standards as digital mammography develops, special procedures and test equipment for MQSA, and will be used to inform decisions on marketing clearance for new products and in compliance actions."

OST scientists develop consensus evaluation methodology for diagnostic medical imaging systems such as mammography and other film-screen x-ray systems, computed tomography, nuclear medicine, diagnostic ultrasound, magnetic resonance imaging, and digital imaging including fluoroscopy and digital mammography. The goal of the program is to characterize and optimize medical imaging systems and components through application of quantitative measures of imaging performance and dose. This program also supports development of mammography equipment standards and special procedures and test equipment for MQSA.

Digital Image Display System Evaluation

Key words: digital radiography, soft-copy display, cathode-ray tube (CRT), flat panel, active-matrix liquid-crystal display (AMLCD)

The purpose of this project is to develop measurement and analysis procedures to evaluate the performance of image display devices for digital diagnostic imaging systems. While CRT-based monitors are currently the most commonly used devices for soft-copy display, flat panel monitors using AMLCD technology are entering the marketplace for medical image display applications. These devices exhibit a strong orientation dependence of brightness and contrast as well as significant sub-pixel structure. Both of these effects require developing new measurement and analysis methods for evaluating flat panel displays. These new methods and the data they produce are essential to the science base on which regulatory decisions are made.

During FY 2001 efforts were directed toward comparing a variety of methods for evaluating the sensitometric, sharpness and noise properties of both CRT-based displays and a new AMLCD flat panel display. Normal and off-axis luminance for the flat panel were measured with a specially designed photometric probe with a very small acceptance angle. The results were evaluated by studying the display compliance to the DICOM grayscale function at off-normal viewing angles. Resolution and noise properties of both CRT and flat panel systems were measured using a CCD camera. Resolution was evaluated using line patterns as well as with the broadband transfer method. Noise was analyzed in terms of two-dimensional noise power and signal-to-noise ratios.

Imaging System Performance Evaluation

Key words: digital imaging, detective quantum efficiency, resolution, machine observer

OST scientists continued to extend the quantitative assessment of the dose and imaging performance of imaging systems from the analog to digital imaging domain. Specifically, the research included the following topics: evaluating inefficiencies in imaging performance using laboratory implementations, when possible, of the inefficient imaging system; investigating the validity of the underlying assumptions for imaging performance measures based on analog imaging systems; and bridging the gap between subjective evaluations using imaging phantoms and objective measures of imaging performance.

With respect to inefficiency in the image formation process, imaging systems comprised of a small detector which "looks" at a large area of a light-emitting phosphor with a lens coupling system are commercially available and have been the subject of premarket submittals. These imaging systems can be quite inefficient depending on the characteristics of the optical coupling. OST scientists set up a lens-coupled imaging system in the laboratory to use as a test bed to investigate the efficiency question. The laboratory system provided the ability to investigate when a system of this configuration starts to develop a "secondary quantum sink" as a result of poor optical coupling. Experimental measurements have been made of the large area gray-scale transfer, resolution, and noise properties of the imaging system at two different values of optical demagnification factor and three levels of x-ray exposure. The values of a fundamental measure of imaging performance, the detective quantum efficiency (DQE), calculated from these measurements demonstrate the significant increase in imaging performance when the demagnification decreases by a factor of approximately two. These data reinforce the need for a lens-coupled digital imaging system to be carefully designed to avoid low coupling efficiencies and the attendant decrease in imaging performance. These data have been and will be useful in premarket approval reviews for digital imaging systems using lens coupling as a component of system design.

The consensus measures of imaging performance such as the DQE, which are part and parcel of premarket submittals, are based on assumptions germane to analog imaging systems, i.e., noise stationarity or imaging system shift-invariance. Yet, researchers know that these assumptions are violated when considering digital imaging systems. OST began investigating the impact of these assumptions on imaging task performance through a series of computer simulations. Scientists used a 2-D Monte Carlo simulation of a digital x-ray imaging system including adjustable parameters for the lesion to be detected, the characteristics of the background noise, the size of the image pixel, and electronic noise. The steps in the simulation closely follow the physical image-formation process commonly encountered in a digital radiography system. The OST approach has no requirement for making assumptions about noise stationarity or imaging system shift-invariance. It is based on the spatial-domain evaluation of task performance for model observers previously shown to bracket human performance. Using simple signals with known characteristics, scientists compared observer figures of merit derived from the OST simulation with theoretical predictions and obtained excellent agreement. In addition to computer simulation, OST also purchased a representative digital detector for experimental verification of simulation results in the laboratories using an actual digital imaging system. Fundamental measures such as resolution, gray-scale transfer, noise, and DQE have already been made on this system. Beside the close association with the computer simulations, these data have been and will be used to gain knowledge and to provide constructive input to the standards-setting organizations dealing with these physical parameters.

In the future, research will also include investigations on replacing the coarse human readings of imaging phantoms or the more sophisticated and expensive laboratory measurements on imaging systems with machine observers reading an appropriate set of test images at the clinical facility. As a start, OST scientists used a lens-coupled digital radiography system (described above) for machine-observer scoring as a task-based measure of performance. Estimates of observer performance in terms of signal-to-noise-ratio were obtained from one algorithmic observer (DC-suppressed matched filter) and from one analytic observer (pre-whitening matched filter) for the task of detecting a known object (CDMAM phantom) on a flat background. For the special case of a large object at the center of the field-of-view, estimates of observer performance were essentially the same whether based on analytic or algorithmic observer performance. OST intends to continue this research track using more realistic test objects and imaging phantoms.

As stated in previous annual reports, this work will open up new approaches to imaging system performance evaluation. For example, a good understanding of the impact of the digital image formation process should have some impact on the imaging communities’ selection of consensus measures. The use of these new consensus measures will then provide descriptive information on imaging system performance in premarket submittals. Additionally, quantitative and precise measures at the clinical facility can replace the subjective evaluation of imaging phantoms such as the scoring of the imaging phantom currently used in enforcement of the MQSA program. (See the section on X-ray Physics Laboratory Studies, above.) Developing staff expertise in implementing these evaluation tools will provide for more informed product reviews of all types of imaging systems, not just digital imaging systems. In addition, a stable protocol for quantitative measures at the clinical facility can provide the Agency with the means of making quick, efficient, inspections to regulatory criteria at the facilities.

Some of the products of this research activity included a presentation and two posters at the SPIE 2001 Medical Imaging Symposium and a poster at the FDA Science Forum.

Ultrasound Bone Densitometry

Key words: ultrasound, bone density, osteoporosis

OST plays a significant role in the approval of PMAs for ultrasound bone densitometers. This is a new technology that is likely to undergo much technological evolution and regulatory activity in the near future. Currently there is a considerable lack of standardization among devices. Pre-clinical experiments, clinical trials, and theoretical analysis are important in understanding this technology and anticipating future trends. This project provides an independent source of data in OST in support of regulatory decision making. OST has explored fundamental mechanisms underlying the interaction between ultrasound and bone. These investigations increase understanding of how and why ultrasound bone densitometry is effective and therefore lead to better and more thorough reviews of these devices. This technology is currently entering an exciting new generation with the first device to perform ultrasonic imaging (rather than simply bulk measurements) of bone obtaining FDA approval in 2001.

In FY 2001, this work resulted in five refereed publications and six presentations at scientific conferences. The work included a clinical trial to investigate a new method (backscatter) for characterizing bone in vivo. In addition, theoretical modeling and experimental measurements (in vitro) regarding scattering, attenuation, and sound speed were reported. A poster describing a model for ultrasonic scattering from bone won the Cum Laude Award at the SPIE Medical Imaging conference in San Diego in February 2001.

X-ray Physics Laboratory Studies

Key words: x-ray spectroscopy, phantom,

The purpose of this project is to evaluate equipment and materials used in medical radiography and in quality assurance of medical radiography systems, to support OST research efforts, other Center programs, and the general radiology community, when appropriate. The conduct of this project requires establishing and maintaining the capability to generate x-ray beams typical of those used both for mammography and for general diagnostic radiography, as well as state-of-the-art capability for x-ray measurements, including high-resolution x-ray spectroscopy.

OST responded to increasing interest in the medical imaging community in the use of higher-voltage x-ray spectra for digital mammography and iodine-contrast-enhanced mammography, and the absence in the literature of measurements of such spectra for the common mammography x-ray tube anode materials. In FY 2000, OST initiated a program to measure spectra from all three anode materials used in mammography at tube voltages ranging from 20 to 50 kVp. That program was continued in FY 2001, and initial results were reported at the annual meeting of the American Association of Physicists in Medicine in Salt Lake City in July. Representative spectra from a rhodium-anode tube are shown in figures 4a and 4b.

Figure 4a. Graph measuring x-ray spectra in full-scale

Fig. 4a. Measured x-ray spectra from a rhodium-anode mammography x-ray tube. Full-scale plot shows the relative intensities of characteristic lines and Bremsstrahlung continuum.

Figure 4b. Graph measuring the x-ray spectra in expanded scale

Fig. 4b. Measured x-ray spectra from a rhodium-anode mammography x-ray tube. Expanded scale shows the details of the Bremsstrahlung continuum.


The MQSA program requires that mammography facilities provide images of the phantom used in the American College of Radiology (ACR) Mammography Accreditation Program (MAP), and that the images allow the identification of a minimum number of test objects included in the phantom. Over the years, there has been a continuing criticism of the phantom on the basis that some of the test objects, particularly the small grains of crystalline matter that simulate microcalcifications, vary significantly from one phantom to the next. As part of an informal collaboration with the Pan-American Health Organization (PAHO), which is starting a pilot program to evaluate image quality and patient dose in mammography in Latin America, OST agreed to evaluate a group of 30 ACR MAP phantoms that were produced and purchased at the same time and that are representative of current production methods.

The approach is to produce multiple, extremely high quality images of the test object insert of each phantom using non-screen film, to digitize the films at high resolution, and to analyze the images by computer, using techniques already implemented. The techniques--the computerized analysis of mammographic phantom images (CAMPI) method developed by Chakraborty at the University of Pennsylvania and the RIT 315 ACR scoring routine (part of a commercial software package--both return a quantitative, numerical score for the visibility of each test object, so that the range of scores within a phantom and between phantoms for grains of the same nominal size can be determined. The range of scores on different films of the same phantom will provide information on the variability of the scoring process. The filming of the phantoms was completed during FY 2001. The analysis will be completed in FY 2002. The results will provide valuable information on the reliability of the phantom imaging portion of the ACR MAP and the FDA MQSA facility inspection program.