2021 FDA Science Forum
Evaluation of CADt Devices Using Queueing Theory
- Authors:
- Center:
-
Contributing OfficeCenter for Devices and Radiological Health
Abstract
Keywords
Computer-aided triage devices, CADt, queueing theory, clinical workflow, waiting time
Background
Radiological Computer-Aided Triage and Notification (CADt) software is an image processing prescription device powered by artificial intelligence (AI) to process, prioritize, and triage radiological medical images based on the likelihood of disease condition. When an effective CADt software is utilized, patients with more life-threatening conditions are more likely to be evaluated sooner, improving radiologist workflow and facilitating earlier diagnosis and treatment of time-sensitive diseases. For example, in patients suspected to have large vessel occlusion (LVO), a notification from an effective CADt device can allow a neuro-interventionalist to emergently remove the clot, reducing the associated morbidity and mortality (“time is brain”). However, as CADt devices become more common in daily clinical workflow, questions and concerns remain regarding to a rigorous, quantitative assessment of CADt effectiveness.
Purpose
To improve clinical evaluation of CADt devices, the authors investigate the use of queueing theory, an established mathematical framework that studies properties of waiting in line, for characterizing CADt performance under various clinical environments. Simulation and theoretical computation tools will be developed and made publicly available to evaluate and potentially optimize the effectiveness of future CADt devices.
Methodology
Queueing theory is applied to quantitatively assess the amount of time saved for diseased patients detected by the CADt device and the time delay for diseased patients undetected by the device. The relationship between time performance and a wide range of clinical parameters, such as disease prevalence, device accuracy, patient arrival rate, and the number of radiologists, is investigated. Evaluations are made in a simulated yet realistic clinical environment.
Results
The simulation model suggests that CADt devices are most effective when a small subset of images is prioritized, over a relatively large population, in a busy clinical environment. These preliminary results are consistent with both clinical intuition and the theoretical computation using queueing theory under different simulation conditions.
Conclusion
Queueing theory is a well-established methodology in computer science and can be applied to improve clinical evaluation of CADt devices.
