2021 FDA Science Forum
Software Pipelines for AI-Based Radiologic Image Analysis
- Authors:
- Center:
-
Contributing OfficeCenter for Devices and Radiological Health
Abstract
Research on segmentation, classification, and analysis of medical images in DICOM format is often hindered by the time-consuming and computationally intensive groundwork (preprocessing, exploratory analyses, feature extraction, etc.) that must be laid prior to further steps. To combat this, we are developing a Python package for the analysis of DICOM datasets, consisting of preprocessing and visualization methods, as well as deep learning and radiomics approaches to extract image features. We demonstrate the functionality of our software by preprocessing, visualizing, and analyzing the Lung1 dataset [1], which consists of CT DICOM scans from 422 non-small cell lung cancer (NSCLC) patients. In particular, our package first loads and displays the DICOM image files for interactive viewing, then performs several preprocessing and normalization steps, and finally extracts image features using different approaches as described below. We use deep learning convolutional neural networks (CNNs), pre-trained on natural images from the ImageNet dataset, to extract features. We also extract various radiomics image features such as voxel intensity distribution, 2D/3D size and shape of a region of interest, distance, symmetry and weight statistics, mesh volume analysis and neighborhood pixel analysis. In addition, we train variational autoencoder models as another feature extraction approach. Subsequently, we compare the predictive ability of these diversely extracted features with respect to classifying mortality, when different types of features are used as input to classification models such as logistic regression or XGBoost. To compare classification accuracy, we use the area under the receiver operating characteristic curve, which provides an aggregate measure of performance across all possible classification thresholds. We demonstrate these diverse preprocessing, visualization, feature extraction, and analysis processes to illustrate the use and adaptability of our Python package, which could be used on various other datasets and imaging modalities. In the future, our software package can aid the regulatory science research efforts at the FDA by providing a ready-to-use solution to preprocess, visualize and extract information from medical images.
[1] Aerts, Hugo JWL, et al. ""Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach."" Nature communications 5.1 (2014): 1-9.
