Using Machine Learning to Identify a Suitable Patient Population for Anakinra for the Treatment of COVID-19 Under the Emergency Use Authorization
FDA recognizes the increased use of artificial intelligence and machine learning (AI/ML) throughout the drug development life cycle and its potential to accelerate this process. For example, AI/ML approaches can be applied to clinical trial patient selection strategies during drug development to predict a patient’s clinical outcome after receiving the investigational treatment based on baseline characteristics (e.g., demographic information, clinical data, vital signs, labs, medical imaging data, and genomic data). These predictive models can identify patients more likely to have worse prognoses or patients more likely to benefit from a treatment, ultimately helping to demonstrate the effectiveness of a drug.
Background
On November 8, 2022, FDA issued an Emergency Use Authorization (EUA) for anakinra (Kineret) for the treatment of COVID-19 in hospitalized adults with pneumonia requiring supplemental oxygen (low- or high-flow oxygen) who are at risk of progressing to severe respiratory failure (SRF) and likely to have an elevated plasma soluble urokinase plasminogen activator receptor (suPAR). Anakinra is the first interleukin-1 inhibitor authorized to treat COVID-19.
The clinical efficacy and safety data used to support the issuance of an EUA for anakinra were primarily based on the SAVEMORE trial, a randomized, double-blind, placebo-controlled study conducted at 37 sites in two countries. The SAVEMORE trial enrolled adult patients with COVID-19 pneumonia who were at risk of progressing to SRF, defined as a respiratory ratio (partial oxygen pressure/fraction of inspired oxygen) below 150 mmHg, necessitating high-flow oxygen, noninvasive ventilation, or mechanical ventilation. Patients enrolled in the trial were required to have a suPAR level ≥ 6 ng/mL assessed by a test available in the two countries where SAVEMORE was conducted. suPAR is a blood protein that rises in patients with COVID-19 and has been proposed as a predictor of disease severity and outcomes.
However, an approved suPAR commercial test is not available in the U.S., which created a challenge during CDER’s review of the EUA application when identifying the patient population most likely to benefit from anakinra. To ensure patients’ timely access to this treatment, the CDER review team used AI/ML to facilitate the identification of patients who could receive the drug under the EUA1. The goal was to develop a scoring rule that would ensure a high proportion of patients meeting the criteria would have a suPAR ≥ 6 ng/mL. This was the first time that CDER used AI/ML for a regulatory decision, in this case to identify a population for a drug therapy.
Developing and Validating the Scoring Rule
The CDER review team developed the scoring rule using data from the SAVEMORE trial. The team used two AI/ML algorithms (elastic net regression and artificial neural network) independently to predict whether a patient in the SAVEMORE trial had suPAR ≥ 6 ng/mL based on baseline characteristics. The elastic net regression was used to select contributing features (clinical characteristics and common laboratory tests), and a neural network-based model was applied to independently select features and the related cutoff values. This approach was taken to ensure that patients identified by the scoring rule more closely aligned with those in the SAVEMORE trial. A final scoring rule was developed and externally validated using data from the SAVE trial, a prospective, open-label, single-arm, interventional study in which patients with lower respiratory tract infection with the virus that causes COVID-19 and having suPAR ≥ 6 ng/mL were treated with anakinra at 100 mg once daily for 10 days.
The CDER team conducted additional exploratory analyses using data from the SAVEMORE trial to evaluate whether the scoring rule could help identify patients at risk for progressing to SRF, and to evaluate the efficacy of anakinra in patients likely to have suPAR levels ≥ 6 ng/mL and worse outcomes (positive for the scoring rule) and patients who were likely to have suPAR levels < 6 ng/mL and better outcomes (negative for the scoring role).
Findings
Based on exploration of the 30 available baseline variables in the SAVEMORE trial, the CDER review team identified eight criteria for the clinical scoring rule. Patients meeting at least three criteria in Table 1 were considered likely to have suPAR ≥ 6 ng/mL at baseline. Both the elastic net regression model and neural network-based model selected the same criteria independently. In both datasets, the scoring rule showed a low false-positive rate and overall was considered appropriate to identify patients likely to have an elevated suPAR.
Table 1: Eight Criteria in the Scoring Rule to Identify Patients with suPAR Levels of 6ng/ml or Higher
Age ≥ 75 years |
Severe pneumonia by WHO criteria |
Current/previous smoking status |
Sequential Organ Failure Assessment (SOFA) score ≥3 |
Neutrophil-to-lymphocyte ratio (NLR) ≥7 |
Hemoglobin≤10.5g/dl |
Medical history of ischemic stroke |
Blood urea ≥50 mg/dl and/or medical history of renal disease |
suPAR, soluble urokinase plasminogen activator receptor; WHO, World Health Organization
The CDER team also conducted exploratory efficacy analyses in the SAVEMORE trial to measure illness severity and all-cause mortality by day 28 and all-cause mortality by day 60, comparing anakinra to placebo in the subgroups of patients defined by the scoring rule status. In both the score-positive and score-negative subgroups, participants treated with anakinra had lower odds for more severe disease at day 28 and day 60 compared with those who received placebo.
Based on these exploratory analyses, patients in the SAVEMORE trial who were score-positive appeared to benefit from treatment with anakinra consistent with the overall studied population. It was unclear if patients who were negative for the scoring rule would also benefit. This means identifying patients using the scoring rule to treat with anakinra could potentially exclude patients who might also benefit from anakinra treatment. However, the scoring rule also identifies patients who have a higher risk of progressing to SRF.
Conclusion
In this case, the CDER review team combined the predictive ability of AI/ML with appropriate validation processes to develop a method to identify the patient population who will likely benefit from anakinra treatment under this EUA. These findings show that identifying patients using the scoring rule can increase the probability that these patients will experience the benefits seen in the SAVEMORE trial.
As a result, a patient identification method was developed and described in section 1.1 (Patient Population Identification) of the Fact Sheet for Healthcare Providers: Emergency Use Authorization for anakinra.
One limitation to this approach is the exploratory nature of the development of the scoring rule and its low sensitivity, meaning that some patients who may benefit from treatment with anakinra will not be identified by this scoring rule.
The CDER review team expects the scoring rule to be easy to understand and implement by healthcare providers. Similar approaches could potentially be applied in other situations during drug development, for example, to help with clinical trial patient selection strategies.
As this research shows, AI/ML can be powerful tools to facilitate drug development and regulatory decision-making.
1 Liu, Q, Nair, R, Huang, R, Zhu, H, Anderson, A., Belen, O, Tran, V, Chiu, R, Higgins, K, Chen, J, He, L, Doddapaneni, S, Huang, SM, Nikolov, NP, & Zineh, I, 2024, Using Machine Learning to Determine a Suitable Patient Population for Anakinra for the Treatment of COVID-19 Under the Emergency Use Authorization, Clin Pharmacol Ther, 115(4): 890–895. doi.org/10.1002/cpt.3191