Deep Learning-enabled Natural Language Processing to Identify Directional Pharmacokinetic Drug-Drug Interactions
CDER researchers are developing large language model-based approaches to identify and characterize drug-drug interactions.
Background
Over the past decade, there has been a surge of interest in developing natural language processing (NLP) methods to automatically extract and process information from biomedical literature (including regulatory drug labels). One particular application for these methods that is under active research is the automated identification of drug-drug interactions (DDIs), which are highly prevalent and may lead to significant adverse events in clinical settings. The rapid expansion of biomedical documents containing established DDI information in natural language format, and recent advances in machine learning techniques, especially those relying on deep learning, have made it possible to extract DDIs from biomedical documents automatically.
One kind of DDI for which automated methods are clearly needed occurs when there is a change in a patient’s exposure to a drug of interest (referred to as the object drug) due to the presence of other (precipitant) drugs (Figure 1). This kind of pharmacokinetic (PK) DDI information is not only important in a clinical setting when prescribing medications, but is critical during drug development: for example, in evaluating a drug’s potential to cause heart arrhythmias, clinical and nonclinical studies are required by international regulatory guidelines to cover the so-called “high clinical exposure” scenario (defined as the expected exposure when the drug is used in the presence of intrinsic or extrinsic factors, e.g. organ impairment, DDIs, food effect etc.). Given a specific object drug, gathering information from existing biomedical literature and regulatory labels about all other drugs that could change the object drug’s clinical exposure through DDI is an important step towards determining the scenarios where there is high clinical exposure.
Figure 1: An example sentence about pharmacokinetic (PK) drug-drug interaction (DDI) involving verapamil. Green indicates object drug and magenta indicates precipitant drug. The fine-tuned model developed by CDER investigators (the BioBERT_directionalDDI model) can automatically identify the sentence as containing a PK DDI and then identify the precipitant drug(s) and the object drug.
Model
The method developed by CDER investigators is designed to automatically identify the directionality of PK DDI (i.e., which drug is the object and which is the precipitant) from natural text. The method is based on the pre-trained neural network language model BERT (Bidirectional Encoder Representations from Transformers). The researchers annotated a dataset of natural text from regulatory drug labels to label object vs. precipitant drugs in each sentence, and then fine-tuned a previously published BERT model that was pre-trained on biomedical literature (BioBERT). The resulting model (BioBERT_directionalDDI) is designed to finish the two steps sequentially: i.e., first identify a sentence that refers to a PK DDI, and then label the object drug and precipitant drug in the sentence. The first step is a sentence classification task, and the second step is a named entity recognition task: i.e., identify which entities in the sentences identified by the first step are objects or precipitants. The model is publicly available and can be found here.
Results
For the sentence classification task, the BioBERT_directionalDDI model achieved a precision of 82.7%, a recall of 80.6%, and an F-score of 81.6%.[1] This suggests that for all sentences that actually carry PK DDI information about 81% will be correctly classified by the model while the remaining 19% will be mistakenly classified as other or no DDI (meaning either no DDI information or DDI of other types such as pharmacodynamic interactions).
For the second step (identifying object vs precipitant drugs in PK DDI sentences), the BioBERT_directionalDDI model resulted in a precision of 100% for both object and precipitant entities (there were no false positives). The recall for object entities was 93.7%, and for precipitant entities it was 94.6%. The F-score for object entities was 96.7% and was 97.2% for precipitant entities. Therefore, about 94% of all entities (object and precipitant combined) are correctly identified by the model. Such high precision and recall suggest that, given a PK DDI sentence, it is very likely that this model will correctly identify the object and precipitant drugs.
Next, the CDER investigators applied the model to a specific use case: identify DDI-mediated clinical exposure changes of some reference drugs that were proposed to support the development of new cardiac safety regulatory guidelines. A sample of the results for each of the 28 reference drugs after scanning all FDA labels for prescription drugs is shown in Table 1. The number of sentences mentioning the reference drugs ranged from around 150 (bepridil) to over 30,000 (quinidine). After applying the two-step approach with the model, most of the reference drugs have anywhere between a few to over a hundred unique sentences identified where the drug appears as the object in a PK DDI. These sentences form the knowledge base that was used to provide evidence and facilitate discussion for the high clinical exposure scenario of the drug.
| Drug | Total Sentences | PK DDI Sentences | Object Sentences (Unique sentences) | Example Sentence |
|---|---|---|---|---|
| Quinidine | 31266 | 7682 | 1583 (118) | Diltiazem significantly increases the AUC of quinidine by 51%, T1/2 by 36%, and decreases its CL by 33%. |
| Dofetilide | 8346 | 645 | 601 (60) | Cimetidine at 400 mg BID (the usual prescription dose) co-administered with TIKOSYN (500 mcg BID) for 7 days has been shown to increase dofetilide plasma levels by 58%. |
| Bepridil | 149 | 0 | 0 (0) | N/A |
Table 1: Sample results from BioBERT_directionalDDI applied to all human prescription drug labels. The first column indicates the drug of interest. The second column (total sentences) shows the total number of sentences that the drug of interest appears. The third column (PK DDI sentences) shows the number of sentences where the drug appears that also contain some PK DDI information. The fourth column (object sentences) shows the total number of sentences where the drug of interest appears as the object in the PK DDI. There is a, potentially large, number of repeated sentences for each drug across all the drug labels so the number in parentheses in this column indicates the number of unique sentences. Where possible, the fifth column shows an example sentence identified by the model. Most of these example sentences have quantitative information that can help facilitate the determination of the high clinical exposure of the reference drug. Complete results for all 28 reference drugs are provided in the article referenced below.
Conclusion
This model was developed to automatically identify precipitant and object drugs involved in PK DDIs from natural text. Initially this project began to facilitate the gathering of high clinical exposure information for reference drugs during the discussion of cardiac safety regulatory guidelines. However, the model has other potential uses; for example, a comprehensive scanning of drug labels and/or literature to gather information about DDI-associated clinical exposure increases of a drug of interest could be used to help select a target clinical exposure for the drug in a first-in-human QT assessment. Also, natural text mining using this model could be used for postmarketing pharmacovigilance surveillance for specific drugs.
Further work on improving this model could involve adding the ability to query external sources during the classification steps. For example, in the case of sentences from drug labels that allude to the label drug without explicitly naming it in the sentence, we could pull the label drug name from other parts of the drug label or from a database such as RxNorm. Additionally supplementing the existing BERT-based pipelines (e.g. including FDA drug labels in the pre-training materials), may improve the model’s classification ability.
How does this research advance drug development?
This model was developed to facilitate the gathering of high clinical exposure information for reference drugs during the discussion of cardiac safety regulatory guidelines. In addition, our model could be used in specific drug development programs when the drug of interest has relevant information in other drug labels or scientific literature. For example, a comprehensive scanning of all drug labels and/or literature to gather information about DDI-associated clinical exposure increase of a drug of interest could potentially be used to help the selection of a target clinical exposure for this drug in a first-in-human QT assessment to fulfill the International Council for Harmonisation (ICH) E14 Q&A 5.1 requirement. Additionally, natural text mining using the model could be used for postmarketing pharmacovigilance surveillance for specific drugs.
[1] Precision is the number of true positives (i.e., correct identification of a sentence referring to a DDI) divided by the sum of true positives and false positives. Recall or sensitivity refers to the ability of a model to find all the relevant cases within a data set (true positives divided the sum of true positives and false negatives). The F-score combines the precision and recall scores of a model and is calculated as 2 X (precision X recall)/(precision + recall).
Reference
-
Zirkle, J., Han, X., Racz, R. et al. Deep learning-enabled natural language processing to identify directional pharmacokinetic drug–drug interactions. BMC Bioinformatics 24, 413 (2023)External Link Disclaimer.
