U.S. flag An official website of the United States government

U.S. Food and Drug Administration
  1. Home
  2. Drugs
  3. Science and Research | Drugs
  4. Regulatory Science in Action
  5. Impact Story: Capturing Patient Experience Through Deep Learning
  1. Regulatory Science in Action

Impact Story: Capturing Patient Experience Through Deep Learning

Regulatory Science Impact Story

 

CDER researchers are developing methods based on deep learning, a kind of artificial intelligence (AI), to capture patient experience of medical products in a way that allows essential information to be communicated, retrieved, and analyzed.

The development, evaluation, and regulation of drugs, biologics, and medical devices hinges on capturing and understanding information about how patients experience these products. This information is both complex and vast. For example, regulators receive information about patient responses or reactions to drug products from thousands of adverse event reports that physicians make to FDA each year through the FDA Adverse Event Reporting System (FAERS). All these reports must be carefully evaluated to ensure the long-term safety of drug products. To cite another example, the results of a clinical trial in an application for a new drug could consist in part of the experiences of thousands of patients, each with individual differences that should be considered.

Figure 1. In coding into MedDRA, lower-level terms (LLTs) are selected from texts in regulatory submissions (e.g., descriptions of patient experiences in a trial, physician reports of adverse events). LLTs are linked to a smaller set of preferred terms (PTs), each of which has a predetermined link to one or more system or organ classes (SOCs) via a high-level term (HLT) and a high-level group term (HLGT). Using this logical structure and consistently selecting correct LLTs from narratives enable retrieval, sharing, and analysis of what could otherwise be unmanageable information in the free text. But this depends on accurate translation into the MedDRA terminology through correct choices of LLTs. As an example of a coding challenge, both narratives at left describe a patient not receiving a scheduled drug, but the term “drug dose omission” in MedDRA would apply only to a medication error and not to a clinical decision not to administer the dose. The example is from MedDRA® Term Selection: Points to Consider.Figure 1. In coding into MedDRA, lower-level terms (LLTs) are selected from texts in regulatory submissions (e.g., descriptions of patient experiences in a trial, physician reports of adverse events). LLTs are linked to a smaller set of preferred terms (PTs), each of which has a predetermined link to one or more system or organ classes (SOCs) via a high-level term (HLT) and a high-level group term (HLGT). Using this logical structure and consistently selecting correct LLTs from narratives enable retrieval, sharing, and analysis of what could otherwise be unmanageable information in the free text. But this depends on accurate translation into the MedDRA terminology through correct choices of LLTs. As an example of a coding challenge, both narratives at left describe a patient not receiving a scheduled drug, but the term “drug dose omission” in MedDRA would apply only to a medication error and not to a clinical decision not to administer the dose. The example is from MedDRA® Term Selection: Points to Considerdisclaimer icon.

MedDRA, a common terminology for patient information

If patient information is to be communicated, retrieved, and analyzed efficiently and accurately, a common language—one that regulatory authorities and the drug development community share—is needed. For example, if the clinicians conducting drug trials around the world used inconsistent language to describe cardiac arrhythmia, it would be impossible to quantify the arrhythmic effects of drugs based on reports the clinicians submitted to FDA. MedDRA (Medical Dictionary for Regulatory Activities)disclaimer icon is widely used as a standardized dictionary of terms to describe adverse events of drugs and therapeutic biologic products. In MedDRA, tens of thousands of medical terms are linked in a way that captures logical relationships among them, making it possible to create meaningful summaries and conduct quantitative analysis (e.g., determining the frequency of various categories of adverse events and how they are associated with certain drugs). For regulatory purposes, patient information must be coded into MedDRA. Currently, for MedDRA coding via correct term selection (Figure 1), unstructured data in various documents needs to be aggregated and reviewed by humans, in a process that is ineffective and time-consuming. FDA reviewers devote considerable time and resources to checking the accuracy of the MedDRA-coded information submitted to the agency by sponsors.

Deep learning to code patient information accurately

Figure 2. In deep learning, artificial neurons (white circles) are connected in neural networks with one or more “hidden” layers of neurons between the input layer (where the data is first entered) and the output layer that signals which choice is to be made in a learning task (in this case, recognition of human faces). By using a training set in which the desired final output is known, inputs and thus outputs (represented by the lines connecting circles) in these networks can be adjusted stepwise so that the network learns to make good decisions consistently. A key characteristic of this learning approach is that successive hidden layers learn to recognize features of greater and greater complexity by combining simpler features from the previous layer. Examples of features captured in successive hidden layers might be edge, eye, and face (as shown here for facial recognition) or waveform, phoneme, word, and sentence for speech recognition. In practice, these networks can consist of thousands of neurons with many hidden layers and features.

Figure 2. In deep learning, artificial neurons (white circles) are connected in neural networks with one or more “hidden” layers of neurons between the input layer (where the data is first entered) and the output layer that signals which choice is to be made in a learning task (in this case, recognition of human faces). By using a training set in which the desired final output is known, inputs and thus outputs (represented by the lines connecting circles) in these networks can be adjusted stepwise so that the network learns to make good decisions consistently. A key characteristic of this learning approach is that successive hidden layers learn to recognize features of greater and greater complexity by combining simpler features from the previous layer. Examples of features captured in successive hidden layers might be edge, eye, and face (as shown here for facial recognition) or waveform, phoneme, word, and sentence for speech recognition. In practice, these networks can consist of thousands of neurons with many hidden layers and features. Learn moredisclaimer icon.

FDA researchers are developing computational approaches to use deep learning, a form of artificial intelligence (AI), to extract standard MedDRA terms automatically from large-scale free-text documents. Deep learning - machine learning that relies on artificial neural networks—has been applied to many activities performed by humans, including image and speech recognition and game playing. In fact, in many tasks of great complexity, deep learning has surpassed human performance. Like neurons in the brain, artificial neurons in deep learning receive inputs, process the inputs, and transmit signals to other neurons. “Deep” in deep learning refers to the organization of the artificial neurons into multiple layers with “hidden layers” learning to identify various kinds of attributes or “features” that can be composed into a set of more complex features in the next layer (Figure 2). Essentially, these networks can learn to recognize patterns in data.

This learning approach depends on large data sets that allow for the training of the neural network to achieve a task. For example, if a set consists of thousands of photographs, some of which are of cats and some of which are not, the connections between neurons in a neural network can be modified in a systematic way based on the accuracy of the outputs so that eventually, the network can reliably distinguish between images that contain cats and those that do not.

Deep learning to code patient information accurately

Figure 3. Natural language processing encompasses multiple techniques (as indicated in the outer circles) for processing unstructured data into a form that can be used by a deep learning model. Combinations of these techniques are used, depending on the type of analysis that needs to be implemented. For instance, sentiment analysis can be used to analyze customer satisfaction (e.g., happy versus sad). After being structured with these techniques, the data are ready as input to a deep learning model for completion of the learning task (e.g., the coding of patient narratives into MedDRA terminology).

Figure 3. Natural language processing encompasses multiple techniques (as indicated in the outer circles) for processing unstructured data into a form that can be used by a deep learning model. Combinations of these techniques are used, depending on the type of analysis that needs to be implemented. For instance, sentiment analysis can be used to analyze customer satisfaction (e.g., happy versus sad). After being structured with these techniques, the data are ready as input to a deep learning model for completion of the learning task (e.g., the coding of patient narratives into MedDRA terminology).

CDER researchers will use thousands of correctly coded patient narratives in new drug application (NDA) submissions to train a neural network to make correct MedDRA coding decisions. Before this can be done, the unstructured data in patient narratives must be extracted, processed, and structured so that it can be used by a deep learning model. CDER researchers are using diverse natural language processingdisclaimer icon techniques (Figure 3), including sentiment analysis, machine learning, summarization, and entity and fact extraction, to parse the free text and convert the data in a patient narrative to a usable and analyzable form for deep learning.

Developing a user-friendly tool for MedDRA coding

In applying deep learning to MedDRA coding, CDER researchers will assess a variety of network architectures to optimize coding performance, including various kinds of recurrent networks that can be thought of as neural networks with a memory capability (these networks may be especially suitable for natural language processing tasks). The coding results from a variety of network architectures will be compared and their performance measured against that of expert reviewers selecting MedDRA terms from patient narratives.

Ultimately, a user-friendly tool will be developed to let users automatically choose preferred terms from free-text descriptions in newly submitted regulatory documents. This tool could also be used to check the coding accuracy of previous submissions, such as adverse event reports from drug developers.

How can this effort advance drug development and evaluation?

Much of FDA’s regulatory oversight involves the interpretation of patient narratives that capture how patients experience drug products. For this information to be analyzed accurately, it must be correctly encoded into a common language. This coding is currently done by sponsors and reviewers, but the process is laborious and prone to error. Recent successes at human tasks by a kind of AI known as deep learning suggest that this approach could lead to better coding and thus improved knowledge of patient experience of drug products in both pre- and post-marketing phases of drug development. CDER’s efforts to develop the capability to code patient narratives through deep learning is one of several CDER projects in the category of knowledge management, whereby researchers are developing new capabilities and systems to interpret the huge amount of data that FDA receives and creates in evaluating the safety and efficacy of medical products.

 

 
Back to Top