Real World Evidence - From Safety to a Potential Tool for Advancing Innovative Ways to Develop New Medical Therapies

 

Real world evidence has the potential to enhance the efficiency of drug development and provide new evidence on risks and benefits of medical products.

What is real world evidence?

Jacqueline: To understand real world evidence (RWE), you have to first understand real world data (RWD). RWD, according to the FDA’s definition, are data that are routinely collected about a patient's health status, including data that the doctor is recording during an office visit, information submitted to an insurer, and data from product and disease registries. RWD can also be data that are collected outside the health care setting — for instance, data from mobile technologies, including wearables and sensors that gather information on our movements and what we're doing, as well as data from social media platforms.

This RWD can then potentially be used to generate scientific evidence that the FDA can use to make decisions about the safety and effectiveness of medical products — such as finding new safety issues with a drug after it is approved and on the market or helping to determine the effectiveness of a drug for a new indication. I say that RWD can potentially be used for assessments of effectiveness because we are still evaluating and developing guidance on its role in doing this. The use of RWD to determine effectiveness of medical products is an emerging science that many people in the drug development world believe can help bring safe and effective new drug therapies to market more efficiently.

What types of study designs can researchers use to generate RWE?

David: Traditionally, FDA evaluates the safety and effectiveness of a drug based on what are known as “randomized controlled clinical trials.”

Such clinical trials work to isolate the effect of the drug on a disease or condition and try to reduce any “external” influences on these effects, such as other treatments, illnesses that a patient may have, and patient and physician expectations. The people in a clinical trial, often called subjects or research participants, are carefully selected. In order to more precisely assess the effect of a treatment in a trial, researchers often exclude people from the trial who have other conditions that may make it difficult to assess the effect of the drug.

On the other hand, FDA encourages investigators to diversify the subjects by gender, age, ethnicity, and, in some cases, by medical conditions, such as kidney disease, when medically appropriate. Despite these efforts, however, participants in many clinical trials do not fully represent the patient population that will take the drug when approved.

Jacqueline: RWE studies and clinical trials are not necessarily an “either-or” situation. In fact, researchers may combine the use of RWD and randomized clinical trials when investigating medical products. For instance, for a clinical trial involving a rare disease, researchers may use RWD to help recruit participants by identifying the geographic location of patients; determining whether the trial as designed can enroll enough participants; and providing other important data about the patient population. The identified patients can then be enrolled in a conventional trial.

A clinical trial can also generate RWE by making use of the RWD in clinical practice to assess study outcomes. By integrating a clinical trial into the setting where patients receive their clinical care, sponsors can both randomly assign participants and capture relevant clinical endpoints from the records of their clinical care. Such trials can also integrate digital technologies to capture data on patients as they go about their daily lives.

David: When randomized controlled clinical trials are not feasible, other uses of RWD to generate RWE can come into play. Researchers seeking to study drugs to treat serious rare diseases and some types of cancer may run into difficulty trying to design randomized controlled clinical trials because they cannot find enough people for a trial or there is no available comparison drug to use in a second arm (and patients are reluctant to enter a trial with a no-treatment [placebo] arm). Researchers may also be reluctant to use a no-treatment control arm when there is early evidence that the investigational drug is highly effective. In such cases, researchers may want to compare how users of the investigational drug fare compared to the normal course of the disease using an "external control," a control that can be persuasive when the drug effect is large. In a case like this, researchers might find medical records of similar patients who did not receive the investigational drug and compare these patients to research participants in the trial who received the investigational drug. With electronic medical records, it may be easier to generate such external controls when appropriate.

Finally, researchers are interested in using “non-interventional study designs” to generate RWE, sometimes called epidemiologic or observational studies. In these studies, the provider and patient choose the treatment — in other words, treatment is not assigned at random as is done in most traditional trials. This could be done prospectively; in other words, a decision would be made to identify patients who take a particular drug and then follow their clinical course over time, or retrospectively, where patients have already received treatment and the outcome is known and the researchers go into patients’ records and obtain the clinical information.

We have seen an example of this from FDA’s Center for Biologic Research and Evaluation (CBER). When CBER approved a vaccine for shingles, there was limited long-term data on effectiveness in the 50-59-year-old population. The sponsor undertook a prospective, observational study in which patients who received the shingles vaccine as part of normal care were followed over time to see if the vaccine prevented shingles. The data were provided to FDA and the approved label was updated with longer-term effectiveness data.

Of course, comparing outcomes of patients who receive one medication versus another in clinical practice can be misleading. There are a number of reasons a clinician may choose a certain medication for a patient, including a preexisting belief that the drug will work well for that particular patient, other medical conditions or treatments, concerns about medication adherence and appropriate use, concerns about financial burden, and formulary restrictions. Some of these factors may also influence the clinical outcomes with the medication, which are subsequently observed. Randomization is designed to prevent any of these factors from influencing who receives the particular medication and thus distribute these potential confounding factors evenly to each trial arm at the start of the trial (baseline). Without randomization, other statistical methods must be employed, but they often cannot address the comparability of the treatment groups as well as randomization can. Because of the possibility of not accounting for all these factors, such studies may potentially be convincing only when the effect sizes are sufficiently large that any such bias due to non-random treatment assignment would not be expected to alter the conclusion.

How do we know if results from RWE studies are providing answers that are valid?

Jacqueline: There are different considerations depending upon the study design. For randomized trials, there is less concern about bias due to differences in the characteristics between the populations being compared than in non-interventional studies. However, in real world trials, there is often not masking (blinding) of the intervention in clinical practice, and the patients and their providers know which drug they are taking. This might bias outcomes if patients or their providers have preexisting beliefs about the benefit of one drug versus another. In addition, when a clinical outcome is derived from RWD, FDA must be confident that the RWD sources provide an accurate description of this clinical outcome. FDA currently relies on full data access to assure the capacity to evaluate clinical trial analyses. Data access and replication will remain important considerations for RWE studies.

David: As we said, it is more difficult to evaluate RWE studies that lack randomization. Advanced epidemiologic and statistical methods are used to address important differences among the populations to be compared. Some researchers have expressed concern that these methods are not sufficient to address differences that might remain because potentially important differences may not be measured in RWD and therefore cannot be balanced. In order to examine if, and under what circumstances, non-interventional RWE can provide credible evidence of drug effectiveness, FDA and other stakeholders are designing RWE studies that ask questions similar to ongoing or completed clinical trials and comparing results. One such project, RCT Duplicate, is attempting to duplicate the results of recently completed clinical trials using RWE studies. Approximately 40 trials have been identified for potential duplication. Another 10 ongoing trials will be duplicated before the clinical trial results are reported. A separate project with the Yale-Mayo CERSI will attempt to duplicate several more trials using medical claims and electronic health record data. This work may increase or decrease confidence in the validity of non-interventional RWE, and it may also suggest which techniques are best aligned with different types of drug effectiveness questions.

Can you share any specific examples of how real world evidence has been used by FDA to make a regulatory decision?

Jacqueline: RWE has been used for many years to evaluate safety issues for drugs already approved by the FDA. For example, researchers studied whether FDA-approved drugs used to treat attention deficit-hyperactivity disorder (ADHD) could cause serious cardiovascular events in children and young adults. A clinical trial to test these drugs for this adverse event presented many challenges. For instance, because serious cardiovascular events are rare in this patient population — only about 3 in every 100,000 children per year — a clinical trial would have had to have been unfeasibly large to assess a potential drug effect. Moreover, it is unlikely that caregivers would consent to randomization to no treatment in patients presumed to be in need of ADHD treatment. Instead, using electronic health care data from four health plans, researchers studied RWE for more than a million children and young adults and the results suggested that current users of drugs for ADHD did not have an increased risk of serious cardiovascular outcomes.

Many studies conducted through FDA’s Sentinel Initiative inform regulatory decisions on safety. The Sentinel Initiative is FDA’s electronic surveillance tool that started in pilot form in 2008 and is now a fully functional program that monitors FDA-regulated medical products.

RWE to establish effectiveness is still an emerging science, but there are several examples where historical controls were derived from RWD.

For instance, a drug called blinatumomab, approved in 2014 under the trade name Blincyto for a type of blood cancer called acute lymphoblastic leukemia (ALL), was approved initially under accelerated approval based on a single-arm trial where everyone received the experimental therapy, and evidence of effectiveness was based on complete remission (CR) and duration of CR. This response to the drug was compared to the records of a population of patients not treated with the drug at several U.S. and European clinical centers. The clinical benefit of the drug was subsequently established with a randomized clinical trial against standard of care establishing a benefit in overall survival. In addition, when Myozyme was approved for Pompe disease, it was based on a study of comparing survival and rate of ventilator support in 18 patients who received Myozyme compared to a historical control group of 62 patients derived from medical charts.

Disease registries are another source of RWD. Disease registries capture clinical outcomes of a defined population of patients. In the approval of cerliponase alfa for a rare pediatric neurologic disease that leads to early death, the evidence for approval came from a single-arm trial of 24 patients, based on an assessment of ability to prevent motor function decrease using a disease specific measurement tool. Compared to 42 untreated children from a natural history disease registry, the drug was shown to significantly slow the progression of motor function decline.

Sponsors continue to approach FDA with novel ideas for using RWE in applications. In these situations, FDA’s RWE Subcommittee of CDER’s Medical Policy and Program Review Committee will usually meet with the clinical divisions evaluating the proposal to help evaluate it.

So, RWE has long been used to assess safety issues but it’s relatively new for studying the effectiveness of drugs. Why is that?

Jacqueline: RWE has been an effective tool for safety, although it is not the only tool. Initial safety reports may come through spontaneous reports. These are useful for identifying a potential safety signal but one of the advantages of RWD is that we typically have a denominator — in other words, if RWE from a health system tells us there 20 adverse events from a drug, it can usually also tell us if that is 20 out of 100 or 20 out of a million, which makes a big difference. In addition, the adverse events of interest are often captured reliably in RWD, particularly serious adverse events that lead to a patient seeking medical care to address the event. Of course, RWE may not always be sufficient for safety, and clinical trials with prospective data collection may be needed, especially for less serious or obvious adverse events that may not be reliably reported in RWD. In the area of effectiveness, one challenge that limits the use of RWE is that the clinical measures that are used to determine whether a drug is effective are either not used in clinical practice or are not captured consistently and at time points that enable an accurate assessment of effectiveness.

Can you comment on the FDA MyStudies App? How can this open source resource be used?

David: The FDA MyStudies App was designed to allow data to be collected from patients or other reporters (such as research staff) in a regulatory compliant manner in traditional clinical trials or RWE studies. Because the code and technical documentation are open source, sponsors, CROs, technology companies, and others may re-brand and re-configure the app for use in their trials and studies with reduced software development effort relative to building a similar system from the ground up. A configuration portal enables use across various conditions and health outcomes. The code was recently updated in conjunction with re-branding and configuration for use in the LimitJIA clinical trial supported by the Patient Centered Outcomes Research Institute and a registry supported by the Crohn’s and Colitis Foundation. Public and private sector groups that re-configure FDA MyStudies are not obligated to notify FDA. However, individuals who visit the GitHub site may post questions regarding the technical materials and will receive responses from the FDA MyStudies team. Recently, Google announced that it will provide a “click-to-deploy” FDA MyStudies option, which may reduce technical hurdles for organizations with limited software development capacity.

It’s clear why real world evidence offers so much potential, but why has it become such a hot topic lately?

David: The FDA is mandated by Congress to develop guidance for RWE under the 21st Century Cures Act, which was passed in late 2016. But that is not really the only factor driving it. The increased focus is also tightly connected to technological advances. Since the Health Information Technology for Economic and Clinical Health (HITECH) Act was passed in 2009, many more physicians and medical facilities are using electronic health records. We are also better able to gather information from images, pathology reports, radiology reports, cardiac imaging reports, next generation sequencing, and other data sources. This has given the clinical research world and health systems access to rich sources of data in an electronic format that in the past existed only in clinical trial data sets. And the explosion in social media and mobile technology, sensors, and wearables is obviously revolutionizing the capacity to obtain data directly from patients. In addition, data sources such as continuous glucose monitoring for diabetes patients have the potential to replace “snapshots” of a patient’s status with a potentially broader view of how the medication is working. The questions about just how to use these sources for evidence generation are under consideration.

You mentioned RWE is still an emerging science, what do you mean by that?

Jacqueline: There is still a long way to go before the scientific world can maximize the potential uses of RWD. Currently, there are many limitations to its use. I think the biggest challenge is that RWD is not intended for research. Much of the RWD that we are seeking to use comes from medical records designed for physicians taking care of their patients — and what they put in the chart is what they think is important. Although there is some consistency with how physicians record their patient care activities, there's probably more inconsistency than consistency. Unstructured data, like doctor’s notes, and scanned reports are more difficult to harmonize across systems without additional curation of some kind, and in many cases, we may need to connect the dots (really datapoints) to get a full picture of a patient’s experience. Sharing this data among different electronic systems holds promise but is not fully developed. As this field grows, hopefully the quality of the data will improve opening up more opportunities to generate RWE and improve everyday clinical care. Of course, study design remains critical. The “easiest” use is use of RWD for enrollment criteria and to assess potential enrichment characteristics in a randomized trial. Whether the data can, in a non-randomized study, support effectiveness conclusions is a critical question.

What does FDA envision for the future of RWE?

Jacqueline: If RWE evolves as an accurate assessment of effectiveness in certain clinical cases, then its role in supporting approvals for new indications for drugs already on the market will grow more robust. It also will continue to highlight potential safety issues in approved drugs and potentially be used as a hybrid with clinical trials. And as the FDA uses RWE studies in its decision-making process, more companies may be likely to conduct RWE analyses, strengthening this area of research.