2004N-0181 - Critical Path Initiative; Establishment of Docket
FDA Comment Number : EC9
Submitter : Dr. Jonathan Morris Date & Time: 08/02/2004 06:08:54
Organization : ProSanos Corporation
Private Industry
Category :
Issue Areas/Comments
Comment Issues
6. For each solution identified, please indicate which could be accomplished quickly, in less than 24 months, and which require a long-term approach?
During the first six months integration of data could take place on a small scale, as a series of small projects. Critical initial steps include: identifying appropriate data sources; obtaining access to the needed data, de-identifying the data, preparing the data repository, and implementing the analysis data model for the system. This phase could start with the horizontal integration of data (i.e. similar drugs in a single class) or the vertical integration of data (single drug across development phases). Small scale implementation would allow for quicker results, and would provide an opportunity to show value before full-scale integration was pursued. Additional drug classes and drug experience data could be added once proof of concept was accomplished.

An additional six to twelve months would be needed to develop the production version of the described analytical tools.

Establishment of the network would take 12 to 24 months, and could begin during analytical tool development. This would include recruiting members, obtaining approval from the institutions, developing standard operating procedures, soliciting feedback from the community, etc.

At the end of twenty-four months, it would be possible for the Agency to begin to pose questions raised by analysis of integrated data to the Confirmatory Safety Network. This safety assessment mechanism would be an enhancement to the analysis of ongoing submissions to AERS.
7. For each problem identified, what role should FDA play and what role should be played by others?
Implementation of the solution would require cooperation among three parties: the FDA, the vendor, and the Confirmatory Safety Network.

The FDA would:
?Xassist the vendor(s) with identification of high value subject areas
?Xwork with the vendor(s) to determine the most useful databases to integrate
?Xassist the vendor(s) with obtaining access to appropriate data sets
?Xprovide scientific input to help in understanding the data and the regulatory issues
?Xleverage existing relationships with industry thought leaders to assist the vendor in establishing the Confirmatory Safety Network

The vendor(s) would:
?Xcollect and integrate the relevant clinical data
?Xdevelop new tools to analyze the integrated data
?Xperform analysis of the integrated data
?Xidentify/develop tools to support distribution of questions and subsequent data collection from the Confirmatory Safety Network
?Xdevelop, install, and support a technology solution that would link the FDA and the Confirmatory Safety Network
?Xprovide training to users

The Confirmatory Safety Network would:
?Xcapture and report safety data to address questions posed by the FDA
?Xinput data into vendor-supplied data collection tools
?Xuse their expertise and knowledge to assist in addressing questions posed by the FDA
4. For each problem identified, if a solution would facilitate the development of drugs, biologics, and/or devices for a particular disease or categories of disease, please indicate which diseases would be affected?
Integration and analysis of safety data has general applicability to most, if not all, drug types and diseases, as well as to biologics and devices.
3. For each problem identified, please indicate the type of drug, biologic, or device to which the hurdle applies.
Integration of relevant clinical data and improved analysis of the data could potentially benefit most types of drugs, biologics and devices. Integration of initial data sets could focus on areas with the largest potential benefit, such as:
(1)products with frequent safety questions and concerns
(2)products with large volumes of data
(3)products used by a large percentage of the population (e.g., statins)
(4)products used in generally healthy populations where the risk to benefit ratio is questionable
1. Hurdle Identification. Please describe the product development issue, the nature of the evaluation tool that is out-of-date or absent, how this problem hinders product development, and how a solution would improve the product development process.
Product development issue:
The design of clinical trials to evaluate drug safety depends critically on prior information and assumptions about risks associated with a drug or class of drugs. Relevant information that could lead to optimal trial design or identification of safety signals may have been captured in the tremendous data resources that exist throughout the drug development industry, the NIH, and within the regulatory agencies (e.g. FDA and EMEA). These resources are not being effectively used because of the lack of integration of the data, and the lack of mathematical tools to perform signal and pattern detection in integrated data.

The absence of an analytical environment which: a) supports the integration of wide ranges of clinical databases that contain potential safety signals, b) leverages advanced mathematics and analytical tools to identify patterns of signals in the data, and c) pragmatically translates these observations back into the drug development process (pre- or post-launch), severely limits the use of prior information. This information could be used to design, implement, and evaluate efficient trials targeted in the areas of greatest risk. Innovation in clinical trial design could be spurred by the analysis of larger databases, particularly for rare events, and the development of more sophisticated analytical tools to examine the data.

Nature of the evaluation tool that is out-of-date or absent:
There is a need for a comprehensive system and process for integrating and analyzing clinical data, and for using the results of the analysis to more quickly address safety concerns and improve future trials. The incorporation of prior clinical information into the design of clinical trials now relies on a patchwork of informal methods, where both sponsors and regulators evaluate final study reports and other documents without the easy ability to integrate and query the original electronic data. There may be important patterns that are not detected as a result of a lack of data integration and appropriate analytical tools. This is a consequence of the number and variety of systems, formats, languages, etc. that exist for collecting, storing and analyzing clinical trials and registry data.

For these reasons, there is benefit to be gained from the development of a comprehensive system and set of processes that integrates and analyzes relevant clinical data sets in order to more broadly examine potential safety issues. Combining, for example, all data relating to one product or one class of products, or all data relating to one indication, could provide an opportunity to identify previously undetected patterns of safety events, or
could create a clearer picture of potential drug/outcome associations.

Examples of clinical databases that should be combined and analyzed in this type of integrated fashion include:
?XFDA (US) with EMEA (Europe) experience with a particular compound;
?XPhase II and III (Pre-market) with Phase IV (Post-market) safety data;
?XPhase III and IV for one drug of a certain class (e.g. a statin), with Phase III and IV for another drug of that class.

The integration of widely disparate clinical data across products and within classes of drugs, a challenging task in and of itself, will create the need for new analytical tools that will allow for performance of statistical comparisons of products and signals, tailored to this different type of data. The analytical tools that exist today, which have been optimized for randomized, controlled clinical trials are not designed to detect the patterns that will become apparent once the disparate clinical data is integrated.

Once the data is integrated, and the appropriate tools are applied to identify possible associations between products and safety events, safety concerns and cause/effect questions may be raised by the Agency.

Please see "General Comment" section for the rest of the answer to this question.
5. Nature of the Solution. For each problem identified, please describe the evaluation tool that would solve the problem and the work necessary to create and implement the tool/solution. For example, would a solution come from scientific research to
There are three components to the proposed solution: 1) the integration of disparate clinical data relevant to safety; 2) the creation of analytical software tools designed for the complexity and the nuances of the combined data, and 3) the formation and management of a Confirmatory Safety Network to address questions and collect data that can quickly test new safety associations and drug/outcome questions in clinical practice.

The first component, the integration of safety databases, involves a variety of challenges. Differences in format, terminology, units of measurement, etc., as well as more subtle discrepancies that are the result of differences in medical practice in various places, times, and settings, will need to be addressed. Format and terminology issues can partially be addressed through harmonization with HL/7 RCRIM, CDISC, and ICH standards development efforts. Yet, these standards alone are not enough to integrate the databases. There will be a need to include legacy data. And integration will also require scientific and clinical expertise to ensure that appropriate integration decisions are made. Automated software Extract, Transform, & Load (ETL) tools can also be used to enhance data integration and quality assurance. For example, statistical software can be written to !?flag!? large differences in the rate of a particular adverse event / drug combination between two safety databases. Such a difference could indicate a data quality problem, or a difference in medical practices underlying the two data sources.

The second component, new analytical and visualization tools and techniques, will be needed to digest these combined safety databases. Such databases can be exploited in two ways. Multiple databases can be used to reinforce each other, to strengthen conclusions by confirming the conclusions on several independent data sources, with an increased overall number of events for consideration. Alternatively, discrepant reporting rates among data sources may indicate under-reporting, counter-detailing, or a similar data quality problem. These discrepancies may also indicate differences in demographics or medical practice that may shed light on complex safety issues such as drug-drug, drug-gene, or drug-lifestyle interactions.

Improvements in the quality of safety data are needed. Adverse event reports must be filtered to ensure that they relate to true safety problems, and not to the indication for the drug. Statistical techniques must be developed to distinguish drugs which cause adverse events from !?innocent bystanders!?, concomitant medications that are frequently used with drugs in question. A combination of standardization procedures and statistical algorithms must be used to ensure that safety signals are not washed out because several categories are used to encode a particular adverse event.

There are additional dimensions of integrated data to consider which preclude standard statistical analysis methods, and which have been used successfully in other non-life science disciplines. Examples include synchronous detection, contextual classification, relationship modeling, and pattern-class discrepancies.

The third component of the solution is the establishment of a post-market Confirmatory Safety
Network linked together electronically, which can be used to quickly and efficiently address specific safety questions. Questions would be addressed through web-based specific data questions targeted at the therapeutic area and the prescribing community which can best address the suspected associations. Data could be gathered through the use of simple and secure web-based Case Report Forms, and the results could be analyzed and reported back to the FDA quickly and efficiently.

The result of the successful implementation of this !YConfirmatory Safety System!| could be the evolution of new industry processes to effectively evaluate and use post-market clinical data.
2. Please rank each hurdle identified in Question 1, above, in priority order according to which hurdles create the most severe product development problems.
We have identified only one hurdle.
8. What factors should guide FDA in setting priorities among the hurdles and solutions identified?
We have suggested a progressive solution, starting with the integration of data, and continuing with the development and testing of new analytic tools, and the formation of a Confirmatory Safety Network.
Continuation of Hurdle Identification answer:

A complete and comprehensive safety system would include a mechanism to quickly address concerns and questions identified through the data analysis described. A network of treatment providers (a Confirmatory Safety Network) established specifically to address questions and test possible associations being seen in clinical practice could make a significant contribution to reducing the response time required to determine if events are isolated or in fact represent a true safety concern.

These three components: (1) integrated data, (2) tools to determine patterns and associations, and (3) a provider network to address subsequent questions that are a result of analysis, can be combined to create a Confirmatory Safety System. The Confirmatory Safety System will provide improved safety signal detection through the identification of previously undetected patterns as integrated data is analyzed, and will act as a feedback loop to address questions and improve future trials.

How this problem hinders product development:

In the absence of an integrated database of relevant data, and the tools to go with it, product risk assessment is less efficient, less organized, and less accurate than it could be. Important early-warning safety signals can be missed when single databases are analyzed in isolation from one another. Integration of data would provide a larger database for analysis, which could lead to improved identification of previously undetected patterns, such as drug interactions. Specific patient populations which need to be studied further or which should be specifically excluded from future trials may also be more easily identified.

These newly-identified safety signals can be used to design more specific and focused trials. As a result, the size of studies may be reduced, the time required to take a product to market may be shorter, and the drug may be approved having had a more comprehensive safety assessment than is currently achievable. The current methods of data analysis contribute to products that take too long to get to market, which may cost lives, and which may ultimately make the product too expensive.

How a solution would improve the product development process:

If relevant clinical databases were integrated and the analysis of data was conducted across data sets, true safety signals (versus background noise) would be more effectively identified. This is the result of a larger number of patients to analyze, the combined longitudinal effect of multiple products, and the ability to review data covering a more diverse population, which provides increased statistical power and higher specificity to conclusions.

Improved signal detection could have three important implications:
1) immediate questions or additional data elements regarding the safety signals (e.g. associations between products and events) could be requested of the provider community, whether via conventional mechanisms or via an integrated community network;
2) products and product classes with low and no safety-signal profiles might have smaller, more focused studies (decreasing cost for the developers and less distraction and focus time for the Agency to review); and
3) the cumulative learning from the integrated safety and product experience could be fed into future clinical trial design, influencing endpoint selection and inclusion criteria without jeopardizing the safety of patients.

Examples of improvements that could be incorporated into a future trial design include optimized inclusion and exclusion criteria, clinical evaluations which target the areas of greatest risk, shortened duration of the trial, and reduction in the number of patients needed for the trial.