U.S. flag An official website of the United States government

On Oct. 1, 2024, the FDA began implementing a reorganization impacting many parts of the agency. We are in the process of updating FDA.gov content to reflect these changes.

U.S. Food and Drug Administration
  1. Home
  2. Science & Research
  3. Science and Research Special Topics
  4. Advancing Regulatory Science
  5. Developing Frameworks and Tools for Integration of Digital Health Tools into Clinical Practice
  1. Advancing Regulatory Science

Developing Frameworks and Tools for Integration of Digital Health Tools into Clinical Practice

CERSI Collaborators: University of California at San Francisco (UCSF): Andrew Auerbach, MD

FDA Collaborators: Center for Devices and Radiological Health(CDRH): Bakul Patel, MS, MBA (Formerly of CDRH); Vinay Pai, PhD; Leeda Rashid, MD, MPH, ABFM; Catherine Bahr; Arti Tandon, PhD; Charlie Yongpravat, PhD; Anindita Saha, PhD; Jiping Chen, MD, PhD, MPH 

Project Start Date: May 1, 2018
Project End Date: February 28, 2022

While FDA funding of this project has ended, the research continues to evolve and result in additional findings, as described below. Outcomes/results from this project informed the development of The Digital Variome: Understanding the Implications of Digital Tools on Health.

Regulatory Science Challenge

This research aimed to develop methodologies to improve the quality and safety of FDA-regulated products by using consensus methods to identify gaps in regulation perceived by health systems and healthcare providers, and developing initial blueprints and recommendations for regulators and vendors to follow. Researchers developed methods and tools to improve and streamline clinical and post market evaluation of FDA-regulated products, including scientifically valid approaches to incorporating patient input and approaches to complex and multisource data to inform regulatory decision-making, including use of “real world” data (RWD/RWE).

Project Description and Goals

This project extends the ongoing research work of FDA’s national network of leading academic medical centers, researchers, and innovators to yield information about which real world measures can be used across types of software used in health, and the eventual data sources required to carry out real world performance measurement and post market surveillance of digital health tools.

Research Outcomes/Results

Caring for patients in the electronic era requires multiple people and systems to collaborate, which in turn requires interoperability among connected health records, and integration between myriad digital health tools and devices. Few data or approaches exist to assist with adoption and integration of innovative digital health tools (DHTs) in ways that are safe and effective.

To explore these questions and develop best practices for addressing them, UCSF founded the Accelerated Digital Clinical Ecosystem (ADviCE). The collaborative, founded in 2018, includes academic and non-academic health systems, single-site and multicenter health systems, medical software developers, payors, and patient groups. Investigators carried out a series of in-person and virtual meetings during which, using a consensus-building framework, we identified problems with Digital Health Technology (DHT) adoption and potential solutions.

Investigators identified the following challenges to DHT adoption: (1) Variable definitions of which DHTs are relevant to clinical care delivery; (2) Lack of consistent, common terms to describe DHTs during selection, (3) Wide variability in how health systems integrate DHTs into practice and, (4) Lack of a framework and tools to evaluate DHTs’ real-world performance through post-market surveillance. Investigators further framed collaborative opportunities that could support solutions to each challenge.

Challenge 1 and 2 - Defining and selecting DHTs: To address the first and second challenges, investigators prototyped a tool known as the ‘Digital Health Common Application’ (DHCA), a framework that gathers a core set of information needed by health systems and patients to make DHT selection choices. In use, the DHCA can increase transparency of DHT’s functions and pitfalls to health system stakeholders as well as becoming a potential ‘package insert’ resource for patients and families. Work on the DHCA continues. Investigators have continued to test the applicability of the Common Application through a series of ADviCE-sponsored internships where ADviCE has hosted between 7 and 18 fellows who have done outreach to digital health companies to gather information using the Common Application format.

Over the course of the last 4 years more than 40 fellows who have contacted more than 180 companies gathering information on 190 total applications. As of Spring 2024, investigators are re-assessing those companies after initial follow-up (between 6 months and 3 years later) to see if they have added functionality, pursued regulation, or accumulated additional evidence. Investigators are also exploring use of large language model chatbots as an approach to gathering information which might later be used to populate the Common Application or regulatory tools.

Challenge 3 - DHT adoption: To address challenges in how health systems adopt DHTs, investigators gathered intake processes and questionnaires from ADviCE participating sites and mapped them into several broad domains – clinical use case, security and privacy review, and governance. These domains are in turn becoming the focus of discussions around best practices for DHT onboarding while also helping frame considerations of RWE generation (as DHT performance may be powerfully impacted by how it is implemented and where).

Challenge 4 – Real world performance: For the fourth challenge identified, investigators took the FDA measurement framework proposed as a general temlaplate for PreCert post market surveillance and undertook consensus work to identify specific domains and measures relevant to each broad domain. For example, within the area of Product Performance à Cybersecurity investigators developed subdomains where metric identification was recognized as a key next step; metric identification was continued in additional findings, as described below. Outcomes/results from this project informed the development of The Digital Variome: Understanding the Implications of Digital Tools on Health.

Domain Subdomain Measure concept
Product Performance Cybersecurity
  • Data stored and transmitted in a secure fashion
  • Privacy of user data protected properly
  • User data sharing is appropriate
  • Ongoing process for ensuring security

Research Impacts

In the absence of empiric data on safety or effectiveness, multistakeholder collaboratives have a key role in setting safety and adoption standards for DHTs. ADviCE is a first example of how collaboratives can identify and employ best practices for adoption, improve provider and patient experience, and substantially increase limited data on DHT clinical effectiveness and safety.

Publications

No peer-reviewed publications to date; Investigators plan to analyze and publish follow-up study results.

Dr. Auerbach has published invited editorials in JAMA IM on digital health regulation based in part on his experiences with ADviCE.

 

Back to Top