Building trust in in silico modeling

As technology spreads across healthcare, the trust question becomes more urgent. Should we trust what a digital health app tells us? Can we rely on a diagnosis provided by an algorithm? Can drug developers trust the results of an in silico clinical trial?

The trust question is not new, it is simply being asked of new things. We trust what human doctors say because they have certain qualifications. We swallow the medicine they prescribe because regulators have approved it.

It’s the same for the many aspects of digitalised health. Tools that achieve accepted quality standards merit our trust.

Ⓒ Novadiscovery

# FDA seeks in silico trial standards

Regulators are working out what those standards should be for in silico trials. For digital therapeutics and other technologies that help us access care, standards are already largely in place: FDA has issued multiple digital health rules and guidances (opens new window). Germany classifies and regulates (opens new window) some digital health tools as medical devices.

Rules for assessing the credibility of in silico models are also evolving fast, as these progress from “simple” PK/PD models (already used in most drug submissions, and subject to their own set of guidelines) to more complex predictive models with the potential to inform or replace clinical evidence. FDA’s Model Informed Drug Development (MIDD) (opens new window) pilot program was set up in 2018 precisely so the agency could better understand the scope and potential of in silico modeling, and the kinds of questions it needed to ask about them.

NOVA was among the first participants in the MIDD pilot, presenting a cardiovascular disease model (opens new window) used to study the effects of a drug on certain disease markers. “The regulators were really interested in what we were doing”, recalls Eulalie Courcelles, Biomodeler - Regulatory Science Tech Lead at NOVA. FDA now has hundreds of modeling focused experts.

# Core principles of model-assessment

In another clear sign of FDA’s commitment to using modeling and simulation in the regulatory process, the Agency in December 2020 promoted Tina Morrison (opens new window), its key advocate for in silico medicine, to Director of Regulatory Science and Innovation. Morrison drove the standards for computer models used in medical device submissions, which emerged in 2018 (opens new window). The core “validation and verification” principles underpinning those standards (known as “V&V 40”) will likely apply to models used in drug development as well.

Context matters, too. A model may be verified as an accurate representation of a system or pathway, and it may be validated using data that stress-tests assumptions underpinning the model (though deciding just how accurate the model must be (opens new window) is tricky). But the model might not be appropriate to address the question in hand. Risk-assessment will also vary according to the impact that inaccurate or misleading model output may have on patients. That’s why “defining context” is the first of ten rules underlying credible use of modeling in healthcare (opens new window) outlined in a September 2020 paper co-authored by Morrison.

# Rigorous, yet flexible rules

These ten rules – which include using appropriate data, listing model limitations, conforming to (context-specific) standards, and documenting appropriately – sound like common sense, as the authors acknowledge. Yet they’re still critical to establishing a framework for modeling and simulation that all healthcare stakeholders can understand and interrogate. They’re critical to building trust.

The rules for in silico modeling in drug development will not be a static list of do’s and don’ts. As modeling methodologies and biomedical knowledge evolve, as case studies and experience accumulate - see, for example, two recent papers, below - guidelines must combine use-case-flexibility with rigorously upheld core principles. It’s not an easy mix. But this stamp of approval is fast coming together (opens new window), ensuring in silico modeling is trusted as a valuable component of drug development.

Note: NOVA has co-authored two recent peer-review publications providing further evidence of in silico model credibility:

Musuamba, F. et al. (2021) “Scientific and regulatory evaluation of mechanistic in silico drug and disease models in drug development: building model credibility”, CPT:Pharmacometric and Systems Pharmacology, June 2021. (opens new window)

Viceconti, M. et al. (2021) “Possible Contexts of Use for In Silico trials methodologies: a consensus-based review,” IEEE Journal of Biomedical and Health Informatics, June 2021 (opens new window)