A: It is evident that we are not learning sufficiently from the many clinical trials and drug development programs done in the past. While we as humans can excel at understanding one particular drug or target at great depth, we are at the same time unable to process large amounts of data and fairly account for them in our decision making. This is indeed an area where unbiased insights from machine learning can complement human experts to making better decisions.

In comparison to other areas of drug development the advance of AI to support the clinical phase of the drug development process has only gained traction recently. Several engines are available now commercially with different targets for prediction (e.g. likelihood of approval vs. likelihood of success). The main challenges for this problem are clearly on obtaining the right data and transforming them in the most meaningful way (feature selection and engineering), as the feature domains that could influence success of a clinical trial or drug development program are very broad and diverse, ranging from drug properties to clinical operations.

Because the problem is so multidimensional there is a strong desire in this area to understand driving factors that contribute to trial success or failure. We have therefore implemented some detailed feature analyses approaches into our latest MH Predict release that make use of some of the latest developments in the field of explainable AI.

A: Pharma and biotech company can use MH Predict to assess the likelihood of success of ongoing and future trials, individually and in the context of a portfolio, and to understand possible drivers of success or failure.

For example, a biotech company has used MH Predict to assess the likelihood of success of a trial in its design phase. In this case, the prediction was positive. Various trial parameter changes were simulated, identifying a particular feature that showed high sensitivity to feasible changes. Our analysis helped the company with their decision to move forward with the trial plan. The company also uses this analysis to support their presentation to prospective investors.

Other customers focused on portfolio analysis in a particular indication area, or on failure analysis of trials.

MH Predict is also used outside of the drug development space in the financial space.

A: Yes, predicting clinical development outcome is a truly multidimensional problem. For machine learning, this is clearly a problem that weighs heavily on data rather than on algorithm optimization. I see different principal challenges: One is that we have very limited knowledge about why an individual trial actually fails. This brings uncertainty into feature selection, we have to consider many feature domains. We do not know where the relevant feature space ends. This is in contrast to areas like image or speech recognition where we know that the majority of information that is needed for building a model is contained within the image or speech recording for example, it is “only” a problem of featurization of extracted parameters.

Second, obtaining relevant data is a problem. Many data are proprietary to pharma companies and may not even be easily accessible in a structured format from within the company.

Third, in order to be useful for training an algorithm, features need to be sufficiently generalizable across the whole training space for the majority of instances (trials). This is a non-trivial problem for trial parameters that are very specific to indications, such as inclusion criteria and outcome measurements.

At Molecular Health we are addressing these challenges by constantly exploring and evaluating new features that enrich the information space by adding orthogonal information dimensions to the existing feature space that defines MH Predict. We are also spending considerable efforts on algorithmically generalizing complex and diverse features such as the definitions of endpoints for example.

A: A great advantage of machine learning models in this space is that they supply probabilities for technical success (in the case of MH Predict) using the same “framework” applied to all trials. This provides consistency and reliability to predictions; you know that the same “reasoning” has been applied to trial A compared to trial B. In contrast, a human expert will use quite different and individual criteria for different drugs and trials based on her/his expertise and experience relevant for trial A vs. B. Thus, a particular strength of those ML derived probabilities is comparability. This makes these models especially attractive to decision making where comparisons are involved, e.g. for portfolio management, competitive intelligence, or search for attractive in licensing candidates (or M&A targets).