When Prism first started out, our product vision was a centralized, easily searchable platform that had aggregated all the metadata on biomedical research. However, we quickly discovered two things: (1) there are many other products that describe themselves like this; and (2) the purchasers of these products are rarely satisfied.
Discovery 2 was especially intriguing. Intuitively, it seems like this kind of product should be a slam dunk. But when we spoke to the users, we heard time and time again that they found the data in these products to be rife with errors or gaps. Despite the sometimes massive corporations behind them and often six-figure price tags for access to these platforms, users ultimately didn't trust what they were getting. So users just ended up defaulting to doing their landscape analysis the hard way—with manual database searching, screening, and data extraction. (Or they paid a bunch more to have another research firm do this work for them.)
Taking these reports at face value, it might seem like Prism could differentiate itself with a simple value proposition: “Our competitors’ data is wrong and incomplete. Our data is more complete and more accurate!”
But we’ve learned that accuracy and completeness are not the real problem with the existing products. Their data isn’t even wrong per se. The problem is that whenever one tries to aggregate disparate, noisy data, assumptions must be made about how to transform and normalize data so that it can be easily searchable and valuable. These assumptions necessarily introduce some systematic biases that will simultaneously make the data suitable for some kinds of uses, and totally unsuitable for others.
Therefore, the problem with the existing data platforms is that their assumptions essentially “lock them in” to having static data assets that are only good for a limited set of use cases—i.e., those use cases where the client shares exactly the same assumptions as the platform’s creators.
But for domains like biomedicine, where many assumptions are hotly contested and experts disagree about even the basic ontologies, these static assets just won’t do. Users need a way to modify the assumptions underlying the data and craft an analysis that is both grounded in data and suited to their specific use case.
We were recently invited to write an article on this topic, wherein we describe how systematic bias (and the resulting errors) in data assets will arise even for something as fundamental as classifying the types of outcome measurements in a clinical trial. Outcome measurements can be described in all sorts of different ways and the expert community has not settled on single, “right” ontology. This makes searching for or analyzing outcomes a notoriously frustrating challenge. But if you try to force one outcome ontology onto the entire research landscape, your data and analysis is likely to be dismissed by many domain experts as inaccurate.
This is why we’re taking a different approach at Prism. Our research platform is designed to let experts apply their own assumptions to the data and analyses they undertake. Rather than just serving up a static dataset, we give our clients the tools to apply, test, and refine their own assumptions.
Going beyond this, we’ve also developed workflows that allow us to create powerful AI modules that surface novel concepts and insights from the data that are hard to find. This is how we developed our digital biomarker tracker for a major pharmaceutical company.
Because of the flexibility in our platform, we can help our clients to see the world like never before, offering them the power to track trends and developments that are so new that the existing data sources don’t even have codes or tags or fields corresponding to the concepts of interest.
The take-home message here is that we’ve learned that truly powerful, decision-making tools need to be dynamic. Static data assets, no matter how comprehensive, can only reflect one set of assumptions. There are just so many ways of “seeing” the biomedical research landscape, and each different way may illuminate a different facet of the user’s question or problem.
Prism's products are different. Our focus is on building tools that empower our clients to see and explore data in different ways and let their analysis evolve and grow along with their understanding.
Hey SP, Kuiper J, Fleck C. Metascience Solutions for the Paradox of Evidence-Based Decision Making. Signal. 2021:02. https://www.ispor.org/publications/journals/value-outcomes-spotlight/vos-archives/issue/view/navigating-the-changing-heor-publishing-landscape/metascience-solutions-for-the-paradox-of-evidence-based-decision-making