How to Restore Trust in Medical Research

The second of a two-part series on what needs to be done to restore public trust in the world of scientific research

Defining trust

Before I launch into how to restore trust, I should clarify what I am talking about when I use the word. Trust, simply put, is confidence in or reliance on the ability, faithfulness, or accuracy of someone or something.

While trust is often found in the company of “truth”, the two are not the same. “Truth” is a result, whereas “trust” is focused on the means and methods used to arrive at a result. It is an important distinction because in the world of scientific research we don’t always discover a truth, but we should always be able to determine the level of faithfulness and accuracy in the means and methods used to perform that research.

In other words, it needs to be trustworthy.

Components of Trust

Trust is a sort of cognitive shortcut that allows us to rely on something without a deep analysis of that thing. If it meets certain criteria, it can be trusted. Psychologists have studied the concept extensively and identified a number of cognitive and social factors that influence how people perceive someone or something as trustworthy (and vice versa).

Even if you have not studied the concept, you are still probably familiar with many of these factors (aka “heuristics”): source credibility or authority, bandwagon effect, perceived motive and intent, consistency and coherence, willingness to engage and revise, confirmation bias, etc. In all, there are around a dozen of these elements; too many for the purposes of this article.

Instead, I have borrowed a simplified framework for trust from computer science and neural networking where they have taken the various components of trust and put them into three interrelated conceptual buckets: Transparency, Explainability, and Significance (TES).

• Transparency can be summed up as “show your work.” In scientific research it is the clarity and fidelity of the methodologies and data sources, affiliations of the researchers, funding sources, a level of detail that allows others to repeat the research described, and so on.
• Explainability is the consistency and coherence of the information and the ability to support it if it contradicts other information. It also factors in elements like the level of uncertainty, the ability to address questions and criticisms, and details that help shed light on how the conclusions were reached.
• Significance is a measure of the relative importance of both the components and results of a piece of research. Components include the type and nature of study, the reputation of the authors, institutions and publications, etc. It also factors in peer review commentary, recency, citations, and other elements that help determine the overall impact of the information.

Each component of TES has many interrelated sub-components so the weighting of each element varies based on the information being analyzed. However, placed on a scale of 0 to 100, the closer each component is to 100, the more “trustworthy” it is (though scientific skepticism makes it extremely unlikely that anything or anyone could ever have a perfect 100 across all three categories).

Restoring Trust in Science

In my previous post I identified a number of areas where the scientific research and medical communities had significant failures in each area of TES, leading to an erosion of public trust in science. Whether it was a failure to disclose financial incentives (transparency), the repeated use of fraudulent data (explainability), or the resistance of highly regarded scientists in acknowledging their inaccuracies or errors (significance), or all of them collectively, the damage done has been significant and as such, will take some time to repair.

The good news is the scientific community is more aware of these problems than ever, and many groups are now working on a variety of solutions, both traditional and technological. The quickest fixes lie in a return to the scientific principles of transparency (especially funding sources), open analysis, and a willingness to address conflicting or contrary information in a timely manner.

Longer term the solutions will most likely be technological. Open access and pre-publishing platforms have turned the research spigot wide open, doubling the amount of available research in medicine alone every two months. Parsing and processing that volume of information will require methods faster and more robust than the traditional, more measured and time-consuming methods of peer review and journal publication.

Not surprisingly, AI is playing a large role in attempting to get more control over the research deluge with numerous tech firms devising innovative and useful ways to search, process and digest information. Other organizations like RetractionWatch are doing important work in surfacing corrections and retractions in the pursuit of scientific progress.

For our part, here at Siensmetrica, we are focused on applying deep data science and multi-vector analysis to identify and score research based on TES principles. Our aim is not to supplant the rigors of the scientific method, but to provide scientists and medical professionals with an objective way to assess the trustworthiness of a piece of research—a tool for guiding further exploration. The goal is not to always be right, but to be transparent and consistent so that we can restore belief in a system which, though not always perfect, can be relied upon to be heading in the right direction.

If we are successful, “trust the science” will be the gold standard once again.