The last few decades have witnessed prominent voices articulate growing concerns on reproducibility in science. At the core of these concerns is the observation that subsequent attempts to replicate certain findings often lead to significantly different results. This problem—which appeared to increase in magnitude—was termed the replication crisis.
Among the culprits are the unprecedented rate at which scientific manuscripts are produced, the ever-growing size and interdisciplinary character of dataset acquisition, and the tremendous demand for computational power, often for a single experiment. All these factors have impinged on the reproducibility of scientific results and, at the same time, on the possibility to effectively peer-review manuscripts. Perhaps with even more dire consequences, the blueprint of this growing crisis also affects regulatory bodies, which need comprehensive and exhaustive access to all past and present research findings pertaining to the assessment of new products.
In an effort to address this problem, Philip Morris International R&D has created INTERVALS, a growing online platform open to scientists from both academia and industry, meant to enable third-party collaboration and data analysis. The website and the associated data warehouse are developed in collaboration with Emakina and Edelweiss Connect GmbH.
With a foundation built by using the latest standards in data sharing and reproducible research, the continuously updated INTERVALS is driven by the idea of proactive sharing of protocols, computational tools, and data from assessment studies. In a single place, it shares results, software, raw data files, and detailed information on the design and protocols used in studies. This facilitates the review process and allows, at the same time, for the reuse of data for generating and testing new hypotheses. We believe that all these traits enhance the transparency of the scientific process several fold and accelerate scientific research.
Initially, the platform was meant to present studies and share data from inhalation toxicology assessments. Owing to its continuous development, INTERVALS has evolved to include clinical study designs and results, various population studies, and even results from the fields of physics and chemistry. Although now enhanced, its core mission remains the same: to establish high standards for transparency and reproducibility by sharing data on scientific assessment of products.
In addition, the INTERVALS platform harbors basic research within the framework of systems biology—leading to a mechanistic understanding of diseases and pathways—as well as research on various aerosols and chemicals.
Taken together, these studies contribute significantly to the understanding of tobacco smoke-induced perturbations in human signaling pathways, detail the scientific assessment of new products, and represent an important step forward in addressing the reproducibility crisis in science by making data and software freely available.
The aforementioned mission of INTERVALS is in line with the vision of the 21st century toxicity testing established by The National Research Council following commission by the United States Environmental Protection Agency. This vision steers away from the sole use of traditional toxicity testing methods and calls for the study of adverse health effects sourced from induced perturbations in human signaling pathways by biologically active substances or their metabolites (Committee on Toxicity Testing and Assessment of Environmental Agents, 2007; Hartung, 2010; Krewski et al., 2010).
While INTERVALS was created as a platform for sharing (raw and processed) datasets and assessment results, the scientific community is invited to use the portal to share their own datasets and results and adhere to our vision of accelerated innovation through scientific transparency.
The portal allows users to browse the data by study, disease, pathway, endpoint, or product and obtain relevant information related to study design, methods, and, most importantly, results from preclinical as well as clinical studies.
The last two decades have seen an increase in scientific output and sharing of industry-sponsored and -conducted research, which is met with skepticism from the scientific community. Indeed, reports detailing concerns about data manipulation and misrepresentation have led to a decline in the credibility of industry-sponsored research. Engraved in public opinion, this lack of trust is damaging to innovation and makes the work of regulatory bodies even more difficult. In a now-landmark paper published in 2013, Bauchner and Fontanarosa presented a possible solution for restoring the credibility of the industry (Bauchner and Fontanarosa, 2013).
The INTERVALS platform has implemented the suggestions therein by:
Candidate and potential MRTPs are largely developed and, so far, assessed by tobacco companies. In order to establish the validity of the findings, the INTERVALS platform enables in-depth access to all scientific data and findings and acts as a study repository. Hence, the INTERVALS platform aims to complement the peer-review publication process by offering a comprehensive medium for sharing data, analyses, and software and to allow researchers to find all relevant information, detailed protocols, and, most importantly, interoperable data files to facilitate independent reanalysis of key findings, meta-analysis, and data reuse.
The current state of risk assessment employed for environmental toxicants and therapeutic drugs relies on correlative analyses performed within the framework of epidemiological studies that are run years or even decades after a product is released on the market or after a certain public policy is enacted. This state of affairs leads to irrecoverable delays, well past the point when a change in therapeutic regimen, lifestyle, or environmental exposure can prevent the onset of disease. In addition, post-hoc epidemiological studies do not aim to elucidate the mechanisms that link perturbations in molecular signaling to diseases.
One way to address these issues is to develop a computational approach capable of quantifying the risk posed by exposure to an active substance well in advance (Hoeng et al., 2012). To this end, the INTERVALS platform constitutes a gateway to a comprehensive data structure meant to provide a detailed picture of diseases, pathways, and interaction of active substances with tissues. It is intended to allow exploration of the Data Cube in relation to multiple biologically active substances that can constitute perturbations of biological networks, test systems, and samples.
The scientific process relies heavily on the peer-review mechanism, which plays a key role in controlling scientific quality, improving performance, and providing credibility. The lack of scientific transparency leads to a peer-review process that is less than ideal and often results in irreproducible science that might have an unpredictable impact on public health or socioeconomic or political factors. To choose just one example, a 2012 report has shown that the results of 47 of 53 published peer-reviewed cancer studies could not be reproduced (Begley and Ioannidis, 2015). Consequently, the last decade has seen increasing awareness and recognition of the weaknesses that have infiltrated the biomedical research peer-review system, and it has been widely acknowledged that this system, under its current implementation, has its limitations.
In terms of journals, the number of signatories of the Transparency and Openness Promotion (TOP) guidelines, as of March 2019, was close to 5000, with an average increase of approximately 120 journals every month over the last three years.
Several impactful studies have detailed the reasons behind the current crisis (Begley and Ioannidis, 2015; Couchman, 2014; Drubin, 2015; Frye et al., 2015; Iorns and Chong, 2014) and identified a spectrum of causes, including inappropriate study design, lack of reagent validation, inadequate documentation of methods and datasets, and insufficient sharing of data and methods – essential for detailed analysis and replication.
Consequently, the replication crisis calls for significant improvements to current practices (McNutt, 2014) that are ultimately meant to restore confidence and facilitate the peer-review process. This implies that the data must be readily available in a comprehensive format that allows the rapid testing of new hypotheses, meaningful meta-analysis, and the development of robust methods. The following points constitute an excellent starting point (Freedman et al., 2017):
Beyond experiments and reporting, adequate disclosure of potential conflicts of interest (COI), and proper credit to the researchers involved are two of the critical components of scientific transparency. Although often met with skepticism, we believe that sponsored research must always be properly identified.
There is no shortage of publications suggesting that industry funding increases the likelihood of pro-industry conclusions (Babor and Miller, 2014; Pisinger et al., 2019), and the genre has certainly been of help in identifying many publication biases and cases of misconduct in both industry and non-industry funded research (Allison and Cope, 2010; Boutron et al., 2010; Cope and Allison, 2010; Golder and Loke, 2008). However numerous, these publications do not grant industry-wide generalizations. In 2011, a study found that, over the last few decades, only 3.8% of all misconduct cases that led to the retraction of medical and scientific publications were associated with support from industry (Woolley et al., 2011). In a perspective article, Barton and colleagues reported that they could not find any data showing that financial conflicts of interest lead to a drop in scientific quality (Barton et al., 2014). In fact, on average, industry-funded clinical trials have been found to be qualitatively superior in their reporting and adherence to regulations when compared with non-industry funded clinical trials run by academic institutions (Del Parigi, 2012).
It is noteworthy to point out that scientific transparency is never a concluded process, and there will always be adjustments and more things we can do in order to facilitate innovation and cooperation. We are only at the beginning of an era of big data, high-throughput experiments, and massive parallel calculations, and our scientific tools are in the process of being reshaped for this new reality. Initiatives like INTERVALS spearhead this transformation and encourage transparent sharing of data to allow easy review and understanding, which will facilitate the objective evaluation of the evidence (Carlo et al., 1992).
Smoking is addictive and causes a number of serious diseases, such as cardiovascular disease, lung cancer, and chronic obstructive pulmonary disease (COPD).
Despite existing strategies to reduce smoking-related harm (i.e., preventing initiation and promoting cessation of smoking), it is estimated that more than one billion people worldwide will continue to smoke in the foreseeable future (Eriksen et al., 2015).
More recently, Tobacco Harm Reduction has emerged as an additional approach to address the health risks of smoking (World Health Organization, 2015; Zeller et al., 2009).
Tobacco Harm Reduction was defined by the U.S. Institute of Medicine (IOM) as “decreasing total morbidity and mortality, without the complete elimination of tobacco and nicotine use.” The IOM referred to “potentially reduced-exposure products (PREPs) as having reductions in exposure to one or more tobacco toxicants” (Institute of Medicine, 2001).
Tobacco Harm Reduction is based on encouraging smokers to switch to less harmful products that emit significantly lower levels of toxicants while providing levels of nicotine comparable to cigarettes. The U.S. Family Smoking Prevention and Tobacco Control Act embraces the concept of Tobacco Harm Reduction and defines a modified risk tobacco product (MRTP) as any tobacco product that is sold or distributed for use to reduce harm or the risk of tobacco-related disease associated with commercially marketed tobacco products (Food and Drug Administration (FDA), 2009).
Candidate MRTPs include products such as e-cigarettes and heat-not-burn products as well as other smokeless tobacco products.
Effective Tobacco Harm Reduction requires that a significant number of smokers adopt available MRTPs, meaning that MRTPs must be designed to minimize product risk while maximizing product acceptance by smokers. Therefore, an effective MRTP must meet two conditions:
To address a range of consumer needs and thereby ensure the broadest possible adoption of reduced-risk alternatives, the industry is developing a portfolio of products, including heated tobacco products (i.e., heat-not-burn products), e-vapor products (i.e., e-cigarettes), oral tobacco, and nicotine products.
Evaluating candidate and potential MRTPs for their potential to significantly reduce smoking-related disease and death requires a robust, science-based assessment framework (Kozlowski and Abrams, 2016; Morven Dialogues, 2015), implemented by companies in the industry (Murphy et al., 2017; Smith et al., 2016).
As a result, numerous studies conducted with various candidate and potential MRTPs have been published by the companies that develop them.
Recently, several independent reports and studies reviewing the available science on heated tobacco and e-vapor products have been published:
In our view, while evidence regarding the harm reduction potential of candidate MRTPs has accumulated rapidly in the scientific literature, it is essential to share the scientific basis of product assessment and the available methods as well as the data and results of assessment studies to enable their independent review and analysis.
The quantitative assessment of the risk reduction potential of candidate and potential MRTPs involves (i) the use and/or development of state-of-the-art methods in regulatory and systems toxicology, (ii) a deep knowledge of the mechanisms that lead to smoking-related diseases, and (iii) expertise in the design and conduct of clinical studies aimed at substantiating reduced exposure and risk in adult smokers.
Toxicity testing is at a turning point now that long-range strategic planning is in progress to update and improve testing procedures for potential stressors. The U.S. Environmental Protection Agency (EPA) commissioned the U.S. National Research Council (NRC) to develop a vision for toxicity testing in the 21st century (Committee on Toxicity Testing and Assessment of Environmental Agents, 2007; Krewski et al., 2010; Thomas et al., 2013) to base the new toxicology primarily on Pathways of Toxicity (PoT) (Basketter et al., 2012). The report by the NRC envisions a shift away from traditional toxicity testing and toward a focused effort to explore and understand the signaling pathways perturbed by biologically active substances or their metabolites that have the potential to cause adverse health effects in humans.
This understanding should allow researchers to:
Systems toxicology (Sturla et al., 2014), or 21st century toxicology (Hartung, 2010), aims to create a detailed understanding of the mechanisms by which biological systems respond to toxicants so that this understanding can be leveraged to assess the potential risk associated with chemicals, drugs, and consumer products. For example, to determine whether a candidate MRTP has the potential to reduce disease risk, its biological impact is compared with that of a reference cigarette (e.g., 3R4F cigarette) on a mechanism-by-mechanism basis.
The identification of pathways of toxicity is imperative in order to understand the mode of action of a given stimulus and to group together different stimuli based on the toxicity pathways they perturb. The first component of the vision focuses on pathway identification , which is preferably derived from studies performed in human cells or cell lines using omics assays. The second component of the vision involves targeted testing of the identified pathways in whole animals and clinical samples to further explain toxicity pathway data. This two-component toxicity testing paradigm, combined with chemical characterization and dose-response extrapolation, delivers a much broader understanding of the potential toxicity associated with a biologically active substance. Systems biology plays an essential role in this paradigm, consolidating large amounts of information that can be probed to reveal key cellular pathways perturbed by various stimuli (Sturla et al., 2014). Of importance, moving to mechanism-based assessment comes with its own challenges and requires new ways to assess literature data (Health and Services, 2014; Smith et al., 2016).
Importantly, omics technologies used to characterize the effect of different exposures molecularly also allow detailed descriptions of what each individual is exposed to (i.e., the exposome). For example, the EXPOsOMICS consortium (Vineis et al., 2017) aims at characterizing the external and internal exposome in relation to air pollution and water contaminants (Turner et al., 2018). Methods developed in this field are relevant to Tobacco Harm Reduction science, and vice versa, so a conversation between the relevant stakeholders through community platforms, such as INTERVALS, is highly relevant.
Furthermore, 21st century toxicology has identified the promise of new technologies and the need for large-scale efforts. Aligned with the 3Rs strategy — which states that animal use in scientific research should be reduced, refined, and replaced — in vitro studies using relevant test systems and systems biology approaches offer new prospects in the field of human toxicology (Daneshian et al., 2011). It is important that synergies are established between different laboratories to ensure that the best possible methods are developed and validated for regulatory consideration.
In drug development and toxicological risk assessment, compound testing usually starts with in vitro experiments and is followed by in vivo testing using rodents as a mammalian model. This practice assumes that animal models respond to active substances through similar mechanisms. Recent inconsistencies in translating findings from animal models to human clinical outcomes (Knight, 2011), as well as animal welfare concerns, have led to the development of increasingly more sophisticated in vitro systems mimicking even the most complex tissue structures (Iskandar et al., 2017; Zanetti et al., 2016).
In vitro models capture events efficiently at molecular, cellular, and, at best, tissue levels and can be used to study transport rates across epithelia, toxicity, and mechanism of action (Fernandes and Vanbever, 2009). However, the fourth level of biological organization, which includes the physiological functions of an organism, can so far only be mimicked in the whole animal (Barré-Sinoussi and Montagutelli, 2015; Choy et al., 2016).
In addition to physiology, the organ of a living animal has advantages over in vitro models, especially in a disease context. For example, no in vitro model can adequately mimic COPD, an important adverse outcome caused by cigarette smoking. The complex pathogenesis of COPD includes inflammatory cell infiltration, goblet cell hyperplasia, cilia dysfunction, squamous cell metaplasia, and emphysema of the lung (Adamson et al., 2011). While ciliary dysfunction, mediator release, and goblet cell hyperplasia can be achieved in an in vitro setting, the C57BL6 mouse model comes much closer to mimicking the structural tissue alterations characteristic of emphysema (Sasaki et al., 2015). The role of the immune system, inflammatory cell infiltration into the lung, and lung function as a whole can also be better addressed in an animal model (Fricker et al., 2014). The C57BL/6 model is also suitable for testing aerosols generated by e-vapor and heat-not-burn products (Ansari et al., 2016; Phillips et al., 2015).
Therefore, despite criticism, animal models continue to serve an important role in human disease research and toxicological assessment (Denayer et al., 2014; Simmons, 2008; Vandamme, 2014).
However, considering the genetic differences between species, it is crucial to gain a detailed understanding of the similarities and differences between the experimental test system and the human disease or mechanism(s) that it is intended to model.
Inter-species comparisons conducted solely at the molecular level are generally inadequate because not all genes/proteins are conserved between species (Lin et al., 2014). Cross-species analysis of co-regulated genes can detect evolutionary conservation and provide functional information beyond sequence alignment (Berg and Lässig, 2006; Djordjevic et al., 2016; Rhrissorrakrai et al., 2015).
Hence, the mechanism-level conservation between species becomes more important than gene-level conservations. Importantly, mechanism-level comparisons will enable the identification of so-called “translational biomarkers”, which are representative of the behavior of key mechanisms. Equally important, the identification of the dissimilarity in disease pathways between species can highlight the value and limitations of a given model (Miller et al., 2010). The co-expression analyses can be conducted using an in vitro setting to establish convergence and divergence in cellular processes that contribute to specific pathologies in humans and animal models (Mueller et al., 2017; Rhrissorrakrai et al., 2015). This way, the in vitro research serves as the “adapter” between rodents and humans.