Publications

Different tools have been developed that facilitate systematic and transparent evaluation and handling of toxicity data in the risk assessment process. The present paper sets out to explore the combined use of two web-based tools for study evaluation and identification of reliable data relevant to health risk assessment. For this purpose, a case study was performed using in vivo toxicity studies investigating low-dose effects of bisphenol A on mammary gland development. The reliability of the mammary gland studies was evaluated using the Science in Risk Assessment and Policy (SciRAP) criteria for toxicity studies. The Health Assessment Workspace Collaborative (HAWC) was used for characterizing and visualizing the mammary gland data in terms of type of effects investigated and reported, and the distribution of these effects within the dose interval. It was then investigated whether there was any relationship between study reliability and the type of effects reported and/or their distribution in the dose interval. The combination of the SciRAP and HAWC tools allowed for transparent evaluation and visualization of the studies investigating developmental effects of BPA on the mammary gland. The use of these tools showed that there were no apparent differences in the type of effects and their distribution in the dose interval between the five studies assessed as most reliable and the whole data set. Combining the SciRAP and HAWC tools was found to be a useful approach for evaluating in vivo toxicity studies and identifying reliable and sensitive information relevant to regulatory risk assessment of chemicals.

Background
The issue of endocrine disrupting chemicals (EDCs) is receiving wide attention from both the scientific and regulatory communities. Recent analyses of the EDC literature have been criticized for failing to use transparent and objective approaches to draw conclusions about the strength of evidence linking EDC exposures to adverse health or environmental outcomes. Systematic review methodologies are ideal for addressing this issue as they provide transparent and consistent approaches to study selection and evaluation. Objective methods are needed for integrating the multiple streams of evidence (epidemiology, wildlife, laboratory animal, in vitro, and in silico data) that are relevant in assessing EDCs.

Methods
We have developed a framework for the systematic review and integrated assessment (SYRINA) of EDC studies. The framework was designed for use with the International Program on Chemical Safety (IPCS) and World Health Organization (WHO) definition of an EDC, which requires appraisal of evidence regarding 1) association between exposure and an adverse effect, 2) association between exposure and endocrine disrupting activity, and 3) a plausible link between the adverse effect and the endocrine disrupting activity.

Results
Building from existing methodologies for evaluating and synthesizing evidence, the SYRINA framework includes seven steps: 1) Formulate the problem; 2) Develop the review protocol; 3) Identify relevant evidence; 4) Evaluate evidence from individual studies; 5) Summarize and evaluate each stream of evidence; 6) Integrate evidence across all streams; 7) Draw conclusions, make recommendations, and evaluate uncertainties. The proposed method is tailored to the IPCS/WHO definition of an EDC but offers flexibility for use in the context of other definitions of EDCs.

Conclusions
When using the SYRINA framework, the overall objective is to provide the evidence base needed to support decision making, including any action to avoid/minimise potential adverse effects of exposures. This framework allows for the evaluation and synthesis of evidence from multiple evidence streams. Finally, a decision regarding regulatory action is not only dependent on the strength of evidence, but also the consequences of action/inaction, e.g. limited or weak evidence may be sufficient to justify action if consequences are serious or irreversible.

From the lectures presented at the 2nd International Workshop on Obesity and Environmental Contaminants, which was held in Uppsala, Sweden, on 8–9 October 2015, it became evident that the findings from numerous animal and epidemiological studies are consistent with the hypothesis that environmental contaminants could contribute to the global obesity epidemic. To increase awareness of this important issue among scientists, regulatory agencies, politicians, chemical industry management, and the general public, the authors summarize compelling scientific evidence that supports the hypothesis and discuss actions that could restrict the possible harmful effects of environmental contaminants on obesity.

It is a great loss, both in terms of knowledge and resources and in terms of credibility of the scientific community, when peer-reviewed studies are of insufficient quality. Reliability evaluations of recently published ecotoxicity studies show incomplete and inadequate reporting, regarding both description of methodology and presentation of results [1, 2]. For a reader of these publications, it can be difficult to determine whether the missing information is the result of insufficient reporting only or if it is the result of inadequate design and performance of the experiment. Regardless, this obstructs the evaluation process and decreases the chance that the studies are used in future research and for regulatory processes.

The medical journal Lancet recently presented a series on how to increase value and reduce waste in biomedical research and mentioned standards for reporting of studies in scientific journals as an important factor [3, 4]. The editor of Toxicological Sciences recently stressed the problem of low reproducibility and pointed out 3 things that journals can do to improve the situation: promote proper reporting of studies, make sure statistical analyses are accurately described and appropriately used, and request disclosure of all potential conflicts of interest [5]. Until recently, word limits in peer-reviewed journals caused authors to focus on short and concise publications, discussing mainly their results and economizing on the description of methods. Because it is now possible to publish supporting information online, for which word limits do not apply, raw data can be provided and all aspects of a study can be described in sufficient detail [6].

In several research areas, systematic reporting recommendations have been developed to guide researchers, reviewers, and editors during the publication process. In the field of epidemiology, for example, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement is used to increase the quality of peer-reviewed publications [7]. So far, the statement has been endorsed by more than 100 biomedical journals, and it provides checklists for a variety of applications: cohort studies, case–control studies, cross-sectional studies, and conference abstracts. The National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs) has developed the Animal Research: Reporting In Vivo Experiments (ARRIVE) guideline to improve the reporting of animal experiments. The guideline has been adopted by more than 300 journals and provides a checklist with 20 items, ranging from an ethical statement to sample size and funding [8]. Minimum Information About a Microarray Experiment (MIAME) is a reporting standard created by the Functional Genomics Data Society for microarray experiments. It specifies the information necessary to interpret results from experiments clearly and to potentially reproduce the experiment [9]. Currently, there are no generally accepted reporting guidelines for peer-reviewed ecotoxicity studies, but some journals have introduced a few specific requirements such as confirmation of exposure concentrations.

As a response to the need for a more robust and complete reporting of ecotoxicity studies, reporting recommendations have been developed in a collaboration between the Dutch National Institute for Public Health and the Environment, the Swiss Centre for Applied Ecotoxicology, the Swiss Federal Institute of Aquatic Science and Technology, and Stockholm University [10]. To ensure reliability and reproducibility of studies, the recommendations are based on reporting requirements for Organisation for Economic Co-operation and Development test guidelines and reliability evaluation methods for ecotoxicity studies. By considering these recommendations, preferably already when designing experiments, researchers will ensure that crucial aspects of the methodology and results are reported and that others, within and outside academia, can use the study results.

Background
The regulatory evaluation of ecotoxicity studies for environmental risk and/or hazard assessment of chemicals is often performed using the method established by Klimisch and colleagues in 1997. The method was, at that time, an important step toward improved evaluation of study reliability, but lately it has been criticized for lack of detail and guidance, and for not ensuring sufficient consistency among risk assessors.

Results
A new evaluation method was thus developed: Criteria for Reporting and Evaluating ecotoxicity Data (CRED). The CRED evaluation method aims at strengthening consistency and transparency of hazard and risk assessment of chemicals by providing criteria and guidance for reliability and relevance evaluation of aquatic ecotoxicity studies. A two-phased ring test was conducted to compare and characterize the differences between the CRED and Klimisch evaluation methods. A total of 75 risk assessors from 12 countries participated. Results show that the CRED evaluation method provides a more detailed and transparent evaluation of reliability and relevance than the Klimisch method. Ring test participants perceived it to be less dependent on expert judgement, more accurate and consistent, and practical regarding the use of criteria and time needed for performing an evaluation.

Conclusions
We conclude that the CRED evaluation method is a suitable replacement for the Klimisch method, and that its use may contribute to an improved harmonization of hazard and risk assessments of chemicals across different regulatory frameworks.

Predicted-no-effect concentrations (PNECs) and environmental quality standards (EQSs) are derived in a large number of legal frameworks worldwide. When deriving these safe concentrations, it is necessary to evaluate the reliability and relevance of ecotoxicity studies. Such evaluation is often subject to expert judgment, which may introduce bias and decrease consistency when risk assessors evaluate the same study. The Criteria for Reporting and Evaluating Ecotoxicity Data (CRED) project attempts to address this problem. It aims to improve the reproducibility, transparency, and consistency of reliability and relevance evaluations of aquatic ecotoxicity studies among regulatory frameworks, countries, institutes, and individual assessors. In the present study, the CRED evaluation method is presented. It includes a set of 20 reliability and 13 relevance criteria, accompanied by extensive guidance. Risk assessors who participated in the CRED ring test evaluated the CRED evaluation method to be more accurate, applicable, consistent, and transparent than the often-used Klimisch method. The CRED evaluation method is accompanied by reporting recommendations for aquatic ecotoxicity studies, with 50 specific criteria divided into 6 categories: general information, test design, test substance, test organism, exposure conditions, and statistical design and biological response. An ecotoxicity study in which all important information is reported is more likely to be considered for regulatory use, and proper reporting may also help in the peer-review process.

This paper presents 10 recommendations for improving the European Medicines Agency’s guidance for environmental risk assessment of human pharmaceutical products. The recommendations are based on up-to-date, available science in combination with experiences from other chemical frameworks such as the REACH-legislation for industrial chemicals. The recommendations concern: expanding the scope of the current guideline; requirements to assess the risk for development of antibiotic resistance; jointly performed assessments; refinement of the test proposal; mixture toxicity assessments on active pharmaceutical ingredients with similar modes of action; use of all available ecotoxicity studies; mandatory reviews; increased transparency; inclusion of emission data from production; and a risk management option. We believe that implementation of our recommendations would strengthen the protection of the environment and be beneficial to society. Legislation and guidance documents need to be updated at regular intervals in order to incorporate new knowledge from the scientific community. This is particularly important for regulatory documents concerning pharmaceuticals in the environment since this is a research field that has been growing substantially in the last decades.

(Eco)toxicity studies conducted according to internationally standardized test guidelines are often considered reliable by default and preferred as key evidence in regulatory risk assessment. At the same time regulatory agencies emphasize the use of all relevant (eco)toxicity data in the risk assessment process, including non-standard studies. However, there is a need to facilitate the use of such studies in regulatory risk assessment. Therefore, we propose a framework that facilitates a systematic and transparent evaluation of the reliability and relevance of (eco)toxicity in vivo studies for health and environmental risk assessment. The framework includes specific criteria to guide study evaluation, as well as a color-coding tool developed to aid the application of these criteria. In addition we provide guidance intended for researchers on how to report non-standard studies to ensure that they meet regulatory requirements. The intention of the evaluating and reporting criteria is to increase the usability of all relevant data that may fill information gaps in chemical risk assessments. The framework is publically available online, free of charge, at the Science in Risk Assessment and Policy (SciRAP) website: www.scirap.org. The aim of this article is to present the framework and resources available at the SciRAP website.

In this study we assess the applicability of a set of reliability criteria proposed by Ågerstrand et al. This was done by evaluating the reliability of 12 non-standard peer-reviewed ecotoxicity and toxicity studies for Bisphenol A. There was an overall agreement between the evaluator and the authors of the papers regarding the result of the evaluations. This suggests that the criteria offer enough guidance to be a useful and consistent evaluation tool. It provides a transparent and structured approach, and ensures that a minimum and similar set of criteria is used. The evaluation of the peer-reviewed ecotoxicity and toxicity studies concludes that important information is sometimes missing, and therefore the studies do not always meet common regulatory requirements regarding reporting. Whether this is due to insufficient reporting or due to poorly performed studies is not known. To improve the reporting, and thereby promote reliability and reproducibility, researchers, reviewers, and editors are recommended to use the suggested criteria as a guideline. In conclusion, in order to improve the reliability of peer-reviewed studies, and to increase their use in regulatory risk assessments of chemicals, the dialog between regulators, researchers, and editors regarding how to evaluate and report studies needs to be strengthened.

Ecotoxicity data with high reliability and relevance are needed to guarantee the scientific quality of environmental risk assessments of pharmaceuticals. The main advantages of a more structured approach to data evaluation include increased transparency and predictability of the risk assessment process, and the possibility to use non-standard data.

In this collaboration, between the research project MistraPharma and the German Federal Environment Agency, a new set of reporting and evaluation criteria is presented and discussed. The new criteria are based on the approaches in the literature and the OECD reporting requirements, and have been further developed to include both reliability and relevance of test data.

Intended users are risk assessors and researchers performing ecotoxicological experiments, but the criteria can also be used for education purposes and in the peer-review process for scientific papers. This approach intends to bridge the gap between the regulator and the scientist’s needs and way of work.

Background
Standard test data are still preferred and recommended for regulatory environmental risk assessments of pharmaceuticals even though data generated by non-standard tests could improve the scientific basis of risk assessments by providing relevant and more sensitive endpoints.

The aim of this study was to investigate if non-standard ecotoxicity data can be evaluated systematically in risk assessments of pharmaceuticals. This has been done by evaluating the usefulness of four reliability evaluation methods, and by investigating whether recently published non-standard ecotoxicity studies from the open scientific literature fulfill the criteria that these methods propose.

Results
The same test data were evaluated differently by the four methods in seven out of nine cases. The selected non-standard test data were considered reliable/acceptable in only 14 out of 36 cases.

Conclusions
The four evaluation methods differ in scope, user friendliness, and how criteria are weighted and summarized. This affected the outcome of the data evaluation.

The results suggest that there is room for improvements in how data are reported in the open scientific literature. Reliability evaluation criteria could be used as a checklist to ensure that all important aspects are reported and thereby increasing the possibility that the data could be used for regulatory risk assessment.