The Scientific Excellence Working Group takes a broad approach towards scientific excellence as excellence in all aspects of the scientific endeavor. “Scientific excellence” in this sense not only includes excellence in scientific research, but also excellence in connecting science to society, in teaching and mentoring scientists, in science management, and in science advice to policy makers, to name only a few.
In this working group, we are interested in defining, describing and measuring scientific excellence and its impact on society. Naturally, there is a key focus on how to evaluate scientific activities and to assess the quantitative and qualitative indicators that are currently used for measuring excellence in science. What is more, we are keen on identifying what kind of environment would be most suitable to unleash curiosity and creativity in science, as well as to foster the development of human potential. We also aim to identify new ways to take into account often implicit personal qualities of a person which are nevertheless vital professionally.
For GYA members and alumni only: connect to the group, collaborate and discuss with other members on the GYA-internal Agora platform here.
Currently, the group engages in five on-going projects:
The idea arose from the discussion at the 2017 GYA Annual general Meeting within the working group and was preliminary implemented in 2018. We prepared an anonymous relational database of the applicants and the reviewers with the gender, nationality, and discipline attributes. We also designed a short questionnaire to be filled out during the member selection process 2018 and 2019 by the selection committee members. The questionnaires were collected and preliminary analysis was performed. We did few overall preliminary analyses and planning to perform further analysis using the 2018 and 2019 applications. The ultimate aim is to initiate a new project for identifying novel factors which reflect the implicit personal qualities of a person which are yet vital professionally. Read more here.
Koen Vermeir spearheaded and led the first global study of research evaluation practices since 2016. We need this data-informed approach to be able to propose evidence-based recommendations that can improve current evaluation practices and research culture as a whole. In order to have a comparable data set on a global scale, we started the Analysing of Global Views of Institutional Promotion Criteria, focusing on criteria to become a full professor. The first results and a provisional report of this project was presented at the AGM 2018. By now, we have come out with an analysis from 80 documents we obtained across many countries. More documents will be further analysed. We expect to publish a report in the end of 2019.
The group is looking into following up on the 2018 report on “Publishing Models, Assessment, and Open Science“. There are some questions in the problem framework that are worth exploring further.
In the spring of 2018, the working group was invited by the United States National Academies of Sciences, Engineering, and Medicine to provide expert input on the factors that affect reproducibility and replication of data produced from Science and Engineering research. A survey of GYA members provided input for a presentation in May 2018, which provided a set of comments and recommendations on replicability, based on GYA members’ input and on their personal expertise. A report on “Young Scientist Perspectives on Replicability and Reproducibility in Science and Engineering” is forthcoming.
Koen Vermier was nominated to join the InterAcademy Partnership working group on “Strengthening Research Evaluation: A Global Assessment and Dialogue”.
The Scientific Excellence Working Group continues to work together with the InterAcademy Partnership (IAP), ICS, ALLEA, the JRC, and several European Commission Expert Groups to drive change in the perception and measurement of scientific excellence. These collaborations have led to significant results that are impacting policy on a local, regional as well as global level (see the section “outcomes”).
Currently, we are working with IAP on a proposal on “Evaluating Research: Assessment and Improvement of Current Practice” and we are collaborating with the JRC on a document on measuring the impact of science advice.