Scientific Excellence

Aims & Objectives

Contact

Manage (your) group membership

Further information

Progress

For GYA members and alumni only: connect to the group, collaborate and discuss with other members on the GYA-internal Agora platform here.

++++++++++++++++++++++++++++++++

Currently, the group engages in five on-going projects:

Incorporating Intangible Information for Assessment Optimisation

The idea arose from the discussion at the 2017 GYA Annual general Meeting within the working group and was preliminary implemented in 2018. We prepared an anonymous relational database of the applicants and the reviewers with the gender, nationality, and discipline attributes. We also designed a short questionnaire to be filled out during the member selection process 2018 and 2019 by the selection committee members. The questionnaires were collected and preliminary analysis was performed. We did few overall preliminary analyses and planning to perform further analysis using the 2018 and 2019 applications. The ultimate aim is to initiate a new project for identifying novel factors which reflect the implicit personal qualities of a person which are yet vital professionally. Read more here.

Analyzing Global Views of Scientific Excellence

Koen Vermeir spearheaded and led the first global study of research evaluation practices since 2016. We need this data-informed approach to be able to propose evidence-based recommendations that can improve current evaluation practices and research culture as a whole. In order to have a comparable data set on a global scale, we started the Analysing of Global Views of Institutional Promotion Criteria, focusing on criteria to become a full professor. The first results and a provisional report of this project was presented at the AGM 2018. By now, we have come out with an analysis from 80 documents we obtained across many countries. More documents will be further analysed. We expect to publish a report in the end of 2019.

Assessment, Publishing Models, and Open Science

The group is looking into following up on the 2018 report on “Publishing Models, Assessment, and Open Science“. There are some questions in the problem framework that are worth exploring further.

Study on Reproducibility and Replicability in Science and Engineering

In the spring of 2018, the working group was invited by the United States National Academies of Sciences, Engineering, and Medicine to provide expert input on the factors that affect reproducibility and replication of data produced from Science and Engineering research. A survey of GYA members provided input for a presentation in May 2018, which provided a set of comments and recommendations on replicability, based on GYA members’ input and on their personal expertise. A report on “Young Scientist Perspectives on Replicability and Reproducibility in Science and Engineering” is forthcoming.

IAP Research Excellence Study: “Strengthening Research Evaluation: A Global Assessment and Dialogue”

Koen Vermier was nominated to join the InterAcademy Partnership working group on “Strengthening Research Evaluation: A Global Assessment and Dialogue”.

The Scientific Excellence Working Group continues to work together with the InterAcademy Partnership (IAP), ICS, ALLEA, the JRC, and several European Commission Expert Groups to drive change in the perception and measurement of scientific excellence. These collaborations have led to significant results that are impacting policy on a local, regional as well as global level (see the section “outcomes”).
Currently, we are working with IAP on a proposal on “Evaluating Research: Assessment and Improvement of Current Practice” and we are collaborating with the JRC on a document on measuring the impact of science advice.

Ongoing Projects

Raising awareness on good research practices through dissemination via Young Academies (project leads: Lydia Rhyman & Martin Dominik)
IAP-GYA-ISC Initiative: a proposal draft to be worked on by members during early summer (Project Lead: Carlo D’Ippoliti)
Led by the group, the GYA has joined the European Commission’s coalition for reforming research assessment and has become an active participant of this initiative (Project Leads: Sibel Eker and Yensi Flores)
An ideal conceptual model of the research environment that supports diversity and inclusion: Assess what is limiting us to get there and find interventions that have worked to get us on that direction (Project lead: Sibel Eker)
Group member Stefania Mondello acts as the GYA’s representative for the organising committee for a joint online workshop on Diversity in Peer Review with ALLEA and STM Publishers in November 2022
“Global trends governing career progression in research institutions and its impact on inclusion and diversity”
- This study collates 238 institutional & government progression evaluation documents spanning 56 countries
- At the moment, the group is disseminating the study in world regions with young research institutions (Project leads: Sandra Lopez-Verges, Alma Hernandez Mondragon)
An interactive panel session at The Trielennial Conference of the InterAcademy Partnership (IAP) and the Worldwide Meeting of Young Academies has been awarded to the group
An interactive panel session has been submitted to the World Science Forum and the group awaits results. Upon acceptance the group will start preparing material for this panel.

Outcomes

2022

In November, group members Stefania Mondello and Pradeep Kumar contributed to the second joint event by the GYA with ALLEA, the European Federation of Academies of Sciences and Humanities, and STM Publishers. The online webinar on ‘IDEA (Inclusion, Diversity, Equity and Accessibility) in scholarly peer-review’ examineed ways to expand Inclusion, Diversity, Equity and Accessibility in scholarly peer review. A report will be forthcoming.

2021

The InterAcademy Partnership, the GYA, and the International Science Council have created a scoping group on possible initiatives to be conducted together. The scoping group produced a menu of alternatives and are deciding on which activities to start.

Furthermore, the group provided feedback to the European Comission as part of their stakeholder consultation on problems with research evaluation and its conflicts with open science. These contributed to their scoping report, which proposes an alliance within the research community, based on an MoU.

2020

In November, group members participated in a joint online workshop with ALLEA, on “Research Assessment“. A brief workshop report can be read here.
In December, group member Boon Han Lim (Malaysia) presented the interim results of the “A Global View of Institutional Promotion Criteria for Professorship” project at the Young Korean Academy of Science and Technology event on “Measures to improve university faculty evaluation system”.

2018

In May, group members Laura Fierce (USA) and Koen Vermeir (France) presented the results of a survey on the “Study on Reproducibility and Replicability in Science and Engineering” in Washington, DC. A report “Young Scientist Perspectives on Replicability and Reproducibility in Science and Engineering” is currently under review.
In October, the report from the 2016 workshop on “Publishing models, assessment, and open science” was published, detailing the results of the workshop, which include a framework of questions and a set of concrete recommendations.

2017

Participation in “Policy impact of knowledge and knowledge organisations: From understanding impact towards measuring it”, June 2017 (Koen Vermeir)
On a request by ALLEA (ALL European Academies) we gave expert feedback on “The European Code of Conduct for Research Integrity – Revised Edition”, published in 2017. For the factsheet by the European Commission on this document, see: Factsheet on the new European Code of Conduct for Research Integrity. (Martin Dominik, Robert Lepenies, Koen Vermeir)

2016

In 2016, the Working Group submitted a well-received expert opinion for the Open Science Policy Platform of the European Commission report on “Next-generation metrics: Responsible metrics and evaluation for open science”, published in 2017.
Participation in “Open access and research evaluation: towards a new ecosystem”, Toulouse, October 2016 (Koen Vermeir)
In May 2016, group members organised a workshop on “Publishing models, assessment, and open science” in Eindhoven, The Netherlands, as a side event to the GYA Annual General Meeting. A brief workshop report was published in October 2018.

2015

In 2015, the group was invited to present the perspective of young scientists around the world on “Rewards, careers, and integrity” in a panel discussion “Research assessment and quality in science: Perspectives from international science and policy organisations” convened by ICSU as part of the “4th World Conference on Research Integrity”, held in Rio de Janeiro, Brazil, from 31 May to 3 June 2015. As a result of a critical analysis by the WG, concrete recommendations for implementation both in the short and long term were brought forward, embracing the link between integrity, transparency, openness, and reproducibility, while warning about pressures that young scientists are facing, and pointing to the effect of reward systems on brain drain.
Participation in “The Future of Scholarly Scientific Communication”, The Royal Society, London, Apr/May 2015 (Martin Dominik)

2014

Participation in “Beyond Bibliometrics – Identifying the Best“, 8^th Humboldt Forum, Berlin, Nov 2014 (Catherine Beaudry)
“What is the role of a scientist and what environment is required to fulfill it?”, session organised as part of the careers programme at the 2014 Euroscience Open Forum (ESOF) in Copenhagen, Denmark (Martin Dominik, Catherine Beaudry). As young scientists face a multitude of challenges in their research environments, there is increasing pressure to perform, to conform. Sometimes, diversifying beyond traditional niches is the only way forward. Early-career scientists possess an immense drive to engage, but this potential is not always met by opportunities, and they feel that their wings are clipped. Looking towards solutions for overcoming this dilemma, the session engaged the participants in an interactive debate on the role of a scientist and on the environment required to fulfill it. The dialogue between the two GYA members, Liudvika Leisyte (TU Dortmund) and Alaa Ibrahim (American University in Cairo), as well as the participants of the session touched on issues regarding the size of research grants, mobility, collaboration, gender issues, promotion criteria, peer review, to name a few.

2013

“Perceptions of Research Excellence in Thailand and Japan”, STI Policy Review 4, 113, 2013 (Orakanoke Phanraksa, Mitsunobu Kano, Shoji Komai, Wibool Piyawattanametha, Martin Dominik, Rob Jenkins). This publication is the result of the Working Group’s first project, started in 2012. We conducted a survey on the perceptions of research excellence at one leading research institute in Thailand. The findings were used as a basis for improvements in the institutional policy of that institute as well as to further compliment the GloSYS precursor study. In the publication, we assessed the research environment, the promotion of opportunities, and the structures that ensue from assessment criteria.

2012

Editorial in “Future of Medical Education Journal” on “Science, Education, and the World’s Future” by Bruce Alberts (FMEJ 2, 2, 2012), soliciting views on “How could young scientists unleash curiosity and creativity to support excellence in science education?“ (Reza Afshari, Marc Creus, Rob Jenkins)

The group also produced a video on “Unleashing Creativity”; see here.