Algorithms and data behind Leiden Ranking in public domain
Unlike other major university rankings, the Leiden Ranking – Open Edition is completely transparent. The ranking compiled by the Centre for Science and Technology Studies (CWTS) uses open data and publishes the algorithms that are used.
The Leiden Ranking is viewed worldwide. It has been in existence for over fifteen years and takes a very different approach from other major rankings such as the QS World University Rankings and the Shanghai Ranking. The Leiden Ranking does not purport to give a complete picture of a university’s performance. Rather, it deliberately chooses to restrict itself to scientific articles published by universities.
Ludo Waltman (CWTS): ‘Besides that, we don’t just lump all the figures together to say what’s the best university in the world. We show figures side by side, which means we actually make several rankings.’ In the Leiden Ranking you can search, for example, how often a university’s articles are cited, the degree to which they’re available open access and how much they engage in international cooperation.
Leiden Ranking – Open Edition
The Leiden Ranking – Open Edition is compiled by CWTS in collabarotion with Curtin Open Knowledge Initiative (COKI), Sesame Open Science and OurResearch.
No more black box
CWTS always used data from Web of Science for the Leiden Ranking. Waltman: ‘It is in many ways a high-quality data source, but the data is not accessible by everyone so we are no longer using it. From now on we are using data from OpenAlex that everyone can access and use without any restrictions. Previously, our ranking was something of a black box, and people couldn’t properly check our work. Now that, with the Open Edition, the data is public and we also publish our algorithms, it can be checked. Someone can now compile an alternative ranking themselves, making different choices. Everyone can decide for themselves what they think is important to measure a university’s performance. That’s unique.’
When CWTS announced that an Open Edition of the Leiden Ranking was going to be made, there were a lot of positive reactions, but also some concerns. In some countries rankings have a big influence on research policy and financing. Waltman: ‘That’s why we also feel obliged to publish the Open Edition at the moment alongside the traditional Leiden Ranking (with data from Web of Science). The Open Edition is still in the experimental stage and we’re hoping for a lot of feedback.’
The transparency of the Open Edition can help users of rankings to think more critically about how universities perform.
Transition
Waltman believes that the transparency of the Leiden Ranking – Open Edition can help users of rankings to think more critically about how universities perform. A transition towards openness of the data on which rankings and other indicators are based is needed for a university as an academic community to be able to arrive responsibly at decisions. ‘Universities to some extent delegate the taking of important decisions to data-driven indicators sourced from commercial parties. These parties have interests that are not necessarily in line with our own. If data and methodologies are in the public domain, you are better able to discuss with one another what can and can’t be concluded from all kinds of data-driven indicators. You can then make better decisions.’
Discussion about rankings
The Open Edition of the Leiden Ranking is a reflection of the current discussion in the Netherlands about rankings. Dutch universities have agreed to take a more critical approach to university rankings. Many of the lists falsely claim to be able to summarise the performance of a university in a single figure. The rankings are also at odds with the commitment to new ways of ‘Recognition and Rewards’. Universities want to assess their staff not only on publications in scientific journals (that weigh heavily in the rankings), but, for example, also on their teaching, leadership, team performance or science communication.
Academia in Motion
Ludo Waltman is ambassador for open science within Leiden University’s Academia in Motion-programma. The aim of the programme is to look for new ways of recognition and rewards and to put open science at the heart of academic work. The Leiden Ranking – Open Edition programme is a good example of open science.
Text: Dagmar Aarts
Image: CWTS