A 'power law' based method to reduce size-related bias in indicators of knowledge performance: An application to university research assessment. Issue 4 (November 2018)

• Record Type: Journal Article
• Authors: Calabrese, Armando; Capece, Guendalina; Costa, Roberta; Di Pillo, Francesca; Giuffrida, Stefania
• Is Part Of: Journal of informetrics. Volume 12:Issue 4(2018)
• Page Start: 1263
• Page End: 1281
• Publication Date: 2018-11
• Subjects: Knowledge performance -- Research assessment -- Research productivity indicators -- Power laws -- Dimensional bias -- Scale-free property; Library statistics -- Periodicals; Information science -- Statistical methods -- Periodicals; Bibliometrics -- Periodicals; Bibliothèques -- Statistiques -- Périodiques; Sciences de l'information -- Méthodes statistiques -- Périodiques; Bibliométrie -- Périodiques; 020.727
• DOI: 10.1016/j.joi.2018.10.005
• Languages: English
• ISSNs: 1751-1577

Abstract

Highlights: The method reduces size-related bias affecting knowledge performance indicators using the scale-free property of power laws. The paper analyses the issue of size-related bias introduced in evaluation exercise outputs by varying university size. A scaled performance indicator is introduced for the normalization of the evaluation average score distributions. The method has policy implications and gives a contribution for the future design of research assessment exercises. Abstract: The knowledge production provided by universities is essential to sustaining a country's long-term economic growth and international competitiveness. Many nations are thus driving to create sustainable and effective funding environments. The evaluation of university knowledge, productivity and research quality becomes critical, with ever increasing share of public funding allocated on the basis of research assessment exercises. Nevertheless, the existing methods to assess the universities' knowledge production are often affected by limits and biases, extensively discussed in the scientific literature. In this paper we study how to reduce the effect of size-related bias due to university size on the indicators of knowledge performance used in evaluation exercises. We propose an innovative utilization of the scale-free property of the power laws as a scaling relationship, to normalize research productivity indicators, and provide results independent by the university size. Our method has evident policy implications and gives a contribution for the future design of assessment exercises. We apply our findings in a recent Italian research assessment exercise.