From rankings to funnel plots: The question of accounting for uncertainty when assessing university research performance. Issue 3 (August 2016)

• Record Type: Journal Article
• Authors: Abramo, Giovanni; D'Angelo, Andrea Ciriaco; Grilli, Leonardo
• Is Part Of: Journal of informetrics. Volume 10:Issue 3(2016:Jul.)
• Page Start: 854
• Page End: 862
• Publication Date: 2016-08
• Subjects: Bibliometrics -- Universities -- FSS -- Productivity -- Funnel plot -- Italy; Library statistics -- Periodicals; Information science -- Statistical methods -- Periodicals; Bibliometrics -- Periodicals; Bibliothèques -- Statistiques -- Périodiques; Sciences de l'information -- Méthodes statistiques -- Périodiques; Bibliométrie -- Périodiques; 020.727
• DOI: 10.1016/j.joi.2016.07.005
• Languages: English
• ISSNs: 1751-1577

Abstract

Highlights: We apply the funnel plot display for measuring uncertainty in research performance. We compare research performance of Italian universities in the science disciplines. We use the FSS research productivity indicator. Most universities have a performance not significantly different from the mean. There are only few instances of markedly outstanding performance. Abstract: The work applies the funnel plot methodology to measure and visualize uncertainty in the research performance of Italian universities in the science disciplines. The performance assessment is carried out at both discipline and overall university level. The findings reveal that for most universities the citation-based indicator used gives insufficient statistical evidence to infer that their research productivity is inferior or superior to the average. This general observation is one that we could indeed expect in a higher education system that is essentially non-competitive. The question is whether the introduction of uncertainty in performance reporting, while technically sound, could weaken institutional motivation to work towards continuous improvement.