Multi-criteria group individual research output evaluation based on context-free grammar judgments with assessing attitude. (December 2015)

• Record Type: Journal Article
• Authors: Li, Zongmin; Xu, Jiuping; Lev, Benjamin; Gang, Jun
• Is Part Of: Omega. Volume 57:Part B(2015:Dec.)
• Page Start: 282
• Page End: 293
• Publication Date: 2015-12
• Subjects: Multi-criteria group evaluation -- Individual research output -- Assessing attitude -- Weight determination; Management -- Periodicals; 658.4005
• DOI: 10.1016/j.omega.2015.09.001
• Languages: English
• ISSNs: 0305-0483

Abstract

Abstract: Individual research output (IRO) evaluation is a multi-criteria problem often conducted in groups. In practice, it is necessary to concurrently apply both bibliometric measures and peer review when evaluating the IRO. During the peer review process, different evaluators may use different linguistic terms because of individual differences in cognitive styles, and therefore, they may give ratings based on different assessing attitudes. Further, the weights between bibliometric measures and peer subjective judgments are difficult to determine. Motivated by these difficulties, this paper proposes a quantitative context-free grammar judgment description with an embedded assessing attitude. The proposed method quantitatively handles the assessing attitude and increases the flexibility of the linguistic information. Accordingly, this paper develops a multi-criteria group IRO evaluation method with context-free grammar judgments which concurrently considers bibliometric measures and peer review opinions. To overcome the weighting difficulties and achieve the maximum consensus, this paper proposes a distance-based method to determine the evaluators' weights and a weighted averaging operator to compute the criteria weights. After that, a TOPSIS-based aggregation method is applied to aggregate the objective and subjective ratings. A practical case study is then used to test the feasibility of the methodology. Finally, we discuss the effectiveness of the proposed method. Abstract : Highlights: A multi-criteria group individual research output evaluation method is studied. Bibliometric measures and peer review are concurrently applied in the evaluation. Context-free grammar judgments with assessing attitude are considered. A distance-based weighting method and a weighted averaging operator are developed.