Combining internal and external evaluations within a multilevel evaluation framework: Computational text analysis of lessons from the Asian Development Bank. (July 2019)

• Record Type: Journal Article
• Authors: Goyal, Nihit; Howlett, Michael
• Is Part Of: Evaluation. Volume 25:Number 3(2019:Jul.)
• Page Start: 366
• Page End: 380
• Publication Date: 2019-07
• Subjects: external evaluation -- internal evaluation -- international development aid -- multilevel evaluation framework -- text mining; Evaluation research (Social action programs) -- Periodicals; 361.61068
• DOI: 10.1177/1356389019827035
• Languages: English
• ISSNs: 1356-3890

Abstract

Although the literature on evaluation has theorized about the distinction between internal and external evaluation, hardly any research has compared them empirically. This article examines whether the lessons of internal evaluations differed from those of external evaluations in the case of international development aid. It analyzes internal evaluations of the Asian Development Bank for nearly 1000 sovereign interventions across 38 countries in the Asia-Pacific during 1996–2016, using computational text analysis or text mining techniques. The results show that internal evaluations focused more on micro- and meso-level characteristics, while external evaluations laid more emphasis on meso- and macro-level constructs, such as dimensions of policy and the institutional environment in the recipient country, or its level and rate of economic growth. The article concludes that internal and external evaluations can be combined to create a multilevel evaluation framework that integrates micro-, meso-, and macro-level lessons to facilitate better learning.