Empty rituals? A qualitative study of users' experience of monitoring & evaluation systems in HIV interventions in western India. (November 2016)

• Record Type: Journal Article
• Authors: Shukla, Anuprita; Teedon, Paul; Cornish, Flora
• Is Part Of: Social science & medicine. Volume 168(2016)
• Page Start: 7
• Page End: 15
• Publication Date: 2016-11
• Subjects: India -- New managerialism -- Monitoring -- Philanthropy -- HIV/AIDS -- Sex workers -- Avahan -- NGO management; Social medicine -- Periodicals; Medical anthropology -- Periodicals; Public health -- Periodicals; Psychology -- Periodicals; Medicine -- Periodicals; Medicine -- Periodicals; Médecine sociale -- Périodiques; Anthropologie médicale -- Périodiques; Santé publique -- Périodiques; Psychologie -- Périodiques; Médecine -- Périodiques; Electronic journals; 362.105
• DOI: 10.1016/j.socscimed.2016.08.041
• Languages: English
• ISSNs: 0277-9536

Abstract

Abstract: In global health initiatives, particularly in the context of private philanthropy and its 'business minded' approach, detailed programme data plays an increasing role in informing assessments, improvements, evaluations, and ultimately continuation or discontinuation of funds for individual programmes. The HIV/AIDS literature predominantly treats monitoring as unproblematic. However, the social science of audit and indicators emphasises the constitutive power of indicators, noting that their effects at a grassroots level are often at odds with the goals specified in policy. This paper investigates users' experiences of Monitoring and Evaluation (M&E) systems in the context of HIV interventions in western India. Six focus groups (totalling 51 participants) were held with employees of 6 different NGOs working for government or philanthropy-funded HIV interventions for sex workers in western India. Ten donor employees were interviewed. Thematic analysis was conducted. NGO employees described a major gap between what they considered their "real work" and the indicators used to monitor it. They could explain the official purposes of M&E systems in terms of programme improvement and financial accountability. More cynically, they valued M&E experience on their CVs and the rhetorical role of data in demonstrating their achievements. They believed that inappropriate and unethical means were being used to meet targets, including incentives and coercion, and criticised indicators for being misleading and inflexible. Donor employees valued the role of M&E in programme improvement, financial accountability, and professionalising NGO-donor relationships. However, they were suspicious that NGOs might be falsifying data, criticised the insensitivity of indicators, and complained that data were under-used. For its users, M& E appears an 'empty ritual', enacted because donors require it, but not put to local use. In this context, monitoring is constituted as an instrument of performance management rather than as a means of rational programme improvement. Highlights: Studied frontline users' experience of M&E systems in HIV interventions in India. NGO staff experience a major gap between their 'real work' and indicators used. Donors value M&E in principle but are critical of the information produced. Without users' confidence in the system, M&E risks becoming an 'empty ritual'. Weak M&E systems may undermine staff morale and data quality.