Dealing with small sets of laboratory test replicates for Improved Cooking Stoves (ICSs): Insights for a robust statistical analysis of results. (August 2018)

• Record Type: Journal Article
• Authors: Lombardi, Francesco; Riva, Fabio; Colombo, Emanuela
• Is Part Of: Biomass and bioenergy. Volume 115(2018)
• Page Start: 27
• Page End: 34
• Publication Date: 2018-08
• Subjects: Cookstove -- Statistic -- Test -- Protocol -- Performance -- ICS; Biomass energy -- Periodicals; Biomass -- Periodicals; Energy-Generating Resources -- Periodicals; Bioénergie -- Périodiques; 333.9539
• DOI: 10.1016/j.biombioe.2018.04.004
• Languages: English
• ISSNs: 0961-9534

Abstract

Abstract: Improved Cooking Stoves (ICSs) represent the most commonly promoted solution to alleviate the burden associated with the use of traditional biomass in a short-term perspective. However, criticism is raising about the methodologies used for assessing their performance, with a particular focus on laboratory-based testing protocols. One of the key weaknesses of current protocols consists in the inaccurate and biased approach adopted for reporting and statistically analysing test results, which can lead to misleading conclusions about the actual improvements ensured by ICSs. This study proposes a robust procedure to deal with the statistical analysis of small sample sizes, and subsequently verify it through its application to an experimental comparison – based on the Water Boiling Test – between three models of stove. The results show that the current practice based on 3 or 5 replicates often produces biases in the analyses, as at least 13 replicates might be needed to achieve reliable results. Moreover, the study shows how the t-test is in most cases improperly applied, while the proposed procedure allows to deal both with normally and of non-normally distributed data sets in a robust way. In one case, the apparent improvement of an ICS model as compared to the three-stone fire, is refuted by the application of our procedure. Highlights: We propose a robust methodology to justify or refute the actual improvement of ICSs. We prove that current practice in lab tests analysis leads to biased conclusions. Results based on 3 or 5 replicates are not reliable. A prolonged lab campaign shows that at least 13 replicates might be needed. An apparently improved stove was not significantly improved when rigorously analysed.