Analysis of methods commonly used in biomedicine for treatment versus control comparison of very small samples. (April 2018)

• Record Type: Journal Article
• Authors: Ristić-Djurović, Jasna L.; Ćirković, Saša; Mladenović, Pavle; Romčević, Nebojša; Trbovich, Alexander M.
• Is Part Of: Computer methods and programs in biomedicine. Volume 157(2018)
• Page Start: 153
• Page End: 162
• Publication Date: 2018-04
• Subjects: Biomedical statistics -- Computer simulation -- Errors -- Qualitative comparison -- Very small samples; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2018.01.026
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: Unstandardized comparison of samples smaller than ten is not uncommon in biomedical research. The sample comparison error depends strongly on the sample size, comparison method and the intensity of the studied effect, whereas the influence of the event distribution is negligible. The recommended minimal sample size is 9 and the t -test with p = 5% as the comparison method. Use of smaller sizes and the method of their comparison should be justified by the objective of the experiment. A broad consensus among biomedical research community and a firm requirement regarding the minimal acceptable biomedical sample size is needed. Abstract: Background and objective: A rough estimate indicated that use of samples of size not larger than ten is not uncommon in biomedical research and that many of such studies are limited to strong effects due to sample sizes smaller than six. For data collected from biomedical experiments it is also often unknown if mathematical requirements incorporated in the sample comparison methods are satisfied. Methods: Computer simulated experiments were used to examine performance of methods for qualitative sample comparison and its dependence on the effectiveness of exposure, effect intensity, distribution of studied parameter values in the population, and sample size. The Type I and Type II errors, their average, as well as the maximal errors were considered. Results: The sample size 9 and the t - test method with p = 5% ensured error smaller than 5% even for weak effects. For sample sizes 6–8 the same method enabled detection of weak effects with errors smaller than 20%. If the sample sizes were 3–5, weak effects could not be detected with an acceptable error; however, the smallest maximal error in the most general case that includes weak effects is granted by the standard error of the mean method. The increase of sample size from 5 to 9 led to seven times more accurate detection of weak effects. Strong effects were detected regardless of the sample size and method used. Conclusions: The minimal recommended sample size for biomedical experiments is 9. Use of smaller sizes and the method of their comparison should be justified by the objective of the experiment.