Work practices and challenges in continuous integration: A survey with Travis CI users. (11th September 2018)

• Record Type: Journal Article
• Authors: Pinto, Gustavo; Castor, Fernando; Bonifacio, Rodrigo; Rebouças, Marcel
• Is Part Of: Software, practice & experience. Volume 48:Number 12(2018)
• Page Start: 2223
• Page End: 2236
• Publication Date: 2018-09-11
• Subjects: build breakage -- continuous integration -- Travis CI -- user survey; Computer software -- Periodicals; Computer programming -- Periodicals; Computer programs -- Periodicals; 005.3
• DOI: 10.1002/spe.2637
• Languages: English
• ISSNs: 0038-0644

Abstract

Summary: Continuous integration (CI) is a software development practice that has been gaining increasing popularity in the last few years. However, we still miss a collection of experiences regarding how software developers perceive the idea of CI, in terms of its fundamental concepts, the reasons that motivate the adoption of this practice, the reasons for build breakage, and the benefits and problems related to CI. To shed light on this direction, we conducted a user survey with 158 CI users. Through a mostly qualitative investigation, we produce a list of findings that are not always obvious. For instance, we observed that (1) developers are not sure whether a job failure represents a failure or not; (2) inadequate testing is the most common technical reason related to build breakage, whereas lack of time plays a role on the social reasons; and (3) although some respondents reported that CI systems increase the confidence that the code is in a known state, some respondents also reported that there is a false sense of confidence when blindly trusting tests. This empirical study is particularly relevant to those interested in better understanding and fostering CI practices either in an open‐source or industrial setting.