Test Dependencies and the Future of Build Acceleration

JONATHAN BELL

Programming Systems Laboratory, Department of Computer Science, Columbia University

With the proliferation of testing culture, many developers are facing new challenges. As projects are getting started, the focus may be on developing enough tests to maintain confidence that the code is correct. However, as developers write more and more tests, performance and repeatability become growing concerns for test suites. In our study of large open source software, we found that running tests took on average 41% of the total time needed to build each project – over 90% in those that took the longest to build. Unfortunately, typical techniques for accelerating test suites from literature (like running only a subset of tests, or running them in parallel) can’t be applied in practice safely, since tests may depend on each other. These dependencies are very hard to find and detect, posing a serious challenge to test and build acceleration. In this talk, I will present my recent research in automatically detecting and isolating these dependencies, enabling for significant, safe and sound build acceleration of up to 16x.

Jon is a fourth year PhD candidate at Columbia University studying Software Engineering with Prof Gail Kaiser. His research interests in Software Engineering mostly fall under the umbrella of Software Testing and Program Analysis. Jon’s recent research in accelerating software testing has been recognized with an ACM SIGSOFT Distinguished Paper Award (ICSE ’14), and has been the basis for an industrial collaboration with the bay-area software build acceleration company Electric Cloud. Jon actively participates in the artifact evaluation program committees of ISSTA and OOPSLA, and has served several years as the Student Volunteer chair for OOPSLA.

• Poster
• Slides: