Improving Regression Testing in Continuous Integration Development Environments - Gregg Rothermel

• Report rights infringement
• published: 14 Mar 2016
• views: 578

ISR Distinguished Speaker Gregg Rothermel Professor and Jensen Chair of Software Engineering University of Nebraska-Lincoln “Improving Regression Testing in Continuous Integration Development Environments” Friday, March 11, 2016 FACULTY HOST: James A. Jones ABSTRACT: In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. Merged code is then regression tested to help ensure that the codebase remains stable and that continuing engineering efforts can be performed more reliably. Continuous integration is advantageous because it can reduce the amount of code rework that is needed in later phases of development, and speed up overall development time. From a testing standpoint, however, continuous integration raises several challenges. Chief among these challenges are the costs, in terms and time and resources, associated with handling a constant flow of requests to execute tests. To help with this, organizations often utilize farms of servers to run tests in parallel, or execute tests ``in the cloud'', but even then, test suites tend to expand to utilize all available resources, and then continue to expand beyond that. We have been investigating strategies for applying regression testing in continuous integration development environments more cost-effectively. Our strategies are based on two well-researched techniques for improving the cost-effectiveness of regression testing -- regression test selection (RTS) and test case prioritization (TCP). In the continuous integration context, however, traditional RTS and TCP techniques are difficult to apply, because these techniques rely on instrumentation and analyses that cannot easily be applied to fast-arriving streams of test suites. We have thus created new forms of RTS and TCP techniques that utilize relatively lightweight analyses, that can cope with the volume of test requests. To evaluate our techniques, we have conducted an empirical study on a large data set from Google, one of the many organizations that currently employs continuous integration. In this talk, I describe our techniques and the empirical results we have obtained in studying them. ABOUT THE SPEAKER: Gregg Rothermel is Professor and Jensen Chair of Software Engineering at the University of Nebraska-Lincoln. He received the Ph.D. in Computer Science from Clemson University working with Mary Jean Harrold, the M.S. in Computer Science from SUNY Albany, and a B.A. in Philosophy from Reed College. His research interests include software engineering and program analysis, with emphases on the application of program analysis techniques to problems in software maintenance and testing, end-user software engineering, and empirical studies. He received a National Science Foundation CAREER Award in 1996 for his research on software maintenance and testing. He co-founded and leads the development of the Software-Artifact Infrastructure Repository (SIR), a repository of software-related artifacts that support rigorous controlled experiments with program analysis and software testing techniques, and has been utilized, to-date, by more than 1000 persons from over 300 institutions around the world, supporting over 200 scientific publications. His research has also been supported by NSF, DARPA, AFOSR, Boeing Commercial Airplane Group, Microsoft, Lockheed Martin, and Rogue Wave Software. Dr. Rothermel is a member of the Editorial Boards of the Empirical Software Engineering Journal and Software Quality Journal. Previous positions include Associate Editor in Chief for IEEE Transactions on Software Engineering, General Chair for the 2009 International Symposium on Software Testing and Analysis, Program Co-Chair for the 2007 International Conference on Software Engineering, and Program Chair for the 2004 ACM International Symposium on Software Testing and Analysis.