Configuration
To assess the usability of our approach, we also performed a study by submitting pull requests to popular open source Java projects on GitHub. We ensured that projects compiled correctly and had identical unit test results and compiler warnings before and after the refactoring. In some instances, minor manual intervention was required, such as to merge javadoc, fix build (e.g., maven, gradle) module dependencies for source methods that crossed module boundaries.
Results
Of the pull requests for which developers have since responded at the time of this writing, 4 have been successfully merged, thus, the refactoring results have been integrated into these projects. Ranging in size and domain, they include projects and frameworks from large companies and organizations such as Eclipse, AOL, and NHL (National Hockey League), and include the popular Eclipse (formerly Goldman Sachs) Collections framework. In all, the merged projects total 163 watches, 1071 stars, and 180 forks , indicating that they have significant usage. The change sizes totaled the addition of 349 lines of code and the removal of 647 lines of code.
Several other projects, although enthusiastic about the approach, rejected our pull requests, citing reasons such as that they had not yet moved to Java 8 (or were in the process of moving), needed to support older Java clients at the time, were in the process of a major release, or wanted to force clients to implement particular interface methods despite the existence of a corresponding skeletal implementation class.
Below is a listing of GitHub pull request URLs used for our study, along with statistics (as of this writing) and whether the pull request has been merged by the project. Real-time statistics may be retrieved by visiting any of the projects on GitHub. Pull requests with no merge indication signifies that we are waiting for a response from the developers.
The complete dataset may be viewed below or directly as a Google Sheet.
Some of the summary data present in the “Statistical Tests” sheet is generated via the following R script:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| changed_files_non_merged <- c(1, 5, 6, 16, 16, 22, 23, 29, 51, 135) | |
| changed_files_merged <- c(2, 4, 4, 21) | |
| changed_lines_non_merged <- c(22,27,62,729,681,283,161,413,922,2528) | |
| changed_lines_merged <- c(23,53,441,526) | |
| concrete_non_merged <- c(1,1,0,0,0,0,1,0,0,0) | |
| concrete_merged <- c(0,1,0,0) | |
| KLOC_non_merged <- c(25,7,20,309,244,136,154,329,1266,506) | |
| KLOC_merged <- c(99,5,2,1266) | |
| stars_non_merged <- c(865,1688,17234,2048,14721,1614,21792,1225,21063,12463) | |
| stars_merged <- c(554,744,4,258) | |
| forks_non_merged <- c(215,173,5808,373,3502,411,3819,811,7275,9575) | |
| forks_merged <- c(54,183,1,63) | |
| watches_non_merged <- c(146,114,1783,215,1568,127,1677,196,1928,2299) | |
| watches_merged <- c(68,103,2,40) | |
| contribs_non_merged <- c(41,28,71,16,98,40,142,61,784,200) | |
| contribs_merged <- c(21,5,2,18) | |
| # changed files | |
| wilcox.test(changed_files_non_merged, changed_files_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # changed lines | |
| wilcox.test(changed_lines_non_merged, changed_lines_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # concrete | |
| wilcox.test(concrete_non_merged, concrete_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # KLOC | |
| wilcox.test(KLOC_non_merged, KLOC_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # stars | |
| wilcox.test(stars_non_merged, stars_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # forks | |
| wilcox.test(forks_non_merged, forks_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # watches | |
| wilcox.test(watches_non_merged, watches_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) | |
| # contribs | |
| wilcox.test(contribs_non_merged, contribs_merged, paired = FALSE, alternative = "greater", mu = 0.0, | |
| exact = TRUE, correct = TRUE, conf.int = TRUE, conf.level = 0.95) |
