There and Back Again: Can you Compile that Snapshot?

This web page is a companion to our JSME 2016 journal paper entitled "There and Back Again: Can you Compile that Snapshot?".

Abstract

A broken snapshot represents a snapshot from a project’s change history that cannot be compiled. Broken snapshots can have significant implications for researchers, as they could hinder any analysis of the past project history that requires code to be compiled. Noticeably, while some broken snapshots may be observable in change history repositories (e.g., no longer available dependencies), some of them may not necessarily happen during the actual development. In this paper we systematically study the compilability of broken snapshots in 219,395 snapshots belonging to 100 Java projects from the Apache Software Foundation, all relying on Maven as an automated build tool. We investigated broken snapshots from two different perspectives: (i) how frequently they happen and (ii) likely causes behind them. The empirical results indicate that broken snapshots occur in most (96%) of the projects we studied and that they are mainly due to problems related to the resolution of dependencies. On average, only 38% of the change history of project systems is currently successfully compilable.

Analyzed Projects


This link provides the following data for each analyzed project:
  • Commpilation Log: the compilability results for every single project's snapshot
  • Evolution Phases: intervals of commits sharing the same compilability status
  • Commit and Developer Factors: metrics analyzed for every single project's snapshot
  • Releases: distance between breaking changes/fixes and the next release

The repository of each project can be found at the Apache website, which provides a GitHub link for each project.

Note: The compilation logs contain partial data for some projects with long change history. However, such data represents most of the data used in the study and contain the sequentially continuous list of snapshots on the master branch from the beginning of the change history of each analyzed project.

Data Analysis

This link provides the following data:
  • Data: aggregated analyzed data
  • Scrips: R scripts used to analyze the data and generate plots
  • Results: output and plots

Build Error Classification

The following error classification has been performed:

ExceptionCategory
AetherClassNotFoundOther
ArtifactResolutionExceptionResolution
ConnectExceptionOther
DependencyResolutionExceptionResolution
DuplicateProjectExceptionParsing
InternalErrorExceptionOther
InvalidPluginDescriptorExceptionParsing
InvalidRepositoryExceptionParsing
LifecycleNotFoundExceptionParsing
LifecyclePhaseNotFoundExceptionParsing
LocalRepositoryNotAccessibleExceptionResolution
MalformedInputExceptionParsing
MissingProjectExceptionResolution
ModelParseExceptionParsing
MojoExecutionExceptionOther
CompilationFailureExceptionCompilation
MojoFailureExceptionOther
MojoNotFoundExceptionParsing
MultipleArtifactsNotFoundExceptionResolution
NoGoalSpecifiedExceptionParsing
NoPluginFoundForPrefixExceptionParsing
OutOfMemoryErrorOther
PluginConfigurationExceptionOther
PluginContainerExceptionOther
PluginDescriptorParsingExceptionParsing
PluginExecutionExceptionOther
PluginIncompatibleExceptionParsing
PluginParameterExceptionParsing
PluginResolutionExceptionResolution
PluginVersionResolutionExceptionParsing
ProjectBuildingExceptionOther
UnresolvableModelExceptionResolution
ProjectCycleExceptionParsing

The documentation for each exception is available on the Apache Maven website.

Authors

  • Michele Tufano - The College of William and Mary, VA, USA.
    E-mail: mtufano at cs dot wm
  • dot edu
  • Fabio Palomba - University of Salerno, Salerno, Italy.
    E-mail: fpalomba at unisa
  • dot it
  • Gabriele Bavota - Free University of Bozen-Bolzano, Bolzano, Italy.
    E-mail: gabriele.bavota at unibz
  • dot it
  • Massimiliano Di Penta - University of Sannio, Benevento, Italy.
    E-mail: dipenta at unisannio
  • dot it
  • Rocco Oliveto - University of Molise, Pesche (IS), Italy.
    E-mail: rocco.oliveto at unimol
  • dot it
  • Andrea De Lucia - University of Salerno, Salerno, Italy.
    E-mail: adelucia at unisa
  • dot it
  • Denys Poshyvanyk - The College of William and Mary.
    E-mail: denys at cs dot wm dot edu