Jardeps

Per-tree Java dependencies in Makefiles — This library allows you to write Makefiles for projects that include Java source trees. You only need to specify the root classes of each tree, and the dependencies between trees, and you get incremental builds at tree granularity. Trying to get incremental builds at file or class granularity is ill-advised.


Description

Most advice on-line about using makefiles with Java says the following:

Don't! Use Ant instead.

I disagree that a tool newer than GNU Make is necessary, and have some comments about why I don't use Ant.

Other advice, that which advocates the use of Make, often naïvely suggests something like this:

%.class: %.java
	$(JAVAC) $<

This results in many more invocations of the compiler than necessary (undermining one of the benefits of an incremental build), and potentially generates an incorrect build.

Jardeps is an attempt at incremental builds for Java projects using makefiles, and I believe that it is a better approach than using the <javac> task in Ant (prior to Java 8, or without <depend>), or a naïve rule in GNU Make.

Here's what is expected from the developer using this scheme:

  • The source should be split into source trees, such that there are no dependency cycles spanning more than one tree. (Cyclic dependencies within a tree are permitted.)

  • The trees which make up a jar are specified. (It is reasonable to have one tree per jar, and this is the default.)

  • The root classes of each tree should be identified. These are the ones which the developer positively requires in the resultant jar.

    You have the option to compile all source files, with specific exclusions.

  • Dependencies between trees should be specified manually. For example, the developer should know that tree impl depends on tree api.

Here's what the developer can expect in return:

  • A built tree will only contain its root classes, plus everything in the same tree which those classes need, i.e., the implicit classes.

  • A tree will only be recompiled if any of its source files changes, or the public profile of one of the trees it depends upon changes. This will allow you to make modifications on the internals of a tree without triggering a recompilation of dependent trees.

    A profile is a summary of the public aspects of the classes in a tree, automatically generated when it is compiled. You can also express dependencies on package-private (default) profiles, or even on the implementation, if necessary.

In summary, you get an incremental build at the granularity of trees (which is feasible both efficiently and correctly), rather than of classes (which is not).

The main problem with Jardeps is speed. The compiler must keep track of source files it used, and class files it generated, though the overhead for these should be low. However, it also has to perform a post-compilation analysis of the class files, which might significantly increase total processing time. Furthermore, these additional activities are bolted onto the compiler by invoking it from a Java process, which should save time by avoiding duplication of work done by invoking a separate process after the native compiler, but which could also increase overhead due to having to start a Java process just to run the compiler, an overhead which a native invocation of javac might avoid. Additionally, under Jardeps, separate trees are intentionally compiled with separate invocations of the compiler, so this overhead could be incurred several times.

I have not measured how bad this overhead is, and my Java projects are probably too small to amortize the overhead by not having to recompile everything, so if you try Jardeps on a bigger project, please let me know how well it performs. Also, Jardeps could be seen as a proof-of-concept, demonstrating that all the information required to generate rules for make can be obtained by augmenting the compiler, and so some of the speed can be recovered by augmenting the native compiler. These augmentations come in the form of some additional options. Despite my opposition to Ant, such options could also be exploited by an enhanced <javac> task.

Changes

  • 2021-02-13Class-Path set in manifest to indicate dependency of one jar on another, based on tree dependencies. Annotation type added to mark entry points, and select one per jar for inclusion as Main-Class in manifest. Added support for CARP export. Fixed space macro for GNU Make 4.3.
  • 2021-01-09 — Annotation processors' reading of non-Java files on the source path is detected as dependency. package-info.java is submitted for compilation so that its annotations are processed. Annotation processors' non-Java outputs in the destination directory are added to jar contents. Directories with intermediate files are marked as caches to be skipped by tar.
  • 2019-03-13 — Roots can be expressed as ‘all Java files except…’, using roots_foo=$(found_foo) and listing exceptions in exroots_foo.
  • 2017-05-02 — Renamed jlink to jrun.
  • 2016-08-10 — More annotations are recorded in tree profiles, so changes to annotations on extended classes and interfaces should trigger recompilation.
  • 2016-05-15 — Added support for CORBA IDL code generation. (Dependence on gawk has been reintroduced, as idlj does not provide much help in detecting dependencies. Also, revised formation of classpath during compilation, so that external parts that don't trigger recompilation are separate from other parts that do, and the ordering of composition between per-tree and global parts is clarified.)
  • 2015-08-29 — Dropped use of separate per-jar/per-tree makefiles.
  • 2015-04-16 — Added service annotation.
  • 2015-04-12 — Included jlink.
  • 2014-10-15 — Works with parallel builds.
  • 2014-09-28 — Added installation procedure for versioned jars.
  • 2014-09-21 — Generation of complex rules performed with evaluated macro expansion instead of creation of external files.
  • 2014-05-15 — Fixed and improved the creation of manifests.
  • 2014-04-16 — By using the Java compiler API in javax.tools, Jardeps now uses its own compiler so that it can extract the full list of source files and class files accessed. Previously, a post-processor made its best effort to achieve this, more slowly, and with incomplete results.
  • 2013-05-07 — Improved support for merging lists of service implementations, for per-tree settings, for annotation-processor dependencies, and for dependencies of trees on other jars.
  • 2013-03-07 — Annotation processing is now possible on root classes, and some experimental support for processing of implicit classes can be switched on.
  • 2012-05-14 — A jar can now be constructed from multiple source trees. Previously, a jar and its source tree were synonymous.
  • 2012-05-13Jardeps is now installable. This removes the dependence on your project being maintained in a Subversion repository (although you can still do it that way).
  • 2012-01-05javap and gawk have been dropped in favour of a single Java progam doing the classfile analysis. This should make things a lot more robust, and is a little faster too!

Files

File Size Last modified Description Requirements and recommendations
Source (Git) GNU Make Java 1.6 Linux