How to use Jardeps


Jardeps is a library for GNU Make for compiling Java. Make sure you understand how to install Jardeps before proceeding. We'll assume you've already got a working directory, with a file called Makefile containing one of these lines, according to your choice of installation method:

include jardeps.mk
include jardeps/jardeps.mk

Source tree definition

Suppose you have a source tree, where the source for each top-level class is in a directory hierarchy that reflects the class's package, so the class MyApp in the package com.example is defined in a source file com/example/MyApp.java. Jardeps supports multiple source trees, each with a user-defined name. You define a tree called core (say) by placing it in your working directory under the name src/core/. Source for the class com.example.MyApp would then go in src/core/com/example/MyApp.java.

Define the contents of tree core, either by whitelisting the root classes:

# In Makefile
roots_core += com.example.MyApp

…or by finding all source files, and optionally blacklisting some:

# In Makefile
roots_core=$$(found_core)
exroots_core += com.example.IncompleteClass
exroots_core += com.example.RedundantClass

If you now run make out/core.jar in the working directory, the selected classes will be compiled to form the requested jar. Other classes in the source tree, even excluded ones, will be compiled if they are referenced directly or indirectly by the root classes. These are the implicit classes.

If you run make out/core.jar again without changing any sources, no action will be taken, because re-compilation is unnecessary. However, if you modify any of the source files used in the previous compilation, or alter the set of root classes, the whole source tree is recompiled, and the jar rebuilt.

Blacklisting has the following advantages:

  • You don't have to maintain the list as much, as excluding classes is rare, while adding and removing classes is quite common.

  • All root classes get annotation-processed. (Excluded classes will only get compiled as implicit classes, if at all.)

Note that you always list fully-qualified top-level class names, not filenames.

Resource bundles

You can specify families of resource bundles:

# In Makefile
dlps_foo += com.example.project.Messages

All files matching src/foo/com/example/project/Messages*.properties are converted to ASCII and copied with the class files. Then Messages.properties absorbs defaults from Messages_en.properties, or from some other language specified by DEFAULT_LANGUAGE.

Note that resource bundles could be implemented by classes, so you always list fully-qualified top-level names of the equivalent classes, not filenames, even if there is no such matching class.

Static files

You can specify files to be copied across unchanged:

# In Makefile
statics_core += com/example/project/defaultConfig.properties

Note that you specify a filename, not a class name, as the resource doesn't necessarily correspond to a class. The filename is relative to the tree root.

Inter-tree dependencies

When you have multiple trees, you can express dependencies between them. Typically, one tree will depend on the public profile or signature of another, i.e., the set of signatures of all public/protected members of public/protected classes, as well as those classes' inheritance, the values of inlined constants, and annotations of all those elements. Changes to a source tree that do not affect its signature should also not affect other trees that depend on it.

Suppose you have two trees, api (defining the API of a library) and impl (providing a reference implementation of it). You want to rebuild impl if any public part of api changes, so you express that as follows:

# In Makefile
deps_impl += api

Now, if you run make out/impl.jar, that will build out/api.jar first, and then out/impl.jar. If you change any source of impl and re-run, only out/impl.jar will be recompiled and rebuilt. If you make an internal or trivial change to api, (like changing a method body, or simply reformatting), and re-run, only out/api.jar will be rebuilt. If you make a more substantial change, both jars will be rebuilt.

Dependencies can be chained to any depth, so the implementation of foo can depend on the signature of bar, and the implementation of bar can depend on the signature of baz. There is no implicit transitivity, however, so changes to the signature of baz do not trigger a rebuild of foo. Also, there can be no circular dependencies.

You can specify stronger inter-tree dependencies:

  • ppdeps_foo lists trees whose package-private profiles foo depends on. This is intended for test suites.

  • rtdeps_foo lists trees that will be executed in order to compile foo. This is intended for annotation processors and other code generators, but you shouldn't need to use it directly.

  • apdeps_foo lists trees which contain annotation processors applicable to classes in foo. Its contents are automatically added to rtdeps_foo. (Perhaps this makes rtdeps_foo redundant. What use is it, other than to run annotation processors?)

apdeps_foo allows you to write an annotation processor in one tree, and apply it to source in another.

Compiler options

Source code is compiled using the classpath specified by CLASSPATH and PROJECT_CLASSPATH. Spaces are converted to colons, so you can extend the classpath using make syntax:

# In Makefile
PROJECT_CLASSPATH += /usr/lib/graphics.jar

Changes to PROJECT_CLASSPATH trigger a recompilation of all Java trees. Changes to CLASSPATH do not. If you include local configuration for your project like this:

# In Makefile
-include myproject-env.mk

…you should set CLASSPATH in that file, so that changes to the local configuration do not trigger recompilation:

# In myproject-env.mk
CLASSPATH += /usr/lib/graphics.jar

If you want to set the classpath just for tree foo, just suffix the variable name:

# In Makefile
CLASSPATH_foo += /usr/lib/graphics.jar

deps_foo and ppdeps_foo automatically extend CLASSPATH_foo.

There are also variables PROCPATH, PROJECT_PROCPATH and PROCPATH_foo, used to set the classpath for annotation processors. PROCPATH_foo is automatically extended by apdeps_foo, if you want to use annotation processors provided by trees in the same project.

If you need to set any other options, JAVACFLAGS and PROJECT_JAVACFLAGS can contain them. There is also JAVACFLAGS_foo for options specific to tree foo.

Multiple trees per jar

By default, each jar consists of a single tree of the same name. If you want a jar to consist of any other set of trees, you must list them with trees_jar. For example, if jar quux should contain code from trees yan, tan and tither, you might write:

# In Makefile

# Jar quux consists of trees yan, tan and tither.
jars += quux
trees_quux += yan
trees_quux += tan
trees_quux += tither

Or you could have one tree per jar, and just use this mechanism to shorten the tree name (which is internal to your project):

# In Makefile
jars += foo_api
jars += foo_plugin
jars += foo_impl1
jars += foo_impl2

trees_foo_api=api
trees_foo_plugin=plugin
trees_foo_impl1=impl1
trees_foo_impl2=impl2

Export of CARP IDLs

If you can build out/foo.jar, you can also build out/foo-carp.zip. It will contain only the carp.rpc files in the source trees that make up the jar. This is intended to allow export to other target languages. (CARP also supports import of IDL source by adding such a zip to the class path.)

CORBA IDL compilation

IDL files should go in idl/. To add the files generated from an IDL to a tree foo, list it under idls_foo:

# Compile idl/contol.idl.
idls_foo += control

# Compile idl/sub/control.idl.
idls_foo += sub/control

For example, this generates source for sub.Controller, sub.ControllerOperations, etc:

# In idl/sub/control.idl

module sub {
  interface Controller {
    oneway void start();
    oneway void stop();
  };
};

The Java files are generated from IDL first, then compiled with your main files. Changing the list of IDL files, their contents, or the contents of anything they locally include, causes the tree to be recompiled.

Fully qualified IDL namespaces need not map exactly to Java packages. To map IDL module sub to Java package foo.bar.baz.sub, you can use either of these forms:

idlpfx_sub=foo.bar.baz
idlpkg_sub=foo.bar.baz.sub

Changing the mapping will cause all trees that contain IDLs to be recompiled.

You can also set IDL compilation flags for all trees:

IDLPATH += /usr/local/include
IDLJFLAGS += -fallTIE

Changes to those variables do not trigger recompilation, but changes to the following do:

PROJECT_IDLPATH += /usr/local/include
PROJECT_IDLJFLAGS += -fallTIE

The following also trigger recompilation, but apply to a specific tree foo:

IDLPATH_foo += /usr/local/include
IDLJFLAGS_foo += -fallTIE

Merging files in different trees

Normally, two separate trees would not contribute files of the same name. If both appeared on the classpath together, one would usually hide the other; if they were to be combined in a single jar, one would overwrite the other. Even your IDE might grumble about the same file appearing in two source trees within one project. Indeed, it doesn't make a lot of sense, unless the intention is to merge their contents in a single jar, which is sometimes what you want.

To achieve this with Jardeps, you should list such files as part of the description of your tree:

# In Makefile
merge_foo += path/to/file

To prevent your IDE from complaining, keep the mergeable files in a separate tree that the IDE is unaware of, namely merge/foo for tree foo.

Files are combined by simple catenation of the constituents, in no particular order, so it is obviously only useful in situations where the order doesn't matter, such as listing services.

Services

If your tree provides services using ServiceLoader, then your jar needs to include a file in META-INF/services/ with the same name as the service provide by the tree, and containing the names of the classes that implement the service.

Jardeps provides a source annotation Service in which you can list classes or interfaces extended or implemented by your class, and an annotation processor will automatically generate the file in META-INF/services/ for you. Annotate your class like this:

import uk.ac.lancs.scc.jardeps.Service;
import com.example.Plugin;

@Service(Plugin.class)
class MyPlugin implements Plugin {
  ...

You'll get an error if the class does not implement or extend the specified service type, for example:

Service type java.lang.String is not implemented/extended

You need the jar jardeps-lib.jar, to be found in either jardeps/ if you're using an embedded installation, or $$PREFIX/share/java/ for a separate installation. The jar is automatically added to the classpath, but you will probably have to tell your IDE where to find it to keep it happy.

I intend to improve this annotation type by allowing it to be placed directly on the implements/​extends type, like this:

class MyPlugin implements @Service Plugin

However, that will make it incompatible with Java 7, which I'm not prepared to abandon yet. Please contact me if you can suggest a version-independent solution.

You can also provide the service file manually as merge/foo/META-INF/services/com.example.Plugin, and declare it with:

# In Makefile
merge_foo += META-INF/services/com.example.Plugin

…or:

# In Makefile
services_foo += com.example.Plugin

…but this older technique should now be redundant.

Dependencies on jars instead of trees

In addition to the dependency variables deps_foo, ppdeps_foo, rtdeps_foo and apdeps_foo (which list trees on which the tree foo depends), there are corresponding variables jdeps_foo, jppdeps_foo, jrtdeps_foo and japdeps_foo, which list jars on which tree foo depends.

japdeps_jar is recommended over apdeps_mod, as it ensures you're using a properly packaged jar to load your annotation processor.

Diagnostics

If things aren't building correctly, use this to get a summary of all jars and trees:

$$ make summary

Tree jar1-api:
         Root demo.jar1.Jar1Demo
         Root demo.jar1.SimpleAnnotation
         Root demo.jar1.HiddenConstantClass
      Service javax.annotation.processing.Processor
        Merge META-INF/services/javax.annotation.processing.Processor
        Flags 

Tree jar1-proc:
         Root demo.jar1.proc.TestProcessor
      Service javax.annotation.processing.Processor
        Merge META-INF/services/javax.annotation.processing.Processor
 Tree API dep jar1-api
    Classpath classes/jar1-api
        Flags 

Jar jar1:
         Tree jar1-api
         Tree jar1-proc

Tree jar2:
         Root demo.jar2.Jar2Demo
 Tree API dep jar1-api
    Classpath classes/jar1-api
        Flags 

Jar jar2:
         Tree jar2

Tree jar3:
         Root demo.jar3.Boot
      AP Root demo.jar3.SourceImplicit
      AP Root demo.jar3.Implicit
 Tree API dep jar1-api
   Jar RT dep jar1
   Jar AP dep jar1
    Classpath classes/jar1-api
     Procpath out/jar1.jar
        Flags 

Jar jar3:
         Tree jar3

Project:
          Jar jar1
          Jar jar2
          Jar jar3

You can also get information on a specific jar foo with make jarsummary-foo, or on a specific tree with make treesummary-foo.

Manifests

You can contribute to a jar's manifest by providing content explicitly. For jar foo, put the contents in src/jar-foo.manifest:

# In src/jar-foo.manifest
Main-Class: org.example.Foo
Class-Path: bar.jar baz.jar

If you want to add other attributes explicitly associated with a tree rather than a jar, you can put them in src/tree-foo.manifest:

# In src/tree-foo.manifest
Main-Class: org.example.Foo
Class-Path: bar.jar baz.jar

Identifying the main class

Annotate your class with @Application. This will automatically add it as the Main-Class of the manifest, after checking that the class has a suitable public static void main(String[]) method:

import uk.ac.lancs.scc.jardeps.Application;

@Application
public class MyProg {
  public static void main(String[] args) {
    ...

The annotation takes an argument, true by default. If you set it to false, the check will still be made, but the class won't be deemed the entry point of the jar. This allows you to check multiple entry points.

Extending the classpath of a jar

You don't need to do anything. Jardeps will put Class-Path: into your jars based on your inter-tree dependencies.

OSGi exports

You can specify packages to be exported from a tree (for OSGi bundles, for example) by listing them in the tree's makefile:

# In Makefile
exports_foo += com.example.jangle
exports_foo += com.example.jangle.util

This results in an Export-Package attribute being added to the manifest of the jar containing the tree:

Export-Package: com.example.jangle,com.example.jangle.util

If several trees in the same jar specify exports, they will be merged into a single line.

OSGi imports

OSGi imports are generated automatically after compilation of a tree, by scanning the generated class files and injecting an Import-Package attribute. When several trees form a jar, their imports are merged, after excluding packages provided by other trees in the same jar. If you want additional imports, you can specify them explicitly:

# In Makefile
imports_foo += com.mysql.jdbc

You can specify packages not to be imported, even if it appears that some classes in the bundle use them:

# In Makefile
excluded_imports_foo += com.mysql.jdbc

That doesn't prevent another tree from causing the bundle to import the same package. If you need that, you can specify it for a jar:

# In Makefile
jexcluded_imports_foo += com.mysql.jdbc

Directory structure

Jardeps assumes that your project has a certain structure, but you can modify it by overriding some variables:

Variables controlling project structure
File/directory Variable default Meaning
$$(JARDEPS_SRCDIR)/mod/ src Root of source tree mod
$$(JARDEPS_DEPDIR)/deps-mod.mk $$(JARDEPS_SRCDIR) Dependency file for tree mod
$$(JARDEPS_DEPDIR)/tree-mod.manifest Explicit manifest for tree mod
$$(JARDEPS_DEPDIR)/manifest-mod.mk Manifest rules for tree mod
$$(JARDEPS_IDLDIR)/ idl Root of IDL files
$$(JARDEPS_MERGEDIR)/mod/ merge Root of files to be merged from tree mod
$$(JARDEPS_JDEPDIR)/jar-jar.manifest $$(JARDEPS_DEPDIR) Explicit manifest contents for jar jar
$$(JARDEPS_JDEPDIR)/jmanifest-jar.mk Manifest rules for jar jar
$$(JARDEPS_CLASSDIR)/mod/ classes Root of generated class files for tree mod
$$(JARDEPS_OUTDIR)/jar.jar out Name of jarfile for jar jar
$$(JARDEPS_TMPDIR) tmp Internal state of Jardeps

For example:

JARDEPS_SRCDIR=src/java/tree
JARDEPS_DEPDIR=src/java
JARDEPS_IDLDIR=src/idl
JARDEPS_MERGEDIR=src/java/merge
JARDEPS_TMPDIR=var/java

include jardeps.mk

Integration with javadoc

If you can make out/foo.jar, you can also make out/foo-src.zip. This file will contain all your source so that IDEs can read your Javadoc comments from a local file. In Eclipse, this is referred to as a source attachment.

Jardeps defines variables which you can use to help build a command to invoke javadoc. For example, if your source exists in three trees foo, bar and baz, and JARDEPS_SRCDIR is java/src, and JARDEPS_TMPDIR is tmp, then $$(jardeps_srcdirs) will expand to:

java/src/foo tmp/apt/foo java/src/bar tmp/apt/bar java/src/baz tmp/apt/baz

Similarly, $$(jardeps_srcpath) will expand to:

java/src/foo:tmp/apt/foo:java/src/bar:tmp/apt/bar:java/src/baz:tmp/apt/baz

Note that these lists include the directories in which annotation processors can write their generated classes, so they can be documented too.

Those variables list directories for all trees, whether you want to document them or not. You might want to exclude trees containing only test cases, for example. You can select only specific trees with $$(call jardeps_srcdirs4trees,mod1 mod2 mod3 ...), or all trees of specific jars with $$(call jardeps_srcdirs4jars,jar1 jar2 jar3 ...). There are also similar variables jardeps_srcpath4trees and jardeps_srcpath4jars to build colon-separated lists.

Since Jardeps knows which trees will form which jars, it can generate the information to pass on to javadoc. The variable $$(jardeps_ssdocargs) generates a string of the form -jardirs jar1 src/tree1:src/tree2 -jardirs jar2 src/tree3:src/tree4 ..., listing each jar with colon-separated directories of each tree. This is intended for use with my Polydoclot, which can report which jar a package or class belongs to.

Installation of user-generated jars with version numbers

Jardeps can help you to install jars with version numbers. Its use is designed to allow an archive of your source to be installed on systems that don't have Jardeps installed. All you have to do is safely include jardeps-install.mk:

-include jardeps.mk
-include jardeps-install.mk

Jardeps will create it in your working directory, along with a BASH script jardeps-install.sh. The variable JARDEPS_INSTALL is defined to call that script, and you should add a rule to use it:

install-jar-%:
	@$$(call JARDEPS_INSTALL,$$(PREFIX)/share/java,$$*,$$(version_$$*))

When you make install-jar-foo, it will attempt to install out/foo.jar as $$(PREFIX)/share/java/foo.jar. If you've also specified version_foo=1.0.0 (for example), it will instead install the same jar as $$(PREFIX)/share/java/foo-1.0.0.jar, and then make $$(PREFIX)/share/java/foo-1.0.jar, $$(PREFIX)/share/java/foo-1.jar and $$(PREFIX)/share/java/foo.jar point to it as symbolic links. However, it will first check whether each of these already exists as a symbolic link, and avoid replacing it if it links to a newer jar.

The command also installs foo-src.zip and foo-carp.zip, using the same versioning scheme.