Implementing Java library management with the Ant tasks for Apache Maven
Filed under: build scripting java tools
There are 0 comments on this article.
The Java dependency management problem
If you took a typical enterprise Java application and decomposed it down into the libraries that were developed or reused to implement it, you could easily find upwards of 100 different libraries (or .jar files). A number of these libraries might be open source implementations, such as the Jakarta commons libraries, others might be commercial business components that you have sourced from a third party. However, since many commercial components also make extensive use of open source components themselves (and why wouldn't they!), you can quite easily find yourself with duplicate sets of libraries at different version levels!
When you are developing your own common libraries, you will obviously also be creating multiple versions of them and will probably need to support different versions for different projects, or for projects in different phases. Following on from this, your library dependencies might have dependencies on other libraries themselves (so called transitive dependencies), you will also need to manage and capture this kind of metadata too.
The typical ways that organizations solve this problem are to create a shared repository of libraries - sometimes in version control system, sometimes not. This approach can work but requires a significant amount of administration to keep it up to date. This is especially true for transitive dependencies where you would need to somehow store their associative metadata alongside the components. Alternately you would need to create a significant amount of baselines in your version control system to refer to a compatible (and collective set) of libraries.
Fortunately, there are now a number of Java tools that can help you solve this problem. These include Apache Maven (in the form of Ant tasks) and Apache Ivy. Both can be made to work with Ant, however since Maven could be seen as the next generation of Java build tool, it is worth looking at its implementation first - doing so will also give you a possible future upgrade path. In this article I will therefore describe in detail how to implement library management with the Ant tasks for Apache Maven.
Installing the Ant tasks for Apache Maven
The Ant tasks for Apache Maven are available as a Java library from the Apache Maven website. You can download the latest version of the tasks from http://maven.apache.org/download.html. There should be a single Java library. You can either install this library to your Ant lib directory (for example, %ANT_HOME%\lib) or to a directory alongside your top-level build.xml file. I prefer to install the Java library into a directory called lib alongside the build.xml file, and then place it under version control.
Once you have done this you will need to change your build.xml to reference the tasks "namespace". You can achieve this by adding the following to your top-level <project> element:
Namespaces allow you to reference third party libraries and their tasks. In this case the namespace being created is called "artifact" and refers to the Uniform Resource Name (URN) "maven-artifact-ant". In order for Ant to be able to locate this URN, you will also need to define a Classpath where Ant can locate the library as follows:
<typedef uri="urn:maven-artifact-ant"
resource="org/apache/maven/artifact/ant/antlib.xml">
<classpath>
<pathelement location="${dir.lib}/
maven-ant-tasks-2.x.x-dep.jar"/>
</classpath>
</typedef>
This <typedef> element references the Ant tasks for the Apache Maven Java library that you have downloaded and installed – in this case in the ${dir.lib} directory.
Using Apache Maven repositories
Apache Maven uses file system repositories to store and organize the Java libraries that you will be using as part of your build process. Alongside the Java libraries themselves, it also stores metadata representing their versions and dependencies. By default this repository is located on the Internet at http://ibiblio.org/maven2 (and is mirrored around the world), however you also can implement internal repositories for your own organization. One obvious reason for wanting to do this is for security purposes – you probably would not want to publish your companies internally developed libraries on the Internet.
The basic structure of any Maven repository is relatively simple and is illustrated in the diagram below:
In this example I am focusing on a specific directory in the Maven ibiblio repository for the Jakarta "commons-net" component. This is a popular component which contains a collection of network utilities and protocol implementations such as FTP, SMTP and Telnet. Inside the commons-net directory of the Maven repository is a sub-directory for every version of "commons-net" that has been released. Inside these sub-directories are three types of files:
- The Java libraries which contain the implemented functionality. In the diagram above there is only a single library called commons-net-1.40.jar (although the directory could contain several libraries).
- The Project Object Model (POM) file for the component, in this case commons-net-1.4.0.pom.
- The maven-metadata.xml which is an internally managed file used to mark out the fact that this is a Maven repository
There will also be a number of checksum files that are managed by Maven as well.
Maven uniquely identifies each version of a library via its filename, for example, commons-net-1.4.0.jar. However, it also allows you to manage snapshots, which are the latest but unreleased versions of libraries. Snapshots are marked via a "–SNAPSHOT" identifier in the version name. Although Maven will help you manage these files in its repository, creating checksums, directory structures and so on, it does not automatically know about dependencies between libraries. This is where the POM file comes in.
In the full version of Apache Maven, POM files replace build.xml files as the build scripts - defining how to build a project or component. Included in this file is the list of dependent libraries that are required for the build to be successful. For example, if you looked at this file for commons-net you would see that it contains the following lines:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<name>Jakarta Commons Net</name>
<version>1.4.0</version>
<dependencies>
<dependency>
<groupId>oro</groupId>
<artifactId>oro</artifactId>
<version>2.0.8</version>
</dependency>
</dependencies>
</project>
In this example you can see that the initial elements define information about the library and how to access it from within the repository - via its "groupId" and "artifiactId". You can also see the set of additional libraries that this library is dependent on; in this case it is the Jakarta ORO (regular expression) library version 2.0.8. You only really need to be concerned with the content of POM files if you are going to be creating your own Java libraries and publishing them to the Maven repository. If this is the case then you will need to create POM files for each component that you wish to publish. I will discuss how to achieve this later.
The way that the Apache Maven dependency management feature works is that when you carry out a build, it cycles through all these dependencies and downloads the appropriate versions from the site repository to a user’s local repository (by default this repository is called ".m2" and is located in your home directory). The local repository is essentially your own personal cache – and is automatically maintained – you do not need to do anything to update or refresh it.
Implementing your own repository
If you want to make use of the Internet repository then you need to do nothing to specify the location of the library repository. However, if you wish to implement your own site based repository, along side or instead of the Internet repository, then you need to specify the location of this repository in your build.xml file. You can achieve this by including a line similar to the following:
In this example an internal repository is being used which is accessible via a Web server on the local Intranet at http://myhost.maven2. This repository is going to be referenced using the id "internal.repository". If you do not have a Web server you can still specify a URL for a file system location, for example, file:///shared/maven2.
You can initialize your own repository by simply creating an empty directory on a file system. If you have an existing set of internally developed Java libraries then you can import them into your repository using the Apache Maven deploy command. For example, to install the Java library HotelEJBClient.jar at version 1.0.1 you would use the following command:
-DartifactId=hoteldejavaejbclient -Dversion=1.0.1
-Dpackaging=jar -Dfile=./HotelEJBClient.jar
-DrepositoryId=repository -Durl=file:///shared/maven2
In order to use this command you will need to download Apache Maven itself since it requires the "mvn" binary. One particular point to consider is that only open source libraries are contained in the ibiblio repository, not those with click-through licenses like Sun’s. Therefore if you are going to build anything which is based on or uses Sun’s libraries then you will need to download and install the relevant Sun Java libraries using this installation process (see here for more information).
Resolving Java libraries
Once you have installed the Ant tasks for Apache Maven, in order to specify the versions of libraries you wish to build against I recommend creating a new properties file called library.properties. You can then populate this file with the versions of libraries that you wish to build against, for example:
ver.hotel.ejbclient = 1.1 ver.commons-net = 1.4.0
Since you are going to compile against these libraries, you will need to setup a Java Classpath to contain them. Fortunately, the Ant tasks for Apache Maven can do this for you automatically. To specify and resolve the Java libraries that you need and to automatically create a new Classpath you would include the following in your build.xml:
<property file="build.properties"/>
<property file="library.properties"/>
<artifact:dependencies pathId="maven.classpath">
<remoteRepository refid="internal.repository"/>
<dependency
groupId="hoteldejava"
artifactId="ejbclient"
version="${ver.hotel.ejbclient}"/>
<dependency
groupId="commons-net"
artifactId="commons-net"
version="${ver.commons-net}"/>
</artifact:dependencies>
In this case I have specified that the Java libraries are to be downloaded from the repository "internal.repository". The Ant tasks for Apache Maven will therefore download the versions of the libraries from this repository to the ".m2" directory in your home directory – if they do not exist there already. If the libraries that you require have dependencies on other libraries, these will be downloaded too – this is the transitive dependencies feature in action. You can then use the generated maven.classpath in your Ant build script, for example to compile your project you could use a line similar to the following:
If at a later date you require a different version of a library then you can simply override the relevant entry in the library.properties file. Additionally, if as a developer you were doing some testing and did not want to update this file, you could override the version on the command line (for example, using " ant –Dver.commons.cli=1.4.1") or make use of your own build.properties file.
Publishing Java libraries
As well as downloading pre-built libraries to build against, you can also publish your own libraries to an Apache Maven repository. This allows you to create a single consistent area for re-using libraries, and to take advantage of Apache Maven’s dependency management capabilities. In order to publish libraries, you will first need to create a pom.xml file. For example, for the Hotel EJBClient project this file would contain the following:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>hotel</groupId>
<artifactId>ejbclient</artifactId>
<version>1.0.1</version>
</project>
These are the only lines that you will need to be able to publish a new library. However if applicable, you can also add dependencies to the POM file similar to that for commons-cli component described above.
To deploy your built library to the Maven repository, you would then include a target in your build.xml file similar to the following:
<target name="maven-deploy"
description="deploy to maven repository">
<artifact:pom id="maven.project"
file="pom.xml" />
<artifact:deploy file="dist/HotelEJBClient-${rel.num}.jar">
<remoteRepository refid="internal.repository"/>
<pom refid="maven.project"/>
</artifact:deploy>
</target>
In this example the library version is referenced by the ${rel.num} property and it is being published to the repository "internal.repository".
In practice, this is all you will need to start using Apache Maven repositories to implement Java library re-use and dependency management. However there are additional settings and configurations that you can make (such as specifying the location of your own local repository). These are described in more detail on the Apache Ant tasks for Apache Maven website.
References