From be3b48e2d35933918d86a3c684ffa22951c81fba Mon Sep 17 00:00:00 2001 From: Peter Donald Date: Thu, 11 Jan 2001 11:56:34 +0000 Subject: [PATCH] Good docs to the recue !!! Submitted By: "Steve Loughran" git-svn-id: https://svn.apache.org/repos/asf/ant/core/trunk@268433 13f79535-47bb-0310-9956-ffa450edef68 --- docs/ant_in_anger.html | 793 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 793 insertions(+) create mode 100644 docs/ant_in_anger.html diff --git a/docs/ant_in_anger.html b/docs/ant_in_anger.html new file mode 100644 index 000000000..640647f42 --- /dev/null +++ b/docs/ant_in_anger.html @@ -0,0 +1,793 @@ + + + Ant in Anger + + + + +

Ant in Anger: +

+

+ Using Ant in a Production Development System +

+ +

+Steve Loughran (steve_l@iseran.com) +

+ + + +

Introduction

+ + +Ant + can be an invaluable tool in a team development process -or it can +be yet another source of problems in that ongoing crises we call +development . This +document contains some strategies and tactics for making the most of +ant. It is moderately frivolous in places, and lacks almost any actual +examples of ant xml. The lack of examples is entirely deliberate -it +keeps document maintenance costs down. Most of the concepts covered +don't need the detail about XML representations, as it is processes we +are concerned about, not syntax. Finally, please be aware that the +comments here are only suggestions which need to be customised to meet +your own needs, not strict rules about what should and should not be +done. + +

+Firstly, here are some assumptions about the projects which this +document covers +

+ +What that all means is that there is no time to spend getting things +right, you don't have that tight control on how the rest of the team +works and the development process is often more one of chaos minimisation +than anything else. The role of ant in such projects is to ensure that +the build, test and deploy processes run smoothly, leaving you with all +the other problems. + + +

Core Practices

+ +

+Clarify what you want ant to do

+ + +Ant is not a silver bullet. It is just another rusty bullet in the armory of +development tools available at your disposal. Its primary purpose is to +accelerate the construction and deployment of Java projects. You could certainly +extend ant to do anything Java makes possible -it is easy to imagine writing an +image processing task to help in web site deployment by shrinking and +recompressing jpeg files, for example. But that would be pushing the boundary of +what ant is really intended to do -so should be considered with care. + +

+ +Ant is also a great adjunct to an IDE -a way of doing all the housekeeping of +deployment and for clean, automated builds. But a good modern IDE is a +productivity tool in its own right -one you should consider keeping using. Ant +just lets you give the teams somewhat more freedom in IDE choice -"you can +use whatever you want in development, but ant for the deployment +builds" + +

+Define standard targets +

+ + +When you have multiple sub projects, define a standard set of targets. +Projects with a split between interface and implementation jar files +could consider impl and intf targets -with separate +debug-impland debug-intf targets for the debug version. +And of course, the ubiquitous clean target. + +

+ +With standard target names, it is easy to build encompassing ant build +files which just hand off the work to the classes below using the +ant +task. For example. the clean target could be handed down to the intf and +impl subdirectories from a parent directory + +

<target name="clean"  depends="clean-intf, clean-impl">
+</target>
+
+<target name="clean-intf" >
+	<ant dir="intf" target="clean" />
+</target>
+
+<target name="clean-impl">
+	<ant dir="impl" target="clean" />
+</target>
+ +

+ Extend ant through new tasks +

+ +If ant does not do what you want, you can use the +exec and +java tasks or +inline scripting to extend it. In a +project with many build.xml files, you soon find that having a single +central place for implementing the functionality keeps maintenance +overhead down. Implementing task extensions through java code seems +extra effort at first, but gives extra benefits:- + + + +

+Embrace Automated Testing +

+ +(alternatively "recriminate early, recriminate often") +

+ +Ant lets you call JUnit tasks, which unit test +the code your team has written. Automated testing may seem like extra +work at first, but JUnit makes writing unit tests so easy that you have +almost no reason not to. Invest the time in learning how to use +JUnit, write the test cases, and integrate them in a 'test' target from +ant so that your daily or hourly team build can have the tests applied +automatically. + +

+ +Once code fetches from the code control system are added as another ant +target, the integration test code can be a pure ant task run on any box +dedicated to the task. This is ideal for verifying that the build and +unit tests work on different targets from the usual development +machines. For example, a Win95/Java1.1 combination could be used even +though no developer would willingly use that configuration given the +choice. + +

+ +System tests are harder to automate than unit tests, but if you can +write java code to stress large portions of the system -even if the code +can not run as JUnit tasks- then the java +task can be used to invoke them. It is best to specify that you want a +new JVM for these tests, so that a significant crash does not break the +full build. + + + +

+Cross Platform Ant +

+ + +Ant is the best foundation for cross platform Java development and +testing to date. But if you are not paying attention, it is possible to +produce build files which only work on one platform -or indeed, one +single workstation. + +

+ +The common barriers to cross-platform ant are the use of command line +tools (exec tasks) which are not portable, path issues, and hard coding +in the location of things. + +

Command Line apps: Exec/ ExecOn

+ +The trouble with external invocation is that not all functions are found +cross platform, and those that are often have different names -DOS +descendants often expect .exe or .bat at the end of files. That can be +bad if you explicitly include the extension in the naming of the command +(don't!), good when it lets you keep the unix and DOS versions of an +executable in the same bin directory of the project without name +clashes arising. + +

+ +Both the command line invocation tasks let you specify which platform +you want the code to run on, so you could write different tasks for each +platform you are targeting. Alternatively, the platform differences +could be handled inside some external code which ant calls. This can be +some compiled down java in a new task, or an external script file. + +

Cross platform paths

+ +Unix paths use forward slashes between directories and a colon to +split entries. Thus +"/bin/java/lib/xerces.jar:/bin/java/lib/ant.jar" is +a path in unix. In Windows the path must use semicolon separators, +colons being used to specify disk drives, and backslash separators +"c:\bin\java\lib\xerces.jar;c:\bin\java\lib\ant.jar". + +This difference between platforms (indeed, the whole java classpath +paradigm) can cause hours of fun. + +

+ +Ant reduces path problems; but does not eliminate them entirely. You +need to put in some effort too. The rules for handling path names are +that 'DOS-like pathnames are handled', 'Unix like paths are handled'. +Disk drives -'C:'- are handled on DOS-based boxes, but placing them in +the build.xml file ruins all chances of portability. Relative file paths +are much more portable. Semicolons work as path separators -a fact which +is useful if your ant invocation wrapper includes a list of jars as a +defined property in the command line. In the build files you may find it +better to build a classpath by listing individual files (using location= +attributes), or by including a fileset of *.jar in the classpath +definition. + + +

+Note that DOS descended file systems are case insensitive (apart from +the obscure aberration of the WinNT posix subsystem run against NTFS), +and that Windows pretends that all file extensions with four or more +letters are also three letter extensions (try DELETE *.jav in your java +directories to see a disastrous example of this). + +

+ +Ant's policy on case sensitivity is whatever the underlying file system +implements *VERIFY*, and its handling of file extensions is that *.jav does not +find any .java files. The Java compiler is of course case sensitive -you can +not have a class 'ExampleThree' implemented in "examplethree.java". + +

+ +Some tasks only work on one platform - +Chmod being a classic example. These tasks usually result in just a +warning message on an unsupported platform -the rest of the target's +tasks will still be called. Other tasks degrade their functionality on +platforms or Java versions. In particular, any task which adjusts the +timestamp of files can not do so properly on Java 1.1. Tasks which can +do that - Get, Touch and +Unjar/Unwar/Unzip for example, degrade their functionality on +Java1.1, usually resorting to the current timestamp instead. + + +

+ +Finally, Perl makes a good place to wrap up Java invocations cross +platform, rather than batch files. It is included in most Unix +distributions, and is a simple download for Win32 platforms from +ActiveState. A Perl file with .pl extension, with the usual Unix +path to perl on the line 1 comment and marked as executable can be run +on Windows, OS/2 and Unix and hence called from Ant without issues. The +perl code can be left to resolve its own platform issues. + + +

Team Development Processes

+ + +Even if each team member is allowed their choice of IDE/editor, or even +OS, you need to set a baseline of functionality on each box. In +particular, the JDKs and jars need to be in perfect sync. Ideally pick +the latest stable Java/JDK version available on all developer/target +systems and stick with it for a while. Consider assigning one person to +be the contact point for all tools coming in -particularly open source +tools when a new build is available on a nightly basis. Unless needed, +these tools should only really be updated monthly, or when a formal +release is made. + +

+ +Another good tactic is to use a unified directory tree, and add on extra +tools inside that tree. All references can be made relative to the tree. +If team members are expected to add a directory in the project to their +path, then command line tools can be included there -including those +invoked by ant exec tasks. Put everything under source code control and +you have a one stop shop for getting a build/execute environment purely +from CVS or your equivalent. + + + +

Deploying with Ant

+ + +One big difference between ant and older tools such as make is that the +processes for deploying java to remote sites are reasonably well +evolved in ant. That is because we all have to do it these days, so +many people have put in the effort to make the tasks easier. +

+ +Ant can Jar, +Tar or Zip files for deployment, while +the War task extends the jar task for +better servlet deployment. Jlink is a jar +generation file which lets you merge multiple sub jars. This is ideal +for a build process in which separate jars are generated by sub +projects, yet the final output is a merged jar. Cab can be used on Win32 boxes to build a cab file +which is useful if you have to target IE deployment. + +

+ +The ftp task lets you move stuff up to a +server. Beware of putting the ftp password in the build file -a property +file with tight access control is slightly better. The FixCRLF task is often a useful interim step if +you need to ensure that files have unix file extensions before upload. A +WebDav task has long been discussed, which would provide a more secure +upload to web servers, but it is still in the todo list. If DAV is your +required upload mechanism, why not take up the challenge? + +

+ +EJB deployment is aided by the ejb tasks. At the +time of writing, only WebLogic was supported with these tasks -if your +EJB server is not supported, extending the ejb tasks would benefit your +project and the rest of the ant community. + +

+ +Finally, there are of course the fallbacks of just copying files to a +destination using Copy and Copydir , or just sending them to a person or +process using Mail. + + +

Directory Structures

+ + +How you structure your directory tree is very dependent upon the +project. Here are some directory layout patterns which can be used as +starting points. + +

Simple Project

+ +The project contains sub directories + + + + + + + + + + + + + + + + + + + + + + + + + + +
bin + common binaries, scripts -put this on the path. +
build + This is the tree for building; ant creates it and can empty it + in the 'clean' project. +
dist + Distribution outputs go in here; the directory is created in ant + and clean empties it out +
doc + Hand crafted documentation +
lib + Imported Java libraries go in to this directory +
src + source goes in under this tree +
+ +The bin, lib, doc and src directories should be under source code control. +Slight variations include an extra tree of content to be included in the +distribution jars -inf files, images, etc. Javadoc output can be +directed to a doc/ folder beneath build/, or to doc/javadoc. + +

Interface and Implementation split

+ +If the interface is split from the implementation code then this can be +supported with minor changes just by having a separate build path for +the interface directory -or better still just in the jar construction: +one jar for interface and one jar for implementation. + + +

Loosely Coupled Sub Projects

+ +In the loosely coupled approach multiple projects can have their own +copy of the tree, with their own source code access rights. +One difference to consider is only having one instance of the bin and +lib directories across all projects. This is sometimes good -it helps +keep copies of xerces.jar in sync, and sometimes bad -it can update +foundational jar files before unit testing is complete. + +

+ +To still have a single build across the sub projects, use parent +build.xml files which call down into the sub projects. + +

+ +This style works well if different teams have different code +access/commitment rights. The risk is that by giving extra leeway to the +sub projects, you can end up with incompatible source, libraries, build +processes and just increase your workload and integration grief all round. + +

Integrated sub projects

+ +Tightly coupled projects have all the source in the same tree; different +projects own different subdirectories. Build files can be moved down to +those subdirectores (say src/com/iseran/core and src/com/iseran/extras), +or kept at the top -with independent build files named core.xml and +extras.xml + +

+ +This project style works well if everyone trusts each other and the +sub projects are not too huge or complex. The risk is that a split to a +more loosely coupled design will become a requirement as the projects +progress -but by the time this is realised schedule pressure and +intertwined build files make executing the split well nigh impossible. +If that happens then just keep with it until there is the time to +refactor the project directory structures. + + +

+ Ant Update Policies +

+ + +Once you start using ant, you should have a policy on when and how the +team updates their copies. A simple policy is "every official release +after whatever high stress milestone has pushed all unimportant tasks +(like sleep and seeing daylight) on the back burner". This insulates you +from the changes and occasional instabilities that ant goes through +during development. Its main disadvantage is that it isolates you from +the new tasks and features that ant is constantly adding. + +

+ +Often an update will require changes to the build.xml files. Most +changes are intended to be backwards compatible, but sometimes an +incompatible change turns out to be +necessary. That is why doing the update in the lull after a big +milestone is important. It is also why including ant.jar and related +files in the CVS tree helps ensure that old versions of your software +can be still be built. + +

+ +The most aggressive strategy is to get a weekly or daily snapshot of the +ant source, build it up and use it. This forces you to tweak the +build.xml files more regulary, as new tasks and attributes can take +while to stabilise. You really have to want the new features, enjoy +gratuitous extra work or take pleasure in upsetting your colleagues to +take this approach. + +

+ +Once you start extending ant with new tasks, it suddenly becomes much +more tempting to pull down regular builds. The most recent ant builds +are invariably the the best platform for writing your extensions, as you +can take advantage of the regular enhancements to the foundational +classes. It also prevents you from wasting time working on something +which has already been done. A newly submitted task to do something +complex such as talk to EJB engines, SOAP servers or just convert a text +file to uppercase may be almost exactly what you need -so take it, +enhance it and offer up the enhancements to the rest of the world. This +is certainly better than starting work on your 'text case converter' +task on Ant 0.8 in isolation, announcing its existence six months latter +and discovering that instead of adulation all you get are helpful +pointers to the existing implementation. + +

+ +You should also get on the ant-dev mailing list +, as it is where the other developers post their work, problems and +experience. The volume can be quite high: 40+ messages a day, so +consider routing it to an email address you don't use for much else. And +don't make everyone on the team subscribe; it can be too much of a +distraction. + + + +

+Tips and Tricks

+ +
+
+ get +
+ +The get task can fetch any URL, so be used +to trigger remote server side code during the build process, from remote +server restarts to sending SMS/pager messages to the developer +cellphones + +
+i18n +
+ + +Internationalisation is always trouble. Ant helps here with the native2ascii task which can escape out all non +ascii characters into unicode. You can use this to write java files +which include strings (and indeed comments) in your own non-ASCII +language and then use native2ascii to convert to ascii prior to feeding +through javac. The rest of i18n and l12n is left to you... + +
+Use Property Files +
+ +Use external property files to keep per-user settings out the build +files -especially passwords. Property files can also be used to +dynamically set a number of properties based on the value of a single +property, simply by dyamically generating the property filename from the +source property. They can also be used as a source of constants across +multiple build files. + +
+Faster compiles with Jikes +
+ +The jikes compiler is usually much +faster than javac, and does dependency checking. Get it. Then set +build.compiler to "jikes" for it to be used in your build files. + +
+#include targets to simplify multi build.xml projects +
+ +You can import XML files into a build file using <?import?> +an element of XML syntax which the standard parsers use. This lets +a multi-project development program share code through reference, +rather than cut and paste re-use. It also lets one build up a file of +standard tasks which can be reused over time. Because the import +mechanism is at a level below which ant is aware, treat it as +equivalent to the #include mechanism of the 'legacy' languages C and +C++. + +
+Implement complex Ant builds through XSL +
+ +XSLT can be used to dynamically generate build.xml files from a source +xml file, with the Style task controlling +the transform. This is the current recommended strategy for creating +complex build files dynamically. + + +
+Change the invocation scripts +
+ +By writing your own invocation script -using the DOS, Unix or Perl +script as a starting point- you can modify a ant behavior for an +individual project. For example, you can use an alternate variable to +ANT_HOME as the base, extend the classpath differently, or dynamically +create a new command line property 'project.interfaces' from all .jar +files in an interfaces directory. + +

+ +Having a custom invocation script which runs off a CVS controlled +library tree under PROJECT_HOME also lets you control ant versions +across the team -developers can have other copies of ant if they want, +but the CVS tree always contains the jar set used to build your project. + +

+ +You can also write wrapper scripts which invoke the existing ant +scripts. This is an easy way to extend them. The wrapper scripts can add +extra definitions and name explicit targets, redefine ANT_HOME and +generally make development easier. Note that "ant" in Windows is really +"ant.bat", so should be invoked from another batch file with a "CALL +ant" statement -otherwise it never returns to your wrapper. + + +

+Write all code so that it can be called from Ant +
+ +This seems a bit strange and idealistic, but what it means is that you should +write all your java code as if it may be called as a library at some point in +future. So do not place calls to System.exit() deep in the code -if you +want to exit a few functions in, raise an exception instead and have +main() deal with it. + +
+Use Antidote as the invocation tool +
+Even if you edit ant files by hand, Antidote makes a good execution tool +because it eliminates the startup time of the JVM, perhaps even some of +the XML parsing delays. + +
+ + +

+ Putting it all together +

+ + +What does an ant build process look like in this world? Assuming a +single directory structure for simplicity, the build file +should contain a number of top level targets + +Sub projects 'web', 'bean-1', 'bean-2' can be given their own build +files -web.xml, bean-1.xml, bean-2.xml- with the same entry points. +Extra toplevel tasks related to databases, web site images and the like +should be considered if they are part of the process. + +

+Debug/release switching can be handled with separate initialisation +targets called before the compile tasks which define the appropriate +properties. + +

+Internal targets should be used to structure the process +

+ +The switching between debug and release can be done using the 'if' and +'unless' conditional flags on the targets, so that debug gets called +unless 'project.mode.release' is defined. + +

+ +It is useful to define a project name property which can be echoed in +the init task. This lets you work out which ant file is breaking in a +multi file build. + +

+ +What goes in to the internal ant tasks depends on your own projects. One +very important tactic is 'keep path redefinition down through +references' - you can reuse paths by giving them an ID and then +referring to them via the 'refid' attribute you should only need to +define a shared classpath once in the file; filesets can be reused +similarly. + +

+ +Once you have set up the directory structures, and defined the ant tasks +it is time to start coding. An early priority must be to set up the +automated test process, as that not only helps ensures that the code +works, it verifies that the build process is working. + +

+ +And that's it. The build file shouldn't need changing as new source +files get added, only when you want to change the deliverables or part +of the build process. At some point you may want to massively +restructure the entire build process, restructuring projects and the +like, but even then the build file you have should act as a foundation +for a split build file process -just pull out the common properties into +a properties file all build files read in, keep the target names unified +and keep going with the project. Restructuring the source code control +system is often much harder work. + +

The Limits of Ant

+ +Before you start adopting ant as the sole mechanism for the build +process, you need to be aware of what it doesn't do. +

+ +

It's not a scripting language

+ +Ant lets you declare what you want done, with a bit of testing of the +platform and class libraries first to enable some platform specific +builds to take place. It does not let you specify how to handle things +going wrong (a listener class can do that), or support complex +conditional statements. + +

+ +If your build needs to handle exceptions then look at the sound listener +as a simple example of how to write your own listener class. Complex +conditional statements can be handled by having something else do the +tests and then build the appropriate ant task. XSLT can be used for +this. + +

It's not Make

+ +Some of the features of make, specifically inference rules and +dependency checking are not included in ant. That's because they are +'different' ways of doing a build. Make requires you to state +dependencies and the build steps, ant wants you to state tasks and the +order between them, the tasks themselves can do depedency checking or +not. A full java build using Jikes is so fast that dependency checking +is relatively moot, while many of the other tasks (but not all), compare +the timestamp of the source file with that of the destination file +before acting. + +

It's not meant to be a nice language for humans

+ +XML isnt a nice representation of information for humans. It's a +reasonable representation for programs, and text editors and source code +management systems can all handle it nicely. But a complex ant file can +get ugly because XML is a bit ugly, and a complex build is, well, +complicated. Use XML comments so that the file you wrote last month +still makes sense when you get back to it, and use Antidote to edit the +files if you prefer it. + + +

Endpiece

+ +Software development is meant to be fun. Being in the maelstrom of a +tight project with the stress of integration and trying to code +everything up for an insane deadline can be fun -it is certainly +exhilirating. Adding a bit of automation to the process may make things +less chaotic, and bit less entertaining, but it is a start to putting +you in control of your development process. You can still have fun, you +should just have less to worry about, a shorter build/test/deploy cycle +and more time to spend on feature creep or important things like skiing. +So get out there and have fun! + + +

Further Reading

+ + +
+

Copyright © 2000 Apache Software Foundation. All rights +Reserved.

+ +