# Apache Myrmidon - Get Involved

|  | Apache Myrmidon

Myrmidon

- Introduction
- Get Involved
- Building Myrmidon
- Todo List

User Guide

- Build file
- Virtual File System
- Ant1 Compatibility Layer
- Differences to Ant1
- My First Task

Extending Ant

- Configuration HOWTO
- Converter HOWTO
- Type HOWTO
- ClassLoader HOWTO
- Library HOWTO

Container Design

Todo List

> The broad goal is to grow Myrmidon from a prototype task engine into a fully fledged build system, that can serve as the basis for Ant 2. The following sections describe some of the many things which still need to be done to achieve that goal. This list is currently under construction.

> Integrate XDocs proposal

> > Integrate with the XDocs proposal that generates XML documentation for tasks. Rework that proposal so that it knows about the myrmidon specific patterns and features. Also rework it so that it can support reading documentation and examples from side-by-side the task.

> TaskInfo

> > Consider allowing task writers to write their own TaskInfo objects (or at least have them generated from XDoclet directives). This would encompass both documentation and perhaps introspection of the types.

> XML Catalog to load XML Fragments

> > When including fragments of XML we are currently forced to use relative paths. However this is sometimes undesirable when a single fragment needs to be used across several projects in several different locations. Instead we could use a Catalog to name the fragment and then each developer would only need to install the fragment once and it would be accessible from all the projects.

> Refactor Java Infrastructure into a Service

> > Much like Exec should be decoupled from Ant runtime, so should classes to implement java task for the same benefits.

> Structural Dependency Utils

> > In the present ant, it is required that each task manage dependency separately. This makes it a lot of work to implement even simple dependency checking. To this day many of the core tasks do not implement it correctly. I am specifically talking about "structural" dependency information. The main reason is that it is painful to implement.

> > Some tasks do no dependency checking and will recompile/transform/etc everytime. Others may perform a simple dependency checking (ie if source file is newer than destination file then recompile). Ideally a dependency system would actually calculate the dependencies correctly. So we need to have some mechanism to determine that `foo.c` actually depends upon `foo.h`, `bar.h` and `baz.h`. As this information is particular to each different task we need to allow tasks to implement this behaviour. Possibly by supplying an interface of the form;

> >     public interface DependencyGenerator
> >     {
> >       File[] generateDependencies( File file );
> >     }

> > Generating the dependency information is a costly operation and thus we do not want to be doing it everytime you run ant. We want to generate it on the initial build and then persist somewhere. Everytime a file is out of date, it's dependency information would be regenerated and stored in the dependency cache. Ideally this cache would also store the above mentioned coloring information. So the entry for `foo.c` may declare that it is dependent upon `foo.h`, `bar.h` and `baz.h`, aswell as being compiled with -O2 flag. If any of the dependencies have changed or are out of date then foo.c would need to be recompiled.

> > A possible API would be

> >     DependencyManager dm = ...;
> >     dm.setFileSet( myFileSet );
> >     dm.setDependencyCache( myDependencyCacheFile );
> >     File[] files = cm.getOutOfDate();

> Antlibs storing templates

> > After a templating system is formalized it would useful to define a mechanism via which you can store templates in an antlib. These templates could then be loaded and used by build files through a "standard" mechanism. This may need to be merged with the XML catalog system.

> Antlibs Storing General Resources

> > Add a system via which ant libs can store general resources for consumption by build users. This could be used to store the XML fragments for the XML catalog, the template fragments for templates, images for documentation system and so forth.

> Coloring API

> > When you execute a task such as "javac" there is two types of dependency information that is important to analyze before we determine if we need to recompile a file. Say we are compiling `Foo.java`, it may depend on the `Bar.java` file. We call this "structural" dependency information - the structure of the source file determines what other files it depend upon. However there is also "environmental" dependency information. For instance if the `Foo.java` file was compiled with `debug="true"` last run and this time needs to be compiled with `debug="false"` then it is out of date and needs to be recompiled. We call this "environmental" dependency information "coloring".

> > So we need to create an infrastructure that allows tasks to manage "coloring". So a task should be able to add coloring information for each resource processed. When the task comes to process the resource again it will detect if the coloring has changed and if it has will force a recompile.

> > An API for such a bean has yet to be established but an example API would be.

> >     ColoringManager cm = ...;
> >     cm.addColor( "debug", "true" );
> >     cm.addColor( "optimize", "false" );
> >     cm.setFileSet( myFileSet );
> >     File[] files = cm.getOutOfDate();

> Create Task/Element/Attribute Naming guidelines

> > Currently Ant has a mixture of tasks from various stages it's evolution, with different authors and each utilizing different naming patterns. Some tasks use names such as "src" and "dest" while others use "file" and "tofile". It would be preferrable if consistent naming patterns were used. It is recomended that we come up with a "best practices" document to document our recomended naming patterns.

> > Before we can come up with such a document we need to identify common patterns through out the tasks. Several tasks have the notion of transforming input from a "source" to a "destination". So we should have consistent naming schemes for these attributes and elements. Analysis of existing tasks will likely bring out other similar patterns. Once we have identified and documented these similarities then we can establish conventions.

> Rethink Notification/Event scheme

> > We need to rethink the whole notificaiton scheme. Should tasks be able to raise events? Probably as long as we have ContainerTasks. Should tasks be able to query state of run? ie Can a task request "are we paused?" or "are we stopped?" ? Probably as that way long running tasks are given the opportunity to be gracefully halted by the end users (primarily aimed at IDE vendors here).

> XPath-like Locators for tasks

> > Most tasks are grouped into some sort of task container. The task containers can be things like workspaces, projects, targets or other tasks. Each of these containers usually has a name. Thus we could refer to tasks via a path such as "/avalon/compile/javac" would refer to the task "javac" in the target "compile" in the project "avalon". In the past it has been requested that a task get access to this path programatically - other people have also asked for access to things like the currently running target. We need to assess this and decide whether we wish to support it.

> > Another point to think about is that we could use XPath-like string to designate to other tasks to execute. ie antcall would refer to a path rather than a target name

> Embeddor HOWTO

> > Assigned To: Peter

> > Write a HOWTO describing how to embed Myrmidon into other applications.

> Optional Dependencies

> > Assigned To: Peter

> > Extend Myrmidons library management facilities so that optional dependencies may be declared for a library. ie The library will still operate in absence of such libraries but can provide further features if these libraries are present. Most likely this will be done via a new manifest entry "Optional-Extension-List:" that behaves similar to "Extension-List:" except that the extensions are optional.

> Facade task HOWTO

> > Currently we have a few tasks that have multiple implementations. For instance Javac task can actually call jikes, jvc, classic javac or modern javac. Similar things will be seen with the jspc task and the cc task (if it ever gets written). We have a base class that is meant to facilitate this sort of task and make it easy to develope. See AbstractFacadeTask. However we need to write up a HOWTO so people can use it.

> Mail tasks

> > Convert the ant1.x Mail tasks to myrmidon.

> Security Manager

> > Add the ability to run java programs that call System.exit() by adding a security manager. Should look something like:

> >     public class MyrmidonSecurityManager
> >         extends SecurityManager
> >     {
> >         public void checkExit( final int status )
> >         {
> >             throw new ExitException( status );
> >         }

> >         public void checkPermission( final Permission permission )
> >         {
> >         }
> >     }

> Self Hosting

> > Myrmidon must be able to build itself. Currently, it is built using Ant 1.x. Ultimately, Myrmidon should be able to build itself from exactly the same build file. To start with, however, there is no need for Myrmidon to be able to do this. Myrmidon should also be able to be bootstrapped (that is, be able to be built from scratch, without using Ant 1.x at all).

> Validation Pass

> > Consider calling validate() on task prior to execute(). This would allow us to have a "make -k" mode that actually did basic validation and would also encourage task writers to do validation properly.

> Paths

> > Consider allowing the user to configure the ant system by setting the following path types.

> > - ant.type.path: path that is used to search for the type libraries
> > - ant.ext.path: path that is used to search for "Optional Packages" or extensions.

> > The default search path will probably include a per-user path element, a workspace path element and a system path elemtn that are searched in that order. Some possible defaults;

> > - Unix Per-user: ${user.home}/.ant/lib, ${user.home}/.ant/ext
> > - Windows Per-user: ${user.home}/ant/lib, ${user.home}/ant/ext
> > - MacOSX Per-user: ${user.home}/Library/Ant/lib, ${user.home}/Library/Ant/ext
> > - Unix System-wide: /opt/ant/lib, /opt/ant/ext
> > - Unix System-wide: /usr/local/ant/lib, /usr/local/ant/ext
> > - Windows System-wide: %SYS_DRIVE%/Program Files/ant/lib, %SYS_DRIVE%/Program Files/ant/ext
> > - MacOSX System-wide: /Library/Ant/lib, /Library/Ant/ant/ext

> Ant 1.x Compatibility

> > The Ant1 Compatibility layer is still in early stages of development.

> > - Get a version of `<antcall>` working.
> > - Provide hooks between Ant1 references and Myrmidon properties. May use converters for adapting Ant2 objects (like Ant2 `<path>` or `<fileset>`) as Ant1 types.
> > - Missing tests:
> >   - Make sure properties are shared between Ant1 and Myrmidon tasks.
> > - Get GUMP runs going using Myrmidon.
> > - Add protected accessors for get/set/list properties in Ant1 Project, to minimise the amount of code duplication in Ant1CompatProject.

> Virtual File System

> > The VFS needs plenty of work:

> > - Move files/folders.
> > - Recursive folders copy.
> > - Search through a file hierarchy, using Ant-style wildcards.
> > - Search through a file hierarchy, using a Selector interface.
> > - The in-memory caching mechanism is pretty rudimentary at this stage. It needs work to make it size capped. In addition, some mechanism needs to be provided to release and refresh cached info.
> > - Convert files/folders into local files, for handing off to external commands, or legacy tasks.
> > - Refactor the replication mechanism out of ZipFileSystemProvder, and make more general pluggable.
> > - Capabilities discovery.
> > - Attributes and attribute schema.
> > - Handle file canonicalisation better (for cases like case-insensitive file systems, symbolic links, name encoding, etc).
> > - File system layering. That is, the ability for a file system to sit on top of another file system, or a file from another file system (e.g. Zip/Jar/Tar file systems, gzip/encoding file systems, virtual file systems).

> File Data-Types and Tasks

> > The file data-types, such as `<fileset>` and `<path>`, are some of the most widely used parts of Ant 1.x. Unfortunately, they aren't particularly extensible.

> > - Redesign the file data-types, replacing them with an interface-based API, plus a set of implementations. The API should use the VFS file `FileObject`, rather than `java.io.File`. This process has started, in the `antlib.vfile` package.
> > - File Selectors:
> >   - Change `AbstractNameFileSelector` to use Ant 1 style patterns matches, rather than Globs patterns.
> >   - Add 'defaultexcludes' to `DefaultFileSet`. Also add a file selector implementation that matches everything except the default excludes.
> >   - Add a name selector that loads patterns from a file.
> >   - Add more selector implementations: size and last-modified comparisons, checksum comparison, byte-wise content comparison.
> > - File conditions:
> >   - Add more condition implementations that perform checks on files. One that searches a path for a file would be useful.
> > - File Name Mappers:
> >   - Change FileNameMapper.mapFileName() to take vfs.FileName objects.
> >   - Move the current mapper implementations across to antlib.
> > - File Sets:
> >   - Add a file set implementation that provides the union of several nested file sets (that is, a file set that merges several file sets together).
> >   - Add a file set implementation that filters files that are up-to-date WRT some other file. Alternatively, this might be better done as a file selector.
> > - Paths:
> >   - Add path implementations that evaluate to the system classpath, and the ant runtime classpath. Or, more generally, combine this with `ClassLoaderManager` to evaluate to the classpath of any 'library' (e.g. system classpath, ant runtime, tools.jar, an antlib, an installed extension, or the system classes of another JVM for cross-compiling).
> > - Port across the Ant 1 file filter proposal, once it is complete.
> > - Copy task:
> >   - Implement 'preservelastmodified', 'overwrite', and 'includeemptydirs'.
> >   - Support a file name mapper.
> >   - Support file filters.
> >   - Detect and handle destination file name collisions.
> > - Implement the `<move>`, `<delete>`, `<touch>` and `<mkdir>` tasks on top of the VFS and the new file data-types. Might be some scope for generalising 'touch' and 'mkdir' into a single task.

> Command-line and Configuration Files

> > One of the goals of Ant 2 is to allow the user to easily customise and extend Ant. The command-line and local configuration files, are two places where this would be done. Currently, Myrmidon some customisation from the command-line, but does not support configuration files.

> > - Load configuration from system (from $ANT_HOME) and user (from $HOME) configuration files.
> > - Allow the following via config files:
> >   - Add (or override) the `lib` and `ext` directories.
> >   - Enable more than one listener, and configure the listeners from the conents of the config file.
> >   - Import libraries, and set properties.
> >   - Execute tasks.
> >   - Install and configure runtime services.

> Scripting

> > Add the ability to extend Ant using languages other than Java:

> > - Define a task using a scripting language such as Javascript.
> > - Use Rhino's ability to implement Java interfaces, to implement and define types, such as FileSelector, or Condition.
> > - Define a task using template.
> > - Add some lightweight scripting tasks.

> Documentation

> > Everyone loves writing documentation, and so a goal for Ant 2 is to generate a lot of reference documentation for tasks and other types directly from the source. Unfortunately, there's still plenty of tutorial material to write. In particular we need these:

> > - User documentation - describing things like the build file format, how properties work, how to use references, how sub-builds work, how to customise Myrmidon, and so on.
> > - Task writer documentation - describing things like an overview the task API, how configuration works, task lifecycle, how to assemble an antlib, a catalog of the runtime services, and so on.
> > - Myrmidon developer documentation - a broad outline of the architecture, how to build, how to test, and so on. Also, this todo list needs plenty of filling out.

> Miscellaneous

> > A completely unordered list of items, big and small:

> > - Search through the code for 'TODO' items and fix them.
> > - Allow service factories to be configured from the contents of the `ant-services.xml` descriptor.
> > - Add verbosity level to ProjectListener LogEvent
> > - Fire ProjectListener events taskFinished(), targetFinished() and projectFinished() events on build failure, adding indicator methods to ProjectEvent.
> > - Fire ProjectListener events projectStarted() and projectFinished() events on start and finish of referenced projects, adding indicator methods to ProjectEvent.
> > - Detect duplicate type names.
> > - Add fully qualified type names, based on antlib name and type shorthand name. Allow these to be used in build files in addition to the shorthand names.
> > - Move the `<http>` and `<socket>` conditions to an antlib. Need to resolve how these will be passed a logger.
> > - Make the `<uptodate>` task a condition, and move to an antlib.
> > - Split up `<is-set>` condition into is-set and is-true conditions.
> > - Allow the `<if>` task to take any condition implementation.
> > - Add an else block to the `<if>` task.
> > - Split the `<available>` condition into separate conditions that test for the availability of a class, or a resource.
> > - Move `crimson.jar` to `bin/lib` in the distribution, and make available to other jars via the extension mechanism.
> > - Change `DefaultPropertyResolver` to ignore '$${'.
> > - Add a `--type` command-line option, to allow the project builder to be manually selected.
> > - Unit tests.

Copyright © 2000-2002, Apache Software Foundation