<document>

    <properties>
        <author email="adammurdoch@apache.org">Adam Murdoch</author>
        <author email="peter@apache.org">Peter Donald</author>
        <title>Get Involved</title>
    </properties>

    <body>

        <section name="Todo List">

            <p>The broad goal is to grow Myrmidon from a prototype task engine into a fully
            fledged build system, that can serve as the basis for Ant 2.  The following
            sections describe some of the many things which still need to be done to
            achieve that goal.  This list is currently under construction.</p>

            <!--
            <subsection name="">
            </subsection>
            -->

            <subsection name="Integrate XDocs proposal">
                <p>Integrate with the XDocs proposal that generates XML documentation for
                tasks. Rework that proposal so that it knows about the myrmidon specific
                patterns and features. Also rework it so that it can support reading
                documentation and examples from side-by-side the task.</p>
            </subsection>

            <subsection name="TaskInfo">
                <p>Consider allowing task writers to write their own TaskInfo objects
                (or at least have them generated from XDoclet directives). This would
                encompass both documentation and perhaps introspection of the types.</p>
            </subsection>

            <subsection name="XML Catalog to load XML Fragments">
                <p>
                  When including fragments of XML we are currently forced to use relative paths.
                  However this is sometimes undesirable when a single fragment needs to be used
                  across several projects in several different locations. Instead we could use
                  a Catalog to name the fragment and then each developer would only need to install
                  the fragment once and it would be accessible from all the projects.
                </p>
            </subsection>

            <subsection name="Refactor Java Infrastructure into a Service">
                <p>Much like Exec should be decoupled from Ant runtime, so should classes
                to implement java task for the same benefits.</p>
            </subsection>

            <subsection name="Structural Dependency Utils">
                <p>
                  In the present ant, it is required that each task manage dependency separately.
                  This makes it a lot of work to implement even simple dependency checking. To this
                  day many of the core tasks do not implement it correctly. I am specifically
                  talking about "structural" dependency information. The main reason is that it is
                  painful to implement.
                </p>
                <p>
                  Some tasks do no dependency checking and will recompile/transform/etc everytime.
                  Others may perform a simple dependency checking (ie if source file is newer than
                  destination file then recompile). Ideally a dependency system would actually
                  calculate the dependencies correctly. So we need to have some mechanism to determine
                  that <code>foo.c</code> actually depends upon <code>foo.h</code>, <code>bar.h</code>
                  and <code>baz.h</code>. As this information is particular to each different task
                  we need to allow tasks to implement this behaviour. Possibly by supplying an interface
                  of the form;
                </p>
<source>
public interface DependencyGenerator
{
  File[] generateDependencies( File file );
}
</source>
                <p>
                  Generating the dependency information is a costly operation and thus we do not want to
                  be doing it everytime you run ant. We want to generate it on the initial build and then
                  persist somewhere. Everytime a file is out of date, it's dependency information would
                  be regenerated and stored in the dependency cache. Ideally this cache would also store the
                  above mentioned coloring information. So the entry for <code>foo.c</code> may declare that
                  it is dependent upon <code>foo.h</code>, <code>bar.h</code> and <code>baz.h</code>, aswell
                  as being compiled with -O2 flag. If any of the dependencies have changed or are out of date
                  then foo.c would need to be recompiled.
                </p>
                <p>
                  A possible API would be
                </p>
<source>
DependencyManager dm = ...;
dm.setFileSet( myFileSet );
dm.setDependencyCache( myDependencyCacheFile );
File[] files = cm.getOutOfDate();
</source>
            </subsection>

            <subsection name="Antlibs storing templates">
                <p>After a templating system is formalized it would useful to define
                a mechanism via which you can store templates in an antlib. These
                templates could then be loaded and used by build files through a
                "standard" mechanism. This may need to be merged with the XML catalog
                system.</p>
            </subsection>

            <subsection name="Antlibs Storing General Resources">
                <p>Add a system via which ant libs can store general resources for
                consumption by build users. This could be used to store the XML fragments
                for the XML catalog, the template fragments for templates, images for
                documentation system and so forth.</p>
            </subsection>

            <subsection name="Coloring API">
                <p>
                  When you execute a task such as "javac" there is two types of dependency information
                  that is important to analyze before we determine if we need to recompile a file. Say
                  we are compiling <code>Foo.java</code>, it may depend on the <code>Bar.java</code>
                  file. We call this "structural" dependency information - the structure of the source file
                  determines what other files it depend upon. However there is also "environmental"
                  dependency information. For instance if the <code>Foo.java</code> file was compiled with
                  <code>debug="true"</code> last run and this time needs to be compiled with
                  <code>debug="false"</code> then it is out of date and needs to be recompiled. We call this
                  "environmental" dependency information "coloring".
                </p>
                <p>
                  So we need to create an infrastructure that allows tasks to manage "coloring". So a task
                  should be able to add coloring information for each resource processed. When the task
                  comes to process the resource again it will detect if the coloring has changed and if it
                  has will force a recompile.
                </p>
                <p>
                  An API for such a bean has yet to be established but an example API would be.
                </p>
<source>
ColoringManager cm = ...;
cm.addColor( "debug", "true" );
cm.addColor( "optimize", "false" );
cm.setFileSet( myFileSet );
File[] files = cm.getOutOfDate();
</source>
            </subsection>

            <subsection name="Create Task/Element/Attribute Naming guidelines">

                <p> Currently Ant has a mixture of tasks from various stages it's evolution, with different
                authors and each utilizing different naming patterns. Some tasks use names such as
                "src" and "dest" while others use "file" and "tofile". It would be preferrable if
                consistent naming patterns were used. It is recomended that we come up with a "best
                practices" document to document our recomended naming patterns.</p>
                <p>Before we can come up with such a document we need to identify common patterns through
                out the tasks. Several tasks have the notion of transforming input from a "source"
                to a "destination". So we should have consistent naming schemes for these attributes and
                elements. Analysis of existing tasks will likely bring out other similar patterns. Once
                we have identified and documented these similarities then we can establish conventions.</p>

            </subsection>

            <subsection name="Rethink Notification/Event scheme">

                <p>We need to rethink the whole notificaiton scheme. Should tasks be able to
                raise events? Probably as long as we have <em>ContainerTasks</em>. Should tasks
                be able to query state of run? ie Can a task request "are we paused?" or "are we
                stopped?" ? Probably as that way long running tasks are given the opportunity
                to be gracefully halted by the end users (primarily aimed at IDE vendors here).</p>

            </subsection>

            <subsection name="XPath-like Locators for tasks">

                <p>Most tasks are grouped into some sort of task container. The task containers
                can be things like workspaces, projects, targets or other tasks. Each of these
                containers usually has a name. Thus we could refer to tasks via a path such as
                "/avalon/compile/javac" would refer to the task "javac" in the target "compile"
                in the project "avalon". In the past it has been requested that a task get access
                to this path programatically - other people have also asked for access to things
                like the currently running target. We need to assess this and decide whether we
                wish to support it.</p>

                <p>Another point to think about is that we could use XPath-like string to designate
                to other tasks to execute. ie antcall would refer to a path rather than a target name</p>

            </subsection>

            <subsection name="Embeddor HOWTO">

                <p>Assigned To: Peter</p>
                <p>Write a HOWTO describing how to embed Myrmidon into other
                applications.</p>

            </subsection>

            <subsection name="Optional Dependencies">

                <p>Assigned To: Peter</p>
                <p>Extend Myrmidons library management facilities so that optional
                dependencies may be declared for a library. ie The library will still
                operate in absence of such libraries but can provide further features
                if these libraries are present. Most likely this will be done via a
                new manifest entry "Optional-Extension-List:" that behaves similar
                to "Extension-List:" except that the extensions are optional.</p>

            </subsection>

            <subsection name="Facade task HOWTO">

                <p>Currently we have a few tasks that have multiple implementations. For instance
                Javac task can actually call jikes, jvc, classic javac or modern javac. Similar things
                will be seen with the jspc task and the cc task (if it ever gets written). We have a base
                class that is meant to facilitate this sort of task and make it easy to develope. See
                AbstractFacadeTask. However we need to write up a HOWTO so people can use it.</p>

            </subsection>

            <subsection name="Mail tasks">

                <p>Convert the ant1.x Mail tasks to myrmidon.</p>

            </subsection>

            <subsection name="Security Manager">
                <p>Add the ability to run java programs that call System.exit() by adding a
                security manager. Should look something like:</p>
                <source>
public class MyrmidonSecurityManager
    extends SecurityManager
{
    public void checkExit( final int status )
    {
        throw new ExitException( status );
    }

    public void checkPermission( final Permission permission )
    {
    }
}
                </source>
            </subsection>

            <subsection name="Self Hosting">

                <p>Myrmidon must be able to build itself.  Currently, it is built using
                Ant 1.x.  Ultimately, Myrmidon should be able to build itself from exactly the
                same build file.  To start with, however, there is no need for Myrmidon to be
                able to do this.  Myrmidon should also be able to be bootstrapped (that is, be
                able to be built from scratch, without using Ant 1.x at all).</p>

            </subsection>

            <subsection name="Validation Pass">

                <p>Consider calling validate() on task prior to execute(). This would allow
                us to have a "make -k" mode that actually did basic validation and would also
                encourage task writers to do validation properly.</p>

            </subsection>

            <subsection name="Paths">

                <p>Consider allowing the user to configure the ant system by setting the
                following path types.</p>
                <ul>
                    <li>ant.type.path: path that is used to search for the type libraries</li>
                    <li>ant.ext.path: path that is used to search for "Optional Packages"
                    or extensions.</li>
                </ul>
                <p>
                The default search path will probably include a per-user path element,
                a workspace path element and a system path elemtn that are searched in
                that order. Some possible defaults;
                </p>
                <ul>
                    <li>Unix Per-user: ${user.home}/.ant/lib, ${user.home}/.ant/ext</li>
                    <li>Windows Per-user: ${user.home}/ant/lib, ${user.home}/ant/ext</li>
                    <li>MacOSX Per-user: ${user.home}/Library/Ant/lib,
                    ${user.home}/Library/Ant/ext</li>
                    <li>Unix System-wide: /opt/ant/lib, /opt/ant/ext</li>
                    <li>Unix System-wide: /usr/local/ant/lib, /usr/local/ant/ext</li>
                    <li>Windows System-wide: %SYS_DRIVE%/Program Files/ant/lib,
                    %SYS_DRIVE%/Program Files/ant/ext</li>
                    <li>MacOSX System-wide: /Library/Ant/lib, /Library/Ant/ant/ext</li>
                </ul>

            </subsection>

            <subsection name="Ant 1.x Compatibility">
                <p>The Ant1 Compatibility layer is still in early stages of development.
                </p>
                <ul>
                    <li>Get a version of <code>&lt;antcall&gt;</code> working.</li>
                    <li>
                    Provide hooks between Ant1 references and Myrmidon properties.
                    May use converters for adapting Ant2 objects (like Ant2
                    <code>&lt;path&gt;</code> or <code>&lt;fileset&gt;</code>)
                    as Ant1 types.
                    </li>
                    <li>
                    Missing tests:
                    <ul>
                        <li>Make sure properties are shared between Ant1 and Myrmidon tasks.</li>
                    </ul>
                    </li>
                    <li>Get GUMP runs going using Myrmidon.</li>
                    <li>Add protected accessors for get/set/list properties in
                    Ant1 Project, to minimise the amount of code duplication in
                    Ant1CompatProject.</li>
                </ul>
            </subsection>

            <subsection name="Virtual File System">

                <p>The VFS needs plenty of work:</p>

                <ul>
                    <li>Move files/folders.</li>
                    <li>Recursive folders copy.</li>
                    <li>Search through a file hierarchy, using Ant-style wildcards.</li>
                    <li>Search through a file hierarchy, using a Selector interface.</li>
                    <li>The in-memory caching mechanism is pretty rudimentary at this stage.
                    It needs work to make it size capped.  In addition, some mechanism needs
                    to be provided to release and refresh cached info.
                    </li>
                    <li>Convert files/folders into local files, for handing off
                    to external commands, or legacy tasks.</li>
                    <li>Refactor the replication mechanism out of ZipFileSystemProvder,
                    and make more general pluggable.</li>
                    <li>Capabilities discovery.</li>
                    <li>Attributes and attribute schema.</li>
                    <li>Handle file canonicalisation better (for cases like case-insensitive
                    file systems, symbolic links, name encoding, etc).</li>
                    <li>File system layering.  That is, the ability for a file system to
                    sit on top of another file system, or a file from another file system
                    (e.g. Zip/Jar/Tar file systems, gzip/encoding file systems, virtual file
                    systems).
                    </li>
                </ul>

            </subsection>

            <subsection name="File Data-Types and Tasks">

                <p>The file data-types, such as
                    <code>&lt;fileset&gt;</code> and

                    <code>&lt;path&gt;</code>, are some of the most widely used parts of Ant 1.x.
                Unfortunately, they aren't particularly extensible.
                </p>

                <ul>
                    <li>Redesign the file data-types, replacing them with an interface-based
                    API, plus a set of implementations.  The API should use the VFS file

                        <code>FileObject</code>, rather than
                        <code>java.io.File</code>.
                    This process has started, in the
                        <code>antlib.vfile</code> package.

                    </li>
                    <li>File Selectors:

                        <ul>
                            <li>Change
                                <code>AbstractNameFileSelector</code> to use Ant 1 style patterns
                            matches, rather than Globs patterns.
                            </li>
                            <li>Add 'defaultexcludes' to
                                <code>DefaultFileSet</code>.  Also add a
                            file selector implementation that matches everything except the default
                            excludes.
                            </li>
                            <li>Add a name selector that loads patterns from a file.</li>
                            <li>Add more selector implementations: size and last-modified comparisons,
                            checksum comparison, byte-wise content comparison.</li>
                        </ul>
                    </li>
                    <li>File conditions:

                        <ul>
                            <li>Add more condition implementations that perform checks on files.
                            One that searches a path for a file would be useful.</li>
                        </ul>
                    </li>
                    <li>File Name Mappers:

                        <ul>
                            <li>Change FileNameMapper.mapFileName() to take vfs.FileName objects.</li>
                            <li>Move the current mapper implementations across to antlib.</li>
                        </ul>
                    </li>
                    <li>File Sets:

                        <ul>
                            <li>Add a file set implementation that provides the union of several
                            nested file sets (that is, a file set that merges several file sets
                            together).</li>
                            <li>Add a file set implementation that filters files that are up-to-date
                            WRT some other file.  Alternatively, this might be better done as a
                            file selector.</li>
                        </ul>
                    </li>
                    <li>Paths:

                        <ul>
                            <li>Add path implementations that evaluate to the system classpath,
                            and the ant runtime classpath.  Or, more generally, combine this with

                                <code>ClassLoaderManager</code> to evaluate to the classpath of any
                            'library' (e.g. system classpath, ant runtime, tools.jar,
                            an antlib, an installed extension, or the system classes of another JVM
                            for cross-compiling).
                            </li>
                        </ul>
                    </li>
                    <li>Port across the Ant 1 file filter proposal, once it is complete.</li>
                    <li>Copy task:

                        <ul>
                            <li>Implement 'preservelastmodified', 'overwrite', and 'includeemptydirs'.</li>
                            <li>Support a file name mapper.</li>
                            <li>Support file filters.</li>
                            <li>Detect and handle destination file name collisions.</li>
                        </ul>
                    </li>
                    <li>Implement the
                        <code>&lt;move&gt;</code>,
                        <code>&lt;delete&gt;</code>,

                        <code>&lt;touch&gt;</code> and
                        <code>&lt;mkdir&gt;</code> tasks on top
                    of the VFS and the new file data-types.  Might be some scope for generalising
                    'touch' and 'mkdir' into a single task.
                    </li>
                </ul>

            </subsection>

            <subsection name="Command-line and Configuration Files">

                <p>One of the goals of Ant 2 is to allow the user to easily customise and
                extend Ant.  The command-line and local configuration files, are two places
                where this would be done.  Currently, Myrmidon some customisation from the
                command-line, but does not support configuration files.</p>

                <ul>

                    <li>Load configuration from system (from $ANT_HOME) and user (from $HOME)
                    configuration files.</li>

                    <li>Allow the following via config files:

                        <ul>
                            <li>Add (or override) the
                                <code>lib</code> and
                                <code>ext</code> directories.
                            </li>
                            <li>Enable more than one listener, and configure the listeners from
                            the conents of the config file.</li>
                            <li>Import libraries, and set properties.</li>
                            <li>Execute tasks.</li>
                            <li>Install and configure runtime services.</li>
                        </ul>
                    </li>

                </ul>

            </subsection>

            <subsection name="Scripting">
                <p>Add the ability to extend Ant using languages other than
                Java:</p>
                <ul>
                    <li>Define a task using a scripting language such as Javascript.</li>
                    <li>Use Rhino's ability to implement Java interfaces, to
                    implement and define types, such as FileSelector, or
                    Condition.</li>
                    <li>Define a task using template.</li>
                    <li>Add some lightweight scripting tasks.</li>
                </ul>
            </subsection>

            <subsection name="Documentation">

                <p>Everyone loves writing documentation, and so a goal for Ant 2 is to
                generate a lot of reference documentation for tasks and other types directly
                from the source.  Unfortunately, there's still plenty of tutorial material
                to write.  In particular we need these:</p>

                <ul>
                    <li>User documentation - describing things like the build file format,
                    how properties work, how to use references, how sub-builds work, how to customise
                    Myrmidon, and so on.</li>

                    <li>Task writer documentation - describing things like an overview the task API,
                    how configuration works, task lifecycle, how to assemble an antlib, a catalog
                    of the runtime services, and so on.</li>

                    <li>Myrmidon developer documentation - a broad outline of the architecture,
                    how to build, how to test, and so on.  Also, this todo list needs plenty
                    of filling out.</li>

                </ul>

            </subsection>

            <subsection name="Miscellaneous">

                <p>A completely unordered list of items, big and small:</p>

                <ul>
                    <li>Search through the code for 'TODO' items and fix them.</li>
                    <li>Allow service factories to be configured from the contents of the

                        <code>ant-services.xml</code> descriptor.
                    </li>
                    <li>Add verbosity level to ProjectListener LogEvent</li>
                    <li>Fire ProjectListener events taskFinished(), targetFinished() and
                    projectFinished() events on build failure, adding indicator methods to
                    ProjectEvent.</li>
                    <li>Fire ProjectListener events projectStarted() and projectFinished()
                    events on start and finish of referenced projects, adding indicator methods
                    to ProjectEvent.</li>
                    <li>Detect duplicate type names.</li>
                    <li>Add fully qualified type names, based on antlib name and type shorthand name.
                    Allow these to be used in build files in addition to the shorthand names.</li>
                    <li>Move the
                        <code>&lt;http&gt;</code> and
                        <code>&lt;socket&gt;</code>
                    conditions to an antlib.  Need to resolve how these will be passed a logger.
                    </li>
                    <li>Make the
                        <code>&lt;uptodate&gt;</code> task a condition, and move to
                    an antlib.
                    </li>
                    <li>Split up
                        <code>&lt;is-set&gt;</code> condition into is-set and is-true conditions.
                    </li>
                    <li>Allow the
                        <code>&lt;if&gt;</code> task to take any condition implementation.
                    </li>
                    <li>Add an else block to the
                        <code>&lt;if&gt;</code> task.
                    </li>
                    <li>Split the
                        <code>&lt;available&gt;</code> condition into separate conditions
                    that test for the availability of a class, or a resource.
                    </li>
                    <li>Move
                        <code>crimson.jar</code> to
                        <code>bin/lib</code> in the distribution,
                    and make available to other jars via the extension mechanism.
                    </li>
                    <li>Change <code>DefaultPropertyResolver</code> to ignore '$${'.</li>
                    <li>Add a <code>--type</code> command-line option, to allow
                        the project builder to be manually selected.
                    </li>
                    <li>Unit tests.</li>
                </ul>

            </subsection>

        </section>

    </body>
</document>
