Improve reliability if ant parallel task tests

docs/manual/CoreTasks/parallel.html

@@ -11,6 +11,31 @@
 <h3>Description</h3>
 <p>Parallel is a container task - it can contain other Ant tasks. Each nested
 task within the parallel task will be executed in its own thread. </p>
+<h3>Parameters</h3>
+<table border="1" cellpadding="2" cellspacing="0">
+  <tr>
+    <td valign="top"><b>Attribute</b></td>
+    <td valign="top"><b>Description</b></td>
+    <td align="center" valign="top"><b>Required</b></td>
+  </tr>
+  <tr>
+    <td valign="top">threadCount</td>
+    <td valign="top">Maximum numbers of thread to use.</td>
+    <td align="center" valign="top">No</td>
+  </tr>
+  <tr>
+    <td valign="top">threadsPerProcessor</td>
+    <td valign="top">Maximum number of threads to use per available processor
+(Requires JDK 1.4)</td>
+    <td align="center" valign="top">No, defers to threadCount</td>
+  </tr>
+  <tr>
+    <td valign="top">pollInterval</td>
+    <td valign="top">Maximum number of milliseconds to wait for before checking 
+when waiting for available threads.</td>
+    <td align="center" valign="top">No, default is 1000</td>
+  </tr>
+</table>
 
 <p>Parallel tasks have a number of uses in an Ant build file including:</p>
 <ul>
@@ -41,6 +66,26 @@ In this situation, the parallel task will also fail.</p>
 sequential</a> task to define sequences of tasks to be executed on each thread
 within the parallel block</p>
 
+<p>The threadCount attribute can be used to place a maximum number of available
+threads for the execution.  When not present all child tasks will be executed at
+once.  When present then the maximum number of concurrently executing tasks will
+not exceed the number of threads specified.  Furthermore, each task will be
+started in the order they are given.  But no guarantee is made as to the speed 
+of execution or the order of completion of the tasks, only that each will be
+started before the next.<p>
+
+<p>If you are using J2RE 1.4 or later you can also use the threadsPerProcessor
+and the number of available threads will be the stated multiple of the number of
+processors (there is no affinity to a particular processor however).  This will
+override the value in threadCount.  If threadsPerProcessor is specified using
+any version prior to 1.4 then the value in threadCount will be used as is.</p>
+
+<p>When using threadCount and threadsPerProcessor care should be taken to insure
+that the build does not deadlock.  This can be caused by tasks such as waitFor
+takeing up all available threads before the tasks that would unlock the waitfor
+would occur.  This is not a repalcement for Java Language level thread
+semantics and is best used for "embarasingly parallel" tasks.</p>
+
 <h3>Examples</h3>
 <pre>
 &lt;parallel&gt;
@@ -75,6 +120,31 @@ compiled in one thead and a set of JSPs is being precompiled in another. As
 noted above, you need to be careful that the two tasks are independent, both in 
 terms of their dependencies and in terms of their potential interactions in
 Ant's external environment.</p>
+
+<pre>
+&lt;parallel threadCount='4'&gt;
+  &lt;ant target='TargetThatConsumesLotsOfCPUTimeAndMemory'&gt;
+    &lt;param name='file' value='one.txt'/&gt;
+  &lt;/ant&gt;
+  &lt;ant target='TargetThatConsumesLotsOfCPUTimeAndMemory'&gt;
+    &lt;param name='file' value='two.txt'/&gt;
+  &lt;/ant&gt;
+  &lt;ant target='TargetThatConsumesLotsOfCPUTimeAndMemory'&gt;
+    &lt;param name='file' value='three.txt'/&gt;
+  &lt;/ant&gt;
+  &lt;!-- repeated about 40 times --&gt;
+&lt;/parallel&gt;
+</pre>
+
+<p>This example represents a typical need for use of the threadCount and 
+threadsPerProcessor attributes.  Spinning up all 40 of those tasks could cripple
+the JVM for memory and the CPU for available time.  By limiting the number of
+concurrent executions you can get the task done in about the same assuming
+infinite memory time without needing infinite memory.  This is also a good
+candidiate for use of threadCount (and possibly threadsPerProcessor) because
+each task (in this hypothetical case) is independent and has no dependencies on
+the other tasks.</p>
+
 <hr>
 <p align="center">Copyright &copy; 2001-2002 Apache Software Foundation. All rights
 Reserved.</p>

src/etc/testcases/taskdefs/parallel.xml

@@ -22,103 +22,97 @@
   </target>
 
   <target name="testThreadCount">
-    <parallel threadCount='1' pollInterval="30">
-      <!-- expected start 1, end 1, start 2, end 2, start 3, end 3 -->
+    <echo>|1/</echo>
+    <parallel threadCount='1' pollInterval="60">
       <sequential>
-        <echo message="+1"/>
-        <sleep seconds="1"/>
-        <echo message="-1"/>
+        <echo message="+"/>
+        <sleep milliseconds="30"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <echo message="+2"/>
-        <sleep seconds="2"/>
-        <echo message="-2"/>
+        <echo message="+"/>
+        <sleep milliseconds="60"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <echo message="+3"/>
-        <sleep seconds="3"/>
-        <echo message="-3"/>
+        <echo message="+"/>
+        <sleep milliseconds="90"/>
+        <echo message="-"/>
       </sequential>
     </parallel>
+    <echo>|2/</echo>
     <parallel threadCount='2' pollInterval="30">
-      <!-- expected start 1, start 2, end 1, start 3, end 2, end 3 -->
       <sequential>
-        <echo message="+1"/>
-        <sleep seconds="1"/>
-        <echo message="-1"/>
+        <echo message="+"/>
+        <sleep milliseconds="30"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <sleep milliseconds="200"/>
-        <echo message="+2"/>
-        <sleep seconds="2"/>
-        <echo message="-2"/>
+        <echo message="+"/>
+        <sleep milliseconds="60"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <sleep milliseconds="300"/>
-        <echo message="+3"/>
-        <sleep seconds="3"/>
-        <echo message="-3"/>
+        <echo message="+"/>
+        <sleep milliseconds="90"/>
+        <echo message="-"/>
       </sequential>
     </parallel>
+    <echo>|3/</echo>
     <parallel threadCount='3' pollInterval="30">
-      <!-- expected start 1, start 2, start 3, end 1, end 2, end 3 -->
       <sequential>
-        <echo message="+1"/>
-        <sleep seconds="1"/>
-        <echo message="-1"/>
+        <echo message="+"/>
+        <sleep milliseconds="30"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <sleep milliseconds="200"/>
-        <echo message="+2"/>
-        <sleep seconds="2"/>
-        <echo message="-2"/>
+        <echo message="+"/>
+        <sleep milliseconds="60"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <sleep milliseconds="300"/>
-        <echo message="+3"/>
-        <sleep seconds="3"/>
-        <echo message="-3"/>
+        <echo message="+"/>
+        <sleep milliseconds="90"/>
+        <echo message="-"/>
       </sequential>
     </parallel>
+    <echo>|4/</echo>
     <parallel threadCount='4' pollInterval="30">
-      <!-- expected start 1, start 2, start 3, end 1, end 2, end 3 -->
       <sequential>
-        <echo message="+1"/>
-        <sleep seconds="1"/>
-        <echo message="-1"/>
+        <echo message="+"/>
+        <sleep milliseconds="30"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <sleep milliseconds="200"/>
-        <echo message="+2"/>
-        <sleep seconds="2"/>
-        <echo message="-2"/>
+        <echo message="+"/>
+        <sleep milliseconds="60"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <sleep milliseconds="300"/>
-        <echo message="+3"/>
-        <sleep seconds="3"/>
-        <echo message="-3"/>
+        <echo message="+"/>
+        <sleep milliseconds="90"/>
+        <echo message="-"/>
       </sequential>
     </parallel>
-    <parallel threadsPerProcessor='1' pollInterval="30">
-      <!-- expected result varies, depends on setup -->
-      <!-- this is a smoke test for threadsPerProcessor -->
+    <echo>|4/</echo>
+    <parallel threadsPerProcessor='1' threadcount='4' pollInterval="30">
       <sequential>
-        <!--echo message="+1"/-->
-        <sleep seconds="1"/>
-        <!--echo message="-1"/-->
+        <echo message="+"/>
+        <sleep milliseconds="30"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <!--echo message="+2"/-->
-        <sleep seconds="2"/>
-        <!--echo message="-2"/-->
+        <echo message="+"/>
+        <sleep milliseconds="60"/>
+        <echo message="-"/>
       </sequential>
       <sequential>
-        <!--echo message="+3"/-->
-        <sleep seconds="3"/>
-        <!--echo message="-3"/-->
+        <echo message="+"/>
+        <sleep milliseconds="90"/>
+        <echo message="-"/>
       </sequential>
     </parallel>
+    <echo>|</echo>
     
   </target>
 

src/testcases/org/apache/tools/ant/taskdefs/ParallelTest.java

@@ -53,10 +53,13 @@
  */
 package org.apache.tools.ant.taskdefs;
 import java.io.PrintStream;
+import junit.framework.AssertionFailedError;
+import org.apache.tools.ant.BuildException;
 
 import org.apache.tools.ant.BuildFileTest;
 import org.apache.tools.ant.DemuxOutputStream;
 import org.apache.tools.ant.Project;
+import org.apache.tools.ant.Task;
 
 /**
  * Test of the parallel TaskContainer
@@ -104,17 +107,59 @@ public class ParallelTest extends BuildFileTest {
     }
 
     /** tests basic operation of the parallel task */
-    public void testTreadCount() {
+    public void testThreadCount() {
         // should get no output at all
         Project project = getProject();
         project.setUserProperty("test.direct", DIRECT_MESSAGE);
         project.setUserProperty("test.delayed", DELAYED_MESSAGE);
         expectOutputAndError("testThreadCount", "", "");
         String log = getLog();
-        assertEquals("parallel tasks did't block on threads properly", log,
-            "+1-1+2-2+3-3+1+2-1+3-2-3+1+2+3-1-2-3+1+2+3-1-2-3");
-
+        int pos = 0;
+        while (pos > -1) {
+            pos = countThreads(log, pos);
+        }
+    }
+    
+    /**
+     * the test result string should match the regex 
+     * <code>^(\|\d+\/(+-)*)+\|$</code> for someting like 
+     * <code>|3/++--+-|5/+++++-----|</code>
+     *
+     *@returns -1 no more tests
+     *          # start pos of next test
+     *@throws AssertionFailedException when a constraint is invalid
+     */
+    static int countThreads(String s, int start) {
+        int firstPipe = s.indexOf('|', start);
+        int beginSlash = s.indexOf('/', firstPipe);
+        int lastPipe = s.indexOf('|', beginSlash);
+        if ((firstPipe == -1) || (beginSlash == -1) || (lastPipe == -1)) {
+            return -1;
+        }
+        
+        int max = Integer.parseInt(s.substring(firstPipe + 1, beginSlash));
+        int current = 0;
+        int pos = beginSlash + 1;
+        while (pos < lastPipe) {
+            switch (s.charAt(pos++)) {
+                case '+': 
+                    current++;
+                    break;
+                case '-':
+                    current--;
+                    break;
+                default:
+                    throw new AssertionFailedError("Only expect '+-' in result count, found " 
+                        + s.charAt(--pos) + " at position " + pos);
+            }
+            if (current > max) {
+                throw new AssertionFailedError("Number of executing threads exceeded number allowed: " 
+                    + current + " > " + max);
+            }
+        }
+        return lastPipe;
     }
+    
 
     /** tests the failure of a task within a parallel construction */
     public void testFail() {
@@ -142,5 +187,6 @@ public class ParallelTest extends BuildFileTest {
             System.setErr(err);
         }
     }
+    
 }