Iterating Over A Set of Parameter Values
The way it works is that callover takes a set of parameters, a property name, and a target in the current Ant environment. The target is called (like <antcall>) in the ant file once for each parameter value in the set with the specified property name set to that value.There are three types of parameter sets: one generated from a fileset with an optional mapper, one generated from a dirset, and one generated from a delimited list of values.
Using a FileSet
This example executes an external command for every text file in a directory hierarchy.<target name="fooOver">
<callover target="doFoo" inheritAll="false">
<paramset name="file.name">
<fileset dir="${root.dir}" includes="**/*.txt">
<globmapper from="*.txt" to="*"/>
</fileset>
</paramset>
</callover>
</target>
<target name="doFoo">
<exec executable="foo">
<arg value="${file.name}.txt"/>
<arg value="${file.name}.out"/>
</exec>
</target>
Using a DirSet
This example iterates over a set of directories with a common structure and performs a clean on each one.<target name="cleanall">
<callover target="clean" inheritAll="false">
<paramset name="module.dir">
<dirset dir="${module.root}" includes="*"/>
</paramset>
</callover>
</target>
<target name="clean">
<delete dir="${module.dir}/dist" failonerror="no"/>
<delete dir="${module.dir}/test/results" failonerror="no"/>
<delete dir="${module.dir}/test/plain/classes" failonerror="no"/>
<delete dir="${module.dir}/test/ejb/classes" failonerror="no"/>
<delete dir="${module.dir}/test/web/classes" failonerror="no"/>
<delete>
<fileset dir="${module.dir}" includes="**/*.class"/>
</delete>
</target>
Using a ValueSet
This example echoes each value one at a time.<target name="processInParallel">
<callover target="process" parallel="3">
<paramset name="greek.letter">
<valueset values="alpha beta gamma delta epsilon" delimiter=" "/>
</paramset>
</callover> </target>
<target name="process">
<echo message="Current letter is ${greek.letter}."/>
</target>
The Real Power: Combinations
The best part of <callover> is that if multiple parameter sets are provided, then the target task is called once for every combination of parameter values. (Hopefully you don't have too many combinations!)<target name="postProcess">
<callover target="doPostProcess" inheritAll="false">
<paramset name="program">
<valueset values="foo bar baz" delimiter=" "/>
</paramset>
<paramset name="file.name">
<fileset dir="${root.dir}" includes="*.out"/>
<paramset>
</callover>
</target>
<target name="doPostProcess">
<exec executable="${program}">
<arg value="${file.name}"/>
</exec>
</target>
The above example will run each of the given programs foo, bar, and baz once for each file in the given directory that ends with .out.
The Code
/*Copyright 2005 Steven S. Morgan
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
steven@technicalabilities.com
http://www.technicalabilities.com/java
*/
package com.technicalabilities.ant;
import java.io.*;
import java.util.*;
import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Project;
import org.apache.tools.ant.Task;
import org.apache.tools.ant.taskdefs.CallTarget;
import org.apache.tools.ant.taskdefs.Property;
import org.apache.tools.ant.types.AbstractFileSet;
import org.apache.tools.ant.types.FileSet;
import org.apache.tools.ant.types.DirSet;
import org.apache.tools.ant.types.Mapper;
/**
<p>
The CallOverTask iterates over sets of values calling another task
for each combination of these values.
</p>
<p>
Using explicit values:
</p>
<table border="2" bgcolor="#ffeeee">
<tr><td><strong>Build file</strong></td></tr>
<tr><td>
<pre>
<target name="echoStuff">
<echo message="param1 = ${param1}"/>
</target>
<target name="example1">
<callover target="echoStuff">
<paramset name="param1">
<valueset values="separated,by,commas"/>
</paramset>
</callover>
</target></pre>
</td></tr>
</table>
<table border="2" bgcolor="#ffeeee">
<tr><td><strong>Results</strong></td></tr>
<tr><td>
<pre>
example1:
echoStuff:
[echo] param1 = separated
echoStuff:
[echo] param1 = by
echoStuff:
[echo] param1 = commas
BUILD SUCCESSFUL
Total time: 1 second
</pre>
</td></tr>
</table>
<p>
A FileSet may be used with the ParamSet as well.
</p>
<table border="2" bgcolor="#ffeeee">
<tr><td><strong>Build file</strong></td></tr>
<tr><td>
<pre>
<target name="echoFileNames">
<echo message="filename = ${filename}"/>
</target>
<target name="example2">
<mkdir dir="tmp/subdir"/>
<touch file="tmp/file1.txt"/>
<touch file="tmp/subdir/file2.txt"/>
<callover target="echoFileNames">
<paramset name="filename">
<fileset dir="tmp" includes="**"/>
</paramset>
</callover>
<delete dir="tmp"/>
</target></pre>
</td></tr>
</table>
<table border="2" bgcolor="#ffeeee">
<tr><td><strong>Results</strong></td></tr>
<tr><td>
<pre>
example2:
[mkdir] Created dir: C:\dev\work\ant\tmp\subdir
[touch] Creating C:\dev\work\ant\tmp\file1.txt
[touch] Creating C:\dev\work\ant\tmp\subdir\file2.txt
echoFileNames:
[echo] filename = file1.txt
echoFileNames:
[echo] filename = subdir\file2.txt
[delete] Deleting directory C:\dev\work\ant\tmp
BUILD SUCCESSFUL
Total time: 1 second
</pre>
</td></tr>
</table>
<p>
Multiple parameter sets are possible, as are static parameters, mappers for
file sets, and specified delimiters. When using multiple parameter sets,
every combination of parameters will be performed. This can quickly get
out of hand!
</p>
<table border="2" bgcolor="#ffeeee">
<tr><td><strong>Build file</strong></td></tr>
<tr><td>
<pre>
<target name="echoAll">
<echo message="myValue is ${myValue}"/>
<echo message="filename is ${filename}"/>
<echo message="param1 is ${param1}"/>
</target>
<target name="example3">
<mkdir dir="tmp/subdir"/>
<touch file="tmp/file1.txt"/>
<touch file="tmp/subdir/file2.txt"/>
<callover target="echoAll">
<param name="myValue" value="always the same"/>
<paramset name="filename">
<fileset dir="tmp" includes="**"/>
<mapper type="glob" from="*.txt" to="*"/>
</paramset>
<paramset name="param1">
<valueset values="separated|by|pipes" delimiter="|"/>
</paramset>
</callover>
<delete dir="tmp"/>
</target></pre>
</td></tr>
</table>
<table border="2" bgcolor="#ffeeee">
<tr><td><strong>Results</strong></td></tr>
<tr><td>
<pre>
example3:
[mkdir] Created dir: C:\dev\work\ant\tmp\subdir
[touch] Creating C:\dev\work\ant\tmp\file1.txt
[touch] Creating C:\dev\work\ant\tmp\subdir\file2.txt
echoAll:
[echo] myValue is always the same
[echo] filename is file1
[echo] param1 is separated
echoAll:
[echo] myValue is always the same
[echo] filename is file1
[echo] param1 is by
echoAll:
[echo] myValue is always the same
[echo] filename is file1
[echo] param1 is pipes
echoAll:
[echo] myValue is always the same
[echo] filename is subdir\file2
[echo] param1 is separated
echoAll:
[echo] myValue is always the same
[echo] filename is subdir\file2
[echo] param1 is by
echoAll:
[echo] myValue is always the same
[echo] filename is subdir\file2
[echo] param1 is pipes
[delete] Deleting directory C:\dev\work\ant\tmp
BUILD SUCCESSFUL
Total time: 2 seconds
</pre>
</td></tr>
</table>
<p>
The CallOverTask also supports parallel execution of these tasks.
The <tt>parallel</tt> attribute indicates the number of tasks that
may execute concurrently. Threads are pooled and reused for
efficiency. Note that it is because of this behavior that the
CallOverTask will not abort as a result of errors committed in
the subtasks. The errors are reported via Ant's logging mechanism.
</p>
@author $Author: ssmorgan $
@version $Revision: 1.6 $
*/
public class CallOverTask extends Task
{
/**
The task to be called.
*/
private String target;
/**
Whether or not to inherit properties.
*/
private boolean inheritAll = true;
/**
Whether or not to inherit references.
*/
private boolean inheritRefs = false;
/**
How many tasks may be run simultaneously.
*/
private int parallel = 1;
/**
Static parameters.
*/
private ArrayList params;
/**
Dynamic parameters, the ones from the parameter sets.
*/
private ArrayList paramSets;
/**
Create a new CallOverTask.
*/
public CallOverTask ()
{
params = new ArrayList();
paramSets = new ArrayList();
}
/**
For each combination of parameters in the given <tt><paramset></tt>
tags, call the requested target. This method does not return until
all of these tasks have completed.
@throws BuildException if an error is encountered parsing the XML
(Note that tasks executed as a result of this
<tt><callover></tt> task may generate
their own exceptions, but they will not cause
each other or this task to fail nor will they
cause ant to abort.)
*/
public void execute()
{
if (target == null)
{
throw new BuildException("Attribute target is required.", location);
}
if (paramSets.isEmpty())
{
throw new BuildException("paramSet element required -- otherwise use <antcall>", location);
}
if (parallel < 1)
{
throw new BuildException("Number of parallel threads must be positive", location);
}
TaskThreadPool threadPool = new TaskThreadPool(parallel);
Iterator [] paramSetIters = new Iterator [paramSets.size()];
String [] currentValues = new String [paramSets.size()];
for (int i=0; i<paramSetIters.length; ++i)
{
paramSetIters[i] = ((ParamSet)paramSets.get(i)).values();
if (!paramSetIters[i].hasNext())
{
log("paramset for " + ((ParamSet)paramSets.get(i)).getName() + " is empty.", Project.MSG_WARN);
return;
}
currentValues[i] = (String)paramSetIters[i].next();
}
boolean done = false;
while (!done)
{
CallTarget callee = (CallTarget)project.createTask("antcall");
callee.setOwningTarget(super.target);
callee.setTaskName(getTaskName());
callee.setLocation(location);
callee.init();
callee.setTarget(target);
callee.setInheritAll(inheritAll);
callee.setInheritRefs(inheritRefs);
Iterator paramIter = params.iterator();
while (paramIter.hasNext())
{
Property param = (Property)paramIter.next();
Property p = callee.createParam();
// If only clone() were supported... :)
if (param.getValue() != null)
{
p.setName(param.getName());
p.setValue(param.getValue());
}
else if (param.getFile() != null)
{
if (p.getPrefix() != null)
p.setPrefix(param.getPrefix());
p.setFile(param.getFile());
}
else if (param.getResource() != null)
{
if (p.getPrefix() != null)
p.setPrefix(param.getPrefix());
p.setPrefix(param.getResource());
if (param.getClasspath() != null)
{
p.setClasspath(param.getClasspath());
}
}
else if (param.getEnvironment() != null)
{
p.setPrefix(param.getEnvironment());
}
}
for (int i=0; i<currentValues.length; ++i)
{
Property p = callee.createParam();
p.setName(((ParamSet)paramSets.get(i)).getName());
p.setValue(currentValues[i]);
}
threadPool.getTaskThread().setTask(callee);
boolean incrementPrevious = true;
for (int i=currentValues.length-1; i>=0 && incrementPrevious; --i)
{
incrementPrevious = !paramSetIters[i].hasNext();
if (incrementPrevious)
{
if (i == 0)
{
done = true;
}
else
{
paramSetIters[i] = ((ParamSet)paramSets.get(i)).values();
}
}
if (!done)
currentValues[i] = (String)paramSetIters[i].next();
}
}
threadPool.done();
}
/**
Set the target to be called with each parameter set combination.
There is no default value; this must be set.
@param target the name of the target to be called
*/
public void setTarget(String target)
{
this.target = target;
}
/**
Set the number of tasks that may be run in parallel. The default
value is 1.
@param parallel the maximum number of tasks that may be run in parallel
*/
public void setParallel (int parallel)
{
this.parallel = parallel;
}
/**
Set or clear the flag indicating if properties should be inherited
by the child tasks. The default value is <tt>true</tt>.
@param b <tt>true</tt> to inherit, <tt>false</tt> to not inherit
*/
public void setInheritAll (boolean b)
{
this.inheritAll = b;
}
/**
Set or clear the flag indicating if references should be inherited
by the child tasks. The default value is <tt>false</tt>.
@param b <tt>true</tt> to inherit, <tt>false</tt> to not inherit
*/
public void setInheritRefs (boolean b)
{
this.inheritRefs = b;
}
/**
Like <tt><param></tt> tag for <tt><antcall></tt>, this
creates a Property for use by the subtasks.
@return a new Property
*/
public Property createParam ()
{
Property param = new Property();
params.add(param);
return param;
}
/**
Create a new ParamSet for this CallOverTask.
@return a new ParamSet
*/
public ParamSet createParamSet ()
{
ParamSet paramSet = new ParamSet();
paramSets.add(paramSet);
return paramSet;
}
/**
One of the threads executing a subtask caught something. Report
it via Ant's logging with MSG_ERR priority.
@param t the caught Throwable
@param task the task that generated the caught Throwable
*/
void reportThrowable (Throwable t, Task task)
{
try
{
StringWriter out = new StringWriter();
PrintWriter buf = new PrintWriter(out);
buf.print("Throwable encountered on ");
buf.print(Thread.currentThread().getName());
buf.print(" by ");
buf.println(task.getTaskName());
t.printStackTrace(buf);
buf.flush();
log(out.toString(), Project.MSG_ERR);
buf.close();
out.close();
}
catch (IOException ioe)
{
// highly unlikely that out.close() will throw this, but have to catch it
log(ioe.getMessage(), Project.MSG_WARN);
}
}
/**
A ParamSet defines a set of values over which the CallOverTask
will iterate. A paramset may contain either a ValueSet or a
FileSet with an optional Mapper.
*/
public class ParamSet
{
/**
The name to be assigned the values that this ParamSet will
generate. This is the name that the subtask will find the
value.
*/
private String name;
/**
The set of values over which to iterate as a delimited string.
*/
private ValueSet valueSet;
/**
The set of files over which to iterate. This ParamSet will
generate String values <em>relative to the basedir of the
FileSet</em> with each iteration. Can be a DirSet.
*/
private AbstractFileSet fileSet;
/**
A Mapper for reformatting the filenames generated by the FileSet.
*/
private Mapper filenameMapper;
/**
Set the name by which subtasks will know these values.
@param name a Property name
*/
public void setName (String name)
{
this.name = name;
}
/**
Get the name by which subtasks will know these values.
@return a Property name
*/
String getName ()
{
return name;
}
/**
Create a new ValueSet for this ParamSet.
@return a new ValueSet
*/
public ValueSet createValueSet ()
{
valueSet = new ValueSet();
return valueSet;
}
/**
Add a FileSet to this ParamSet.
@param fileSet the FileSet to be added
*/
public void addFileSet (FileSet fileSet)
{
this.fileSet = fileSet;
}
/**
Add a DirSet to this ParamSet.
@param dirSet the DirSet to be added
*/
public void addDirSet (DirSet dirSet)
{
this.fileSet = dirSet;
}
/**
Create a new Mapper for this ParamSet.
@return a new Mapper
@throws BuildException if a Mapper has already been defined
*/
public Mapper createMapper () throws BuildException
{
if (filenameMapper != null)
{
throw new BuildException("Cannot define more than one mapper", location);
}
filenameMapper = new Mapper(project);
return filenameMapper;
}
/**
Obtain an Iterator for the values represented by this ParamSet.
@return an Iterator that will provide each value from this ParamSet
*/
Iterator values()
{
if (valueSet != null)
{
if (filenameMapper != null)
throw new BuildException("paramset does not use a mapper with a valueset", location);
// When using a ValueSet, return its Iterator.
return valueSet.values();
}
if (fileSet != null)
{
if (filenameMapper == null)
{
/**
This Iterator is created when there is a FileSet
without a Mapper.
*/
return new Iterator ()
{
/**
The files known to the FileSet
*/
private String [] files = (fileSet instanceof FileSet)
? fileSet.getDirectoryScanner(project).getIncludedFiles()
: fileSet.getDirectoryScanner(project).getIncludedDirectories();
/**
Internal index
*/
private int i=0;
/**
Show if values still remain.
@return <tt>true</tt> if values still remain,
<tt>false</tt> if not
*/
public boolean hasNext ()
{
return i < files.length;
}
/**
Return the next value in the FileSet.
Note that this will be relative to the
FileSet's basedir.
@return the next value in the FileSet
*/
public Object next ()
{
return files[i++];
}
/**
Unsupported by this Iterator.
@throws UnsupportedOperationException always
*/
public void remove ()
{
throw new UnsupportedOperationException();
}
};
}
else
{
/**
This Iterator is created when a Mapper is present
with a FileSet.
*/
return new Iterator ()
{
/**
The files known to this FileSet.
*/
private String [] files = (fileSet instanceof FileSet)
? fileSet.getDirectoryScanner(project).getIncludedFiles()
: fileSet.getDirectoryScanner(project).getIncludedDirectories();
/**
Internal index
*/
private int i=0;
/**
The names generated by the Mapper for the
given set of files.
*/
private String [] mappedNames = new String [0];
/**
Internal index
*/
private int j=0;
/**
The next value to be returned by this Iterator
*/
private String nextValue;
/**
Constructor sets the first value.
*/
{
setNextValue();
}
/**
The logic here is a bit funky because a
FileNameMapper can return multiple mappings
for a single file. As long as mappings
remain for the current file, return those
mappings. Once those mappings are exhausted,
then go the mappings for the next file.
Once all the files have been exhausted, then
we are done.
*/
private void setNextValue ()
{
nextValue = null;
while (j < mappedNames.length && nextValue == null)
{
nextValue = mappedNames[j];
++j;
}
if (nextValue == null)
{
while (i < files.length && nextValue == null)
{
mappedNames = filenameMapper.getImplementation().mapFileName(files[i]);
++i;
if (mappedNames != null)
{
j = 0;
setNextValue();
}
}
}
}
/**
Show if values still remain.
@return <tt>true</tt> if values still remain,
<tt>false</tt> if not
*/
public boolean hasNext ()
{
return nextValue != null;
}
/**
Return the next value.
@return the next value from the mapped FileSet
*/
public Object next ()
{
String value = nextValue;
setNextValue();
return value;
}
/**
Unsupported by this Iterator.
@throws UnsupportedOperationException always
*/
public void remove ()
{
throw new UnsupportedOperationException();
}
};
}
}
throw new BuildException("paramset must have a valueset, a fileset, or a dirset", location);
}
}
/**
A ValueSet parses a delimited String.
*/
public class ValueSet
{
/**
The delimited String
*/
private String values;
/**
The value delimiter
*/
private String delimiter=",";
/**
Set the delimited String of values. There is no
default value; this must be set.
@param values the values as a delimited String
*/
public void setValues (String values)
{
this.values = values;
}
/**
Set the value delimiter. The default value is the comma (,).
@param delimiter the value delimiter
*/
public void setDelimiter (String delimiter)
{
this.delimiter = delimiter;
}
/**
Provide an Iterator for accessing the values represented
by this ValueSet.
@return an Iterator that iterates over the defined values
*/
Iterator values ()
{
/**
This Iterator is created when a ParamSet is defined by
a ValueSet.
*/
return new Iterator ()
{
/**
The Iterator simply wraps a StringTokenizer.
*/
private StringTokenizer tok = new StringTokenizer(values, delimiter);
/**
Show if values still remain.
@return <tt>true</tt> if values still remain,
<tt>false</tt> if not
*/
public boolean hasNext ()
{
return tok.hasMoreTokens();
}
/**
Return the next value.
@return the next value from the ValueSet
*/
public Object next ()
{
return tok.nextToken();
}
/**
Unsupported by this Iterator.
@throws UnsupportedOperationException always
*/
public void remove ()
{
throw new UnsupportedOperationException();
}
};
}
}
/**
A thread pool provides an optimal implementation for supporting
multiple parallel threads with minimal overhead.
*/
private class TaskThreadPool
{
/**
The pool of available TaskThreads
*/
private ArrayList pool;
/**
A list of all TaskThreads managed by this pool
*/
private ArrayList references;
/**
Create a new TaskThreadPool with the given number
of available TaskThreads.
@param parallel the number of available TaskThreads
*/
TaskThreadPool (int parallel)
{
this.pool = new ArrayList(parallel);
for (int i=0; i<parallel; ++i)
{
this.pool.add(new TaskThread(this, i));
}
this.references = new ArrayList(this.pool);
}
/**
Get the next available TaskThread. This method blocks the
calling Thread until a TaskThread is available.
@return the next available TaskThread
*/
synchronized TaskThread getTaskThread ()
{
while (pool.isEmpty())
{
try
{
wait();
}
catch (InterruptedException goOn) {}
}
return (TaskThread)pool.remove(0);
}
/**
When a TaskThread comletes its task, it returns to
the pool here.
@param taskThread the newly-available TaskThread
*/
synchronized void taskComplete (TaskThread taskThread)
{
pool.add(taskThread);
notify();
System.gc();
}
/**
The CallOverTask calls done() on the TaskThreadPool when
it is done handing out tasks. This method notifies the
TaskThreads and then calls join() on all the TaskThreads.
As a result, this method will not return until all the
TaskThreads have completed their tasks.
*/
void done ()
{
for (Iterator refs = references.iterator(); refs.hasNext();)
{
TaskThread nextThread = (TaskThread)refs.next();
nextThread.done();
try
{
nextThread.join();
}
catch (InterruptedException goOn) {}
}
}
}
/**
A TaskThread is a resuable Thread that processes tasks handed
out by the CallOverTask.
*/
private class TaskThread extends Thread
{
/**
The task to be performed
*/
private Task task;
/**
Flag indicating if the CallOverTask is finished handing
out tasks
*/
private boolean done;
/**
The TaskThreadPool managing this TaskThread
*/
private TaskThreadPool pool;
/**
This TaskThread's number (for logging, mainly)
*/
private int threadNumber;
/**
Create a new TaskThread.
@param pool the managing TaskThreadPool
@param threadNumber this TaskThread's number (used to set
the Thread's name)
*/
TaskThread (TaskThreadPool pool, int threadNumber)
{
this.done = false;
this.pool = pool;
this.threadNumber = threadNumber;
setName("callover(" + Integer.toHexString(CallOverTask.this.hashCode()) + ") thread-" + threadNumber);
start();
}
/**
Set this task's Thread. The TaskThread immediately goes
to work on this task in its own Thread.
@param task the task to be performed
*/
synchronized void setTask (Task task)
{
this.task = task;
if (task != null)
{
notify();
}
}
/**
Tell this TaskThread that the CallOverTask is done handing
out tasks. When the current task, if any, is complete, then
the run() method will exit.
*/
synchronized void done ()
{
this.done = true;
notify();
}
/**
Needed a synchronized accessor so that the calling Thread
would by synch'ed up with the other Threads. Test for both
done and for no task assignment.
@return <tt>true</tt> if the CallOverTask is done handing
out tasks, <tt>false</tt> otherwise
*/
private synchronized boolean isDone ()
{
return this.done && this.task == null;
}
/**
Performs all assigned tasks until the CallOverTask says
it is done. Any Throwables caught are relayed through
Ant's error log.
@see com.technicalabilities.ant.CallOverTask#reportThrowable(java.lang.Throwable,org.apache.tools.ant.Task)
*/
public void run ()
{
while (!isDone())
{
Task myTask = null;
synchronized (this)
{
while (task == null && !done)
{
try
{
wait();
}
catch (InterruptedException goOn) {}
}
myTask = this.task;
}
if (myTask != null)
{
try
{
myTask.perform();
}
catch (Throwable t)
{
reportThrowable(t, myTask);
}
setTask(null);
pool.taskComplete(this);
}
}
}
}
}
No comments:
Post a Comment