Continuous testing w/Ant

UPDATED - It Works

As we write code, a continuous feedback will help us know how we are progressing. And what code are we breaking as we add functionality. A way to run unit tests, as we code and save java files will be great!

I know there was a plugin for Eclipse, Continuous Testing from MIT. So I immediately downloaded the plugin, and tried to integrate with my eclipse IDE. but unluckily the plugin didn't worked with the version of the eclipse I work with. There also seems to be no activity in that plugin development. So I thought of ways to get this started, through some simple ways.

After checking other option with Eclipse, I came to know that you can create a task in ant build.xml, and assign that task to execute as part of build process (clean/rebuild) from within IDE. (My bad, I didn't realized that you can't trigger the ant task on every save command on resources. you can trigger only by manual build. Before I did checked this I went ahead on trying it out. so I will explain below, how much I reached there) It works!


Ok, so made an simple ant target with JUnit task in it. It executes all the unit tests in the project. As this would be time consuming, you will never use this. So this is not at all worth. Lets create test suites that represent a single unit that would be executed one every save operation. This would be the best approach, as test suites can represent a behavior/specification so it will give a larger perspective of what failed. And each package will have a test suite which could be triggered. But I want something that will work with my current setup.

So thought, how about an ant task that figures out itself what are the Test Cases that are affected by my currently working file. All I need is to find the right test cases that need to be executed, and pass it on the JUnit task. For this we don't even need to write an ant task, instead we just need to create a custom file selector ant component which can be used inside any ant <fileselector> task.

So I checked any existing tool/task that could list all source files that uses given class file. But I couldn't find something out of my inpatient quick search. So I looked into some reflections library so, I could trace if current file is dependent of the given file. I tried out Apache BCEL, as even some the core ant tasks uses the same library to do some bytecode engineering.

Here is the code for the custom selector.

public class DependentClassSelector extends org.apache.tools.ant.types.selectors.BaseExtendSelector {

String changedClassName;

public void setChangedClassName(String changedClassName) {
this.changedClassName = changedClassName.replace('.', '/');
}

@Override
public boolean isSelected(File basedir, String filename, File file)
throws BuildException {
boolean testable = filename.endsWith("Test.java");

if(testable && changedClassName != null)
{
testable = false;
//check if this Unit Test, depends on the changed class
//System.out.println("$$$$$$$$"+filename + "$$$$$$$$" + changedClassName);

filename = filename.replace(".java", "");

//System.out.println(filename);

com.sun.org.apache.bcel.internal.classfile.JavaClass javaFile = com.sun.org.apache.bcel.internal.Repository.lookupClass(filename);
com.sun.org.apache.bcel.internal.classfile.Constant[] constants = javaFile.getConstantPool().getConstantPool();

//System.out.println("loaded java file");

for (Constant constant : constants) {

//check if the constants pool has an entry for given class

if (constant != null && constant.toString().contains("L"+changedClassName+";"))
{
//System.out.println(constant);

testable = true;
break;
}

}

}

return testable;
}

After you coded the custom selector's isSelected method. just drop it in the class path, and add the typedef lines in the build.xml

<property environment="env"/>

<typedef name="selected-tests"
classname="org.countme.ant.tasks.DependentClassSelector" />

<target name="continuous_testing">

<junit  printsummary="yes" haltonfailure="yes">
<classpath>
<pathelement path="${classpath}"/>
<fileset dir=".">
<include name="**/*.jar"/>
</fileset>
<pathelement location="bin"/>
<dirset dir="bin">
<include name="**/*.class"/>
</dirset>

</classpath>

<batchtest fork="yes" todir="reports/">
<fileset dir="src">
<selected-tests changedClassName="${env.java_type_name}"/>
</fileset>
</batchtest>

</junit>
</target>

The ant script gets the current file open in the eclipse, by the environment variable java_type_name. To get this working, you should launch your ant script from within eclipse. The custom selector uses this information to decide where this test should be passed to the unit task or not. This works amazingly good, but still needs some more improvements, like: This code handles only, one level dependency, not the whole chain of dependency. This script need lot of improvements, but this looked like a good way to start.

Test Result: When I ran my script, after coding and saving the changes in just a single file BusinessDomain.java
It picks up the related test cases automatically.

Since I cant get this ant script triggered on every file save within eclipse, I will lose value of this script if I forget to run it on every save. Unfortunately eclipse ant builder cant be triggered on automatic builds (i.e. when eclipse compiles your file). If you know a way to get this fixed, let me know. I will be happy to use it.

Otherwise instead of starting with the files dependent on current file, we could run all tests that dependent on the files that changed since last run. This way I will have some gain over running the whole test suite.

Please comment on any continuos testing approach that worked for you!

UPDATE

After spending some more time today, I am able to get this working.  You should be able to set the Ant Builder to run any task, on Auto-Build (i.e. for every save). If you get NullPointerException, then you are missing some library in the class path. Also configure to export the {java_type_name} to the environment, by adding in the Environment Tab of the Ant Builder Configuration. Probably I will post with screenshots, on my next post

The other feature, which I thought about is to increase the chaining depth in the dependency. But it will be of too much cost to execute TestCases that are more than one block away from your modified class file.

Of the choices between:

• don't execute test case as you modify a class


• execute all of them


• execute all the dependent test cases

This sounds most pragmatic way for me: On every change to your class file, execute the Test Cases that are one-block away from your class.