FEST’s JavaFX Maven plugin 1.0a1

Posted by Alex Ruiz in JavaFX, Open Source on February 1, 2010

I’m proud to announce the release of FEST-JavaFX-Maven 1.0a1!

FEST-JavaFX-Maven is Maven plugin for compiling JavaFX source files. This is a complete rewrite of the plugin, as a Java Mojo.

The original plugin was written as a wrapper for an Ant script. This approach didn’t work well: I couldn’t find a way to make the Ant script see the dependencies declared in a POM and Ant itself is not a programming language (try to do a simple “if-then-else.”) In addition, it seems to me that writing Maven plugins with Ant is discouraged, since the related documentation has been removed from the Maven official online book.

Features

  • Compiles JavaFX sources (currently supports desktop profile only)
  • Does not require JavaFX distribution to be in a Maven repository (according to its license, only Sun/Oracle can distribute JavaFX)
  • Does not require to list all JavaFX libraries as dependencies in your pom.xml

Example

<build>
  <plugins>
    <plugin>
      <groupId>org.easytesting</groupId>
      <artifactId>fest-javafx-maven</artifactId>
      <version>1.0a1</version>
      <executions>
        <execution>
          <phase>compile</phase>
          <goals>
            <goal>compile</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

The example above will call the “compile” goal in the standard Maven “compile” phase.

Release Notes

Bug

  • [FEST-288] – loader constraint violation. Thanks to Viktor Hedefalk.
  • [FEST-289] – Compilation fails if folder ‘target/classes’ is not already created. Thanks to Johannes Schneider.
  • [FEST-290] – Dependencies on other Maven projects (particularly other Maven JavaFX projects) didn’t seem to work. Thanks to William Billingsley.
  • [FEST-294] – Allow configuration of JavaFX home in POM, as well as environment variable. Thanks to nicerobot.

The FEST JavaFX Maven plugin can be obtained from our Maven Repository.

Links

Feedback is always appreciated :)

, , ,

3 Comments

A (overlooked?) use case for the Strategy pattern

Posted by Alex Ruiz in API, Opinion on January 28, 2010

Composition over inheritance. It seems that we all agree and understand this valuable principle. I’ve been reading a good amount of code lately from some open source projects (including mine,) and I think we are still missing at least one use case where to apply it.

This is what I’ve seen: class A has method x. Just after method x is done doing its thing and before it exits, it calls a (sometimes empty) protected method y meant to be overridden to handle some result or side effect of x (is there a name for this anti-pattern?)

Here is a concrete (and oversimplified) example, taken from FEST:

/**
 * Understands a <code>{@link SecurityManager}</code> that does not allow an application 
 * under test to terminate the current JVM. Adapted from Abbot's own 
 * {@code SecurityManager}.
 */
public class NoExitSecurityManager extends SecurityManager {
  @Override public final void checkExit(int status) {
    if (!exitInvoked()) return;
    handleExitRequest(status);
    throw new ExitException(concat(
      "Application tried to terminate current JVM with status ", status));
  }
 
  /**
   * Implement this method to do any context-specific cleanup. This hook is provided since 
   * it may not always be possible to catch the <code>{@link ExitException}</code> 
   * explicitly (like when it's caught by someone else, or thrown from the event dispatch 
   * thread).
   * @param status the status the exit status.
   */
  protected void handleExitRequest(int status) {}
}

At first glance this code looks OK. Subclasses can do whatever they want when “exit” is called, and since the method checkExit is final, there is no way that subclasses can accidentally (or intentionally) change the intended behavior of the super class.

There are, IMHO, some issues with this approach:

Let's take a look at a simple (silly?) subclass of NoExitSecurityManager that writes a message to a file when exit is called:

public class WriteToFileNoExitSecurityManager extends NoExitSecurityManager {
  // The class FileWriter only exists for the purposes of this example
  private final FileWriter fileWriter = new FileWriter("${temp}/log.txt");  
 
  @Override protected void handleExitRequest(int status) {
     fileWriter.write(concat("Exit called with status ", status));
  }
}

This subclass is actually doing too much: it is preventing an application from exiting (inherited behavior) and it is also handling the “exit request.”

WriteToFileNoExitSecurityManager is not really a NoExitSecurityManager, but an “exit request handler.” Inheritance in this case is, IMHO, unnecessary: the implemented functionality (not the inherited one) in WriteToFileNoExitSecurityManager is self-contained, and it does not depend on the internal state (or identity) of the superclass. In addition, since the method checkExit is final, there can only be one implementation. On the other hand, there can be different implementations of handleExitRequest, each of them forced to be a subclass of NoExitSecurityManager.

Testing is also more complicated in this scenario. To test NoExitSecurityManager, we would need to subclass it (more unnecessary inheritance) and set some flag in handleExitRequest to verify it is called (we could also create a mock, but at the end it’s all the same):

public class NoExitSecurityManagerTest {
  private TestNoExitSecurityManager securityManager;
 
  @Before public void setUp() {
    securityManager = new TestNoExitSecurityManager();
  }
 
  @Test public void should_prevent_application_from_exiting_and_handle_exit_request() {
    // test that an application trying to exit is effectively stopped
    ...
    // here we verify that the call to 'handleExitRequest' was made
    assertThat(securityManager.handleExitRequestCalled).isTrue();
  }
 
  private static class TestNoExitSecurityManager extends NoExitSecurityManager {
    boolean handleExitRequestCalled;   
 
    @Override protected void handleExitRequest(int status) {
      handleExitRequestCalled = true;
    }
  }
}

Enter the Strategy pattern

We can clean up our example by using the strategy pattern.

First, we introduce the interface ExitRequestHandler. It only has one responsibility: to handle “exit requests.”

public interface ExitRequestHandler {
  void handleExitRequest(int status);
}

Then, we can replace the call to the method handleExitRequest with a call to an instance of ExitRequestHandler:

/**
 * Understands a <code>{@link SecurityManager}</code> that does not allow an application 
 * under test to terminate the current JVM. Adapted from Abbot's own 
 * {@code SecurityManager}.
 */
public final class NoExitSecurityManager extends SecurityManager {
  private final ExitRequestHandler exitRequestHandler;
 
  public NoExitSecurityManager(ExitRequestHandler exitRequestHandler) {
    this.exitRequestHandler = exitRequestHandler;
  }
 
  @Override public void checkExit(int status) {
    if (!exitInvoked()) return;
    exitRequestHandler.handleExitRequest(status);
    throw new ExitException(concat(
      "Application tried to terminate current JVM with status ", status));
  }
}

Finally, we can rewrite WriteToFileNoExitSecurityManager as a WriteToFileExitRequestHandler:

public class WriteToFileExitRequestHandler implements ExitRequestHandler {
  // The class FileWriter only exists for the purposes of this example
  private final FileWriter fileWriter = new FileWriter("${temp}/log.txt");  
 
  @Override public void handleExitRequest(int status) {
     fileWriter.write(concat("Exit called with status ", status));
  }
}

It looks like I just moved code around and actually created even more code! True, we have more code, but better-quality code:

  • each class has a single, well-defined responsibility
  • there is no tight coupling between the class using the strategy interface and a specific strategy implementation
  • implementing a small interface is easier than extending a class (no need to worry about the state of the superclass in a particular context)
  • testing is simpler

Here is the updated test:

public class NoExitSecurityManagerTest {
  private ExitSecurityManager securityManager;
  private TestExitRequestHandler requestHandler;
 
  @Before public void setUp() {
    requestHandler = new ExitRequestHandler();
    securityManager = new ExitSecurityManager(requestHandler);
  }
 
  @Test public void should_prevent_application_from_exiting_and_handle_exit_request() {
    // test that an application trying to exit is effectively stopped
    ...
    // here we verify that the call to 'handleExitRequest' was made
    assertThat(requestHandler.handleExitRequestCalled).isTrue();
  }
 
  private static class TestExitRequestHandler implements ExitRequestHandler {
    boolean handleExitRequestCalled;   
 
    @Override public void handleExitRequest(int status) {
      handleExitRequestCalled = true;
    }
  }
}

I had a hard time writing this blog post because the points I just made seemed to be too obvious and well-understood by now. But, by reading code (both mine and written by others) I got the impression that although we understand the “composition over inheritance” concept, it is still not easy to apply it in practice.

Am I being too picky? I hope not (although I admit this issue has been annoying me for some time, I finally had the chance to complain.) IHMO, every detail counts when writing clean code :)

Feedback is always welcome!

Update: This post has been published at Javalobby.

(Image taken from etringita’s flickr stream under the creative commons license)

, ,

5 Comments

FEST-Swing 1.2a4: GUI Testing Made Easy

Posted by Alex Ruiz in Java, Open Source, Swing, Testing on January 19, 2010

I’m proud to announce the release of FEST-Swing 1.2a4!

FEST-Swing is a Java library that provides a fluent interface for functional Swing GUI testing. This library provides an easy-to-use API that makes creation and maintenance of GUI tests easy.

This is the fourth and last alpha release planned for version 1.2. This new release includes a good number of bug fixes, new features and improvements. After this release, we’ll have a beta one that will include bug fixes and low-risk improvements only, followed by a release candidate.

Release notes

Bug

  • [FEST-98] – Finding Components is unreliable on Linux. Thanks to Mihael Vrbanec, Julia Koep, Martin Ginkel, Martin Bachmann.
  • [FEST-175] – German umlauts not working since upgrade from 1.1->1.2a2. Thanks to Jan Zuchhold.
  • [FEST-207] – FailOnThreadViolationRepaintManager does not work on Java 5. Thanks to Daniel Dyer.
  • [FEST-210] – JSplitPaneDriver does not respect the limitations of the BasicSplitPaneDivider.DragController. Thanks to Martin Ginkel.
  • [FEST-215] – FEST-Swing does not compile, since project configuration claims Java 1.5 compatibility, but source uses classes/methods from Java 1.6 runtime.
  • [FEST-219] – invokeAndWait() in JApplet.init() causes Exception. Thanks to Dominik Wild, Kyle Murphy.
  • [FEST-227] – JTableFixture.cell(String value) expects a regex. Thanks to Uli Schrempp.
  • [FEST-231] – JScrollPane : up & down inverted. Thanks to Florian SIMON.
  • [FEST-232] – Problems with French keyboard. Thanks to Florian SIMON.
  • [FEST-246] – JTreeDriver: drop() fails when root tree not visible. Thanks to Pavol Marko.
  • [FEST-250] – optionPane() fails to find JOptionPane with null parent. Thanks to Stephen Gade Esven, Morten Breum Mikkelsen.
  • [FEST-254] – JComboBoxFixture.requireSelection(String) method with an editable JComboBox does not use cell reader. Thanks to Glen Schrader.
  • [FEST-262] – fest-swing 1.2a3 swinghelper-debug dependency incorrect in pom.xml. Thanks to Luke Nezda.
  • [FEST-274] – ‘requireSelection‘ in JComboBoxDriver should return value of ‘getSelectedItem‘ if selection index is -1 in editable JComboBox.
  • [FEST-280] – JTextComponentFixture obtains text of JTextComponent outside EDT.
  • [FEST-285] – Typo in error message. Thanks to Steve Farrell.

Improvement

  • [FEST-123] – ItemGroupFixture: separate notion of select/click. Thanks to Jeanette Winzenburg.
  • [FEST-222] – JProgressBar has no Fixture. Thanks to Jacob Qvortrup.
  • [FEST-244] – JListFixture needs a requireRowCount method. Thanks to Harry L.
  • [FEST-248] – Add rightClickPath to JTreeFixture. Thanks to Huib.
  • [FEST-251] – Update FEST-Assert dependency to version 1.2.
  • [FEST-267] – JOptionPaneFixture missing many of ContainerFixture’s methods. Thanks to Morten Breum Mikkelsen.
  • [FEST-269] – add "requireSelection" in JTableFixture. Thanks to Florian SIMON.

New Feature

  • [FEST-8] – Method on JTreeFixture to return a fixture for a cell (issue 185.)
  • [FEST-43] – Add selectedRow() to TableCell and TableCellByColumnName (issue 226.) Thanks to Ken Geis.
  • [FEST-213] – Add method ‘clickRow(int)‘ to JTreeFixture.
  • [FEST-225] – Referencing a JTableFixture row by its cell contents. Thanks to Morten Breum Mikkelsen.
  • [FEST-249] – fest allows system.exit calls that kill running multiple tests. Thanks to Brandon, Jean-Francois Poilpret.
  • [FEST-266] – Add method "rightClickRow(int)" to JTreeFixture.
  • [FEST-283] – Provide a way to install custom AWT Exception Handlers.
  • [FEST-284] – Call scrollToVisible when clicking on a button. Thanks to Steve Farrell.

FEST-Swing can be downloaded here (file fest-swing-1.2a4.zip.) FEST requires Java SE 5.0 or later.

Links

Feedback is always appreciated :)

, , , ,

1 Comment

Dean Iverson joins the FEST team!

Posted by Alex Ruiz in JavaFX, Open Source, Testing on January 18, 2010



I’m so happy to announce that Dean Iverson, member of the open source project JFXtras and co-author of “Pro JavaFX Platform,” has joined the FEST team!

Welcome Dean!

(Image taken from Bob Xu’s flickr stream under the creative commons license)

, ,

No Comments

Thoughts about "UI Test Automation Tools are Snake Oil"

Posted by Alex Ruiz in Open Source, Opinion, Testing on January 18, 2010

I just finished reading "UI Test Automation Tools are Snake Oil" by Michael Feathers. Although I agree with many of the ideas in the article, I also think it contains some hasty generalizations and misplaced blame.

Mr. Feathers points out that “selling UI test automation tools is irresponsible” and these tools are sold “with a dream, a very seductive and dangerous one.” The scenario that he refers to, to my understanding, is the one where tool vendors sell their (expensive) tools promising:

  • “click and type,” quick test generation with a record/playback tool, no coding required
  • test the whole application, including all different use case and data permutations, using this tool

This can be easily interpreted as “with my tool, you will have a comprehensive test suite in no time, and you can hire cheap monkeys to do all the testing.” It is indeed a very seductive promise: spend some money and get what you want fast and with minimal effort, just like “weight loss, without diet and exercise.”

I completely agree with Mr. Feathers that selling something based on a fallacy is irresponsible. At the same time, I also think this is the point where blame is misplaced. Instead of blaming the tool, blame should be shared between tool vendors and the people buying the tool, as long as the business transaction is based on the “seductive dream.” IMHO, this is a people problem: on one hand we have vendors taking advantage of the customers’ lack of knowledge and the customers, that did not do their proper homework on time and by now have a big, untestable mess.

Another good point that Mr. Feathers makes is that UI testing tools are brittle. Selling a UI testing tool promising that the generated test suite will never break is also careless. The fact is, when testing UIs there are many external elements that can introduce false failures. IMHO, it is impossible to fix all of them, but at least, the tool vendor should warn their customers about these limitations and offer ways to overcome them (more on that later.)

I also agree with Mr. Feathers that record/playback doesn’t work. I wish he went into more details to see if we agree based on the same premises. In my opinion, the major weakness of existing record/playback tools is expensive maintenance of the generated tests. Recorded scripts are often long and written in proprietary languages lacking object-oriented features. Modularization/refactoring is hard or impossible, resulting in duplicate test code. The common end result is that changes in the application requires re-recording all tests scenarios, a labor-intensive and error-prone task.

Not everything is black and white. UI test automation tools still have a place. Not every shop has the resources or time to rewrite an application, just to make it more testable. I’ve seen applications with massive code bases, that were not written with testability in mind. Extending or refactoring is scary due to the lack of the safety net that automated tests provide. The only way to test them is through the UI. UI testing can provide the initial safety net, which later on can be enhanced (or replaced) with API-based unit tests.

In his post, Mr. Feathers asks “Developing them open-source? Well, let your conscience be your guide.” I don’t sell a UI test automation tool per se. What I provide is an open source, API-based UI testing library, called FEST (you can find some testimonials here.) I think I’ve been pretty responsible by pointing out that UI testing is fragile and offering ways to overcome this limitation. What I’m missing, and I have to thank Mr. Feathers for it, is a guide on how to test the UI layer in isolation (which I will be doing soon.)

Like I mentioned in a previous post, I’m currently working on my spare time on a playback/recording tool. My intention is to create a tool that generates clean Java code, as if it was hand-crafted, to overcome to what I think are the problems with this technique. We’ll see if I can accomplish this ambitious goal :)

Overall, I liked and enjoyed Mr. Feathers article. It has pretty good points and observations. I wish he backed up his ideas with more details and examples, to sound less emotional and more objective. The problem is in the human factor, not in the tool. Ideally, vendors (in general) should set the right expectations about their products, and customers should have enough knowledge to avoid being taken advantage of. At least, some of us are trying to do the right thing :)

Feedback is always welcome :)

Update: This post has been published at Javalobby.

,

No Comments

Resolutions for 2010

Posted by Alex Ruiz in Personal on January 6, 2010


As an attempt to document my plan for this year, the following are tech-related resolutions for 2010:

  1. Learn one or more programming languages.

    I spent most of 2009 working on FEST. Even though I learned a lot more about Swing, enjoyed writing clean and testable object-oriented code, and implemented many interesting features, all the development was done in Java. I think it’s time to learn a new language!

    My plan is to learn at most two languages. Like I mentioned in a previous post, I think that it takes a considerable amount of time and effort to learn a language well. I have worked in Java for quite some time, and I’d love not only to learn a new syntax, but also new programming concepts/paradigms. My intention is to learn Groovy and Scala.

    I wouldn’t want to learn these languages using simplistic exercises. I’d prefer to learn them by working on a non-trivial, useful project. In fact, my plan is to create alternative APIs for FEST-Swing using both Groovy and Scala.

  2. Refresh and increase my knowledge of algorithms.

    I have enjoyed studying design patterns and have been using them both at my day job and on my open source project. Needless to say, it was been a useful and rewarding experience. For one of my side-projects (and to become a better developer,) I’m going to need a deeper knowledge of algorithms, which are also patterns. I already started studying them. My resolution is to not break the chain.

  3. Finish and release the FEST record/playback tool.

    This tool will record user interactions with a Swing UI and generate FEST-Swing UI tests. The goal is to generate clean code, as if it was hand-written. This is something that I wanted to do since 2008, but I was too busy building the open source API. I started working on this project a few months ago. Progress has been slow but steady. I hope to have a preview in a couple of months.

    Many thanks to my good friends Andy Glover, Kirill Grouchnikov and Andres Almiray for their valuable feedback and advice :)

  4. Figure out a way to create a testing library for JavaFX.

    I’ve been working on a robot-based library for functional testing of JavaFX UIs, similar to FEST-Swing. At this moment I’m stuck. FEST-JavaFX needs to work with compiled JavaFX code, which is different to what we see in a JavaFX script. I have to deal with hierarchies of SGNodes, an implementation detail that can change at any time (most likely with Prism.) It seems to make sense for what JavaFX tries to accomplish: to have the same application executed on different device types. I think it’s time to ask my JavaFX-guru friends for help :)

  5. Create a domain-specific language (DSL) for testing UIs.

    The idea here is to create a simple DSL for writing functional UI tests. The target audience are QA engineers that do not know Java. I plan to explore Xtext to accomplish this task.

IMHO, it is an aggressive plan. I hope to be well-organized and focused enough to accomplish most of it :D

Feedback is always welcome! :)

(Image taken from tnarik’s flickr stream under the creative commons license)

, ,

4 Comments

FEST: new JavaFX Maven plugin

Posted by Alex Ruiz in JavaFX, Open Source on December 9, 2009

I’m back to working on FEST-JavaFX, an open source library for functional testing of JavaFX GUIs. To keep consistency with the rest of FEST modules, I decided to use Maven to compile and build the project.

I blogged some time ago how to compile JavaFX code with Maven. I wasn’t too happy with it, since it was verbose and I’d have to copy/paste the Ant build for every JavaFX project. I found JFrog’s JavaFX Maven plugin and gave it a try. Although configuration was easy, I couldn’t make it work.

The plugin expects the JavaFX jars to be installed in a local Maven repository. This is something that I don’t agree with, for the following reasons:

  • I don’t see the point of having JavaFX jars in a Maven repository, since those jars cannot be distributed (per JavaFX’s license)
  • Installing those jars in a local repository is easy but time-consuming: it needs to be done one by one
  • So far, with each release, the number and some of the names of JavaFX jars change; I wouldn’t want to go through the process of installing them in a local repository every time there is a JavaFX release

Since Maven already supports creating plugins based on Ant, I just had to wrap my existing Ant script with a Maven plugin.

Creating the plugin was really easy, it took no more than 10 minutes. Maven’s documentation and online book are pretty good and they helped me a lot.

The main difference between FEST’s JavaFX Maven plugin and JFrog’s is that FEST’s does not require the JavaFX jars to be installed in a local repository. Instead, it uses an environment variable, JAVAFX_HOME, to locate the path of the JavaFX jars.

I just released version 0.1. Since it is a Maven plugin, I don’t think it is necessary to add a distribution file to the “downloads” page. The plugin can be obtained from our Maven repository.

The following example shows how to use the plugin to compile JavaFX source when the standard “compile” goal is executed:

<build>
  <plugins>
    <plugin>
      <groupId>org.easytesting</groupId>
      <artifactId>fest-javafx-maven</artifactId>
      <version>0.1</version>
      <executions>
        <execution>
          <phase>compile</phase>
          <goals>
            <goal>desktopCompile</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Please note that it is not necessary to specify the JavaFX jars as project dependencies (at least I haven’t had to do it, so far.)

Currently the plugin has only one goal: “desktopCompile”, to compile JavaFX sources under the desktop profile. For more information about configuring the plugin, please visit the plugin’s homepage.

I originally created version 0.1 to satisfy the needs of the FEST-JavaFX project. There is a lot of room for improvement.

Please give the plugin a try. Feedback is always welcome! :)

As an additional note, the Silicon Valley JavaFX user’s group (SvJugFx) will be holding its first meeting, with a live streaming presentation from Richard Bair. The SvJugFx is organized by my good friend and FEST teammate Stephen Chin. See you there!

, , , ,

6 Comments

FEST-Reflect 1.2: Fluent Interface for Java Reflection

Posted by Alex Ruiz in Java, Open Source, Testing on November 30, 2009

We are proud to announce that FEST-Reflect 1.2 is out!

FEST-Reflect is a Java library that provides a fluent interface that simplifies the usage of Java Reflection and JavaBeans Introspection, resulting in improved readability and type safety. It supports class loading, access to static inner classes constructors, methods and fields, and more!

Example:

Person person = constructor().withParameterTypes(String.class)
                             .in(Person.class)
                             .newInstance("Yoda");
 
method("setName").withParameterTypes(String.class)
                 .in(person)
                 .invoke("Luke");
 
field("name").ofType(String.class)
             .in(person)
             .set("Anakin");
 
List<String> powers = field("powers").ofType(new TypeRef<List<String>>() {})
                                     .in(jedi)
                                     .get();

Release notes

Bug

  • [FEST-68] – FEST-Reflect is catching my RuntimeException while it should not! (issue 310.) Thanks to Francis.ANDRE.
  • [FEST-257] – FEST-Reflect should throw exception if static inner class is not found.

Improvement

  • [FEST-256] – Replace constructor calls with fail-fast static factory methods.

New Feature

  • [FEST-235] – Add API to reflect bean properties. Thanks to Jean-Francois Poilpret.

    Example:

    // equivalent to calling "getName()"
String name = property("name").ofType(String.class)
                              .in(person)
                              .get();

FEST-Swing can be downloaded here (file fest-reflect-1.2.zip.) FEST requires Java SE 5.0 or later.

Links

Feedback is always appreciated :)

, , , ,

2 Comments

FEST-Assert 1.2: Fluent Interface for Assertions

Posted by Alex Ruiz in Java, Open Source, Testing on November 2, 2009

We are proud to announce that FEST-Assert 1.2 is out!

FEST-Assert is an “assertThat” library that provides a fluent interface for writing assertions. Its main goal is to improve test code readability and make maintenance of tests easier.

Example:

int removed = employees.removeFired();
assertThat(removed).isZero();
 
List newEmployees = employees.hired(TODAY);
assertThat(newEmployees).hasSize(6)
                        .contains(frodo, sam);
 
String[] newHires = employees.newHiresNames();
assertThat(newHires).containsOnly("Gandalf", "Arwen", "Gimli"); 
 
assertThat(yoda).isInstanceOf(Jedi.class)
                .isEqualTo(foundJedi)
                .isNotEqualTo(foundSith);

Release notes

Bug

  • [FEST-198] – Fix ImageAssert.read to compile successfully against Java 1.5
  • [FEST-218] – Inconsistency in parameter names in assertions
  • [FEST-228] – Error message when comparing Strings is not consistent between JUnit and TestNG
  • [FEST-229] – Descriptions are not evaluated lazily when an assertion fails
  • [FEST-230] – Validation of arguments not throwing correct Exceptions.

Improvement

  • [FEST-147] – Allow reuse of formatting helpers in class Formatting by making them public protected
  • [FEST-157] – Add isNotEqualByComparingTo() for ease of use in BigDecimalAssert
  • [FEST-165] – Make Assertions non-final and make its constructor protected
  • [FEST-166] – Make assertion classes non-final
  • [FEST-168] – Add methods "is" and "isNot" to all assertions
  • [FEST-242] – Nicer debugging output with JUnit’s ComparisonFailure
  • [FEST-245] – Deprecate ‘Delta’ class in FloatAssert and replace it with top-level Delta class

New Feature

Task

For more details about new, updated or removed classes, please read the change report.

You can download the latest release here (file fest-assert-1.2.zip.) FEST-Assert requires Java SE 5.0 or later.

Here are some useful links:

Feedback is always appreciated :)

, , , ,

No Comments

Giving Infinitest and “Continuous Testing” a try…

Posted by Alex Ruiz in Java, Opinion, Testing on October 6, 2009

It has been a week since I started using Infinitest while working on FEST-Assert. I’m very pleased with the results, to the point that I’m getting addicted to it!

Infinitest is an interesting tool that brings Continuous Testing to your IDE (Eclipse or IDEA.)

From its website:

Whenever you make a change, Infinitest runs tests for you. It selects tests intelligently, and only runs the ones you need. It reports unit test failures like compiler errors, and provides additional information that helps you write better tests.

I was first introduced to Infinitest by my good friend Rod Coffin, and I was impressed with it the very first time I saw it!

Benefits

Infinitest provides the fastest feedback possible. It will notify us right away if any changes to the code base break existing tests. This is done without running the whole test suite, just the tests covering the affected code. That is IMHO, priceless, for the following reasons (in no particular order.)

  • Reduced risk of introducing defects: tests are executed immediately after any code changes
  • Less broken builds: there is a good chance Infinitest will catch any broken tests before checking-in
  • Less manual work: Infinitest runs tests automatically
  • (Potentially) less coupled code: if too many tests are running while performing small changes in code, it may be a sign of code that is too tightly coupled

Overall, I’m working faster, with more confidence and having more fun :)

Prerequisites?

Infinitest has been working really well for me. I think this is directly related to the nature of the project I’m working on:

  1. It is a small project: only 48 classes and 2,604 NCSS (Non Commenting Source Statements)
  2. It has a comprehensive test suite: 1,718 tests with 99.9% code coverage
  3. It has unit tests only, no integration or functional tests
  4. Tests run fast

As you can see, given this setup, Infinitest runs my tests in the background pretty quickly. I still don’t know if this will be case when using it in a bigger project, with more functional tests.

A more “real” (and hopefully fair) test

After releasing FEST-Assert (hopefully in November,) I’m going to give Infinitest a try with FEST-Swing.

FEST-Swing is a bigger, more complex project:

  1. It has 389 classes, and 14,383 NCSS
  2. More tests: 3,405
  3. Tests are also slower: most of them are functional tests that interact with a Swing UI

Probably FEST-Swing is not a good candidate for continuous testing, since its tests are slower than unit tests. Nothing wrong with it, it is just the nature of the project. I can expect to see some Swing UIs popping up in front of my IDE, taking over the mouse and keyboard, after making some changes in code. I’m very curious to see how this kind of interruption affects my coding “flow.”

Even if it doesn’t work as I expected, I’m pretty sure this exercise can at least help me reorganize the code base, as a way to minimize the number of functional tests executed.

I’ll be posting my experiences with this scenario as well :)

What Infinitest can’t do

I have found one thing Infinitest doesn’t cover: code that uses reflection. I accidentally broke some tests that use reflection to do some funky (but necessary) stuff. To be honest, asking Infinitest to cover code that is using reflection is asking the impossible.

The lesson I learned is:

I still need to run the whole test suite before checking-in

Some rough edges

Even though I’m a happy Infinitest user, there are things that, in my opinion, can be improved. I’ve been using Eclipse 3.5.1 and Infinitest 5.1.61, and I have a couple of suggestions.

1. Fix: Infinitest executes unrelated tests
When deciding which tests to run, Infinitest is smart, but not smart enough (IMHO.) In my case, I made a small change to the method method read(String) in the class ImageAssert. The only reference to the affected method is the test class ImageAssert_read_Test, which has only two methods. Instead of running ImageAssert_read_Test by itself, Infinitest ran all the tests that have a reference to ImageAssert. Instead of executing only 2 test methods, Infinitest executed 13!

2. Fix: sometimes tests in other projects within a workspace are executed
I haven’t been able to reproduce this issue consistently. For some strange reason, Infinitest tries to run approximately 1,300 tests from the FEST-Swing project, even though there are no dependencies between these two projects. This is particularly painful, since most of these tests pop up a Swing UI and take over the mouse. My only option is to kill all Java processes.

3. Fix: remove “quick fix” for failed test
Infinitest reports any broken tests in the “Problems” view in Eclipse, which is fine. The issue here is that the “quick fix” action for a test failure is to “print stacktrace,” which I find pretty much useless. Generally speaking, I don’t think there is a “quick fix” for a test failure. I’d prefer to have this action removed (I’m not sure if it is technically doable though.)

4. Fix: sometimes, changes in tests do not trigger Infinitest
Once again, I haven’t found a consistent way to reproduce this issue.

5. Improvement: display executed tests in its own view
Currently, Infinitest displays executed tests in a tooltip:

I’d like to see the executed tests in a view similar to the “Problems” view: a table where I can select a test, go its source or just execute it again.

6. Improvement: ability to stop Infinitest once it started running tests
This is related with issue #2. I’d like to have ability to stop Infinitest when it is running tests that are either not related to the change I just made or are simply intrusive.

7. Improvement: change icon for test failures
Infinitest shows test failures using the same icon Eclipse uses for compilation errors:

Although it is really no big deal, it can be confusing sometimes. I’d prefer to see a different icon for test failures.

8. Improvement: integrate with Eclipse’s JUnit plug-in
This is related to issue #4. If Infinitest is not triggered by changes I made to a broken test, I’d like to see the broken test removed from the “Problems” view when such test passes when I manually execute it through the JUnit plug-in.

9. Improvement: documentation
Infinitest is so easy to install and use, that there is really no need for documentation. What I’d like to see is documentation about strategies to reorganize both production code and test code, to minimize both the number of tests executed and the time tests are executed in the background.

10. Wish: a graph showing dependencies between code and executed tests
IMHO, it would be nice if Infinitest provided some graphical representation of the relationship between changes being made and the tests being executed, to have a better understanding of these dependencies and help me reorganize my code.

Conclusion

I’m very satisfied with Infinitest. It is an excellent tool that notifies me instantly if any changes I make break existing functionality. In my case, the main benefits are increased productivity and confidence. It is already in my “must have” tool arsenal. Even though I’ve been using it successfully in a small project with a comprehensive test suite, I still have to see how Infinitest performs in a bigger, more complex project that contains a good number of functional tests. Stay tuned!

Update: this post has been published at Javalobby.

, ,

6 Comments