/customers/iconara.net/iconara.net/httpd.www/blog/wp-content/plugins/wp-super-cache/wp-cache-phase1.php Iconara » Automated testing with FlexUnit (a proposal)

Automated testing with FlexUnit (a proposal)

FlexUnit works quite well for testing ActionScript code, but one thing that has been a problem is how to automate the testing. Currently you have to run the tests manually, or go to such lengths as Peter Martin (whose solution is clever, but complex). In this post I outline a possible solution using Apollo and Ant.

I don’t have any possibility to test these ideas it right now, but I thought I should outline the idea here and perhaps someone else will beat me to it.

What I want is to be able to add a target in my Ant build script which will run my tests and fail on error much like it’s possible to do with JUnit and unit testing in Xcode.

My idea is to wrap the following as an Ant task (using macrodef)

  1. compile an Apollo application which will run the tests
  2. run the Apollo application with exec
  3. the results of the tests are written to a file
  4. when the file is created, it is picked up by a waitfor task which runs in parallel with the exec
  5. the results are printed as build messages
  6. depending on the file contents the build fails or continues

Some comments on the steps:

  1. it would be even better if you could compile your tasks as a runtime shared library which a pre-compiled Apollo app could load.

  2. as long as adl isn’t available as an Ant task exec will have to do, but you can wrap it up in a macrodef, which will be almost the same.

  3. the Apollo app doesn’t need to have a UI, so it would be quite unobtrusive.

3-5. Peter Marins implementation uses a socket connection between the test runner and Ant script, which is a bit over the top for my taste, and the parallel and waitfor tasks exist mainly to support this kind of thing. However, I’m not sure about the format of the output file, it has to be something Ant can work with to determine sucess or failure, and to print it along with the build messages.

  1. One idea is to use one file for failures and one for sucesses, since we can have two parallel waitfor, but perhaps it’s not needed, I’m not sure at this point. It would be good if an Apollo application could exit with a non-zero value, but it doesn’t look like it’s possible to me.

Anyway, I hope to be able to look into this in a few days. Hopefully I can wrap it all up in a nice task using macrodef with minimal external dependencies – but it will have to include a SWC containing the test runner and Apollo application at least.

Suggestions are appreciated.

9 Responses to “Automated testing with FlexUnit (a proposal)”

  1. Sam Robbins Says:

    The source for Peter’s solution is on his site why not just modify that to fit your needs? With some simple changes you could “fail on error much like it’s possible to do with JUnit and unit testing in Xcode.”

  2. Theo Says:

    Well, “fail on error” is not the problem with Peter’s solution. It requires a custom Ant task that starts a socket server and that the application is started in a browser or Flash player. This works, but it seems more straightforward to exec and wait for the results and to be able to do this with only core Ant tasks.

    Perhaps some of Peter’s code could be used for these purposes too, but the problems it solves are the ones I want to do away with =)

  3. zwetan Says:

    I thought of another way (without Ant)

    make an ApolloTestRunner that can: - scan a path to collect *.as class - compile those class using mxmlc - dynamically load this external file into the TestRunner

    if you look at how to run Java unit tests without Ant, you don’t need anything else than Java itself, for AS3 it should be the same imho.

    When this is implemented, then you can hook Ant or any other process.

    take a simple use case:

    “I’m a user, I want to run tests while writing my code”

    • any IDE than can run an external executable tool
    • being able to pass as an argument to that exe the current class we want to test

    then the classic - write tests - run (it fails) - write code and test till it does not fail

    ..repeat

    just a basic stdOut back to the IDE should be enought to provide feedback.

    Now, to make it automated in Ant, the perspective change, we already have an exe that take care of compiling the class and run the test on the class.

    then taking this way, have the ApolloTestRunner exe feedback to Ant the JUnit XML report and use the JUnit task.

    I plan to take this way with ASTUce, but I’m waiting an Apollo beta than can run exe :).

  4. Theo Says:

    Sounds like a good idea. Do we know that Apollo will have exec capabilities? I would really like to see that, together with printing to stdout and exiting with a status other than zero it would make Apollo applications just like any other. The alpha is alpha an a bit crippled in that department. (On the other hand, if this is the alpha, I can’t wait for the final.)

  5. darron Says:

    I’m currently using a solution that combines the Stand Alone flash debug player, a python script, and ANT.

    I launch the debug player via ANT, which writes the output of units tests to the flashlog.txt file. The python script picks that apart and creates an .xml file that can be imported via CruiseControl. ANT picks up a success/failure flag from the python script, and the ANT build fails if the python script reports that the test failed. Otherwise, the build succeeds, and CruiseControl incorporates the test results on the build result page.

    It’s got a few moving parts, but I like it better than the socket server solution. If you’re looking to implement something yourself, have a look at Paul’s CI category: http://www.eyefodder.com/blog/continuous_integration/

  6. Theo Says:

    Seems like a good solution.

  7. darryl west Says:

    Apollo might give you some client capabilities, but FlexBuilder is written in java. ant is written in java. Using java, you can create an eclipse (FlexBuilder) plugin that would give you what you need.

    Also, a superior model for industrial strength test frameworks comes from the ruby/rails world. whenever you create a new class, the platform creates (code generates) unit and functional tests, test fixtures to create test data, and connections to the test database.

    with test data in place, unit testing is just that–no dependency on the backend server. and functional testing that makes round trips to the server, use data in a specially configured test database instance.

  8. Theo Says:

    Darryl: If you can show me how to run ActionScript code in Java you have a point, but since FlashPlayer is the only thing that runs ActionScript code your comment is meaningless. Yes there are better ways of doing what I want to do, but there are not better ways of doing it with the tools available.

  9. darryl west Says:

    yes, after re thinking through the issues, apollo/AIR seems like the likely runner. my bad.

Leave a Reply