Functional testing AIR applications with FlexMonkey, Part 1

FlexMonkey is probably the best tool going for functional testing on Flex and AIR applications.  It’s free (that’s a good start), comes with an AIR-based test authoring tool, integrates nicely with both FlexUnit and Fluint unit-test frameworks, and all the source code is available for you to try.

I recently set up a functional testing framework for my company’s AIR application, including integrating with our automated continuous build and integration system.  It turned out to be a bigger project than I had anticipated, and so I’d like to share a bit of lessons learned.

This discussion will be presented in five parts:

• Part 1 (this article) concerns the overall process of how you run your AIR application with FlexMonkey, and prerequisites for getting started.
• Part 2 covers the task of building a custom version of your AIR application compatible with the FlexMonkey IDE.
• Part 3 presents tips for authoring tests with the FlexMonkey IDE.
• Part 4 covers the task of integrating FlexMonkey tests into your continuous build system using Apache Ant.
• Part 5 (coming soon) provides an alternative to Ant scripting, with a Ruby-based test automation server you can integrate into your build.

Please note: this series focuses specifically on testing Flex-based AIR applications, as opposed to applications built with the Flex framework that run inside a web browser. While FlexMonkey is equally capable of handling in-browser applications as well as desktop applications, the configuration is different (see next section) and so in-browser applications are outside the scope of this discussion.

That said, part 3 should be useful reading for anyone working with FlexMonkey.

Three types of FlexMonkey automation

Part of the confusion of working with FlexMonkey comes from the fact that it tries to cover many use case scenarios for testing, and that you can run your application in FlexMonkey in three different ways.

Though the runtime configuration is probably the most important decision you need to make when deploying FlexMonkey, the documentation doesn’t help much. It does a decent job of laying out the bare facts between the configurations, but doesn’t separate out the use cases, give advice to help you decide, and finally focus on the configuration you’ve chosen once you have decided. In fact, the topic isn’t even broached until halfway into the FlexMonkey User Guide.

Confusing matters even more is that the three methods of running your application are driven by different components of FlexMonkey, all of which have confusing names containing some permutation of the terms “Easy”, “Monkey”, “Agent”, “Link”, and “Library”.  Finding the bits of FlexMonkey you need to accomplish the task at hand from all the irrelevant bits is like solving a 70-Picarat Professor Layton puzzle.

Briefly, the three testing configurations FlexMonkey supports are as follows, along with the names the documentation uses to refer to them:

1. “Target SWF Window“: Loads your unmodified application SWF file directly into the FlexMonkey IDE. You run your test suite, and your Flex application, completely within the FlexMonkey IDE.
2. “MonkeyAgent“: A special web page called MonkeyAgent.html loads your unmodified application SWF file in a web browser.  You still run your test suite in the IDE. The IDE controls the tests, while the application runs remotely in the browser.
3. “MonkeyLink“.  You build a special version of your application that includes both the FlexMonkey runtime, as well as the test suite classes generated by the FlexMonkey IDE. FlexMonkey and the tests run entirely within your application.  The tests are initiated and controlled by an external control program driven by an Ant task.  In this scenario, the FlexMonkey IDE is not used at all to run tests, but only to author them.

While you can debate the relative merits of these choices for Flex applications, really the only viable choice for AIR applications is (3).

You can try option (1) for an AIR application, but trust me, you will be in for a world of hurt.  The FlexMonkey documentation does a good job of listing the things that will break if you try to use option (1) to run your AIR application, which is a large part of what makes it untenable.

I did start with option (1), but quickly ran into an insurmountable roadblock. The path that AIR uses as the base for locating assets is different when you run your AIR application inside the FlexMonkey IDE. This caused my application to choke and die when it could not find the skins, sounds and images it needed to function properly. I quickly concluded option (1) is not a real choice for AIR applications.

Option (2) is intended for Flex apps running inside a web browser. If your application does not use any AIR-specific API features, you may be able to repackage your application to run in-browser and use option (2).  But many AIR applications don’t meet this criteria, such as mine which prominently uses the AIR-specific HTMLLoader and File APIs.

Finally, option (3) is the only available choice if you want to automate testing as part of your builds – and who wouldn’t want to do that?

Before You Start

The MonkeyLink option for testing AIR does have one big downside: you need to have the Flex Automation Libraries from Adobe.  At this writing, they are only available bundled with Flex Builder Professional, which costs $700 from Adobe.  (Hint: Google around for discount codes). On the other hand, if you were working with the raw Flex/AIR SDK and editing Flex code in TextMate, as I was, then you also get a nice IDE as part of the deal.

You should also have some familiarity with the AIR SDK tools for building and running applications from the command line, specifically the amxmlc and adl tools. Adobe has documentation on how to use them.

You should also make a point of checking out the FlexMonkey source code from the FlexMonkey SVN project repository on the Google Code website.  You will more than likely need it in the following articles as we proceed further in this walkthrough of producing a testable AIR application and integrating it into your build.

Last but not least, if you’d like a tutorial introduction to FlexMonkey, Gorilla Logic – the makers of FlexMonkey – have you covered.

In the next article in this series, we will build an AIR application with MonkeyLink and use it with the FlexMonkey IDE.