Thursday, January 30, 2014

A tester workbench, an exploratory testing tool

Murphy can do many things but the starting point for it is to know your application, to do so it will try to visit all the dialogs the application has by looking at the screen, click anything it can, enter values and so.
This is called model extraction, it creates an internal representation of the application and the transition between states / dialogs.

An example of a simple application would be:


Most of it is done automagically, you only need to specify the application you want to explore. It cannot guess many things like specific values (think installation keys and so) or specific sequences of actions, however it is fully customizable and those cases can be handled easily by tweaking the extraction script for that application.

The above example was created with 2 lines of code (slightly styled for display purposes):

extractor = base_extractor.BaseExtractor('7zipWinScraper', '7z920.exe')
extractor.add_boundary_node('Browse For Folder')

The very first thing we can do with it is to interactively use that graph, just click what you want to do and it will do it for you.
Let's suppose for example you want to test what happens if the application tries to install in a folder where it does not have access right, to do so, just do this:



Type c:\readonly_folder in the text field (image at right) and press the run button, that's it.
Murphy will do the rest for you, it will create a virtual machine, launch the application type the value and press the install button.

Obviously something bad is going to happen, to see what we just open a vnc or remote desktop connection to that machine and see the fireworks, in this case we got:

Also murphy will tell us that something went wrong:

MurphyConfused: Cannot recognize view Completing The 7_Zip 9_20 Setup Wizard
2014-01-30 21:40:13,410 - root.web_workbench.planner - INFO - Run finished.

This example is rather simple but enough to give us an idea of the power of the tool, we clicked 2 steps but there's no limit to the complexity.

It also comes with a "GPS" included: you don't need to click consecutive steps, in the above example, you could for example do only 1 click in the close button of the installation finished dialog, that will create a virtual machine with a default installation of the 7zip application (yes, there are limits, but they can also be overcome with a bit of code).

But for the same price, we get a nice extra feature: cloud. In the company I work for, we use a cloud for testing (we use https://github.com/F-Secure/dvmps), you're not limited to 1 run at the time, you could ask for 10 machines in 10 different states at the same time and it will all happen at the same time saving a lot of time in many cases, you can read the news while murphy does a good chunk of the tedious work.

Last but not least, you can proofread the dialogs in thumbnails and screens mode (as opposed to a statechart), you can also export the model as a self contained file, quite handy for things like translators so they can have contextual information of the texts (which helps to avoid typical transliteration mistakes)

This is just the tip of the capabilities of murphy, there are many others to mention in future posts :)

-Mat

No comments:

Post a Comment