Wednesday, February 5, 2014

Testing in the cloud and writting murphy scripts

Murphy was designed to be able to run tests in a remote machine mainly, while it is still possible to run the tests locally there are many advantages to do it remotely.
Obvious advantages are that you can run as many parallel tests as machines you have access to, and the costs nowadays are dirty cheap (I remember something like 70 usd for an hour of 10.000 cpu's in Amazon or something along those lines)
Even when murphy extracts the model of an application, it does so by running the application to analyze in a remote machine, don't worry, you don't need a cloud to use it if you dont have, a virtual machine using VirtualBox, VmWare or KVM will do just fine.
If you're not familiar with it, there's a lot of magic you can do, like saving the state of the machine and restore it later and other very handy things.

Enough of the cloud, the idea should be somewhat clear already, let's go to the scripts.

While the main focus is to try to test as much as possible without writing test scripts it is still needed in some cases (hopefully less and less as murphy evolves), let's explain it over some sample model:


So how do we write a script over that? pretty simple, what we're looking at is the flow with images, if instead we look at the text view what we see is:


Test scripts are written in python so there's no limit to what you can and cannot do, you simply load a model and call methods. Let's suppose we want to write a test that launches the application and in the main dialog cancels the installation, that script is written as:

from murphy.tester import run_remote_test

def test_install_cancelled(worker):
    worker.In("Node 0").GoTo("Were Sorry You Did Not Installed Us")

if __name__ == '__main__':
    run_remote_test(test_install_cancelled, "abc/my_model.json")

That's it, that will assert everything, the test will fail if anything does not match with what you see in the flow.
Under the hood it has a GPS and will find the route from Node 0 to the installation cancelled dialog for you and execute all the needed intermediate steps, of course you can be verbose and explicitly state step by step, for example

worker.In("Node 0").Do("Launch application").Then("Cancel")

Trying out several invalid installation keys is a simple iterator with the values to test, the concept is still the same.
(sorry about the typo in the graph, I need to fix the example in the source code)

The names you see in the nodes are a best guess from the core extraction routines, but as everything is customization you can rename then to your own liking (specially if they're annoyingly long)

Those are the basics, it can get as complicated as the test cases you want to write.

As for automatic generation of test cases, it is a desired feature but absent at the moment, the idea is to export the model into another open source tool that takes care of that.

Ok then, I gotta think what the next post will be about.

-Mat

No comments:

Post a Comment