Friday, March 21, 2014

Freshly baked version and some hard stuff

Yez, the github code was updated today with latest and greatest and hopefully not too broken new version, I still have in my todo list the versioning, I promise next time I commit code I'll do it :)

In any case, let's move forward with some really hardcore stuff, it is probable quite safe for you to not to read beyond this point as i'm going to go very technical, but I MUST write this at least for myself.

One of the most important parts of murphy is the ability to crawl the application and make a map or a model out of it, I had split the problem into 4 stages that go from quite simple and most useful with a notable increase in complexity in each stage.

Before going into the stages we need to first need some basic concepts, to begin with the model is a directed graph, each node represents a state while each edge the transition into another state, that's trivial but we're doing a graph of the UI of an application.
In this case a node represents then the state of the UI, in a windowing environment that usually means a window or dialog, edges will then represent what I can do with that dialog (pressing buttons, enter text, tick checkboxes, etc)
Things are not THAT simple, let's see a simple dialog:


If we take literally the image as a state, then each combination of pressed / unpressed check boxes, radios and so will explode the number of different states and images to store, to reduce that the state is parametrized and so the image too, we can then recognize from a screenshot comparing against our parametrized image, a parametrized image looks like this:


It is then a matter of finding an image inside a screenshot of the whole screen, the red areas are ignored during the comparison (as their content can change)
Without going deeper into how and why (or i'll spend the whole night explaining) that is more or less the basics of recognizing the different states.

As we are interested in doing a human readable statechart or graph of the flow this is good enough for most cases. Let's move on to the crawler and the different stages then.


Stage 1: Big splash

The simplest crawler tries to visit as many dialogs / states as possible with as little work as possible, at this stage we're mostly interested to have a bird view of the application, to do so the crawling uses a very simple process, again some simple flow (in text mode this time):


While this strategy gives us fast results we can see that certain things tends to be confusing, in the center of the graph we can see the 'Cancellation Confirmation' and a mess of arrows around, not even all the paths are valid, if you come from the 'Welcome To SuperApp' state you cannot then go to the 'No__Select Installation Directory', it is an invalid flow.
Separating that state requires a more advanced crawler.
This is the crawler that is at the moment in github.


Stage 2: More details but not exhaustive

As we step up to the challenge we want something nicer and we're willing to spend a bit more of machine time, this crawler can decouple this kind of 'reusable dialogs'.
Internally what it does is to associate a state with a context, if there's some invalid case like the above one (from the welcome, press cancel, then no_select installation directory) it opens up as a separate state, now the graph looks like this:


Now we're talking... all paths are valid (or almost, for most cases), as a convention you can see that in the text the __Blah... is a disambiguation of where does it comes from, this is useful when we write manual scripts (one day I may write about that)
Context means (in the context of this discussion) the path we travel to arrive to such node, more specifically only the predecessor node/edge, as we're still under a manageable complexity we only care about the immediate before node.
(And I am happily and proudly showing you the first results of the soon to come version :))


Stage 3 and 4: when hell freezes

This ones are a real pain, we step up the challenge significantly and now the graph is not closed anymore, at this point we care about all the nodes / edges that affects the possible next step. Wha???
Yes, let me translate that, lets suppose that the 2nd node in the graph has 1 checkbox, it can be on or off, and depending on it the 8 node may vary, detecting the flow dependencies beyond the immediately before node. Context is now the full path we travel!

Heavy stuff, and will come, but the plan is to get there gradually since most of the benefits are collected with simpler crawlers there's no hurry to get to stage 3 and 4.
However, the purpose of the stage 3 and 4 is to allow super cool functionality in the workbench, you will then be able with the mouse to instruct testing / exploration in an extremely powerful way, you could then for example do the following things:

  • Get a quick graph (of stage 2 quality)
  • Select areas of the grap
  • for each area you could specify if you want exhaustive exploration (try many different ways to reach the same state)
  • loop areas, for example try a series of values in a field, for example a collection of valid and invalid inputs

And so on, you get the idea right? the potential is really interesting.

But time is up and post is long, so I'll be on my way.

-Mat

PS: I warned you it was going to be heavy stuff...




2 comments:

  1. Hey Mat, thanks for a good post. I'm doing maybe something like "stage 3" by saving the data values (hidden behind the red color in the pic, and also the selection value from check boxes and such) into the properties of the state transitions. That way the amount of states keeps as low as in "stage 2", but we have infinite amount of possible state transition. But as we have agreed before, we are not looking to find all paths between the states - the target is to somehow visit all the possible states (with minimal amount of transitions to save time in model extraction).

    ReplyDelete
  2. Yeap, so many interesting things to do, so little time... we'll get there... eventually.
    Gotta write you about the multidimensional maps where high level actions are mapped from the low level full map, quite powerful concept... again... when I find the time... :)

    ReplyDelete