More on the workbench, automatic detection and report of application changes, testing with privileged prompts and more.

In short, take a model extraction script as we saw in the previous post, run it against a newer version (presumably the next minor update), hit the 'Compare' link (near the top left corner) and it will tell you what has changed.

Sounds a bit weird but works like a charm, given the model of 2 different versions of the same application it will tell you where it has changed in a (nicely?) formatted report.
What's the use? well, in testing is near a dream tool for me, we work in a continuous integration model, we get several versions of the application per day and we used to have more than 25 persons working over it at the same time, that is without counting UX (user interface) specialist and translators (who also commit or contribute to the changes on a daily basis), keeping track of what changes, where and when was a really difficult thing to do.

Now the paradigm is inverted, when something changes our build management tool is configured to run a model extraction and compares the last 2 versions, if there are differences it generates the report and triggers a notification.
What changes are reported? well if you add a dialog, add / remove a button (or other ui element), change the style like fonts or colors, changes in the texts being displayed, most of the visual changes.

One could say if tests are in place and properly written they should fail and notify, well... in your dreams... most tests did lack a lot of assertions and did little UI asserts. In fact if they did do all the necessary assertions then the maintenance cost would be rather huge.

Anyway the topic can go really deep, enough is to say that there was nothing else needed than the basic extraction script to get alarms displaying what has changed, here's a couple of (not so good) shots of such a report.

Privileged prompts, a pain(?) to automate

Suppose you have your test machine properly configured, at certain point the app may require to do something for which windows security will ask the user to confirm if it wants to proceed:

If it were easy to automate that, security would be a joke (and computer viruses would rejoice at it), the challenge is to automate it in the most realistic way, with real security settings in the test machine.
Writing a kernel driver or other low level solution would be very platform specific, not to mention the care and cost it would require, no problem, murphy does the trick for you.

Here's a simple model that handles the case:

You may notice that from the privileged dialog it only does the Yes / No button, that's because the extraction script instructed to do so, if you want to go crazy you can go as much as you want, for example:

(The script to produce that had about 8 lines of code which I intend to shrink to 3)

Funny thing, after the model was extracted I found 2 things right away:

• In the "Change when to be notified..." dialog, pressing OK without changing anything required me to confirm the changes, however no changes were made (is at least arguably if that's ok)
• The certificate information is expired but there was no warning whatsoever from windows

Here's a closer look at the certificate info:

Checked the date and time and they're correctly set to 2014, also tried the same in this very computer and also says it is valid until 2011. If you know why please let me know, to me it looks like a (serious? minor?) security issue, then again I'm not a security expert.

Wrapping up (the post is getting long) you can have a view slightly simplified to avoid too much arrows crossing:

Ok, time to start my weekend, hope I wasn't too confusing.

-Mat