An Inspired Pairing

I’ve been traveling a lot lately and that has given me the opportunity to knock of some of the books that have been sitting on my to-read list for too long.  Two of those books, which I read back-to-back go together like Gruyere and Chardonnay.  They are great individually, but a truly inspired pairing.

The first is The Visual Display of Quantitative Information by Edward R. Tufte.  This book has received gushing reviews since it was first published and deservedly so.  My favorite review quote was from the Boston Globe:  "A visual Strunk and White", referring to the classic on how to write clearly and concisely.  The Elements of Style, is self-consistent; it is clear and concise.  The same can be said of the The Visual Display of Quantitative Information.  It holds to its own principles in how it presents data. 

This book motivates me to be more contemplative of the charts and graphs I produce to convey information.  Example after example show ways the display of data can add to or take away from the message the data has for its readers.  Excel gives you some tools for aspiring to the former, but makes it oh-so-easy to achieve the latter.

The second book is Presentation Zen: Simple Ideas on Presentation Design and Delivery by Garr Reynolds.  This book seeks to inspire us to deliver better presentations.  I’ve sat through hundreds of PP presentations.  Not many were memorable.  But some were, and those that were follow the guidelines in this book.  It’s about crafting your message, not making yourself obsolete by having the slides stand alone, simplifying and using space effectively.  It is a convicting book because boring presentations are the de facto result when we use built-in templates, cut-and-paste from Word documents and Excel spreadsheets, and throw in some images we find from Google images.  We can do better, and we must do better.  We can serve our coworkers by better valuing their time, and making them delighted to have been at our presentations.

Each of these books is wonderful.  But having by serendipity read them back to back, I realize they are an inspired pairing.  The one guides us to presenting data in a way that is clear and compelling.  The other guides us in using that data in clear and compelling presentations.  If we take these lessons to heart and apply them in our creations, we may not save the world from death by PowerPoint, but we can avoid adding to carnage.

Making Microsoft Test Manager Better

There are three things (two features and a datasheet) that, if Microsoft added to Test Manager, would push me past the hesitation I have to move my defect and test case management from our existing tool to TFS and TM. 

Currently, we are using Rally across the board: requirements, defect management, test case management.  Having all of our data in one unified tool has some definite advantages, and makes traceability easy.  But inadequate customization, poor support for required fields (they are either required or not, regardless of the state a defect is in) and an inability to fail a specific test step are just three examples of why I am open to enduring the pain of transitioning to another tool.

But I want three things from Microsoft.

First, the ability to version test cases.  I was surprised, maybe even shocked that this was missing.  After all, TFS is the base upon which TM stands.  And TFS is all about versioning.  Like most commercial software companies that deliver software (i.e. not hosted), we have multiple parallel development streams being worked on.  As requirements change, as defects are discovered and as gaps are found, our test cases change.  I do not want to have to duplicate my test cases for each stream.  But I need to be able to modify a test case for one streams and still execute the previous “version” of that test case for another streams .  Test case versioning is the obvious way to handle this well and TM doesn’t have it.

Second, and this is not a feature, I want hard data on the system impact of executing test cases within TM.  It has some impressive features.  It will capture a video of the application execution which can be automatically submitted with defects.  It will keep track of what code is executed when a test case (manual or automatic) is run so later, when a change set is introduced it can recommend the test cases which are affected and therefore need to be rerun.  It will collect IntelliTrace data to allow a developer to step through the code as it executed when a defect was discovered.  This is great stuff.  It also sounds like the Heisenberg Principle on steroids.  What is impact to the system?  I cannot get any data when I ask Microsoft this, only vague responses that “I’ve never seen it be an issue.”  That’s not good enough.

The third want is related to the feature I mention above.  I can execute manual and automated tests and tell TM to keep track of the code that gets executed.  Later, when development checks in a change set, I can know which test cases exercised code that was changed.  This is a great feature.  So I asked the following question at a Microsoft presentation:

“Since you keep track of what code gets executed when I run a test case, can I run through all of my test cases and then have TM tell me what code never got executed?”

They have all the data to have a very powerful code coverage feature, but the answer, sadly is no, the system will not do that.  But it makes too much sense not to add in the future.

Microsoft’s focus on testing is exciting and long overdue.  The team has delivered some excellent features and the defect handling in particular is very powerful.  But changing tools is a sometimes painful, and usually a hard sell internally.  The addition of these three things would make me pull the trigger on the change.

Using the Intranet to Implement Shared Steps

One of the really useful features in Microsoft Test Manager is shared steps.  It allows you to specify steps which are shared across multiple test cases, give a name to those steps, and reference them in your test cases. 

Wherever the shared step is referenced, all of the associated steps follow.  This can be used for manual test cases, automated test cases, or a mix, where the shared steps are recorded and referenced in a manual test case.  Microsoft’s example is recording the logging in to your application, and then including that automated snippet in your test cases.

We have not yet embraced Microsoft Test Manager (I’ll give you some reasons in an upcoming post), although we are now on Visual Studio 2010.  As far as sharing steps, we are doing something very simple, but for us, very beneficial.

We have created wiki posts on our intranet site which describe the steps needed to perform our most common tasks.  These include logging in to the application, creating a new case, creating a new patient, performing filtering and organization activities on our cases, etc.  Each wiki post describes these steps in enough detail so it is clear how to perform the actions and what result is expected.

Then, in our manual test cases, we include a one-liner and link it to the associated wiki page.  The test case may have a step which states “Filter Pile on last name starting with ‘D’”.  That is linked to the page which describes in detail how to do that.  When you are executing the test and you come to that shared step, if you know how to perform it, do it.  If you need more information, simply click on the link and the wiki page will give you the details you need.  The benefits:

•  Common activities are centrally located and thus easier to maintain
• Test cases are shorter as detailed is abstracted out
• Testers needing a knowledge refresh can get the detail they need without any searching

We could record these steps although we are not doing that now.  This is a simple technique which we are leveraging to improve the readability, supportability, and usability of our test cases.