Archive for February 2009

Waterfall manual test tools don’t work when the waterfall is broken

February 10, 2009

The Problem:

At Uplogix I’m now working with qatraq after we left testlink, and it is another tool that assumes you have a huge battery of good tests before you start testing.  You can assign them all to neat test scripts and assign the scripts and execute them. All of this falls apart when you realize the test is broken and you need to go fix it or you discover there are areas that are not tested. If you try to write one on the spot and add it to a script you end up loosing results for other tests.

Another problem that came up is how to deal with tests that have failed and rerunning them after the offending bugs have been fixed.

What I need to do

I should download qatraq, testlink and bugzilla/testopia at home and try to set up examples and see for myself if there is no workaround.  I noticed on,the pay-for-version of qatraq’s web site, that they make this claim: “The only Test Management tool based on IEEE Standards” which explains it not working with agile or exploratory testing.  What do IEEE/CMM shops do when they see cracks in their test plans?  Fix them in the next release in 9-18 months?  Spend a month or two on a patch?

Questions I have:

What, if any, test tracking tools deals with an environment where you have

  • Old test cases that may need to be reworked while testing
  • New discovered tests that a tester sees while testing
  • Exploratory or context driven testing where tests are developed and run at the same time without mounds of documentation that need to be maintained
  • Works with automated tests as well

What do you do if you are in an old-school IEEE, CMM, or Waterfall group (or even one that used the word process frequently) and want to migrate to a system that works better how do you deal with your old and new tests?

If the answer is ‘redo everything in the new technology’ you will probably fail.  Running the old system and new system separately for a transition period can be painful especially if the old system doesn’t die.

Test tools need to be more like revision control systems where you can migrate your assets from CVS to Subversion then later go to Git.