Waterfall manual test tools don’t work when the waterfall is broken

The Problem:

At Uplogix I’m now working with qatraq after we left testlink, and it is another tool that assumes you have a huge battery of good tests before you start testing.  You can assign them all to neat test scripts and assign the scripts and execute them. All of this falls apart when you realize the test is broken and you need to go fix it or you discover there are areas that are not tested. If you try to write one on the spot and add it to a script you end up loosing results for other tests.

Another problem that came up is how to deal with tests that have failed and rerunning them after the offending bugs have been fixed.

What I need to do

I should download qatraq, testlink and bugzilla/testopia at home and try to set up examples and see for myself if there is no workaround.  I noticed on http://www.testmanagement.com/,the pay-for-version of qatraq’s web site, that they make this claim: “The only Test Management tool based on IEEE Standards” which explains it not working with agile or exploratory testing.  What do IEEE/CMM shops do when they see cracks in their test plans?  Fix them in the next release in 9-18 months?  Spend a month or two on a patch?

Questions I have:

What, if any, test tracking tools deals with an environment where you have

  • Old test cases that may need to be reworked while testing
  • New discovered tests that a tester sees while testing
  • Exploratory or context driven testing where tests are developed and run at the same time without mounds of documentation that need to be maintained
  • Works with automated tests as well

What do you do if you are in an old-school IEEE, CMM, or Waterfall group (or even one that used the word process frequently) and want to migrate to a system that works better how do you deal with your old and new tests?

If the answer is ‘redo everything in the new technology’ you will probably fail.  Running the old system and new system separately for a transition period can be painful especially if the old system doesn’t die.

Test tools need to be more like revision control systems where you can migrate your assets from CVS to Subversion then later go to Git.

Advertisements
Explore posts in the same categories: Test Harness

One Comment on “Waterfall manual test tools don’t work when the waterfall is broken”

  1. marekj Says:

    I track only failing test. For other tests I don’t have time.

    Every test is in ‘source control’ currently in CVS. Every word about testing is version control. All in ‘text source files’ – NO Binaries, binaries are not greppable.
    My front end for tests is Jedit editor. I can search for phrases and strings in directories.
    All tests presented as Ruby classes and I use Watir for execution.
    Hudson CI for regression testing running with ci_reporter.

    All of this has to be simple and fast.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: