3 scenarios when we should automate (or not)

by | Last Updated On Aug 8, 2014 | Guest | 25 comments

This is a guest article by Shuchi.

Shuchi is a principal consultant in a multi national corporation and has ten years of development/testing experience in the IT industry. She has worked closely with test automation tools and has been involved in test strategy-related decisions.

In her off-work hours she writes about cryptic crosswords. Check out her cryptic crossword guide on the blog, Crossword Unclued.

Why Automate?

A prevalent belief among IT folk is that automation, by default, will enhance the quality of testing.

How true is that belief?

“There is a time and place for everything”, as the popular idiom goes, and that holds true for test automation too.

Certain scenarios in which automation is generally a good option:

1. Repetitive Tests

Repetitive TestsA system that needs the same tests executed periodically is a good candidate for automation. Take for example, support projects that must perform regression tests for every PR/CR. Manual effort in such cases tends to be huge and error-prone. Automation can prove very efficient in such cases.

It makes sense to opt for a testing tool with a version controlled repository to manage the test scripts.

The cost of automation and the initial effort investment for script creation is usually greater than for manual testing, and it must be analyzed how far this investment will pay off in the end. A few pointers to help make that decision:

  • How many repetitions of tests are expected, for how long? A longer “life” of the test scripts tips the scale in favor of automation.
  • How expensive is the tool?
  • How much time/effort will it take to create the test scripts? Automation may not simply be record-and-play, a lot of design and coding may have to go in to achieve the desired result.
  • Are incremental changes expected to the scripts?
  • Do testers on the team require training on the tool?

“Repetitive tests => automation” is a fair thumb rule but not an inviolable one. A costly automation setup on one hand and quick-to-build on-the-fly manual tests on the other, and you might just find that automation will not be useful even if you expect a certain number of repetitions.

2. Manually Infeasible Tests

Before an application is deployed in the production environment, it might be critical to Manually Infeasible Testsdetermine how the application will behave under huge user load or when dealing with millions of records.

Such test scenarios are usually best simulated with automation. In high risk conditions, where the cost of missing a test might be disastrous, even a single-time use of automation might be a worthwhile decision.

3. Low Severity/Probability Of “Human” Bugs

Low Severity/Probability of human bugsAutomation can perform some jobs far better than a human can hope to do. Give the tool and the human tester two excel sheets with 10,000 rows of data each to reconcile with each other. No prizes for guessing who can do it faster, without errors.

But then, humans are far better at another sort of testing. Humans can notice oddities beyond documented tests. Does the UI look too cluttered? Does the mouse flicker strangely when moved over the button? Did the screen behind the modal window just blink?

Automation is limited to testing what it is programmed to do. Not everything that the human mind can observe/analyze can be programmed, and so, automation will miss out on capturing some bugs. The question to ask is: how important are those missed-out bugs? An unstable system might not be ready for automation, as severe “manual” bugs may get ignored. Automation is better suited to systems in which the “human” bugs are expected to be fewer and are of low severity.

 

In Closing

A point worth remembering is that automation vs. manual does not have to be an either-or decision. A mix can work. Take a multi-step workflow – a couple of steps might be fit for automation, not the entire flow. Automate only those steps that will benefit from automation.

Ankur:Thanks Shuchi for the insightful article. I’m sure it will be of great help to the automation testing community.

If you wish to share your experience/insights related to test automation or have discovered some cool new tip related to QTP, contact me using this contact form. [Please select ‘Article’ in the dropdown.]

You Should Read These 3 Related Articles Too!

25 Comments

  1. Lokesh on Mar 29, 2011 at 11:31

    Hi Ankur/Suchi,
    Can we automate unstable web application ? My application doesnt follow any codes standard and the object identified by QTP fails to run. Please guide me on the same.
    Thanks!
    Lokesh

    Reply
  2. Sucheta on Jan 3, 2011 at 15:11

    Hi Ankur/Suchi,

    Thanks, Very nice article which is helpful for me definitely.

    Sucheta

    Reply
  3. Bouhalli Zied on Oct 29, 2009 at 20:58

    Hi Ankur/Suchi

    Thanks a lot for this valuable Article

    Zied

    Reply
  4. Venu Teepireddy on Sep 6, 2009 at 22:46

    Thanks Shuchi, it will definitely be helpful to all beginners.

    Reply
  5. Vijay on Aug 30, 2009 at 00:25

    Helps Alot…Thanks Suchi and Ankur

    Reply
  6. Nitin on Aug 22, 2009 at 05:10

    Hi Ankur/Suchi,

    I am a software developer for last 7 years and now just trying to get into automated testing. Could you please give me tips how do I go about it.

    ta,
    Nitin

    Reply
  7. Aditya on Aug 17, 2009 at 20:39

    Hey thats a really good and helping topic for those who do have a misconception of helped by using Automation testing tool in all scenarios.

    Thanks for explaining your points in a precise way.

    Reply
  8. karthik on Aug 11, 2009 at 23:58

    Shuchi,

    Its a great article. It would be great to see another article which is talks more about ROI(Return on investment) for automation.

    Reply
  9. Vikalp on Aug 4, 2009 at 06:13

    Hi Shuchi,
    Nice thoughts and good comparision……….

    Reply
  10. Avi on Aug 3, 2009 at 03:31

    hi, I got step by step guide to learn QTP………
    Thanks..

    Reply
  11. Avi on Aug 3, 2009 at 03:29

    Hi Ankur,

    I am a novice to QTP..
    I want to learn this tool as deeply as possible..

    But since I am a new to this tool, how should I start learning this tool so that I can learn it quickly…..?

    regards

    Reply
  12. Shrikant on Aug 3, 2009 at 01:29

    This is nice article & there are hundrades of others … Excellent work!!
    From last couple of week this, site made me crazy to know more & more about QTP. I never worked on QTP but on WinRunner.
    Ankur has performed exceptionally outstanding job. Thanks a ton for all these inputs…

    Reply
  13. girija on Jul 31, 2009 at 07:58

    Thanks Ankur. Your website is really good.

    Reply
  14. Srikanth on Jul 31, 2009 at 07:20

    Thanks for Sharing …..

    Reply
  15. anusha on Jul 31, 2009 at 07:15

    Thanks suchi & Ankur..
    You guys are doing very good job,Thanks for the great info…

    Reply
  16. SRI on Jul 31, 2009 at 05:26

    Ankur,
    This article is useful, Thanks for sharing

    Reply
  17. jambo on Jul 31, 2009 at 04:01

    Ankur ! this is good but increase your number of responces.

    Reply
    • Ankur on Jul 31, 2009 at 04:12

      @Everyone: Thanks.
      @Jambo: Can you please explain what do you mean by “increase your number of responses”?

      Reply
  18. Sri on Jul 31, 2009 at 00:33

    Worthfull….

    Reply
  19. Christophe on Jul 31, 2009 at 00:30

    Nice article. Nothing that I didn’t knew yet, but it’s a good start to explain people who aren’t familiarized with automated testing when to autmate and when not to automate.

    Reply
  20. arush jain on Jul 30, 2009 at 23:34

    please u will send me the set up of atomation tool, qtp, i unable to work on qtp, as my lappy has been formated, so it is not taking liscence again. what should i do?

    Reply
  21. arushi on Jul 30, 2009 at 23:30

    hi ankur, this article is really very usefull for me. thanks.

    Reply
  22. Naresh on Jul 30, 2009 at 22:19

    Thanks a lot!!!

    It’s really good article.

    Reply
  23. Avikash on Jul 30, 2009 at 21:09

    Beautiful Article………

    Reply
  24. Anuj on Jul 30, 2009 at 20:39

    Hi Shuchi & Ankur,

    Thanks for sharing such a good article. I found this article really useful.

    Thanks Again

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *