Open Source Published on:

Selenium sucks! QTP rules! Got your attention, didn't I? How many times have you heard this type of statement from overzealous automation engineers? Truth be told, I usually hear statements like this most often made by open source evangelists.

One Test Automation Tool to Rule Them All

The fact is, however, there's no such thing as a perfect test tool. There is no “Test tool to rule them all.” In lieu of that, you have to use the tool that works best for your team and your particular application. And just because you use one tool for one specific project does not mean you should use it for all your projects. Many test tools require an investment in not just dollars but also in learning and infrastructure – so I understand how tempting it can be to stick with one tool solution for all your testing needs. If you have a hammer, everything looks like a nail.

My Test Tool Dilemma

The main application I test in my current position has many layers of proprietary control wrapped in an active x control and displayed in a web browser (IE to be exact; it doesn't run under Firefox). The application uses a hodgepodge of non-open source technologies like .NET, CACHE DB, and MUMPS. On top of all this is a Frankenstein-like build system that I couldn't even begin to explain the workings of.


My company also has an enterprise license for HP's automation tools. So which tool do you think I choose to use? That's right — QuickTest Pro. Why, might you ask? Well first, it's the standard tool used throughout the company. Most test engineers are already familiar with it so there's no need for training and it works (albeit with several modifications) with our application. Also — it doesn't hurt that I've been using the Mercury/HP toolsets for many years.


Some of the groups in my company are moving to AGILE. One group, in particular, is involved in a new application that uses the latest and greatest web2.0 technologies. Also — unlike my group, most of the development is done using open source technologies and uses Hudson for daily builds. This team's application can run under different browsers and the application is fairly straightforward. So, in this case, I would think a test tool like Selenium would be a better fit than QTP.

Allegiance to a Test Tool

The point of my rant is that we should all have an allegiance to a particular test tool. As automation engineers, we should be focused on developing the quickest method with which to create reliable automation solutions that serve a specific business need!

Don't be a Test Tool Fanboy

This leads me to my last point. You should not align yourself with one company's tool. Let me tell you a quick story. Long ago, in a cubicle far, far away lived a Win Runner automation engineer. How he loved the crisp white screen of a new Winrunner test! The engineer spoke the language of TSL. There was no business problem that he couldn't solve with the right Winrunner script. Then, one day, Winrunner was discontinued. No longer would his favorite test tool get updates. Job postings for Winrunner experts became fewer and fewer. The engineer was heartbroken. Don't become like him! Keep learning. Use the right tool for the job. Never become a Selenium or QTP (or whatever company's technology) fanboy.


So what do you think? Am I wrong? Leave a comment below and let me know!

133 responses to “SELENIUM SUCKS! QTP RULES!”

  1. Even in the Agile scenario QTP Rocks ! .. We also usong Hudson and we’re complete opensource (Development) group but since QTP supports all the challenges that we met ..

    • saravana » Great to know. My company just started to use AGILE so I’m glad to hear that QTP worked for you in an Agile environment. For every build do you have Hudson run QTP scripts?

    • I did run, however, into a project that used QTP because the licenses were readily available. However, due to some pretty nasty exceptions QTP was causing in the app (which crashed everytime you clicked on a button or requested an object property), we had to switch to other tools, and ended up with Visual Studio CUIT. You never expect to make such a swift change, but in the context of multiple custom object, layers and layers of ContentPresenters and above all, incompetent HP support (on this instance), you have to change.

  2. Considering the cost savings selenium definitely will have a great future and will over rule QTP in future for sure. Go to any job forums and you will see job postings on selenium more than QTP…. So i agree Joe, we will have to be really on par with pace of the automation trends.

  3. Hi Joe,

    My vote is to open source. i’m not fan of any specific tool but used wide varieties like rational, mercury/HP, WATiR, WATiN, Selenium, sahi etc. every tool has its advantages and drawbacks. Now a days after HP took QTP it lost its significance as i used QTP 6.5 in early days and i never had issues. Now i have some issues i posted in your forum still not resolved by HP SC. i’m exchanging emails everyday to get the problem solved. One of my friend in other group told me that HP is concentrating more on protecting unauthorized usage of QTP (licence areas  lol) than actual QTP functionality. I may not agree with it, but may be he is true with his exerience.

    • sam » Hi Sam – I’m with you, use the tools that works. I agree before HP took over I think Mercury was really just focused on creating killer test tools. With HP not so much. HP’s support is awful and that is suppose to be the big selling point of using a commercial tool rather then an open-source one. To bad Selenium is just for web applications.

  4. wow… interesting article..! even my organization is also following complete agile mode of product development using open source technologies…still for automation, QTP is being used and 1000’s of test scripts are running in test CI :)

  5. from my point of view, QTP is a poor choice since in 2012 it still use VBScript as the only available scripting language, which is kind of retarded if you ask me.
    At least, Selenium let you choose your scripting language of choice.

  6. Hi joe.i am Completely new to automation testing.Wanted to know that ho the decision is made based on the application that which automation tool is to be used? To start with qtp would be good?Market has more jobs for automation than in manual testing ?

    • nik » HI – If you are testing web based application than I would start with Selenium 2 WebDriver— it is free and I’ve been seeing more and more job posting that have it as a requirement. For testing web-services check out soapUI –which is also a free opens source tool.

      For other technologies like .NET,WPF etc.. I would try either HP’s UFT(QTP) or TestComplete.

      Before selecting any tool for a project I would recommend doing a proof of concept first to evaluate each option and choose the best one for your situation. There is no one tool that will be right for every company/project.

  7. Hi Joe,

    From past 3 years i m working on manual testing, i switched to automation, since i m very poor in coding, i found very difficult in learning selenium….But QTP was little easy….So if i continue in QTP, will there be a future for coming 10 years.. Please guide me.

    • Ravishankar » The best way I found to stay current is to do a job search every few months and see what technologies are being listed as job requirements. I compile a list and in my spare time try to learn those skills. I personally have seen an increase in Selenium and Python as requirements for automation positions. Also I believe that all automation engineers need to know how to program and should know how to run/create performance tests. That’s just my opinion

      • QTP Esstentials for web testing (includes HP QTP Essentials supports the following add-in environments:
        ActiveX, Java, Visual Basic, Web, .NET, WPF, and Silverlight. It also
        supports ASP, .NET, and AJAX through Web extensibility)
        and you can buy a seat license for $4K (approx)

        You can contact Checkpoint Technologies or any other HP Partner

        Good Luck
        -Satya Mantena

    • Hi Alex,

      Better you try opensource Selenium
      Selenium one of the popular software testing tools is a portable testing framework for web applications. This is an open source testing tool, developed by Jason Huggins and team and released under the Apache 2.0 license. Since it is an open source tool, it can be downloaded free of cost. Selenium provides a playback tool for authoring tests. Using the tool, the tester need not learn the test scripting language (Selenium IDE). Furthermore, it provides, a test domain specific language (DSL) Selenese that allows to write tests in various languages including C#, Java, Groovy, Perl, PHP, Python and Ruby. Then, these tests can be executed in many latest web browsers and works on everything including Windows, Linux, and Macintosh platforms.

  8. I’ve been using QTP for 10 years, since version 6.5. I used to think it was the greatest thing since sliced bread. In the past couple years, my feelings have changed completely, especially since the HP buy-out of Mercury.

    Although I’m no big fan of Selenium, its just too limited, I hate QTP! It is one of the most difficult to use, poorly designed, unstable, flaky and tempermental pieces of garbage software I’ve ever had the misfortune to use. Licensing is outrageously expensive for a piece of software that just doesn’t work. Our group is slowly phasing out QTP in favor of TestComplete. It can’t happen too quickly for me.

    • Dave P » Hi Dave – Interesting we have an Enterprise license with HP, so the license is free for a group to use and we still have groups that go out and buy TestComplete instead. I really love soapUI by smartBear so I’m sure TestComplete is also a great tool. I agree – Mercury was awesome, HP not so much their to big to really focus on automation tools. Let us know how TestComplete works out for you. Cheers~Joe

      • Our software development team uses SoapUI to validate the structure of their webservice messaging. The way they have it set up is pretty slick. Hudson manages the build process, so whenever they check in code that touches any of the web service libraries, the SoapUI scripts run automatically to verify that they haven’t broken any of the messages.

        That works out really well for us in the QA department. I use TestComplete to generate web services messages from a data-driven library of test files. Then I walk through the application to validate the expected effects of those ws messages, e.g. creating new accounts, etc. The combination works out really well. I know that I don’t have to mess around with validating the structure of the message itself and can concentrate on deeper testing of the application.

        Smartbear has a way to go on some of their test management systems, but their support is awesome. They are very willing to listen to their customers and work to fulfill their needs.

  9. Thanks Joe for your responce!
    I am in process of selecting automation tool for UI testing. As you know, every manager dreams to have test cases automated.
    My team are not very eager to program. So, I am looking for a tool, which would automate but does no require heavy programming. Do you know of any?
    Would be great to hear your recommendations!
    Thank you,

    • In my never to be humble opinion…

      I’ve been down the road a few times with different organizations trying to build automated testing without programming. Organizations seeking this goal are susceptible to the record-and-playback promises of almost every testing package out there. Record and playback is pure marketing BS. Any test script you build this way is nearly always obsolete within a revision or two. Its nearly impossible to modify the recorded scripts to match the application changes, so you have to record the scripts again. Eventually, you give up on automation entirely.

      Automated testing of any significance is going to require programming skills. Test scripts are software just like the application being tested. If you’re serious about test automation, you’re going to have to accept that fact. If you’re not willing to make that commitment, you will not have much success with test automation.

    • Alex,
      if you can tel me the technology stack and scope of your testing or testing activating which you want to automate , i can able to guide you the right tool.

    • I’ve never used Ranorex, so I can’t tell you much there. I have some limited experience with Rational. Typical IBM software, overbloated and confusing. Our department is currently using a mix of Selenium and QTP. I haven’t worked with Selenium that much. It seems to work well enough, but it can only be used on web based apps. QTP is slowly being phased out and replaced by TestComplete. TestComplete is a far superior testing tool for several reasons:

      1. Multiple scripting language support, including JavaScript, C# script and VBScript. Object oriented languages are such an advantage in building test frameworks…

      2. Much more solid. I don’t have the problems with the IDE locking up during debugging like I do with QTP.

      3. Stronger support for distributed testing. The test engine is a separate module named TestExecute which can be installed on a remote machine to execute tests. It doesn’t require the complete IDE like QTP does.

      4. Substantially less expensive than QTP, about 2/3 the cost for the same level of functionality.

      5. Far better support. Lets be honest, HP bought Mercury so they could compete against IBM’s Rational. Now HP can say, “We have test automation too!” but beyond that, they don’t give a damn about the product line. True story: I once called HP’s support for a QTP question. The support tech literally asked me, and I quote, “QTP… is that one of our products?”

      That’s my two cents worth. Good luck on your automation efforts. No matter which system you ultimately choose, my advice would be:

      To begin with, write a few simple test scripts just to learn how the package works, its capabilities, idiosyncracies, etc. Whatever you develop at this early phase will likely all be thrown out once you become more familiar with how the test system works, so don’t spend a lot of time trying to build the ultimate testing system yet.

      Later, step back and figure out exactly what you expect to get from test automation. Plan it out like a software development project, because that’s exactly what it is. Then before you write any actual testing scripts, build a framework around your target application. The framework should be designed to handle things like launching the app, shutting it down, performing simple, commonly repeated validations, encapsulating common structures, etc.

      With your framework in place, thats when you can start writing real test scripts. If you follow this general plan, you will save yourself a lot of frustration and wasted effort.

      Once again, good luck.

  10. I started 5 years ago with Winrunner and hated it. When it was no longer supported we looked for a new tool and ended up with HP QC, QTP and BPT. The licensing cost is an issue but we found the framework to be the selling point. BPT makes writing tests fast, reuseable and an easy environment to maintain. The reporting out of QTP with its recording or screen shots an important tool for defect submission.

    I cannot comment on Selenium but there are others in the company that want to go that route. And that is fine… but my tool of preference is the BPT world.

  11. ThinkSys Inc’ team of Selenium and QTP experts have helped migrate many for our customers from QTP to Selenium in record time and at extremely low costs.

    Using our unique, patent pending Selenium based framework KRAFT- Krypton Regression Automation Framework Testing, ThinkSys Inc has helped clients to create a low cost end-to-end automation solution for web sites and web based applications using Open Source tools.

  12. Helped a major Fortune 1000 client migrate over 8,000 test cases from QTP to Selenium in record time. This saved them hundreds of thousands of dollars in yearly costs and significantly reduced their QA cycle time. They now do multiple releases a week.

    For a company recently gone public on Nasdaq (2012), using our KRAFT framework we created and automated about 1000 test cases in Selenium in 4 weeks.

    • Your comments sounds like a sales pitch for a framework your company is intending to sell.

      Can you describe any problems with the KRAFT framework and what changes you made to resolve those problems?

  13. Hi Joe,

    gr8 article and comments… I am a manual tester with 6 years of experience and looking at current market i really feel that i should learn automation… but i am not sure how to start with…
    Please guide, thanks in advance.

  14. Hi Joe – Completely agree on your thoughts.. I used to suggest my folks the same..
    Also would like to thank you personally for a such wonderful job you are doing here and educating,suggesting,helping and motivating the guys who need your helping hands .. Great going .. keep up your same spirit always high…

  15. I agree with you Joe. In my job role, I actually have to used WinRunner, Selenium WebDriver, and QTP. They each have their advantages and disadvantages just as you said. We had a team that thought WATiN was going to solve everything and only to find out it was no longer being supported. They were frustrated and moved onto Selenium. They have put Selenium on a pedi stool although I’ve made the same argument that one is not superior over the other and could be discontinued at the drop of a hat. We live in ever evolving technology and I’m sure something “better” will come along within a year or two. In my 8 years of Automation Development, I’ve seen the end of life on two products that our company has consumed and I’m sure I will see the end of life on another before all is said and done.

  16. I’ve used a variety of tools over the years but I started with WinRunner and then QTP. I have since used Webdriver, Concordion, FitNesse and Robot Framework/Selenium. And I’ve tried out many more in the course of finding the right tool for a particular audience. There is such a variety of tools available now, that it’s no longer a question of QTP vs Selenium, but QTP vs Selenium vs Telerik TS vs TestComplete vs Axe vs Ranorex vs Canoo etc etc. Determining the best tool for the job needs to take into account various factors such as who will be writing/running the tests, what languages the tool supports and whether they can be developed using a language and IDE that the development team is familiar with, how well the tool abstracts the intent of the test from its implementation etc. Things were certainly a lot easier when there were fewer tools to choose from but, as you say, limiting yourself to one tool is a quick route to redundancy.

  17. Hi Joe, I completely agree that there is no “best tool” as such, you have to select the one best suited to your requirement. I want to add to the Selenium v/s QTP debate. We were asked to do a small comparative report on tools in our project and we found some interesting insights:
    a) Selenium out-of-the-box has very limited capabilities, being limited to web-testing and Selenese. But after integrating with opensource test frameworks and Java packs, its capabilities are unlimited. One of our development teams who are venturing into automated unit testing profess that Selenium supports Java and Java can do anything. I am a QTP guy myself (so far) but find that hard to contest. I cant say that about QTP, though of course QTP can be used to drive JARs.
    b) One advantage of QTP over Selenium and other open-source packs is that the mundane automation tasks – reading test data from excels, maintaing test objects, identifying objects on the UI – are implemented as one-liner standard methods which are mature and bug-free. This means
    b1 – you can get on with thinking about your test algorithm.
    b2 – when debugging a failed test you can quickly check these and move on to checking the actual algorithm.
    b3 – most QTP scripts are readable; you can take-over another person’s automation scripts with little or no handover process if you are reasonably experienced.
    With Selenium or other open-source packs, your skill with the tool gets you only so far when reading other’s scripts as you need to learn how every little task was implemented.

  18. Frist off great discussion and lot of good valid respected opinions
    Funny analogy – So please dont take it personally
    The ones that are Pro QTP are analogous to “the married and have decided to settle down”
    Pro others are analogous to “either single or divorced, not literally, but similar thoughts to explore new and better, adventurous” feel like they made a bad decision to start with, newer the better, maybe because it is better, for which the Pro QTP’ers agree too, but have decided to be happy with what they have
    That being said there are majority on the middle
    My point being, sometimes its the ideology that makes the decisions more than pro’s n con’s

  19. hi good mrng to all..

    Please tell me which tool is best to learn for feature use and good scope in feature..manual testing or automation testing or selenium.which is best…if you dont mind plz forward your suggestions to my mail .please please do this needful for me..i will be waiting for your precious reply.

  20. My company spent big $ on QTP and we can’t even use it, LOL. The show stopper: QTP can’t handle the task of testing applications in a SSO environment using different user ids. This doesn’t seem like a strange or unusual request… test with multiple users, SSO… Hmm, and yet… QTP simply can’t do it.

    Big huge glaring minus.

    • I have written a some functions that solved the SSO problem with QTP (vBScript). I was also able to create a framework where a PC woke up every morning at a set time, and started to execute a set of tests on 52 applications using a different username and password for both Windows Login and application login.

      This framework required a complex design though. It was pulling usernames and passwords from excel sheets and using other 3rd party tools on top of QTP to do login/logout.

      So unless I am mistaken, what you are thinking should be doable.

      • Hi Murathp,

        Can you please eloborate on how did you accomplish
        “I was also able to create a framework where a PC woke up every morning at a set time, and started to execute a set of tests on 52 applications”…..
        Can you please mail me the details to
        It would be of great help.

  21. Hi Joe, Wonderful discussion group over here. I have just switched over from restaurant management to testing( Hoping of having a regular life). I am taking classes for QTP automation testing. I was wondering if I should try to learn Sellenium also while I am picking up on QTP and what would be the best way to go about it. I noticed there are a lot of experienced people over here, if anyone has any suggestions please email me at:
    As I can use all the help and guidance I can get, any feedback would be greatly appreciated. Thanks

    • kavi » Thanks Kavi! Can you do me a favor – when your done taking the course can you let us know what course you took and if it was helpful. I get a lot of request selenium course recommendations so it would be good to know if you find one that you are happy with. Cheers~Joe

      • Joe,

        I just discovered your site and I really like it. Information exchange and all of the contributors are great.

        I have a more general question though related to this topic. You see I picked up testing profession not by choice, but by mistake and economical circumstances of the time about 16 yrs ago. Started out as a manual test engineer, then picked up winrunner and QTP. I sure like to learn Selenium more and many other tools mentioned here, however, to be an expert in something open source, you need to learn something else…for example, Java, PHP, Ruby, C# and whole nine yards…so a test engineer eventually need to be as good as a software developer in my humble opinion if they want to do automation…am I wrong?

        • murathp » I agree – it looks like the direction automation is going with Selenium is that you do need to have solid programming skills to be successful. To get started with Selenium you need to know some programming language (Selenium IDE is not a valid option for serious automation). Also many Selenium books assume that you already know what junit and nunit is. On top of all this you have to also be aware of all the different CI build tools in order to integrated your Selenium test in your companies build process. So if you are coming from a vendor test tool background like QTP there is a bigger learning curve than most people like to admit.

          • Hi Joe
            In fact the learning curve for Selenium is higher than for QTP, but it’s not a nightmare, it’s pure pleasure. You mentioned that a person who wants to work in test automation must know programming, so what’s the deal?
            I can share very fresh thoughts about Selenium2, because my main task for last 2 months was to develop the automation framework. I can say for sure that this time was not wasted and the set of components used for that framework allowed me to get what I want.
            For those who consider Selenium I recommend such components as: Jenkins – as CI environment, Java language for Selenium tests – biggest number of examples on the internet, ant – for facilitating flexible tests run, html reports; log4j or other logging framework, Selenium Grid 2 – fantastic thing! you can easily have your environement distributed; JUnit/TestNG. Community did fantastic job on the whole Selenium and Selenium Grid, kudos to you!!!
            I am still in process of recognition of other good practices and the tool set I have described may have slight modifications. But in general the overall result, effectiveness of the Selenium plus other components has impressed me very much.
            I was using QTP for 6 years and probably came to the high fence where no other progress can be made and the QTP limitation started limiting me.
            Selenium for Web is in my opinion obvious choice, don’t waste your time on crappy, slow, buggy QTP. Save your life!
            For desktop application I am waiting for Test Complete to appear in the company. The opinions are good in general, though I can’t write my own opinion since I didn’t work with that tool yet.

          • Lukasz » Hi Lukasz – I’m not against learning to program. I am a programmer and love doing it! It just that some people are not aware that automation is programming and that Selenium require it. I just accepted a new position in my company where I will be creating a Selenium framework work with BDD tools like SpecFlow and JBehave. I’d love to hear more about your framework. What programming language are you using? I would like to use Ruby but most of the developers want to use Java or .NET.

          • Hi again
            I do not consider myself an expert in Selenium. I am still learning it. Of course the Internet is very helpful and provides many code snippets. Our role it to filter it out and often on the forums there are only questions and no good answer.
            My framework uses native Webdriver, so far I decided not to use any external libraries (Geb, Cucumber).
            The libraries are divided into packages. One package contains common methods that are generic and work for all other classes.
            People ask here about how to start with test automation. I can only say that of course you can start with automation having no knowledge about programming but you will never go beyond some horizon and your tests will more rely on tool functionality, wizards than on real code. This imposes a lot of test maintenance and sometimes test automation simply fails because manual testing would be cheaper.
            The best test automaton engineers, from my past experience, derive from software developers who wanted to develop test scripts.
            The community may be angry but majority of QAs who claim they automate, do crappy record-play tests which fail very often. That way companies may not believe in the power of test automation.
            Recently I had conversation with a VP who said automation is a waste of time. This is a result of low-quality test automation. I was hosting job interviews many times and saw small percentage of candidates have strong programming skills while in their CV only automation is being mentioned. This is sad.

  22. I am using Java for the tests.
    I still think whether to use Page Factory or Geb, but I am cautious because often it is like the tools work well for presentations, on simple page.
    But what will happen when the page is dynamic enough that number of objects is high, the DOM is big? Another concern is what happens when I don’t have unique id I can use but I have to find other query selector which not only has to find the object but be effective in IE (I have to support IE7/8/9 for cross browser testing).
    I am sure people made many interesting solutions but not always want to share it. My framework desing not necessarily is the greatest but for a few tests currently done it works very well. The webpages I was to test are not the simpliest, there is dynamic content in the tables, but I can imagine more complex pages on the other hand.
    I see you want to go the way of BDD, cucumber-like frameworks.

  23. Hi
    My project is developing on java environment i want to know which automation tool (either QTP or Selenium) is best.Please suggest me..I have experience on QTP only……….

    • Pavan » Hi Pavan – Is your application a browser based app? Selenium is only for browser automation. If it is a true java thick client app you need to use a different tool like QTP, TestComplete, or Jemmy

  24. Why , why on the earth company buy QTP for web automation.
    …because tester get very small learning time for programming , I guess….
    Days are not far when QTP will die as VBS is going to be obsolete in future … TDD BDD and OOPS based test are the the future.

    But QTP will not wash away for low salary jobs. It will be there with big company with non-IT core business…
    — God bless Winrunner

  25. Has anyone used Automation Anywhere? how it is comparable to UFT? I am evaluating some tools that can do the job without much programming. I was leaning towards QTP but all your comments here are throwing me off.

  26. I have 5 years exp. In Manual Testing & I will shifted in Automation which will good for long term QTP or Selenium (my programming is not good)

    • saramg » If I was starting to learn automation for the first time I would definitely start with Selenium. The great thing with Selenium is that you can use it with any programming language so if you are new to programming you can use a more user-friendly scripting language like Ruby or Python.

      • hi joe ,
        thanks for good advise but one more question, is this Programming knowledge is compulsory & if compulsory then java script good or not.

      • Joe, what is the use of Selenium 2 VBScript webdriver?
        Can we use Selenium 2 VBScript webdriver for creating test cases in VBScript?
        and is there any specific difference between Selenium and Selenium 2 (Except for just the Version number)?

        Lots of queries!! Coz i am new to this automation thing…
        Hope you understand…. Thanks.

  27. This really is happening to me. I spent years learning to dance six count east coast swing style. I was getting to be one of the better dancers. Along came west coast swing. It was still dancing to the same music. But now all of the movements changed.
    QTP and Selenium and now east/west coast swing. Once I get good at a skill, why can’t the target stop moving? I enjoy learning, but how about letting me enjoy the efforts of my hard work for a little while?

  28. Hi,
    I personally feel that using QTP or Selinium should be to the barest minimum. Imagine an enterprise level application where there is complex business logic involved, and they keep changing.

    1) Can a manual tester be made to learn programming and maintain scripts build on build or release on release.
    1 a) Say if only 40% of the suite of application, can only be automated imagine the effort on the programming front spent (time and money) is it worth the effort?
    1 b) Most of the time will be wasted in reprogramming and negotiating and adapting to ever changing applications. Well manual testers can try various permutations and combination to test and uncover the defects in business logic, again is it worth the effort?

    2) QTP poses huge licence and meager trail period(just about enough to get our hands on features), corporate are reluctant to try their hand on the tool.


  29. You shouldn’t be religiously devoted to a particular tool set, but as with any form of development, you should try to pick 1 or 2 tool stacks and stick with them. As a result of efforts to, “get it done, ” test automation too often becomes a hodgepodge of duplicative code, living on non-inter-operative islands, at best patched together with a series of half-baked parsers and hacked together reporting solutions. If you can stick to a single stack, you can leverage this shared code base to implement rich running and reporting interfaces, and reduce maintenance costs in the long run by maximizing code reuse and knowledge portability between test suites and project teams. You shouldn’t only use Selenium any more than you should only use mongodb, but you also shouldn’t use 5 different tools any more than you should implement each table of your database in a different database engine.

  30. H joe,
    came across your website and looks like its very informative. I am going to dive into it.. I was curious to know which scripting language are you using with selenium.. Is it java or c# and have tried Selenium grid so far?

      • I am at a point where I need to start looking into using Selenium Grid for distributing tests on different machines and think it takes bit of a work to make it work.I looked in to proprietary automation tools and they have very simple and neatly documented ways of achieving the same. Makes me wonder why not use a paid tool which is a lot hassle free when compared to selenium.Would you be able to talk about some of the issues that you are facing. I would also be interested to know the test runner tools used to make grid functionaly work. For example, i heard TestNg has great parallel attributes which helps working with selenium grid easier.What is the equivalent tools/framework used for selenium C# version?

  31. Hii joe..
    I am already join for qtp course of testing. I am really very much confuse that whether I am doing right or not. Plz say me whether there is any futer for qtp testing or not for fresher.

    • there is not right or wrong answer for this. I can’t see into the future but if I was to guess I would say that the future belongs to open source tools like Selenium. Just like there are still jobs for COBOL programmers there will probably still be a small need for QTP engineers. But you should not focus on a specific tool – you should focus on automation testing in general and use what ever is the best tool for the job.

  32. For me, Selenium appears to be a developer testing tool while QTP (and such) are QA tools. But it looks like the QA world has shifted towards Selenium as well. Will programming then become the critical skill in QA? What about excellent analytical skills and penchant for breaking software.

    • HI Haris – I agree – I think most companies are moving away from straight manual QA and towards more of a software developer in test type roles. My own company has started this transformation. So if you are in QA i think you must learn programming. Good or bad this seems to be a software industry wide trend

  33. Good post. I’d like to share some additional insights.

    I think part of this Selenium vs QTP debate, when you cast aside the “just for web applications” aspect is really about web vs non-web applications testing. The problem is in the OSS/free tooling area, there has yet to be a popular widespread tool like Selenium for the desktop & non-web apps. Until there is, QTP will still have an advantage in some areas. There is some headway with things like AutoIt, Microsoft’s UI Automation support, the White OSS project that expands upon MS’s UIA, but none of those are near the level of commonplace use & test coverage capability of QTP/Selenium.

    But I think towards the future, that gap can be closed by bridging those tooling into a Selenium WebDriver (JSONWireProtocol) format. Yes you don’t need one tool to rule them all, but one tooling format to rule them all sure would be nice. There is already things like Appium, ios-driver, etc. that bring Selenium API to mobile testing (for native, hybrid, and web apps). I don’t see why that can’t be expanded to non-mobile desktop, etc. apps as well utilizing a Selenium API but internally drives some other tool behind the scenes. Someone could retrofit a port of Selenium server or Appium server, etc. to interface to a desktop UI automaton tool. There already is such projects out there, just not very popular or gone very far, here’s one example:

    And wanted to close with mentioning that if one wanted to have Selenium support from QTP/VBScript, you can do that without simply using shell/command line calls to some wrapper to Selenium. Get more integration with calling Selenium API in QTP through these:

  34. This is all very interesting but the same things I hear from my company’s QTP fan boys who by the way do not have computer science degrees. They speak of all the tens of thousands of “test cases” programmed in QTP. I came in (one of the new Sr. QA mgrs) and designed a Python tool that auto-generates test cases using a combination of test rules coded in XML (7 variations of boundary value analysis, Cartesian products, relational algebra and decision table analysis) to produce a CSV file of multi-variable test cases that we THEN feed into ONE QTP script where even the test steps are under control of the CSV data, allowing the QTP script to vary its behavior. No need for lots of poor performing QTP scripts. Although sites like this have helped me to optimize our QTP function library, our fan boys don’t understand software design, don’t bother to benchmark QTP performance so don’t bother to research how to speed it up. So thank you Joe for taking the time to share your knowledge and experience!

  35. Completely agree, although now I’m relearning Selenium (Webdriver, actually) after having been away from automation for a while. As well as learning Java more or less for the first time. Your blog rocks by the way – of all the heaps of technical books I’ve splurged on don’t contain a single example of an assert in Java (but your blog does!) Thanks for that, Joe.

  36. Hi joe,
    Is there any book or code…for how to run the qtp scripts in Hudson/cucumber/bamboo for continuous integration tools

  37. I totally agree with you Joe. You should not be limited to one tool. But frankly speaking I worked on Winrunner, QTP/UFT, RFT, Testpartner, selenium for functional testing. I like QTP/UFT more because of ease to use. Lot of help available for any type of issue. User friendly, lot of support for technologies, good for any agile project as well and for many other reasons…QTP/UFT rocks…

  38. Hi I completed my manual testing course.i’m a fresher.then which one is good in automation tools?like QTP or Selenium.plz provide me

    • If I was starting to learn automation for the first time I would definitely start with Selenium since its a free tool. The great thing with Selenium is that you can use it with any programming language so if you are new to programming you can use a more user-friendly scripting language like Ruby or Python.

  39. totally disagree with your thoughts .their is a lot of difference in 100000$ licences and a free of cost tool and the versatility in customization which gives edge to selenium over UFT. you can get every thing in Selenium too by integrating 3rd party open source tools like sikuli for image based recognition Robot for analog steps etc

    • HA -I know you can develop anything from scratch using all kinds of open source libraries. The point is if you have to test legacy applications – for the typical test engineer – developing a tool from scratch it way outside their ability.

  40. //Never become a Selenium or QTP (or whatever company’s technology) fanboy.// True. It’s important to weigh a variety of factors like Cost, Training, Need, etc while we decide the testing tool.

  41. Yes Joe I do agree with you but in that not to depend on tool how can we build test cases/framework or test suites irrespective of tool so that it can be used in any tool?
    Example ,

    Let us say I have a test suite I QTP and it should be easily migratable to other tool say Selenium or Coded UI

  42. Joe,
    I have been following your blogs for quite some time now.

    I totally concurr with you on not beating everyone with one stick.

    The best way as the sutomation industry is focussing more on E2E rather that unit , a framwork, which could use the best of tool features and give a suite which runs through a Web UI , then DB and other applications to give me an end to end result.
    The major challenge however will be switching betlwwen the tool ( however Jenkins is doing a good job there), and a crisp reporting to show where all the test sailed through.


  43. Hi Joe

    I agree with you. I have used QARun, RFT, an in-house WATIR based tool, QTP and Selenium/Appium

    Sometimes I just write a VBS/VBA program to do a job.

    I have always been a generalist and I am not running out of work or recruiters banging on my emails…

    ps – great video on API Automation

  44. Actually, don’t be a fanboy of anything. By the way, it would be nice if there were some sort of guidelines to help people choose between tools. Do you know any such guide which compares selenium and QTP/UFT without bias ?

  45. Hi Joe,
    Actually, Literally i tried, but i am unable to find out the way to prepare that what document should i wanted to prepare differences for automation tools (Selenium, Sahi and UIPath).Could you please help me out…If can your can share to my mail. E-mail:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.