Test Automation Trends for 2015

Test Automation Published on:
test automation trends

Well, it's the beginning of a New Year! Time to set goals and make some New Year's resolutions.

Around this time of year, I always take a quick pulse of the technology trends that are going on and how they might impact me in the coming year.

This helps me to focus on any gaps in training that I might need to fill in order to be effective at my job, and to keep my skills up-to-date and myself employable.

This is the same process I outlined last year in my post Test Automation Trends for 2014 – How to Keep You Skills Up-to-Date. I decided to make this a yearly post since I've gotten a few request asking me to do a 2015 update.

Before we get into which skills to work on this year to make yourself an in- demand test engineer, I want to go over how I did with my 2014 forecast — and make a few new ones for this year.

(NOTE: I don’t claim that this is a scientifically accurate method but rather a quick and dirty way to “guesstimate” what skills I should focus on.)

How did I do last year with Automation Testing Trends?

One of the biggest trends I saw last year was that vendor tools like HP's test tool suite (QTP/UFT) were on a downward trajectory. Selenium appeared to be the clear winner for the top must-learn automation tool, and I believe it will continue to become the must know web based automation tool in 2015.

Looking at Google Trends and selecting the Forecast option shows an upward trend in search popularity with some small dips.


Where I think I was wrong in 2014 was when I said that that tool vendor tools like HP Unified Functional Testing were slowly dying off.

While I don't see vendor- based tools making a comeback anytime soon, I do think we will start to see a new twist and new life breathed into vendor- based test tools in 2015.

Vendor-based Tool Embrace Open Source Tools

During the coming year, I think we're going to see more and more vendor- based test tools embrace open-source tools like Selenium. A great example of this is that in the last few weeks of 2014, SmartBear announced that their latest release of TestComplete 10.5 would support Selenium.

With TestComplete 10.5, you can now call Selenium-based tests from within TestComplete and have all your test results in one test report. Cool!

If I were to break out my 2015 crystal ball, I would definitely see other test tool vendors doing the same. For example, there's no reason I can think of that will stop HP from coming out with something similar that will allow their tools to integrate with Selenium as well.

I don't work for HP, and I don't have any insider knowledge that this is in the works, but I just can't see why HP UFT couldn't have functionality that allows it to easily integrate with Selenium.

Selenium W3C standard

The main reason I think this is going to happen is that in Episode 28 of TestTalks I interviewed Jim Evans, the main developer behind the IE Selenium Driver and .Net bindings, and we talked about the Selenium WebDriver W3C standard.

W3C is basically the standard that all companies follow when they are doing development. One of these standards is for the Selenium WebDriver implementation. Once it's a standard, it will give browser vendors a specification they can use to implement as part of their browser delivery — giving them a web driver that is part of their browser!

So browser vendors like Chrome, Firefox and IE will be supporting their own browser-based implementation of Selenium webdriver out of the box.

Once that occurs, then all the tool vendors will have to support Selenium because it's no longer just their open source tool; it's actually going to be a W3C standard that if you're a tool vendor you will have to support it.

In addition to vendor-based tools embracing open-source technologies, I also predict that more and more development/testing teams will begin embracing headless testing.

Headless Testing

I've been talking about the importance of headless testing for a while now, and I expect it to become even more important in 2015 (and beyond).

When asked what their top automation best practice was, almost every person I spoke to last year on Test Talks pointed to the test pyramid, and the fact that we should avoid having too many UI-based tests.

Instead, we should have more unit and integration tests and less GUI tests.

This is an area that we as automation engineers should be guiding our teams towards. The more UI tests we have, the harder it's going to be to create reliable, maintainable test automation test suites.

It follows, then, that the more you know about xUnit frameworks and REST- based testing library tools, the better equipped you will be to handle this change.

In addition to the test pyramid, another reason for more headless type testing is that the very types of applications companies will be creating are going to be increasingly centered on APIs.


Think of things like NEST, which allows you to remotely monitor your thermostat by using a sensor to communicate information over the internet using http/https.

I think we will see an increase in more and more devices that don't have traditional user interfaces in terms of software development.

IoT – Internet of Things

We are living in an ever-connected world with all kinds of things connecting to the internet. For example, check out the large growth of the term “internet of things” in Simply Hired and Google trends:



I think this is another indication that soon all kinds of crazy things are going to have sensors embedded in them. For an example, check out this video of David Rose, the author of Enchanted Objects: Design, Human Desire, and the Internet of Things.

Also, with more and more devices now connecting to the internet, think of all the possible security issues!

Security Testing

You don't need to look very far to see some recent event in the news exposing the hacking of some high-profile companies.

In August of 2014, Apple's iCloud was hacked into, and the hackers were able to get a hold of a cache of private photos. Many of them were celebrities' intimate photos that were then released to the world on all the websites.

Even more recently, someone hacked into Sony and released some very embarrassing private email communications between some studios big wigs and other sensitive employee data.

I see the need for heightened security being another 2015 trend. More and more companies are going to be focusing on it because they won't want to make headlines for being hacked or having poor security. Companies take notice when other companies are embarrassed this way, and because of this I see more and more organizations jumping on the security bandwagon this year.

What is really scary is that I recently took the Pluralsight course How to Hack Your API-Security Testing by Troy Hunt, and learned just how easy it is hack your API using simple tools like Fiddler.

Also, the Internet of Things sensors we talked about earlier are exactly the type of APIS that can be easily hacked using some simple techniques that Troy demonstrates.

So, if you're a test engineer I see more and more opportunities in security testing in 2015.

Test Automation Skills for 2015

Now that we went over my 2015 automation predications, let's look at some possible skills to work on.

What I like to do is search for the term “test automation” in an indeed.com job search around Boston, MA to see what the requirements for most of the listed positions are.

I then make a list of all those requirements and do a search for trends. This helps me determine if my skills are up-to-date, and I then focus on the ones that are the most popular or that would help me get another position if I needed to.

I look at the first 30 or so job listings, then add some of the prerequisites to a spreadsheet. I then mark the number of times the prerequisite appears. Here is my list for test automation 2015 in the Boston, MA area.

SQL

11

Java

10

AGILE

8

Linux

8

python

8

Selenium

8

REST/SOAP/API

8

perl

6

performance testing

6

javascript

4

BDD/Cucumber

4

C#

3

Git

3

Continuous Integration

3

mobile iOs/Android

3

C

2

ruby

2

jMeter

2

QTP

2

jUnit/TestNG

2

Maven/Ant

2

Hadoop

1

Cassandra

1

Spring Hibernate

1

RDBMS

1

Scala

1

Robot

1

Groovy

1

 

Next I look for each perquisite to see how it is trending in Google and Simply Hired trends.

Prereq # Google Trends SimplyHired Trends
SQL

11

down

-19

Java

10

down

-6

AGILE

8

UP

-23

Linux

8

down

-16

python

8

UP

-3

Selenium

8

UP

-18

API(REST,AWS)

8

UP

29

perl

6

DOWN

-27

performance testing

6

FLAT

-21

javascript

4

DOWN

-12

BDD/Cucumber

4

UP

0

C#

3

DOWN

54

Git

3

UP

37

Continuous Integration

3

UP

-21

mobile testing

3

UP

-3

C

2

FLAT

54

ruby

2

UP

-9

jMeter

2

FLAT

-25

QTP

2

DOWN

-48

jUnit/TestNG

2

DOWN

-31

Maven/Ant

2

UP

-25

Hadoop

1

UP

2

Spring Hibernate

1

FLAT

-40

Scala

1

FLAT

-12

Groovy

1

DOWN

-42

Appium

1

UP

490

Next I take a look at the top ten perquisites and see how my current skills compare and what my weaknesses are. This tells me that if I was ever laid off or had to change jobs, what skills I should work on in the new year. Luckily, most of my current skills and what I'm using every day match up with the top prerequisites — with a few exceptions.

I was surprised by how many local companies listed SQL as a prerequisite. I know SQL, but I would not call myself a SQL guru –so that's one skill I might work on this year. I'm also not that strong in Linux or Perl so those are two other skills I might work on polishing up in 2015.

I also look for future trends, or x-factor skills. Those are the ones that might not have appeared in my prerequisites, but that are nevertheless growing in popularity. Those skills are the ones that I will definitely work on; I'm going to learn APPIUM this year.

Testing Skills for Current Position

The last thing I do is look for keywords that the group I'm in uses and look at their trends. For example, we started doing continuous integration a few years back and it's a subject that comes up quite often in team meetings. So taking a look at some of these keywords turns up the following:

Prereq SimplyHired Trends
Jasmine

94

Puppet

22

Chef

-12

Protractor

29

AngularJS

981

Karma

920

Looking at this, it appears as though researching most of these technologies and working on projects that use these technologies can only help me in my current position. These are the skills that I will noodle with.

What does Gartner Say?

The very last thing I look at is the yearly Gartner top 10 technology trends report to get an overall picture of what the overarching trends might be outside of test automation.

Their top trends for 2015 are:

  • Computing Everywhere
  • The Internet Of Things
  • 3D Printing
  • Advanced, Pervasive and Invisible Analytics
  • Context-Rich Systems
  • Smart machines
  • Cloud/Client Computing
  • Software-Defined Applications and Infrastructure
  • Web-Scale IT
  • Risk-Based Security and Self-Protection

After reading the Gartner article and looking at their top 2015 trends it's funny how well my lower-level test automation skills/trends line up with their big-picture trends.

This at least tells me that the technologies I'm dealing with will be in demand regardless of the industry I work in — or if I have to switch careers completely.

TopThreeAutomationTrends

Automation Awesomeness for your ear buds

If you want to hear more, check out my podcast Episode 33 Recap and Test Automation Trends for 2015, where I cover some of this in more detail.

(NOTE: I don’t claim that this is a scientifically accurate method but rather a quick and dirty way to “guesstimate” what skills I should focus on.)

20 responses to “Test Automation Trends for 2015”

  1. Joe,
    I think one thing to clarify regarding Cohn’s Automation Pyramid, and this is going to ruffle some feathers on people, is that it doesn’t push for less GUI automated tests. It in fact is showing the percentage distribution of all types of automated tests. Meaning that yes we should have a good focus on Unit Level automated tests because they improve a systems code stability/reliability. These are easy (easier) tests to automate because of the xUnit Harnesses and the focus of doing ‘assertion’ type tests at the code level. It has a natural tendency to produce more tests because you are at a more atomic level of the system.

    So when you look at it (according to Cohn) you should have about 50-60 percent of your overall automation be at the Unit test level. The GUI level is about 10-15 percent. Thus if you have 1000 Unit tests you will have 100-150 GUI level automated tests. But here again we have to look at the focus/intent of those GUI tests. They are exercising logic/business processing paths/scenarios. Thus more complex and only cover certain areas of the system versus the Unit Test level.

    So GUI automation shouldn’t be de-emphasized, but better focused to its intent and act as an additional layer to the overall automated testing approach. I met Mike Cohn years ago and talked with him about this (he is here in Denver area like myself) and he pretty much agreed with this. Just his book was already out and people gloming onto the automation pyramid concept. The Automation Pyramid isn’t and “either/or” proposition, it is an “and” one. Don’t get me wrong, I support Unit Test Automation in its principle that it pushes developers to do more to test and make their code robust and remove the stupid functionality errors we could see later on in System Test that waste our time. The more robust the base functionality is the more time I can spend on really pounding the system at the User/Business logic level (which is where Test tends to not have enough time to do and get burned by it).

    Regards,

    Jim Hazen
    Automation Mercenary

  2. Joe,

    I love to read your blog. Thanks for the service your provide and the expertise you bring to the testing community. I miss working with you. I think your predictions are well thought out. I also appreciate the methodology that you shared in how you came up with the industry technology trends and what you plan to work on for the coming year. I think I will use that same approach to identify my training needs.

    Good work my friend!

    Dave Martin

    • Thanks Dave! I wish you were still here, with you and Mike Hammond gone its a lot harder talking to other people about automation that actually get it. Are you going to any test conferences this year?

  3. Joe,

    Every time I read your posts I am only inspired to do more and keep my skills updated. Most of your predictions are bang on!!
    Currently the automation trends I am seeing here show that AngularJS/Polymer/JavaScript/Protractor and of-course Selenium in the web automation space, Appium/SeeTest in the mobile testing space, all CI tools that you can think of :) and services based testing using REST and SoapUI is where the market is heading to…

    Thanks once again and keep up the good work.
    Ady

  4. Nice read! I appreciate the mention of API as automation is moving more and more to a good blend of UI and API/SOA testing. Whether you are using API for pre/post validation of your UI testing or flat out automation of API, it is incredibly handy. We’ve been introducing more and more API style testing as a pre-req for testing UI so we don’t waste hours of time on all our consuming applications when we could have just found the functional issue centralized in the SOA requests.

    I’d say continual focus of Mobile/Mobile Web and Cross-Browser is another area to never forget as accessibility to data is becoming more and more spread out.

    Thanks for the continued posts !

  5. Hi Joe,

    Very good Post. Your post always is a check about where we stand and what to improve.

    Definitely the trend is for API. We have been moving all out Test case more towards API with 80:20 rule favoring API.

    Yes selenium/Appium and headless testing is good but but I encountered another framework call CATJS in Google Techtalk. It is awesome and have very good potential.

    Keep posting – all are very informative :)

    Regards
    Subhabrata

  6. Hi Joe,

    Very good article :)

    Can you please suggest good books that we can read and learn, for following topics :
    (Keeping in mind that the reader DOES NOT have any coding knowledge)

    1. Java Script
    2. Java
    3. C
    4. Phython
    5. QTP Tool
    6. Selenium Tool

    Appreciate your help. Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

test automation trends