Headless Browser Testing Awesomeness Pros and Cons

Automation Testing Published on:
Headless Browser Automation


So what is this “browsing”? It actually is what it sounds like.

Headless testing is when you run a UI-based browser test without showing the browser UI. It's running a test or running a script against a browser but without the browser, UI starting up.

Why would you want to use headless browsers? There are a lot of pros and cons to following this approach. Using a headless browser might not be very helpful for browsing the Web, but for automating tasks and tests it’s awesome.

Why Should You Care About Testing with a Headless Browser?

Follow the money is such a cliché, but it’s a key indicator of what I think is a real trend and something I should pay attention to. For example, Sauce Labs just came out with a new service called Sauce Headless, a cloud-based headless testing solution.

I know the folks at Sauce are smart folks. They don’t develop anything unless they’ve gotten feedback from their users that it’s needed functionality.

I’m sure they will not be alone with their focus on headless testing.

As we shift more and more left in our software development lifecycle, we need to give feedback to our developers faster and faster. One way to do this is using some quick checks leverage a headless browser.

Automation in Software Production

If you know me at all, you also know that I’m very automation inclusive.

To me, it's not just about automation testing.

It's anything that you can automate to save someone time or effort in any part of the software delivery lifecycle–whether it's developing, quality, testing, DevOps or installation; I would refer to any of these as automation. And headless browsers are something you can actually utilize for a lot of these efforts.

Headless Browsers are Faster than Real Browsers

One definite “pro” of headless browsers is that they are typically faster than real browsers; the reason being that since you aren’t starting up a browser GUI you can bypass all the time a real browser takes to load CSS, JavaScript and open and render HTML.

I have to admit although, that it's not like exactly like night and day. But you will typically see a 2x to 15x faster performance when using a headless browser.

So if performance is critical for you, headless browsers may be a way to go. 

Headless Browser Scraping

Another advantage of headless browsers is that they can be used to scrape websites. To do this, you don't necessarily want to have to manually start up a website, however. You can go to it headlessly and just scrape the HTML. You don't need to render a full browser to do that.

For example, say your job needs some data sports statistics, or to compare prices between different sites.

Since its just data you’re looking for, it doesn’t make sense to start up a full instance of a browser; it's just extra overhead–and sometimes the less overhead you have, the quicker you'll get results back.

It may not necessarily be a test, and that's okay. Again, you want to leverage the right tools to do the right things.

I also think that headless browser scraping is not leveraged by many testers – and that's a shame.

So if you want to do some website scraping to help you with a test, later on, you won't necessarily need the overhead of starting a full-blown browser; you can utilize headless browsers to obtain that functionality for you.

Save Your Developers Time

I’m aware that a lot of developers use a headless browser for unit testing code changes for their websites and mobile apps. Being able to do all this from a command line without having to manually refresh or start a browser saves them lots and effort. For example, Rob Friesel, author of the PhantomJS CookBook in a TestTalks interview explained how his developers use the headless browser PhantomJS:

Although PhantomJs in itself is not a test framework, it’s a really good canary  in a coal mine to give you some confidence; if your tests are passing, you can have a high degree of confidence that your code is ok.

Monitor Performance With Headless Browser Scripts

Another common use is to use a headless browser script to monitor network application performance.

Some even use it to automate the rendering and screen capturing of their website images to perform layout checks in an automated fashion.

I think these are some of the reasons why Google has also developed a new headless Chrome API called Puppeteer that was designed to handle many of these developer use cases.

Headless Browser Testing Ideas

Besides the one we’ve already covered, here are some other uses for headless browsers that I’ve run across:

  • Run tests on a machine without a display
  • Setup Data
  • Testing SSL
  • Simulating multiple browsers on a single machine
  • Running tests on a headless system like Linux OS without GUI
  • Retrieve and render PDFs
  • Layout testing – since headless browsers render and interpret CSS & HTML like a real browser you can use it to test style elements.

Examples of When You Might NOT Want to Use a Headless Browser

Of course, there are a number of reasons why you may wish to use a real browser as opposed to a headless browser. For instance:

  • You need to mimic real users
  • You need to visually see the test run
  • If you need to do lots of debugging, headless debugging can be difficult.

Popular Headless Browsers

  • Google Puppeteer- The Headless Browser Puppeteer is a Node library. It gives you a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It also can also be tweaked to use full (non-headless) Chrome or Chromium.
  • Google Chrome since version 59
  • Firefox versions 55 & 56
  • PhantomJS –is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG. *This is no longer being maintained. Because of this, you might want to avoid.
  • HtmlUnit –is a “GUI-Less browser for Java programs”. It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc… just like you do in your “normal” browser.
  • Splinter – Splinter is a headless browser Python-centric option.  It's open sourced and is used for testing web applications using Python.  For example, you can use it to automate browser actions, such as visiting URLs and interacting with their items.
  • jBrowserDriver – A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec — headless, WebKit-based, pure Java

When to Use a Headless Browser for Your Testing?

So when should you use headless browsing for your testing? As you can see, it depends on your testing goals.

People on the left will often say, “Never use a headless browser. A real user would never use it so why should you?” Meanwhile, folks on the right will say, “You should always use headless browsers because they’re always faster, and faster is always better.”

As we well know, however, it’s not always one versus the other, but rather a matter of selecting the right tool for the rights task depending on the situation.

Remember–use the right tool for the job and always ask yourself how it will affect the end users and what the goal(s) of your test are when decided between the two approaches.



8 responses to “Headless Browser Testing Awesomeness Pros and Cons”

  1. “Although PhantomJs in itself is not a test framework, it’s a really good cannery in a coal mine…”

    The phrase is “canary in a coal mine”. A small bird taken into underground mines (especially coal mines) as an early warning for noxious and deadly gasses.

    Cannery is a place where stuff is put in cans.

  2. For me, headles browser (PhantomJS) is taking longer than Chrome to execute test cases.

    Html, JavaScript, images and CSS still has to be loaded by a headless browser.

  3. Re. Performance increases.

    I am really interested in headless testing, but am wondering about the real gains in performance.

    If I have a single page application that makes lots of client side API calls, rather than a server side generated HTML page, is the test still 1.5 to 2 times faster, or are we simply talking about browser interaction and rendering?

    If my test bottleneck is API requests, then I assume headless does not address this?

  4. PhantomJS was a pain when it was being maintained (not behaving like a real browser would in response to some Selenium tests) and now it is not being maintained any more is likely to be best avoided completely.
    See http://phantomjs.org/

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Headless Browser Automation