Headless Browser Testing Awesomeness Pros and Cons

Automation Testing Published on:
Headless Browser Automation


So what is this headless testing “browsing”?

It actually is what it sounds like.

Headless testing is when you run a UI-based browser test without showing the browser UI. It's running a test or running a script against a browser but without the browser, UI starting up.

Why would you want to use headless browsers? There are a lot of pros and cons to following this approach. Using a headless browser might not be very helpful for browsing the Web, but for automating tasks and tests it’s awesome.

Why Should You Care About Testing with a Headless Browser?

Follow the money is such a cliché, but it’s a key indicator of what I think is a real trend and something I should pay attention to.

For example, Sauce Labs just came out with a new service called Sauce Headless, a cloud-based headless testing solution.

I know the folks at Sauce are smart folks. They don’t develop anything unless they’ve gotten feedback from their users that it’s needed functionality.

I’m sure they will not be alone with their focus on headless testing.

Headless browser testing is a shift-left design thinking that is important for software QA.

This means the tests are automated and run in the browser without the need for any user interaction.

As we shift more and more left in our software development lifecycle, we need to give feedback to our developers faster and faster.

One way to do this is using some quick checks leveraging a headless browser.

Automation in Software Production

If you know me at all, you also know that I’m very automation inclusive.

To me, it's not just about automation testing.

It's anything that you can automate to save someone time or effort in any part of the software delivery lifecycle–whether it's developing, quality, testing, DevOps, or installation; I would refer to any of these as automation. And headless browsers are something you can actually utilize for a lot of these efforts.

Headless browser testing is the process of testing an application or website without a human user watching.

This technique has pros and cons that will depend on your particular project.

PRO: Headless Browsers are Faster than Real Browsers

One definite “pro” of headless browsers is that they are typically faster than real browsers; the reason being that since you aren’t starting up a browser GUI you can bypass all the time a real browser takes to load CSS, JavaScript and open and render HTML.

I have to admit although, that it's not exactly like night and day. But you will typically see a 2x to 15x faster performance when using a headless browser.

So if performance is critical for you, headless browsers may be a way to go. 

Headless Browser Scraping

Another advantage of headless browsers is that they can be used to scrape websites. To do this, you don't necessarily want to have to manually start up a website, however. You can go to it headlessly and just scrape the HTML. You don't need to render a full browser to do that.

For example, say your job needs some data sports statistics, or to compare prices between different sites.

Since it's just data you’re looking for, it doesn’t make sense to start up a full instance of a browser; it's just extra overhead–and sometimes the less overhead you have, the quicker you'll get results back.

It may not necessarily be a test, and that's okay. Again, you want to leverage the right tools to do the right things.

I also think that headless browser scraping is not leveraged by many testers – and that's a shame.

So if you want to do some website scraping to help you with a test, later on, you won't necessarily need the overhead of starting a full-blown browser; you can utilize headless browsers to obtain that functionality for you.

Save Your Developers Time

I’m aware that a lot of developers use a headless browser for unit testing code changes for their websites and mobile apps. Being able to do all this from a command line without having to manually refresh or start a browser saves them lots and effort. For example, Rob Friesel, author of the PhantomJS CookBook in a TestTalks interview explained how his developers use the headless browser PhantomJS:

Although PhantomJs in itself is not a test framework, it’s a really good canary  in a coal mine to give you some confidence; if your tests are passing, you can have a high degree of confidence that your code is ok.

Monitor Performance With Headless Browser Scripts

Another common use is to use a headless browser script to monitor network application performance.

Some even use it to automate the rendering and screen capturing of their website images to perform layout checks in an automated fashion.

I think these are some of the reasons why Google has also developed a new headless Chrome API called Puppeteer that was designed to handle many of these developer use cases.

Headless Browser Testing Ideas

Besides the one we’ve already covered, here are some other uses for headless browsers that I’ve run across:

  • Run tests on a machine without a display
  • Setup Data
  • Testing SSL
  • Simulating multiple browsers on a single machine
  • Running tests on a headless system like Linux OS without GUI
  • Retrieve and render PDFs
  • Layout testing – since headless browsers render and interpret CSS & HTML like a real browser you can use it to test style elements.

Examples of When You Might NOT Want to Use a Headless Browser

Of course, there are a number of reasons why you may wish to use a real browser as opposed to a headless browser. For instance:

  • You need to mimic real users
  • You need to visually see the test run
  • If you need to do lots of debugging, headless debugging can be difficult.

Popular Headless Browsers

  • Google Puppeteer- The Headless Browser Puppeteer is a Node library. It gives you a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It also can also be tweaked to use full (non-headless) Chrome or Chromium.
  • Google Chrome since version 59
  • Firefox versions 55 & 56
  • PhantomJS –is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG. *This is no longer being maintained. Because of this, you might want to avoid.
  • HtmlUnit –is a “GUI-Less browser for Java programs”. It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc… just like you do in your “normal” browser.
  • Splinter – Splinter is a headless browser Python-centric option.  It's open sourced and is used for testing web applications using Python.  For example, you can use it to automate browser actions, such as visiting URLs and interacting with their items.
  • jBrowserDriver – A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec — headless, WebKit-based, pure Java

When to Use a Headless Browser for Your Testing?

So when should you use headless browsing for your testing? As you can see, it depends on your testing goals.

People on the left will often say, “Never use a headless browser. A real user would never use it so why should you?” Meanwhile, folks on the right will say, “You should always use headless browsers because they’re always faster, and faster is always better.”

As we well know, however, it’s not always one versus the other, but rather a matter of selecting the right tool for the rights task depending on the situation.

Remember–use the right tool for the job and always ask yourself how it will affect the end users and what the goal(s) of your test are when decided between the two approaches.



Headless Browser Automation