I believe automation engineers are living in a time of great change and API testing is a must for any serious automation testing strategy.
Testers need to stop focusing all their efforts on brittle hard to maintain UI automation tests.
Agile and Continuous Testing (CI/CD) has disrupted standard test automation practices, causing test engineers to rethink how automation is done.
APIs and API testing are needed if you want to succeed with any test automation efforts.
This is a long guide, so here's an overview of what we're going to cover:
1. What Is API Testing?
2. What is API Testing Important
3. API Testing Pyramid
4. API Load Testing
5. API Testing Tools
6. How to Test Web Services
7. Web Service Testing Terminology
8. Create a SOAP Web Service Test
9. HTTP
10. What is REST APIs
11. How to Create a REST API Test
12. What is JSON
In a previous post, I've covered how No longer is the GUI king.
Today’s automation engineer needs to burrow deeper underneath the GUI to the API level.
There are many reasons why API testing is important:
- More test coverage
- Speed of execution
- Continuous Integration and delivery need fast feedback
- Internet of Things (iOT)
Before we get into API testing tools and techniques let take a quick look at what is an API?
What is API testing?
Application Programming Interface (API) is a specification that acts as an interface for software components. While most functional testing involves testing a user interface like a web page or a .NET form, API testing involves bypassing a user interface and communicating directly with an application by making calls to its APIs.
API testing allows the user to test headless technologies like JMS, HTTP Databases, and Web Services.
For example, many modern web applications business logic lives in a backend API.
When interacting with the frontend, calls are being made to these backed microservices.
So APIs allow different parts of your application to talk to each other.
What is Headless Testing
Most headless testing consists of bypassing the GUI and sending a request directly to an application’s backend or service and receiving a response back while validating the response to ensure things are working as one expects them
The above example is often referred to as the client/server relationship. A client makes a request by asking for a resource, and the request goes out to find the server that will fulfill the request. The server locates the desired resource and then sends a response back to the client.
Why is API Testing Important?
With Agile development becoming the standard in most organizations, the ways in which we develop software and automate tests have changed dramatically.
Pre-Agile, most of the time spent on automation was done against a graphical user interface (GUI). This is the piece that tools like Selenium and UFT/QTP handle.
But if you've been doing automation for any length of time you know how time-consuming, fragile and hard-to-maintain these types of tests are.
Entire books have been written on how organizations have invested large sums of money into creating custom functional GUI test automation frameworks, only to become frustrated with their reliability over time; to the point where people stop using it and the software system becomes shelf-ware.
Also, GUI tests that go against a user interface tend to take a long time to run. For certain Agile practices like continuous builds, when new code is checked in, the amount of time it takes to receive feedback from a GUI regression suite of tests is unacceptable.
API Fast Feedback
In those cases, quicker feedback is needed.
The sooner bugs are found the better since a developer instantly knows the code changes they made have broken the build and need to be looked at. In test-driven processes, users need a large percentage of test sets to run fast and frequently and must be able to integrate them into the development lifecycle.
Don't get me wrong — GUI testing is still very important; it's the only test type that truly tests how a user will experience an application during production. Certain defects can only be caught by GUI tests. In other words, while it's crucial, GUI should not be the only type of automation a user focuses on, nor should it be the largest piece of the total amount of automated tests that one creates.
All the reasons I just listed have given test automation a reputation for being unreliable and not worth the effort to create. Right now you may be thinking, “This stinks! Isn't test automation a core agile practice?!?” No worries! Luckily, the type of automation Agile focuses on is Unit testing (and the more reliable API lower-level testing) and less on GUI automation.
What is the API Testing Pyramid
In his book Agile: Software Development Using Scrum, author Mike Cohn introduces the concept of a test automation pyramid:
This image represents the opposite of the way most non-agile development teams perform automated testing.
What is a GUI Test
GUI testing focuses on testing an application user interface to ensure that its functionality is correct. IDE UI tests are at the top of the pyramid and represent a small piece of the total number of automation test types that should be created.
What is a Unit Test
Unit testing makes up the largest section of the pyramid, forming a solid base. A unit test is created to verify a single unit of source code, like a method. By doing this, developers can isolate the smallest testable parts of their code. Unit tests are the easiest to create and yield the biggest “bang for the buck.” Since unit tests are usually written in the same language that an application is written in, developers should have an easy time adding them to their development process.
What is an API Test
The middle Service layer is the “sweet spot” that tools like Rest-Assured and Postman were created for.
Service testing is also known as integration testing. Integration testing focuses on verifying that the interactions of many small components can integrate together without issue. Since API tests bypass the user interface, they tend to be quicker and much more reliable than GUI tests.
And most importantly — since different types of API testing don’t rely on a UI to be ready, they can be created early in the development cycle. (Bonus: API tests — as I will try to demonstrate in the example in this guide — are actually much easier to create than GUI tests!)
What is an API Load Testing
Another benefit of API testing is that you can leverage the same functional API automated test to also use in your performance testing efforts. When I spoke with the folks at BlazeMeter, they mentioned that they found a significant chunk of their customer population was using JMeter to run load tests that had exactly one user in them and maybe even one iteration.
This gave them an idea to create what they call API functional testing.
The basic idea is that you're doing performance testing using a service like BlazeMeter but before you run for example a large scale load test against your API you want to make sure that it actually works at all. So you want to do the functional tests first and then move on to the performance tests.
So API scripts are a big plus in a performance testing workflow.
Also it's quite common to use performance testing to help with automated API testing. Here is an example of using LoadRunner to test a web service
How to Choose an API Testing Tools
There are many tools you can use to help you with your API Testing automation efforts.
Many times I get the question from test engineers asking if they can use Selenium to test an API. The answer is no.
Since Selenium is just for browser-based testing, you may be wondering which tool to use for Rest and Soap web service-based testing?
For a full list of awesome API test tools check out my post on 20 Open Source API Testing Tools For REST & SOAP Services
How to Test Web Services
I’m often asked the question, “How do I go about testing web services?” The answer I always give may surprise you — the same way you would test any other application! In general, the best approach to normal functional testing is the same for web services (except for the fact that unlike most other applications, web services don’t have GUI user interfaces).
So take comfort in knowing the functional testing techniques that you have always used still apply. Simply think of a web service as a business process without an IDE, and write your test case accordingly.
Some good questions to ask when automating a web service:
- Does the service respond with the correct values?
- Does the behavior match the end user’s intended requirements?
- How quickly does the service send a response to the user?
- Can the service handle expected and unexpected user loads?
- Can the service handle invalid values and exceptions caused by bad data?
Web Service Testing Terminology
The biggest hurdle for most testers is acclimating to the terminology used when talking about web services.
For instance:
What is XML
XML is a way of creating a markup language with which you can define your own tags. XML allows the user to share structured data with numerous systems, including over the internet.
What's a WSDL
WSDLs are an XML format that tells you how to access a web service. (Most test tools read in a WSDL and present all the information you need to interact with it.)
What is SOAP
SOAP (Simple Object Access Protocol) is a protocol that uses XML format to exchange info to and from a Web service.
What is SOA
SOA (Service-oriented Architecture) is a way in which companies can organize software that can be quickly changed to respond to the requirements of the marketplace.
Web Services are small units of software that run in a network. They are typically written to handle a specific business process. Web services can be strung together in multiple ways and used by different applications to create the desired functionality.
What is REST or Restful API
REST (Representational State of Transfer) is a lightweight option for developing a web service that uses the HTTP protocol — a fact that makes it simpler with less overhead than a web service that uses the SOAP protocol.
I believe that once the above terms are demystified, the job of testing web services is pretty straightforward. I also feel the best way to demystify something is to break it down into simple, hands-on examples — which is the approach I will take in this book. Now, let’s take a look at calling a simple SOAP Web Service provided by webexerviceX.net.
How to Create a SOAP Web Service Test
A WSDL is one of the most important pieces for testing a SOAP-based service. WSDLs are basically a set of definitions that actually define a contract that a web service uses.
The W3C’s specification on WSDL describes it this way: “WSDL is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information.”
In this example, we'll be using the HolidayWebservice web service’s WSDL, provided by holidaywebservice.com, which will allow us to easily look up which countries celebrate what holidays.
For this first API testing tutorial example, we'll make use of an online app to call our web service.
- Navigate to https://wsdlbrowser.com/
- In the WSDL URL enter: http://www.holidaywebservice.com//HolidayService_v2/HolidayService2.asmx?wsdl
- Under functions click on GetHolidaysAvailable
- Under Request, XML change the <countryCode> from Array to Scotland
- Click the Call function
- The results of the response should return
What is an API WSDL Response
Our HolidayWebservice web service is the most typical kind, in that a client (in this case our test tool) sends a request to the service and waits. The service then processes the request and sends a response.
You might ask, “How does this communication take place?”
Basically, SOAP, which is an XML-based protocol used for communicating with a Web Service, sends info to the request using the HTTP protocol. If we look at the response we received in our test, we’ll see SOAP elements such as SOAP Envelope, Header, and Body.
How to Verify a WSDL Response
Of course, what's the point of creating a test if you can't verify the results. There are many ways to do this depending on what test tool you are using.
For example, here is how you would do it using SOAP UI. Although the video is older it should give you an idea on how to verify a WSDL response. In SoapUI terminology this is called an assertion.
What are SOAP Message Elements
The SOAP XML message is made up of three elements:
1) SOAP Envelope – The SOAP Envelope is always the top element in the message.
2) Header – the Header is optional and is the first child element to appear after the envelope. Headers can contain different types of application-specific information like security authentication or session management info.
3) Body – Sometimes referred to as the payload, the Body contains the actual message that shows the information for the recipient of the message. The UFT Results Viewer always shows the SOAP XML response that was returned from a Web Service. If you want to learn more, W3C.org contains a detailed specification document for the Simple Object Access Protocol 1.1 Before we look at more SOAP or REST services, we should take a quick detour to look at the underlying base for what most services use to transport messages – HTTP.
What is HTTP
HTTP is a communication protocol that transports messages over a network. HTTP is also known as a stateless protocol because each request that it makes is independent of all previous requests.
Cookies are used to keep track of state from a previous request for a session. Cookies are files stored on the client that have information added to them from HTTP header information. When requests are made to a website that a user has already visited, the info stored in the cookie is sent back to the browser. In this way, the website is able to remember a user’s previous activities.
Understanding HTTP will give us a good foundation for understanding the majority of your API testing tool functionality.
For the next few examples, we are going to use the mostly free Insomnia REST client. Follow the straightforward install instructions https://insomnia.rest/download/
How to make an HTTP Request
- In Insomnia choose the New Request option
- Name the request HTTPDemo and choose the GET option
- Under the GET field enter: https://testguild.com
- Click the send button
- It should return the status code of 200 OK
- To look at the raw data click on the Timeline tab
What are HTTP Requests
HTTP client requests consist of three main parts. They are:
1) Request line (HTTP Method) – this tells the server what type of request is being made. In the above example, we made a GET request, but there are many more that you could use, depending on the type of request you need to make. The HTTP method has the following options (the first four methods are the most common):
| API Testing HTTP Methods | Meaning |
|---|---|
| GET | retrieve data from a specified source |
| POST | sends new data to a specified source |
| PUT | updates info for a specified source |
| DELETE | removes data from a specified source |
| TRACE | asks that proxies declare themselves |
| OPTIONS | ask for info about options available on the server |
| HEAD | similar to a GET request, but only sends info about a document |
| CONNECT | used when a client has to use an HTTPS server |
2) Header – contains additional information to send to the server like Browser, OS, Accept and Cookie info. The different types of headers are:
- General — an optional header that contains information such as the current time.
- Request — provides more information about the client to the server.
- Entity — contains specific information about a document being sent, like length and encoding schemes.
3) Body – contains data to use for the methods that require it, like the PUT method. The Get method, as seen in our example, is empty.
The response that was returned from the server also contains three sections, like what we looked at for the HTTP request:
- A response line (status code)
- Header info
- A body which contains all the text from the response
What are HTTP Status Codes
In our example, the status code was 200, which meant everything was OK. The status code will vary depending on what happened with the original request.
The status codes that can be returned from a server are:
- 1xx – Responses in the range of 100-199 mean that the server is working on the request. 1xx codes are un
- 2xx – Responses in the range of 200-299 mean that the request was successful.
- 3xx – Responses in the range of 300-399 mean that the request was not performed– further action is needed.
- 4xx – Responses in the range of 400-499 mean that the request was incomplete and may need more info.
- 5xx – Responses in the range of 500-599 mean that the server encountered an error.
What is a REST API?
REST (Representational State of Transfer) is a lightweight option for developing web services using the HTTP protocol — a fact which makes it simpler with less overhead than a web service that uses the SOAP protocol.
When an API follows REST architecture, it is called a REST API. When a service is designed around a REST standard, it is said to make the service “RESTful”. The main requirements for a web service to be considered RESTful are that it must:
- Have a separate client and server
- Use HTTP
- Be stateless
A REST API is made up of a bunch of resources; this is called a resource model and it makes use of Uniform Resource Identifiers (URIs). The URI syntax allows you to specify a query that returns the information you want from a REST API. The main elements of a REST system are:
- Resources are the request from a client of what it wants to get from a host — like a web page or a database record.
- A resource identifier is a URI which names the resource.
- Representations are when a server sends a response with the resource in a finished format.
Rest API Testing (How to Create a REST API Test)
For our REST test example, I'll be using the cool Star Wars REST API from swapi.co
- Create a new project in Insomnia
- Call it RestDemo and choose type GET
- In the GET field enter: https://swapi.co/api/people/10
- Click send
Under the preview pane notice rather than the website view we had in our HTTP example we instead get back JSON.
The results of the Star Wars API might look strange to you.
Thus far, we’ve only worked with services that return XML; now, we’ll get our first look at what it’s like to work with a service that returns JSON.
Maybe it’s me, but every time I hear the word “JSON,” rather than enjoying a feeling of restfulness, a terrifying image of Jason from the horror movie Friday the 13th pops into my head.
But in actuality, JSON is just another harmless technology acronym, and you’ll soon see it’s nothing to be frightened of.
What is JSON?
JSON stands for JavaScript Object Notation and was designed to be a lightweight data interchange format. JSON is definitely becoming more popular and is now replacing XML in certain situations for API data exchanging. The site www.json.org describes how JSON is built on two structures:
- “A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
- “An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.”
So far we’ve seen how to do a standard GET.
Next, let’s take a look at how to use REST to actually POST data using JSON.
How to Validate a Rest API Test
Once again as we talked about earlier there are many ways to validate a REST response. It all depends on what test tool you are using. Also, the choice of tools can range from just libraries used in a programming language like rest-assured for java or a full-blown vendor API test tools from a vendor like Microfocus UFT API.
For example here is how you would create a rest API test using an open source solution like SOAP UI.
Here is an example of testing a REST API using Karate using a BDD like approach.
For a more programmatic way of testing a REST-ful web service here is an example using rest-assured. Rest-assured is a fluent Java library that you can use to test HTTP based REST services. It was designed with testing in mind.
- Getting Started with Rest Assured
- Testing JSON REST Service with rest assured; Given, When, Then
- Using Rest Assured to POST to a JSON REST Service with rest-assured
As I stated at the beginning, this was not meant to be an exhaustive study of API testing automation; my goal was to give you enough information to help you quickly get started with API testing and creating your own automation awesomeness.
I hope I achieved that goal.
If you feel that anything essential let me know.
