I believe automation engineers are living in a time of great change and API testing is a must for any serious automation testing strategy.
Testers need to stop focusing all their efforts on brittle hard to maintain UI automation tests.
Agile and Continuous Testing (CI/CD) has disrupted standard test automation practices, causing test engineers to rethink how automation is done.
APIs and API testing are needed if you want to succeed with any test automation efforts.
I wrote this way back in 2014 but it still holds true today.
This is a long updated guide, so here's an overview of what we're going to cover:
INDEX
In a previous post, I've covered how No longer is the GUI king.
Today’s automation engineer needs to burrow deeper underneath the GUI to the API level.
There are many reasons why API testing is important:
- More test coverage
- Speed of execution
- Continuous Integration and delivery need fast feedback
- Internet of Things (iOT)
Before we get into API testing tools and techniques let take a quick look at what is an API?
What is API testing?
Application Programming Interface (API) is a specification that acts as an interface for software components. While most functional testing involves testing a user interface like a web page or a .NET form, API testing involves bypassing a user interface and communicating directly with an application by making calls to its APIs.
API testing allows the user to test headless technologies like JMS, HTTP Databases, and Web Services.
For example, many modern web applications business logic lives in a backend API.
When interacting with the frontend, calls are being made to these backed microservices.
So APIs allow different parts of your application to talk to each other.
Free Hands-On Guide to API Testing Course
What is Headless Testing
Most headless testing consists of bypassing the GUI and sending a request directly to an application’s backend or service and receiving a response back while validating the response to ensure things are working as one expects them
The above example is often referred to as the client/server relationship. A client makes a request by asking for a resource, and the request goes out to find the server that will fulfill the request. The server locates the desired resource and then sends a response back to the client.
Why is API Testing Important?
With Agile development becoming the standard in most organizations, the ways in which we develop software and automate tests have changed dramatically.
Pre-Agile, most of the time spent on automation was done against a graphical user interface (GUI). This is the piece that tools like Selenium and UFT/QTP handle.
But if you've been doing automation for any length of time you know how time-consuming, fragile and hard-to-maintain these types of tests are.
Entire books have been written on how organizations have invested large sums of money into creating custom functional GUI test automation frameworks, only to become frustrated with their reliability over time; to the point where people stop using it and the software system becomes shelf-ware.
Also, GUI tests that go against a user interface tend to take a long time to run. For certain Agile practices like continuous builds, when new code is checked in, the amount of time it takes to receive feedback from a GUI regression suite of tests is unacceptable.
What are the benefits of API testing?
API testing offers a wide range of benefits crucial for ensuring the reliability and performance of software applications that rely on APIs. Key advantages include enhancing test coverage and efficiency, guaranteeing functionality, performance, security, and compatibility.
By frequently running API tests as part of DevOps processes or CI/CD pipelines, development teams can streamline their release cycles and focus on core aspects of software development.
Additionally, the shift-left approach of API testing allows for early issue detection, leading to faster bug fixes and overall improved efficiency in identifying performance bottlenecks and scalability issues. Moreover, the support for various programming languages and the provision of clear documentation for API usage promote better collaboration among development teams and help in ensuring data security by identifying vulnerabilities and implementing necessary security measures.
But one of the biggest benefits to me is faster feedback.
API Fast Feedback
In those cases, quicker feedback is needed.
The sooner bugs are found the better since a developer instantly knows the code changes they made have broken the build and need to be looked at. In test-driven processes, users need a large percentage of test sets to run fast and frequently and must be able to integrate them into the development lifecycle.
Don't get me wrong — GUI testing is still very important; it's the only test type that truly tests how a user will experience an application during production. Certain defects can only be caught by GUI tests. In other words, while it's crucial, GUI should not be the only type of automation a user focuses on, nor should it be the largest piece of the total amount of automated tests that one creates.
All the reasons I just listed have given test automation a reputation for being unreliable and not worth the effort to create. Right now you may be thinking, “This stinks! Isn't test automation a core agile practice?!?” No worries! Luckily, the type of automation Agile focuses on is Unit testing (and the more reliable API lower-level testing) and less on GUI automation.
What is the API Testing Pyramid
In his book Agile: Software Development Using Scrum, author Mike Cohn introduces the concept of a test automation pyramid:
This image represents the opposite of the way most non-agile development teams perform automated testing.
What is a GUI Test
GUI testing focuses on testing an application user interface to ensure that its functionality is correct. IDE UI tests are at the top of the pyramid and represent a small piece of the total number of automation test types that should be created.
What is a Unit Test
Unit testing makes up the largest section of the pyramid, forming a solid base. A unit test is created to verify a single unit of source code, like a method. By doing this, developers can isolate the smallest testable parts of their code. Unit tests are the easiest to create and yield the biggest “bang for the buck.” Since unit tests are usually written in the same language that an application is written in, developers should have an easy time adding them to their development process.
What is an API Test
The middle Service layer is the “sweet spot” that tools like Rest-Assured and Postman were created for.
Service testing is also known as integration testing. Integration testing focuses on verifying that the interactions of many small components can integrate together without issue. Since API tests bypass the user interface, they tend to be quicker and much more reliable than GUI tests.
And most importantly — since different types of API testing don’t rely on a UI to be ready, they can be created early in the development cycle. (Bonus: API tests — as I will try to demonstrate in the example in this guide — are actually much easier to create than GUI tests!)
What is an API Load Testing
Another benefit of API testing is that you can leverage the same functional API automated test to also use in your performance testing efforts. When I spoke with the folks at BlazeMeter, they mentioned that they found a significant chunk of their customer population was using JMeter to run load tests that had exactly one user in them and maybe even one iteration.
This gave them an idea to create what they call API functional testing.
The basic idea is that you're doing performance testing using a service like BlazeMeter but before you run for example a large scale load test against your API you want to make sure that it actually works at all. So you want to do the functional tests first and then move on to the performance tests.
So API scripts are a big plus in a performance testing workflow.
Also it's quite common to use performance testing to help with automated API testing. Here is an example of using LoadRunner to test a web service
How to automate API testing?
To automate API testing effectively, start by selecting specialized tools such as Postman, Insomnia, or <Person>Swagger</Person>. These tools provide user-friendly interfaces for creating and managing API test cases. Once you have chosen a tool, begin by defining the API endpoints, request parameters, and the expected responses in your test scripts.
These scripts serve as a blueprint for your automated tests.
Next, integrate your test scripts into an automated testing framework. By using frameworks like Selenium, Robot Framework, or JUnit, you can schedule and execute your API tests automatically. This automation not only saves time but also ensures that your tests are consistently run across different iterations of your software.
Finally, remember to regularly update your test scripts to keep pace with changes in your APIs.
By following these steps and leveraging the right tools and frameworks, you can streamline the process of API testing and improve the overall quality of your software.
How to Choose an API Testing Tools
There are many tools you can use to help you with your API Testing automation efforts.
Many times I get the question from test engineers asking if they can use Selenium to test an API. The answer is no.
Since Selenium is just for browser-based testing, you may be wondering which tool to use for Rest and Soap web service-based testing?
For a full list of awesome API test tools check out my post on 20 Open Source API Testing Tools For REST & SOAP Services
How to Test Web Services
I’m often asked the question, “How do I go about testing web services?” The answer I always give may surprise you — the same way you would test any other application! In general, the best approach to normal functional testing is the same for web services (except for the fact that unlike most other applications, web services don’t have GUI user interfaces).
So take comfort in knowing the functional testing techniques that you have always used still apply. Simply think of a web service as a business process without an IDE, and write your test case accordingly.
Some good questions to ask when automating a web service:
- Does the service respond with the correct values?
- Does the behavior match the end user’s intended requirements?
- How quickly does the service send a response to the user?
- Can the service handle expected and unexpected user loads?
- Can the service handle invalid values and exceptions caused by bad data?
Web Service Testing Terminology
The biggest hurdle for most testers is acclimating to the terminology used when talking about web services.
For instance:
What is XML
XML is a way of creating a markup language with which you can define your own tags. XML allows the user to share structured data with numerous systems, including over the internet.
What's a WSDL
WSDLs are an XML format that tells you how to access a web service. (Most test tools read in a WSDL and present all the information you need to interact with it.)
What is SOAP
SOAP (Simple Object Access Protocol) is a protocol that uses XML format to exchange info to and from a Web service.
What is SOA
SOA (Service-oriented Architecture) is a way in which companies can organize software that can be quickly changed to respond to the requirements of the marketplace.
Web Services are small units of software that run in a network. They are typically written to handle a specific business process. Web services can be strung together in multiple ways and used by different applications to create the desired functionality.
What is REST or Restful API
REST (Representational State of Transfer) is a lightweight option for developing a web service that uses the HTTP protocol — a fact that makes it simpler with less overhead than a web service that uses the SOAP protocol.
I believe that once the above terms are demystified, the job of testing web services is pretty straightforward. I also feel the best way to demystify something is to break it down into simple, hands-on examples — which is the approach I will take in this book. Now, let’s take a look at calling a simple SOAP Web Service provided by webexerviceX.net.
How to Create a SOAP Web Service Test
A WSDL is one of the most important pieces for testing a SOAP-based service. WSDLs are basically a set of definitions that actually define a contract that a web service uses.
The W3C’s specification on WSDL describes it this way: “WSDL is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information.”
In this example, we'll be using the HolidayWebservice web service’s WSDL, provided by holidaywebservice.com, which will allow us to easily look up which countries celebrate what holidays.
For this first API testing tutorial example, we'll make use of an online app to call our web service.
- Navigate to https://wsdlbrowser.com/
- In the WSDL URL enter: http://www.holidaywebservice.com//HolidayService_v2/HolidayService2.asmx?wsdl
- Under functions click on GetHolidaysAvailable
- Under Request, XML change the <countryCode> from Array to Scotland
- Click the Call function
- The results of the response should return
What is an API WSDL Response
Our HolidayWebservice web service is the most typical kind, in that a client (in this case our test tool) sends a request to the service and waits. The service then processes the request and sends a response.
You might ask, “How does this communication take place?”
Basically, SOAP, which is an XML-based protocol used for communicating with a Web Service, sends info to the request using the HTTP protocol. If we look at the response we received in our test, we’ll see SOAP elements such as SOAP Envelope, Header, and Body.
How to Verify a WSDL Response
Of course, what's the point of creating a test if you can't verify the results. There are many ways to do this depending on what test tool you are using.
For example, here is how you would do it using SOAP UI. Although the video is older it should give you an idea on how to verify a WSDL response. In SoapUI terminology this is called an assertion.
What are SOAP Message Elements
The SOAP XML message is made up of three elements:
1) SOAP Envelope – The SOAP Envelope is always the top element in the message.
2) Header – the Header is optional and is the first child element to appear after the envelope. Headers can contain different types of application-specific information like security authentication or session management info.
3) Body – Sometimes referred to as the payload, the Body contains the actual message that shows the information for the recipient of the message. The UFT Results Viewer always shows the SOAP XML response that was returned from a Web Service. If you want to learn more, W3C.org contains a detailed specification document for the Simple Object Access Protocol 1.1 Before we look at more SOAP or REST services, we should take a quick detour to look at the underlying base for what most services use to transport messages – HTTP.
What is HTTP
HTTP is a communication protocol that transports messages over a network. HTTP is also known as a stateless protocol because each request that it makes is independent of all previous requests.
Cookies are used to keep track of state from a previous request for a session. Cookies are files stored on the client that have information added to them from HTTP header information. When requests are made to a website that a user has already visited, the info stored in the cookie is sent back to the browser. In this way, the website is able to remember a user’s previous activities.
Understanding HTTP will give us a good foundation for understanding the majority of your API testing tool functionality.
For the next few examples, we are going to use the mostly free Insomnia REST client. Follow the straightforward install instructions https://insomnia.rest/download/
How to make HTTP Request and Create an Automated Test
- In Insomnia choose the New Request option
- Name the request HTTPDemo and choose the GET option
- Under the GET field enter: https://testguild.com
- Click the send button
- It should return the status code of 200 OK
- To look at the raw data click on the Timeline tab
What are HTTP Requests
HTTP client requests consist of three main parts. They are:
1) Request line (HTTP Method) – this tells the server what type of request is being made. In the above example, we made a GET request, but there are many more that you could use, depending on the type of request you need to make. The HTTP method has the following options (the first four methods are the most common):
| API Testing HTTP Methods | Meaning |
|---|---|
| GET | retrieve data from a specified source |
| POST | sends new data to a specified source |
| PUT | updates info for a specified source |
| DELETE | removes data from a specified source |
| TRACE | asks that proxies declare themselves |
| OPTIONS | ask for info about options available on the server |
| HEAD | similar to a GET request, but only sends info about a document |
| CONNECT | used when a client has to use an HTTPS server |
2) Header – contains additional information to send to the server like Browser, OS, Accept and Cookie info. The different types of headers are:
- General — an optional header that contains information such as the current time.
- Request — provides more information about the client to the server.
- Entity — contains specific information about a document being sent, like length and encoding schemes.
3) Body – contains data to use for the methods that require it, like the PUT method. The Get method, as seen in our example, is empty.
The response that was returned from the server also contains three sections, like what we looked at for the HTTP request:
- A response line (status code)
- Header info
- A body which contains all the text from the response
What are HTTP Status Codes
In our example, the status code was 200, which meant everything was OK. The status code will vary depending on what happened with the original request.
The status codes that can be returned from a server are:
- 1xx – Responses in the range of 100-199 mean that the server is working on the request. 1xx codes are un
- 2xx – Responses in the range of 200-299 mean that the request was successful.
- 3xx – Responses in the range of 300-399 mean that the request was not performed– further action is needed.
- 4xx – Responses in the range of 400-499 mean that the request was incomplete and may need more info.
- 5xx – Responses in the range of 500-599 mean that the server encountered an error.
How can error conditions be tested in API testing to ensure proper handling?
Error conditions in API testing can be thoroughly tested by intentionally triggering scenarios such as sending incorrect requests, simulating server errors, inducing timeouts, and encountering other exceptional situations.
By doing so, testers can verify if the API responds appropriately by returning the correct error codes, displaying meaningful error messages, and providing suitable responses in each case. This ensures the API can handle errors effectively and communicates clearly with users when issues arise.
Testing error conditions systematically helps identify vulnerabilities and improve the overall robustness and reliability of the API.
What is a REST API?
REST (Representational State of Transfer) is a lightweight option for developing web services using the HTTP protocol — a fact which makes it simpler with less overhead than a web service that uses the SOAP protocol.
When an API follows REST architecture, it is called a REST API. When a service is designed around a REST standard, it is said to make the service “RESTful”. The main requirements for a web service to be considered RESTful are that it must:
- Have a separate client and server
- Use HTTP
- Be stateless
A REST API is made up of a bunch of resources; this is called a resource model and it makes use of Uniform Resource Identifiers (URIs). The URI syntax allows you to specify a query that returns the information you want from a REST API. The main elements of a REST system are:
- Resources are the request from a client of what it wants to get from a host — like a web page or a database record.
- A resource identifier is a URI which names the resource.
- Representations are when a server sends a response with the resource in a finished format.
Rest API Testing (How to Create a REST API Test)
For our REST test example, I'll be using the cool Star Wars REST API from swapi.co
- Create a new project in Insomnia
- Call it RestDemo and choose type GET
- In the GET field enter: https://swapi.co/api/people/10
- Click send
Under the preview pane notice rather than the website view we had in our HTTP example we instead get back JSON.
The results of the Star Wars API might look strange to you.
Thus far, we’ve only worked with services that return XML; now, we’ll get our first look at what it’s like to work with a service that returns JSON.
Maybe it’s me, but every time I hear the word “JSON,” rather than enjoying a feeling of restfulness, a terrifying image of Jason from the horror movie Friday the 13th pops into my head.
But in actuality, JSON is just another harmless technology acronym, and you’ll soon see it’s nothing to be frightened of.
How to write test cases for API testing?
When writing test cases for API testing, it is crucial to start by outlining the functionalities and endpoints of the API you are testing. Understand the input data required by the API and what outcomes are expected from various operations.
A robust test case should cover a range of scenarios, including positive and negative inputs, error handling mechanisms, and boundary conditions, to ensure the API behaves as expected under different conditions.
Detail the specific request parameters that need to be included in the test case, along with the API method being called and the expected response from the API. It is important to ensure that the test cases are clear, concise, and consistent in structure and format. By methodically designing test cases that cover a wide range of scenarios and clearly defining the expected outcomes, you can effectively test the functionality and performance of the API.
What is JSON?
JSON stands for JavaScript Object Notation and was designed to be a lightweight data interchange format. JSON is definitely becoming more popular and is now replacing XML in certain situations for API data exchanging. The site www.json.org describes how JSON is built on two structures:
- “A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
- “An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.”
So far we’ve seen how to do a standard GET.
Next, let’s take a look at how to use REST to actually POST data using JSON.
How to Validate a Rest API Test
Once again as we talked about earlier there are many ways to validate a REST response. It all depends on what test tool you are using. Also, the choice of tools can range from just libraries used in a programming language like rest-assured for java or a full-blown vendor API test tools from a vendor like Microfocus UFT API.
For example here is how you would create a rest API test using an open source solution like SOAP UI.
Here is an example of testing a REST API using Karate using a BDD like approach.
For a more programmatic way of testing a REST-ful web service here is an example using rest-assured. Rest-assured is a fluent Java library that you can use to test HTTP based REST services. It was designed with testing in mind.
- Getting Started with Rest Assured
- Testing JSON REST Service with rest assured; Given, When, Then
- Using Rest Assured to POST to a JSON REST Service with rest-assured
Free API Testing Course
This is a long post and some folks would prefer a video instead. If this is you make sure to check out our Hands-On Guide to API Testing with Karate course.
AS you have seen automated testing of APIs is essential for teams to implement robust, reliable and scalable software. However, there is a misconception that API testing is difficult to implement, and teams often don’t know where to start.
This hands-on course introduces API testing using the Karate Framework – a powerful tool designed to implement all types of test automation (including API testing) in a simple and expressive non-technical syntax.
What are the common challenges encountered in API testing?
Common challenges encountered in API testing encompass various aspects that testers often face while ensuring the functionality and reliability of APIs. Some prevalent challenges include overlooking response time assertions, neglecting API dependencies, failing to validate data accuracy, not testing all parameter request configurations comprehensively, and struggling with API call sequencing. One common challenge in API testing is excluding response time assertions.
While testers may focus on checking status codes and response content, they sometimes overlook the importance of assessing the response time of API requests. Failure to consider response time can impact end-user experience, as slow responses may deter customers and affect the application's overall performance. Another challenge is not including API dependencies in the testing process.
Testing only the API without integrating necessary dependencies can lead to incomplete testing scenarios, potentially resulting in failures when the application interacts with external services. Ensuring that all API dependencies are incorporated into the test plan is crucial for comprehensive testing and accurate assessment of system functionality.
Validating data accuracy is also a significant challenge in API testing. Even if API tests return successful results, verifying that the APIs are returning the correct data in their responses is essential. Ensuring the accuracy of data returned by APIs involves checking parameters such as response codes, HTTP headers, and data formats like JSON or XML to guarantee the integrity and reliability of the API.
Additionally, testing all relevant parameter request configurations poses a challenge in API testing. APIs interact with various data values and parameters, making it crucial to test different combinations of parameters to uncover potential faults or inconsistencies. Failing to test all possible parameter configurations can lead to overlooked issues that may affect the functionality and performance of the API in real-world scenarios.
Lastly, API call sequencing challenges QA teams in API testing. Proper sequencing of API calls is essential for ensuring the correct flow of operations within the application. Failing to maintain the sequence of API requests can lead to errors or unexpected behavior in the application.
Visual aids such as flowcharts can help understand and organize the sequence of API calls, enabling developers to align API requests effectively with the application flow. Addressing these common challenges in API testing is essential for ensuring the reliability, performance, and functionality of APIs in diverse software environments.
By recognizing and overcoming these challenges, testers can enhance the quality and usability of APIs, ultimately improving the end-user experience and the application's success.
Next Steps
As I stated at the beginning, this was not meant to be an exhaustive study of API testing automation; my goal was to give you enough information to help you quickly get started with API testing and creating your own automation awesomeness.
I hope I achieved that goal.
If you feel that anything essential let me know.
