Do you have manual tests that are not currently automated, not because they should be manual, but because you don’t have a reliable way to automate them?
I know I do.
Some of the tests I’ve struggled with automating are workflows that require email validation.
That’s why I was happy to learn about Mailosaur from Andy Dobbels at this year’s Testing Guild. This article contains what I learned from Andy’s session and live Q&A.
Before we delve further into email testing with Mailosaur, let’s look into why having some end-to-end (e2e) tests can be beneficial to your overall test automation strategy.
What is End-to-End Testing?
End-to-end testing is the closest you'll get to mimicking a real human being using your website. But if you've ever tried creating your own end-to-end test suite, you might've encountered a few hurdles along the way.
How does E2E Testing Differ from Other Types of Testing?
The goal of an end-to-end test is, unlike unit tests, to prove that the whole product works as intended, from start to finish. As you automate more and more tests, it's easy to forget that you’re trying to emulate a real person in the real world, and sometimes things don't quite go as you expect.
So, what counts as an end-to-end process?
What’s an Example of an End-to-End Test?
One example is a typical user flow when interacting with Amazon. It could be as simple as a password reset, or as complicated as a whole ordering process, where a user has to find the product, order the product, add it to their cart, and purchase it.
Why Automate End-to-End Tests?
So, why should end-to-end tests be part of your overall suite?
First, they complement your existing unit and acceptance tests by giving a more real-world overview of your components interacting with one another. Also, because they simulate real user behavior, end-to-end tests are great for regression testing during deployment, or for ongoing monitoring. They are particularly useful in production systems, as well-designed, end-to-end tests can usually be pointed at any running copy of a system.
Why is it difficult to write e2e tests?
Hurdles for E2E Tests
Most products involve multiple systems, such as email, payment processing gateways, financial and billing systems, and order fulfillment. And many of these systems are hard to automate with tools like Selenium. Also, as businesses become more and more security conscious, stronger authentication mechanisms, such as multi-factor authentication via apps, or SMS, are being introduced, and these are challenging to automate.
But if you’re a good automation engineer, you’re probably asking, “Wouldn’t these things be better handled at the unit level?”
Wouldn’t a Unit Test be Better?
One of the questions folks frequently ask is, “How much end-to-end testing do we need to do, compared to unit testing?”
The answer is that unit tests should generally form the backbone of your coverage. This is due mainly to the fact that they are laser focused by design, and can help developers and QA quickly identify low-level problems with specific portions of their code base.
End-to-end tests, on the other hand, are much higher-level and provide wider oversight of real-world scenarios. This is particularly important for business-critical processes such as accepting new subscriptions or taking payments. Smaller automated acceptance tests fall between these two pillars.
End-to-End Tests for Email
This gets us to Mailosaur.
Sometimes folks automate pieces of their E2E tests at the UI level, which renders them slower and harder to maintain. You should avoid these types of tests whenever possible.
However, many folks fail to leverage APIs when creating their workflow tests. That’s why I was excited to discover Mailosaur.
If you’re doing email testing, Mailosaur provides an API that can easily be consumed by your Selenium tests to give you the coverage you need.
Having a reliable API helps you create fast, reliable E2E tests without the overhead of an extra GUI interaction.
How does Mailosaur Work?
Mailosaur provides an API that can be programmatically accessed in most programming languages, including Java, JavaScript, C#, Python etc.
So if you have a Selenium test written in any of these languages and need to verify email, you can easily add the API to your tests to get this functionality. You also gain the benefit of not having to set up your own email and SMTP servers for testing.
Is this Better than In-House Email and SMTP Servers?
Sometimes when folks want to test in their development environments, they have to set up a local email server.
When you have an online application, however, you might have a development environment, a testing environment a production environment.
When you get to that production environment, you can't simply use your company’s email server anymore; neither can you use a local SMTP server, because you need real email addresses that you can send to via the Internet.
Mailosaur has real SMTP servers, along with an infinite number of email addresses that you can use without the need and trouble of setting them up yourself.
Unlimited Email Addresses are Cool!
If you are using your company’s internal email address, you are limited to that e-mail address.
If you want to test your own products by requiring people to create an account and enter their email addresses, you’ll invariably get stuck.
For example, a typical user signs up consists of them clicking on Sign Up, after which they get a link sent to their email. They then click on the link to verify their email address and then finally into your product.
Say, for example, you want to test what would happen if a hundred users signed up concurrently to your product. You're going to be stuck with that one company email address that you've been using thus far, or maybe the 20 that somebody set up for you. With unlimited emails, you can create more realistic scenarios.
Sounds pretty cool to me.
Mailosaur at the Testing Guild
So if you’re searching for a solution to help you test email, check out Mailosaur. For a full demo or to hear Andy’s full Q&A, consider purchasing an after-event Testing Guild ticket and you’ll have access to all the recordings from all the awesome speakers who participated in the 2018 event.
