Automation Testing

Top 18 Automation Accessibility Testing Tools (Guide 2024)

By Test Guild
  • Share:
Join the Guild for FREE

What is Accessibility Testing?

Accessibility is all about making applications accessible for people who have impairments. It covers people with visual, auditory, or other cognitive impairments.

As we are developing our applications, we should keep these folks in mind as we test to ensure that our software is accessible to all our users.

Why is Accessibility Testing Important?

At least one in five people have some type of impairment, so it's very important to have them in mind when developing software. Recent figures from the Census Bureau show that 18.7% of the US population has some sort of disability and that 54% of those adults go online. From a business perspective, it makes sense for us to ensure them the best experience possible when using our apps.

Furthermore, more governments are passing regulations that make certain disability requirements for software mandatory.

What is AODA Compliance?

I first heard about AODA from Pragati Sharma who has a Test Guild course on Cypress and Accessibility Testing.

She summed it up like this:

AODA Compliance refers to adhering to the Accessibility for Ontarians with Disabilities Act. This mandates that organizations in Ontario, Canada, ensure their environments and services are accessible to individuals with disabilities. This includes web accessibility, ensuring that websites and web content are accessible to people with a range of disabilities.

The standards are similar to the WCAG guidelines, which aim to make online information and services more accessible. And this is just one example of a growing trend of accessibility based standards.

Free Course Accessibility Testing Using Cypress

 

Can Testing be Automated with Accessibility Testing Tools?

Although using an exploratory-type testing approach conducted by folks that actually have impairments will give you some of the best coverage, partial and full automation of accessibility testing is also possible.

But before you fire up your favorite accessibility testing tool, I recommend downloading the Checkpoints for Web Content Accessibility Guidelines (WCAG) to help direct your automation efforts. Keep in mind Compliance with Web Content Accessibility Guidelines (WCAG) is important, but it should be used in conjunction with user research and feedback from users with disabilities.

This list includes three types of accessibility testing tooling: manual, automation, and user acceptance testing. 

  • Manual testing involves using browsers, plug-in tools, and assistive technology. 
  • Automation testing focuses on tools like Ax. 
  • User acceptance testing involves testing with people with disabilities to understand real-world performance and find issues beyond accessibility conformance.

Now on to some tools to help you with testing accessibility.

Top Accessibility Testing Tool List

Top Accessibility Testing Tool List

While there are many free options if you work for an enterprise you probably want a more robust solution.

So lets start off with my favorite and that is the tool provided by BrowserStack

1. BrowserStack

BrowserStack is an industry-leading test platform offering manual and automated accessibility testing. Their App Accessibility product covers testing of iOS and Android apps by running scans on app screens/workflows to identify and capture accessibility issues automatically. Enabling a code flag in your SDK config file makes integrating accessibility testing into your builds possible.

With their accessibility automation, teams can monitor DOM with every build run, triggering accessibility scans wherever any changes are detected.

What important to note is the BrowserStack is already built on WCAG principles, their test engine covers ADA, AODA, Section 508, and EN 301 549 compliance.

All test reports are saved in a central repository, which allows for a quick overview of test results and further investigation with smart issue summaries.

2. Web Accessibility Checker

This first one was mentioned to me by Aparna Gopalakrishnan during a Test Guild Automation Podcast Interview.

Web Accessibility Checker is a tool designed to analyze web pages against the WCAG (Web Content Accessibility Guidelines) to ensure they are accessible to all users, including those with disabilities. These tools automatically scan and identify parts of web content that may not meet accessibility standards. Common features include:

  • Checking for proper use of HTML elements.
  • Ensuring adequate contrast ratios.
  • Verifying the presence of alternative text for images

These checks are crucial for maintaining inclusivity and meeting legal accessibility requirements.

The Web Accessibility Checker is probably the easiest way to perform accessibility checks on any ASP.NET Web application. It’s fully customizable and supports all major international accessibility standards.

3.axe-core-maven-html

axe-core-maven-html is a robust toolset designed to integrate axe accessibility testing with popular testing frameworks like JUnit, Selenium, and Playwright. This repository supports automated accessibility testing, ensuring your web applications meet high accessibility standards.

Key Features:

  • Selenium Integration: Leverage the power of Selenium for automated web accessibility testing.
  • Playwright Support: Utilize Playwright for fast, reliable accessibility testing across different browsers.
  • Automated Testing: Implement rules that ensure zero false positives, quick execution, and compatibility with modern browsers.

I really like the philosophy of this tool, which promotes:

  • Digital Equality: Automated testing plays a crucial role in achieving digital equality.
  • Mainstream Adoption: Designed to inspire trust, be fast, and work everywhere for professional web developers.

4. Axe-WebDriverJs

Axe-WebDriverJs provides JavaScript test automation engineers with a chainable Axe API for Selenium's WebDriverJS and automatically injects it into all frames.

5. Accessibility Insights

Accessibility Insights for Web is a powerful browser extension designed for Chrome and Microsoft Edge. It empowers developers to identify and resolve accessibility issues in web apps and websites efficiently. The tool offers two primary functionalities:

  • FastPass: A quick, two-step process that helps detect common, high-impact accessibility issues in less than five minutes.
  • Assessment: Comprehensive evaluation to ensure compliance with Web Content Accessibility Guidelines (WCAG) 2.1 Level AA.

Key Features:

  • Automated Checks: Automatically verifies compliance with approximately 50 accessibility requirements.
  • Tab Stops: This section provides visual aids and instructions for identifying and correcting keyboard access issues, such as missing tab stops and incorrect tab orders.
  • Manual Tests: Step-by-step instructions, examples, and how-to-fix guidance for about 20 tests, including assisted tests for identifying specific issues.

6. Guidepup

Guidepup is a screen reader driver designed for test automation. It supports VoiceOver on macOS and NVDA on Windows. With Guidepup, developers can ensure their applications are accessible, mirroring real user experiences with screen readers.

Key Features:

  • Full Control: If a screen reader has a keyboard command, Guidepup supports it.
  • User Experience: Assert on what users actually do and hear when using screen readers.
  • Framework Agnostic: Compatible with Jest, Playwright, and can run as an independent script.

7. sa11y (Salesforce Automated Accessibility Testing Libraries)

Need to do accessibility testing again, Salesforce? No worries!

sa11y (Salesforce Automated Accessibility Testing Libraries) offers a comprehensive suite of tools for integrating automated accessibility testing into various testing workflows. Built on axe-core, sa11y supports Jest unit tests, WebdriverIO component/integration tests, and more, making it a versatile solution for developers aiming to ensure their applications meet accessibility standards.

Key Features:

  • Jest Integration: Provides a toBeAccessible() matcher and integrates with the Jest assertion API for seamless accessibility checks.
  • WebdriverIO Integration: Offers assertAccessible() and assertAccessibleSync() APIs to check the accessibility of web pages rendered in browsers.
  • Flexible APIs: Use the assertAccessible API to check DOM or HTML elements for accessibility issues.
  • Results Formatter: Formats raw JSON output of accessibility issues from axe into a more consumable format.
  • Preset Rules: This includes Base, Extended, and Full accessibility preset rules as axe configurations, which can be overridden as needed.
  • Browser Testing: Provides a minified version of selected sa11y libraries for browser injection during integration testing.

8. UI Automation

Microsoft UI Automation is an accessibility framework that enables Windows applications to provide and consume programmatic information about user interfaces (UIs).

It provides programmatic access to most UI elements on the desktop. It enables assistive technology products, such as screen readers, to provide information about the UI to end users and to manipulate the UI by means other than standard input.

UI Automation also allows automated test scripts to interact with the UI.

9. Automated Accessibility Testing Tools (AATT)

The GitHub page for AATT by PayPal describes itself as a browser-based accessibility testing tool, and plugins require manually testing each page, one at a time.

What’s cool is that instead of developing, testing, and using a separate accessibility test suite, you can now integrate accessibility testing into your existing automation test suite using AATT.

AATT integrates with automation frameworks like PhantomJS, SeLion, and NemoJS.

10. Accessibility Developer Tools

The Accessibility Developer Tools (ADT) is a library of accessibility-related testing and utility code.

Its main component is the accessibility audit: a collection of audit rules checking for common accessibility problems and an API for running these rules on an HTML page.

ADT has an Audit API axs_testing.js that you can use for automation. You can also run it at the command line using PhantomJS.

For testers that want to run Selenium-based accessibility tests, they even have an example of using it with Selenium Webdriver and Scala.

11. Protractor-Accessibility-Plugin

Protractor has an Accessibility Plugin that will run each set of audits (depending on your configuration) on your existing end-to-end tests to ensure your site is free of obvious errors.

The creator of this plugin, Marcy Sutton, has written a post on how and why she created the Protractor Accessibility Plugin, along with some examples of how to use it for auditing Selenium WebDriver element objects.

12. Pa11y

Pa11y.org has a bunch of tools you can use to help automate your accessibility tests. They have a command-line interface, a browser-based dashboard for monitoring the accessibility of a number of your sites simultaneously, a web service, and even a tool that integrates with CI tools.

13. Apple’s Accessibility APIs

If you’re testing IOS apps, there are some accessibility APIs, like VoiceOver, that you can use in your scripts to help test accessibility.

Leveraging these APIs will not only make your testing easier, but it will also have the added benefit of improving your user experience.

14. Google’s Accessibility Test Framework for Android

For Android testing check out Google’s Accessibility Test Framework (GATF) has test logic that can detect a bunch of common accessibility issues. It makes use of existing Android UI constructs and is easily integrated with other tools and frameworks.

For example, you can access the GATF functionality in Espresso by enabling AccessibilityChecks.enable.

Get TestGuild Partnership Plans

AR VR Testing

If you’re into browser extensions to help you with your accessibility testing, here are a few of the more popular web accessibility testing tools you should check out:

15. Wave Evaluation Tool

You can pass the Wave Evaluation Tool to your URL, and it will tell you what accessibility practices you might be missing or are not optimized for, along with a summary of errors and alerts. It provides visual feedback about the accessibility of your Web content by injecting icons and indicators into your page.

16. Color Contrast Analyzer

The Color Contrast Analyzer extension allows you to check for text color contrast problems on a Web page according to the WCAG 2 text-color-contrast requirements.

It evaluates the page as it appears in the browser, so it’s able to handle text over gradients and advanced CSS attributes. You can choose to analyze a portion of a Web page, the entire visible contents of a tab, or an entire Web page.

17. ChromeVox

The ChromeVox screen reader is an extension to Chrome that brings the speed, versatility, and security of Chrome to visually impaired users.

18. Color Oracle

Color Oracle is a free color blindness simulator for Windows, Mac, and Linux. It takes the guesswork out of designing for color blindness by showing in real time what people with common color vision impairments will see.

What are some key Metrics used in Accessibility Testing?

Key Metrics used in Accessibility Testing are quantitative measures that assess various aspects of website accessibility.

These metrics play a crucial role in evaluating the level of accessibility, tracking progress, identifying areas for improvement, and ensuring compliance with accessibility standards.

Some of the key metrics include:

  • Error Density, which measures the frequency of accessibility errors on a website
  • Compliance with WCAG Levels, which evaluates the website's adherence to the Web Content Accessibility Guidelines
  • Unique Issues, which identifies specific and distinct accessibility issues present on the website
  • User Impact, which assesses how accessibility barriers affect users interacting with the website
  • Keyboard Accessibility Score, which evaluates the ease of navigating the website using only keyboard controls
  • Screen Reader Compatibility, which measures how well the website is compatible with screen readers for visually impaired users.

These metrics collectively provide valuable insights into the accessibility of a website and help in enhancing the overall user experience for all individuals.

Test Guild Testing Expert Recommended Metrics and Guidelines

Also many experts on my Test Guild Automation podcast have mentioned their view over the years. For example here are a few metrics and guidelines mentioned in some episodes that can help you with accessibility testing:

  1. WCAG 2.1 Compliance: Shweta Sharma mentions that 90% of their clients require their sites to be WCAG 2.1 compliant. The Web Content Accessibility Guidelines (WCAG) are a set of recommendations for making web content more accessible to people with disabilities.
  2. Color Contrast Ratio: Shannon Lee from Kobiton demonstrates how their AI-powered tool, Nova, checks for color contrast ratios to ensure accessibility. WCAG 2.1 Level AA requires a contrast ratio of at least 4.5:1 for normal text and 3:1 for large text.
  3. Touch Target Size: Kobiton's Nova also checks for touch target size, which is important for users with motor impairments. WCAG 2.1 Success Criterion 2.5.5 requires a target size of at least 44 by 44 CSS pixels for touch targets.
  4. Percentage of Issues Found: Crystal Preston-Watson mention in here 2022 Automation Guild session that most accessibility experts agree that between 20% and 40% of accessibility issues can be found through automated testing. This metric can help teams understand the effectiveness of their automated accessibility testing efforts and highlight the need for manual testing.
  5. Percentage of Content with Captions: Pragati Sharma shares an example where a streaming company was required to add closed captions to 100% of their content within two years. This metric can be used to track progress towards making video content fully accessible.

It's important to note that the W3C states that approximately 15% of the world's population has some form of disability. This statistic underscores the importance of accessibility and can be used to emphasize the potential impact of accessibility efforts on a website's user base.

By tracking these metrics and adhering to established guidelines like WCAG 2.1, teams can ensure that their websites are meeting accessibility standards and providing an inclusive experience for all users.

What are the different Types of Accessibility Testing?

Different types of accessibility testing aim to ensure that web and mobile applications are easily usable by individuals with disabilities such as visual, hearing, mobility, and cognitive impairments. The process involves thorough evaluation of the product against recognized accessibility standards and laws, such as WCAG Compliance, to address any existing accessibility issues.

Various key types of accessibility testing include:

  • Color Contrast Testing: This involves assessing the contrast of text against its background to meet specific standards, like the minimum contrast ratio required by WCAG for different types of text sizes.
  • Text Alternatives Verification: Verifying the presence of appropriate alternative text or aria-labels for images to aid users who cannot see the visuals.
  • Accessible Rich Internet Applications (ARIA) Testing: Ensuring proper application of ARIA roles and attributes on interactive elements like buttons, form controls, and live regions to enhance screen reader compatibility.
  • Keyboard Accessibility Testing: Testing the functionality of navigating through a website or application solely using the keyboard, including checking keyboard shortcuts that enable users to interact with links, buttons, and form controls effectively.

By conducting these types of accessibility testing, developers and testers work towards creating a digital environment that is inclusive and free from accessibility barriers, thus ensuring better user experience for individuals with disabilities.

Accessibility Advocate Crystal Preston-Watson breaks down accessibility testing into three main categories:

  • Manual Accessibility Testing: This involves testing applications manually for accessibility issues that may cause problems for users with disabilities. It includes using browser and plug-in tools like WAVE, Lighthouse, or aXe, as well as assistive technologies like screen readers and switches.
  • Automation Accessibility Testing: This involves using automated tools and frameworks to test for accessibility issues. Some examples include aXe, Applitools Contrast Advisor, and Espresso (for Android).
  • User/Usability Testing: This involves testing with people with disabilities to understand how the application or site performs in the real world and to find issues beyond just accessibility conformance.

Diagram illustrating different team roles including QA, Accessibility Specialists, Designers & UX Experts, Developers, Legal & Compliance, Product Managers, Users with Disabilities, and Stakeholders.

Who should be involved in Accessibility Testing?

Thats easy – everyone on the team!

But seriously based on the insights provided by the experts I've spoken with, it's clear that accessibility testing should involve a diverse group of people to ensure a comprehensive and inclusive approach. Here's who should be involved in accessibility testing:

1. Quality Assurance (QA) Teams: QA professionals, play a vital role in ensuring accessibility standards are met. They are responsible for creating test plans, executing tests, and reporting issues.

2. Accessibility Specialists: Experts who have deep knowledge of accessibility guidelines and best practices, should be involved to provide guidance and support throughout the testing process.

3. Designers and User Experience (UX) Professionals: As Shweta Sharma points out, accessibility should be considered from the early stages of design. Designers and UX professionals should work closely with accessibility specialists to create inclusive designs that meet accessibility standards.

4. Developers: Developers play a crucial role in implementing accessible code and fixing identified issues. They should be knowledgeable about accessibility guidelines and work closely with QA and accessibility specialists.

5. Product Managers: Product managers are responsible for ensuring that accessibility is prioritized and included in the product roadmap. They should work with the team to balance accessibility requirements with other product goals.

6. Users with Disabilities: As emphasized by Crystal Preston-Watson involving users with disabilities in the testing process is essential. They can provide valuable insights and feedback on the real-world usability of the website or application.

7. Stakeholders: All experts mentioned the importance of educating stakeholders about the importance of accessibility. Stakeholders, such as business owners and executives, should be involved in the process to ensure that accessibility is given the necessary resources and priority.

8. Legal and Compliance Teams: Given the legal implications of accessibility, it's important to involve legal and compliance teams to ensure that the website or application meets all relevant regulations and standards.

Accessibility testing should involve a collaborative effort from QA teams, accessibility specialists, designers, developers, product managers, users with disabilities, stakeholders, and legal and compliance teams. By involving this diverse group of people, organizations can ensure that their digital products are accessible, inclusive, and compliant with relevant standards and regulations.

Your Top Accessibility Testing Tools

Accessibility testing is an essential part of creating an effective, accessible website. While you automate accessibility testing with tools, you save time and money, as well as improve testing accuracy and reliability.

Do not let a lack of resources stand in your way any longer. Let me know if I missed one of your favorite accessibility test automation tools, and I will add it to the list.

Join the Guild

  1. I’ve recently been heavily involved with Accessibility Testing (AT) manually, using Axe, Wave and Vox. Our organisation has an Accessibility test team who are engaged when major releases need testing, but we have to factor in AT automation for each release. It appears that automated tests are not particularly good at finding Accessibility defects and that would an interesting topic to cover. But adding AT to Agile development teams is proving a challenge.

Comments are closed.

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}

Symbolic AI vs. Gen AI: The Dynamic Duo in Test Automation

Posted on 09/23/2024

You've probably been having conversations lately about whether to use AI for testing. ...

8 Special Ops Principles for Automation Testing

Posted on 08/01/2024

I recently had a conversation, with Alex “ZAP” Chernyak about his journey to ...

Top 8 Open Source DevOps Tools for Quality 2024

Posted on 07/30/2024

Having a robust Continuous Integration and Continuous Deployment (CI/CD) pipeline is crucial. Open ...