Automation Testing

Selenium AutoIt: How to Automate Non-Browser Based Functionality

By Test Guild
  • Share:
Join the Guild for FREE
Selenium Test Automation

Learn more about our Strategic Partner

Learn more about our Strategic Partner

My team is working on a project using Behavior Driven Development (BDD) and Java. Our main application is browser-based, so we're also using the Selenium WebDriver with Java; however, our application also has other thick client applications integrated into it.

Most of our automation efforts are fine when using Selenium with BDD, but issues arise whenever we need to perform actions against the thick client applications when testing certain workflows. What to do?

The best solution I've found around these issues is to use AutoIt, a freeware application designed to help automate Windows-based UIs. The normal procedure is to create AutoIt scripts using the built-in BASIC-like programming language, but that isn't an optimal solution for my team.

Instead, to still be able to develop in Java in our normal IDE we are accessing AutoIt through COM using Jacob, which is a Java COM bridge.

Also — rather than have everyone that needs to run the tests install another product — AutoIt– we can get around it simply by registering the AutoItX3 DLL.

This might sound complicated, but following the steps below should get you up and running in no time.

The Selenium AutoIt Solution

First you need to configure your machine:

  • Fire up Eclipse and Create a new Java Project named AutoIT


  • In the Windows directory where the AutoIT project is saved, create a new directory named lib

‘

  • Download the JACOB Java COM bridge:

http://sourceforge.net/projects/jacob-project/

  • Download the AutoIt Zip file option:

http://www.autoitscript.com/site/autoit/downloads/


  • Download the AutoItXJava.jar file:

https://code.google.com/p/autoitx4java/downloads/list


  • Navigate to where you placed the Jacob zip file and extract it.
  • Copy the jacob.jar, jacob-1.18-M2-x64.dll and jacob-1.18-M2-x64.dll filed and place them in the AutoIT>Lib folder we created earlier.


  • Navigate to where you downloaded the AutoItX4java.jar file and copy it into the AutoIt>Lib directory.
  • Your AutoIT >lib directory should now have these four files:
    • AutoItX4Java.jar
    • Jacob.jar
    • Jacob-1.18-M2-x64.dll
    • Jacob-1.18-M2-x86.dll


    • Create another folder under your project named tools/autoit
    • Navigate to the folder where you extracted AutoIT and copy the AutoItX3.dll and the AutoItX3_x64.dll into the tools/autoit directory.


    • In Eclipse, right click on the AutoIt project and click Refresh
    • Right click on the AutoIt project again and select Properties
    • Under the Java Build Path, click on the Libraries tab.
    • Click on the Add JARS.. button.
    • Navigate to the lib folder we created and add:
      • AutoItX4Java.jar
      • Jacob.jar


Now — so that we can use AutoIt without installing Autoit we need to register the AutoIT dll that we placed in our tools/autoit directory:

If you are using Java 32bit:

  • Click on the Windows start button and enter regsvr32 yourpath/AutoItX3.dll


If you are using Java 64bit (if you don't do this, you will get the error Can't co-create object):

  • Click on the Windows start button and enter regsvr32 yourpath/AutoItX3_x64.dll



  • You should now be able to call AutoItX methods directly from Java. Let's look at how this is done:

Use the Calculator to add two numbers

For a quick sanity check to verify that you can use ActiveIT in Java, let's create a small test that calls the Windows calculator and adds two numbers:

  • In Eclipse, right click on the AutoIt project and click Class
  • In the New Java Class window, name the class CalcTest and select the public static void main option.


  • Click finish
  • At the very top of the class, add the following imports:
import java.io.File;
import autoitx4java.AutoItX;
import com.jacob.com.LibraryLoader;
  • In the CalcTest class, under the main section, add the following code:

First, you'll need to determine whether the machine you're running this on has Java32 bit or 64bit installed. (If you know this already you won't need this code, but if you want to run on multiple machines it might make sense for you to programmatically determine the Java bit version and load up the corresponding Jacob.dll version.)

To do this, add a method under the static void main section:

/**
*
* Returns if the JVM is 32 or 64 bit version
*/
public
static String jvmBitVersion(){
 return System.getProperty("sun.arch.data.model");
}
  • Under static void main, enter:
public static void main(String[] args) throws InterruptedException {
String jacobDllVersionToUse;
if (jvmBitVersion().contains("32")){
jacobDllVersionToUse = "jacob-1.18-M2-x86.dll"
}
else {
jacobDllVersionToUse = "jacob-1.18-M2-x64.dll";
}

File file = new File("lib", jacobDllVersionToUse);
System.setProperty(LibraryLoader.JACOB_DLL_PATH, file.getAbsolutePath());

AutoItX x = new AutoItX();
x.run("calc.exe");
x.winActivate("Calculator");
x.winWaitActive("Calculator");
//Enter 3
x.controlClick("Calculator", "", "133") ;
Thread.sleep(1000);
//Enter +
x.controlClick("Calculator", "", "93") ;
Thread.sleep(1000);
//Enter 3
x.controlClick("Calculator", "", "133") ;
Thread.sleep(1000);
//Enter =
x.controlClick("Calculator", "", "121") ;
}
  • To get the Calculator button ids for the number 3 and = I used the Au3info application that is in the install directory of autoit-v3 that we downloaded and extracted in a previous step.


  • For a list of AutoIt function's available and how to use them, check out the AutoIt online documentation.
  • Right click on the AutoIt project and select Run As > Java Application
  • The Windows calculator should start and enter the values 3 + 3 = 6


More Automation Awesomeness with AutoIt

How cool is that?! Stay tuned for my next post where I will show you how to use this same technique to handle non-browser based dialogs that occasionally occur when testing web applications.

Another option to check it besides AutoIt to help with these types of issues is SikuliX. Check out my post on How to Get Started with SikuliX to learn more.

  1. Hi Joe,

    Congratulations on the success of your website.Lot of helpful articles and Testtalks.Great work.
    My question:-
    Can we use AutoIT for functional automation testing of Windows applications.
    Is AutoIT any different from WHITE framework?

  2. Thanks Malay – how you been!? I think White Framework which is basically a wrapper around the UI Automation Framework is much better and can automate more things then AutoIt. AutoIt is good at simulating keystrokes, and mouse movements and works fine for what I need to automate in my framework right now. If I was developing in .NET I would differently be using White/UI instead.

  4. Pingback: [BLOCKED BY STBV] Selenium: How to Handle Windows-Based Dialogs and Pop-Ups | Joe Colantonio - Selenium-UFT-QTP-SoapUI-ALM-LoadRunner & more

  5. Hi Joe,
    Thanks for the clear instructions. I’m now using this to deal with non browser based dialogs and it is working beautifully.
    Regards,
    Charmaine.

  7. Thank you for clear article on auto it incorporation into automated testing. I am not a developer and getting below shown errors for following lines of code:

    Your help is highly appreciated.

    import java.io.File;
    import autoitx4java.AutoItX;
    import com.jacob.com.LibraryLoader;
    public class calcTest {

    public static void main(String[] args) {

    public static void main(String[] args) throws InterruptedException {
    String jacobDllVersionToUse;
    if (jvmBitVersion().contains(“64”)){
    jacobDllVersionToUse = “jacob-1.18-M2-x64.dll”;
    }
    else {
    jacobDllVersionToUse = “jacob-1.18-M2-x64.dll”;
    }

    Errors:

    Description Resource Path Location Type
    Duplicate local variable args calcTest.java /AutoIT/src line 8 Java Problem
    Syntax error on token “(“, ; expected calcTest.java /AutoIT/src line 8 Java Problem
    Syntax error, insert “;” to complete LocalVariableDeclarationStatement calcTest.java /AutoIT/src line 8 Java Problem
    void is an invalid type for the variable main calcTest.java /AutoIT/src line 8 Java Problem

  8. Hi – you should only have one main section. Your code snippet has main twice. Should look like this:

    public static void main(String[] args) throws InterruptedException {
    String jacobDllVersionToUse;
    if (jvmBitVersion().contains(“32”)){
    jacobDllVersionToUse = “jacob-1.18-M2-x86.dll”
    }
    else {
    jacobDllVersionToUse = “jacob-1.18-M2-x64.dll”;
    }

    File file = new File(“lib”, jacobDllVersionToUse);
    System.setProperty(LibraryLoader.JACOB_DLL_PATH, file.getAbsolutePath());

    AutoItX x = new AutoItX();
    x.run(“calc.exe”);
    x.winActivate(“Calculator”);
    x.winWaitActive(“Calculator”);
    //Enter 3
    x.controlClick(“Calculator”, “”, “133”) ;
    Thread.sleep(1000);
    //Enter +
    x.controlClick(“Calculator”, “”, “93”) ;
    Thread.sleep(1000);
    //Enter 3
    x.controlClick(“Calculator”, “”, “133”) ;
    Thread.sleep(1000);
    //Enter =
    x.controlClick(“Calculator”, “”, “121”) ;
    }

  9. I have been followed the steps which you have provided and finally integrated “Autoit” with eclipse.
    Very clear steps and easily understandable.
    Thanks for the tutorial !!!

  13. Hi Joe,

    I used your ‘Code’ above. It works great. However, I am facing this problem inconsistently….the problem is:
    When ‘File upload’ dialog opens, ‘AutoIt’ script does execute.
    The scenario where I am trying it is:
    Upload an image in ‘Gmail’ mail. i.e. Compose an email with an attachment
    My Observations are:
    1. When ‘File Upload’ dialog opens, no action is performed
    2. If I manually close this ‘File Upload’ dialog and reopen it, code executes.

    My code snippet is as follows:
    AutoItX autoIt = new AutoItX();
    autoIt.winWaitActive(“File Upload”);

    //autoIt.wait(3000);
    autoIt.send(“C:\\vcredist.bmp”);
    // autoIt.wait(3000);
    autoIt.controlClick(“File Upload”, “”,”1″);
    P.S. I tried introducing ‘Wait’, but it didn’t help.

    Thanks and Regards,
    Surbhi

  14. Hi,

    Thanks for your clear information.
    In my current project, i have to automate a web application (it is an desktop app (exe)) before it was a browser application. Now for me the challege is how to automate the desktop app using Selenium ( i have to use selenium webdriver since already the scripts and framework are designed), Can you please let me know how to do this with Selenium Webdriver by using other tools to integrate to it.

  15. Nice article. I have a windows based application which has certain menus. On clicking on menu item it opens IE browser where I need to perform testing.

    By seeing above article, looks like by using autoIT I can open the windows application and click on menu but how will webdriver will be attached to the browser window which was opened by the app ?

    How can we automate such scenarios using webdriver and AutoIT ?

  16. Awesome explanation. I just tried this installing and I could do it flawlessly. Thanks a ton….

  17. Wow good question. I did not think of this scenario – I always assumed that the UI was launched from a Selenium browser instance. I don’t know if the functionality you are looking for is possible. Any one else know?

  18. Great article! Learned how to get Java COM and AutoIt in Java working.

    On a related note, you can use AutoIt through WebDriver instead of over COM over Java to integrate with Selenium. With WebDriver route, you just deal with 2 driver instances in your preferred language. Check this out:

    https://github.com/daluu/AutoItDriverServer

    and I based the calculator demo in part on your article here ;-)

    there’s a demo showing Selenium integration as well. Sorry that the demo scripts are in Python WebDriver client bindings, but one should be able to figure out how you’d map that to the Java equivalent.

  20. Hi Joe,
    Thanks for the very good tutorial. I have tried all the steps mentioned in this blog and hitting the following error message. Can you please help me to resolve the issue?

    ————————————————————-
    Exception in thread “main” com.jacob.com.ComFailException: Can’t co-create object
    at com.jacob.com.Dispatch.createInstanceNative(Native Method)
    at com.jacob.com.Dispatch.(Dispatch.java:99)
    at com.jacob.activeX.ActiveXComponent.(ActiveXComponent.java:58)
    at autoitx4java.AutoItX.(AutoItX.java:181)
    at KCalc.main(KCalc.java:21)
    ——————————————————

    Code :
    ________________________________________
    import java.io.File;

    import autoitx4java.AutoItX;

    import com.jacob.com.LibraryLoader;

    public class KCalc {

    public static void main(String[] args) {
    // TODO Auto-generated method stub

    String jacobDllVersionToUse;
    if (jvmBitVersion().contains(“32”)) {
    jacobDllVersionToUse = “jacob-1.18-M2-x86.dll”;
    } else {
    jacobDllVersionToUse = “jacob-1.18-M2-x64.dll”;
    }
    File file = new File(“lib”, jacobDllVersionToUse);
    System.setProperty(LibraryLoader.JACOB_DLL_PATH, file.getAbsolutePath());

    AutoItX x = new AutoItX();
    x.run(“calc.exe”);
    x.winActivate(“Calculator”);
    x.winWaitActive(“Calculator”);

    }

    public static String jvmBitVersion() {
    return System.getProperty(“sun.arch.data.model”);
    }

    }
    ____________________________________________________

  21. Hi Joe,

    I am using windows 7 . When i tried to register the dll regsvr32 D:\NewRnD\AutoIT\tools\autoit\AutoIt\AutoItX3_x64.dll or regsvr32 D:\NewRnD\AutoIT\tools\autoit\AutoIt\AutoItX3.dll. was getting an error that (Check your windows is compatible or not with regsvr32). i am unable to fix the AutoIt. Can you please help with it. Thanking you in advance.

    Bhuwan

  22. Hi Joe,

    I followed the above steps. There were no compilation errors,
    however, I ended up with the below errors:

    Exception in thread “main” java.lang.NoSuchFieldError: m_pDispatch
    at com.jacob.com.Dispatch.createInstanceNative(Native Method)
    at com.jacob.com.Dispatch.(Dispatch.java:101)
    at com.jacob.activeX.ActiveXComponent.(ActiveXComponent.java:58)
    at autoitx4java.AutoItX.(AutoItX.java:181)
    at CalTest.main(CalTest.java:31)

    I tried to find out the solution over the net but could not get any help.
    Could you please help me with this.

  23. This is really a nice article to start with, specially those are willing to automate a solution which uses browser and windows application.

    Thanks,
    Debasis

  24. Got the solution of my problem.

    The jacob and the dll version were not same.
    After keeping the same version (1.17), its working fine now.

  25. Hi,
    I want to ask that,
    Can we able to click the bookmarks stored on browses bookmark menu bar using AutoIt?

  26. Hi Joe,

    Thanks for your clear instruction. I am able to integrate Autoit with Java code and it is successfully working for demo like calculator. but while I am trying to automate one of my Windows app it is not functioning properly as the ID field is getting generated dynamically. each tym i am opening the window by “run” command, the field and button ids are different. so can you please lemme know how to manage it using autoit. Is it possible to generate the autoit script using AutoitRecorder and integrate it with the java code.

    please find the below generated code for my automation.

    #region — Au3Recorder generated code Start (v3.3.9.5 KeyboardLayout=00000409) —

    #region — Internal functions Au3Recorder Start —
    Func _Au3RecordSetup()
    Opt(‘WinWaitDelay’,100)
    Opt(‘WinDetectHiddenText’,1)
    Opt(‘MouseCoordMode’,0)
    Local $aResult = DllCall(‘User32.dll’, ‘int’, ‘GetKeyboardLayoutNameW’, ‘wstr’, ”)
    If $aResult[1] ‘00000409’ Then
    MsgBox(64, ‘Warning’, ‘Recording has been done under a different Keyboard layout’ & @CRLF & ‘(00000409->’ & $aResult[1] & ‘)’)
    EndIf

    EndFunc

    Func _WinWaitActivate($title,$text,$timeout=0)
    WinWait($title,$text,$timeout)
    If Not WinActive($title,$text) Then WinActivate($title,$text)
    WinWaitActive($title,$text,$timeout)
    EndFunc

    _AU3RecordSetup()
    #endregion — Internal functions Au3Recorder End —

    Run(‘C:\Program Files (x86)\DecTech Solutions\Instinct 4.6.0.12 (QBEAU)\Instinct.exe’)
    _WinWaitActivate(“Instinct Sign-On”,””)
    MouseClick(“left”,374,352,1)
    MouseClick(“left”,-412,-175,1)
    MouseClick(“left”,-323,-2,1)
    _WinWaitActivate(“Application Quick Search”,””)
    Send(“{DOWN}”)
    _WinWaitActivate(“Instinct Sign-On”,””)
    MouseClick(“left”,668,577,1)
    MouseClick(“left”,1082,13,1)
    MouseClick(“left”,-412,-179,1)
    MouseClick(“left”,-332,-10,1)
    MouseClick(“left”,662,579,1)
    MouseClick(“left”,674,583,1)
    MouseClick(“left”,52,430,1)
    MouseClick(“left”,664,582,1)
    MouseClick(“left”,664,582,1)
    MouseClick(“left”,1042,-203,1)
    #endregion — Au3Recorder generated code End —

    Please help… waiting for your response..
    Thanks,
    Sampurna

  27. Hi i was trying this software for zenmap software but it didnt worked at all.
    can you please check and tell me.

  32. this is Great Information. Cant wait to test.

    Pretty Cool Thing you have shared.

    Appreciate for the info

  34. Hi Joe,

    I followed your tutorial and everything worked fine. In this test example you have used Auto3Info application to extract information about the calculator GUI, but in my case how I can extract information for my Java application components like textField,button etc id or name.

    Thanks
    Neeraj

  36. Pingback: Can I use Selenium WebDriver to automate a Windows Desktop Application? » Selenium Simplified

  40. Hi Joe,

    Thanks for this article. Can we use this AutoIT approach to automate the mainframe applications ?

  45. I am getting different behavior while using mouse click options {mouseclick(x,y)}. I recorded a mouse click action to select an option from the menu on my windows form. But when I take the recorded code out and run it through my java program, its not clicking the option I wanted to. The coordnates seems to be changing. How can this be resolved?

  46. Problem in AutoIT.jar

    While using the “ControlClick” Command to click on “Open” Button of Putty , My Putty Screen got disappeared.

    I am sure that i am able to click on the “Open” button, But my Putty Window got disappear. I have not used any Hidden Window Commands.

    Please give me any solution.

  48. Hi. I’m trying to remember how I fixed this. Did you follow the exact steps in this post? I think it might of been related to me not doing the regserv on the write bit version.

  49. When you say it “disappears” does it really close Putty or is it running in the background? Can you do a controlfocus on the main window to bring it forward again?

  50. Hi,

    I tried the steps but when i am registering DLL for 64 bit operating system, it is throwing an error that –

    “The module “”%1″” may not compatible with the version of Windows that you’re running. Check if the module is compatible with an x86 (32-bit) or x64 (64-bit) version of regsvr32.exe”

    Whereas I checked that my system is 64-bit. Please help me with this.

    Thanks.

  51. Hi Joe,

    Thanks for your clearly outlined article.

    Can AutoIT and Selenium be used to test applications like SAP ISU or any SAP software?

    Thanks,
    Zayyad.

  52. Hi Joe,

    Thanks for the article.

    I am trying to automate an SWT app and AutoIt seemed useful, however, every time i launch a new instance of the app, the control ID changes and the instance ID changes.

    Any way around this or any other toolset i could use(Currently looking at Jubula)?

    Thank you.

  55. Can any one please help me on below steps

    where exactly we need to type the below commands?

    Now — so that we can use AutoIt without installing Autoit we need to register the AutoIT dll that we placed in our tools/autoit directory:

    If you are using Java 32bit:

    Click on the Windows start button and enter regsvr32 yourpath/AutoItX3.dll

    If you are using Java 64bit (if you don’t do this, you will get the error Can’t co-create object):

    Click on the Windows start button and enter regsvr32 yourpath/AutoItX3_x64.dll

  56. Hi,

    Thank you so much for the clear explanation, but i’m stuck at below steps could you [please help me understand the below points. where excatly we need to type regsvr & path.

    Now — so that we can use AutoIt without installing Autoit we need to register the AutoIT dll that we placed in our tools/autoit directory:

    If you are using Java 32bit:

    Click on the Windows start button and enter regsvr32 yourpath/AutoItX3.dll

    If you are using Java 64bit (if you don’t do this, you will get the error Can’t co-create object):

    Click on the Windows start button and enter regsvr32 yourpath/AutoItX3_x64.dll

  57. Hi Joe,

    Thank you for the article,

    Is there a way to run this on Remote node using Selenium Grid. As I understand this code runs on a local system. How about running the same on distributed machines using Selenium Grid

    Thank you!
    Shivaji

  58. why i get this excetion
    Exception in thread “main” java.lang.UnsatisfiedLinkError: Can’t load library: D:\workspace\oatester\lib\jacob-1.18-M2-x64.dll
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.load0(Unknown Source)
    at java.lang.System.load(Unknown Source)
    at com.jacob.com.LibraryLoader.loadJacobLibrary(LibraryLoader.java:151)
    at com.jacob.com.JacobObject.(JacobObject.java:110)
    at autoitx4java.AutoItX.(AutoItX.java:181)
    at com.vip.test.CalcTest.main(CalcTest.java:22)

  60. Hi Joe,

    Thanks for the great tutorial.
    Please joe help us to run this code on remote machine as well.

  62. Hi Joe,

    Works great!! Initially wasn’t able to register the .dll as i was not running it as administrator. After i ran it as administrator, its working. Thanks a ton!!! :-)

  63. hey i am not able to press key on windows 10 calculator auto info is not able to detect id on it ??

  65. This is Excellent information.
    Question: Here operation is performed using co-ordinates. Is there any other way it can be identified as regular objects?
    Background: I want to identify fields on Mainframe screen using Jacob.jar but not as co-ordinates.

    I’ll be very thankful if anyone can answer.

  68. Hi,
    I was following you post and when trying to download the zip from http://www.autoitscript.com/site/autoit/downloads/ I could not find any jacob file.
    Maybe they have updated/removed these files.
    Not sure how to proceed and if there is a better way to automate non browser based windows now, since this post is from 2014.
    Thanks

  69. An what do I do when I did all that and FAILS?

    Exception in thread "main" java.lang.NoSuchFieldError: com.jacob.activeX.ActiveXComponent.m_pDispatch I
    at com.jacob.com.Dispatch.createInstanceNative(Native Method)
    at com.jacob.com.Dispatch.(Dispatch.java:99)
    at com.jacob.activeX.ActiveXComponent.    (ActiveXComponent.java:58)
    at autoitx4java.AutoItX.    (AutoItX.java:231)
    at CalcTest.main(CalcTest.java:25)

Comments are closed.

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
A person in glasses and a black jacket gestures with their hand against a blue backdrop with the bold text "Cut Testing Time in Half.
Automation Testing

DevAssure Review: Cut Testing Time Using Low Code Automation?

Posted on 12/11/2024

As you know, testing often becomes a bottleneck in software development due to ...

Two people in a video call enthusiastically discussing AI Playwrights. One person gestures with their hands as they delve into the nuances of test case generation. Text reads "AI Playwright Unlocked!" in bold letters.
Automation Testing

Leveraging AI and Playwright for Test Case Generation

Posted on 11/22/2024

Two BIG trends the past few years in the software testing space been ...

Two men speaking, with illustrations of a robotic head, AI chip, and robot in the background. One man wears a white shirt, the other a black jacket. The scene subtly hints at Symbolic AI advancements while touching upon the future of Test Automation in tech discussions.
Automation Testing

Symbolic AI vs. Gen AI: The Dynamic Duo in Test Automation

Posted on 09/23/2024

You've probably been having conversations lately about whether to use AI for testing. ...

Click to Get 10 Quick Test Automation Tips Now
(865) 272-6088‬

I appreciate the information you send over email. It has really helped me to enhance my testing skills!

Alisha Sheikh (Test Engineering Senior Analyst at Accenture)

Thank you for all the information provided in your newsletter. I am new in automation testing; hence, it is very helpful.

Audrey Pruvot (TagPlay)

Automation Guild '25 Online Event - Registration Kickoff Special (Limited Time) - Elevate your E2E testing skills in 2025 >>