Artificial Intelligence (AI) in test automation

Sometimes I dream about having more spare time, less time spent on work and more time for hobbies, family and friends. And at this point, I start to question where all those robots are that could do our job? Why don’t they exist yet? Why is it taking this long? I mean, we have self-driving cars, so why can’t I have a self-testing system? So, the question is, will I get more time for other testing activities soon or will test automation keep me busy in the upcoming decade?

Let’s first note a big difference between the two fields here: there is the testing of Artificial Intelligence and testing with the help of Artificial Intelligence. The key difference is that when we are testing the AI, we have to somehow check that the AI is doing what we expect it to do, which can be difficult since the AI is basically learning on its own. Testing with AI means that we give our work of writing automated tests to a neural network, and we will wait on the beach till the AI has written the test code for us. Let’s see it as test automation without any coding, clicking or configuration. In this article, we are only focusing on testing with Artificial Intelligence and not testing the Artificial Intelligence itself.

So, what am I looking for? As I mentioned, I want to use AI in test automation. I would like to feed an application to an AI tool, go away and when I come back, the tool has figured out how to test the application and can run a regression test from that point onwards without breaking, and it will update itself. However, hours of searching for a perfect tool have resulted in nothing. There are plenty of tools that claim to use AI for test automation, but none of the tools would actually do the testing for me. So why are they using AI if it’s not taking over my job? Then it’s useless, right?

Let’s start with how AI works in short and why it’s so hard to just build a tool that takes over my job as a test automation engineer. The most used form of AI is machine learning, we feed loads and loads of data to an algorithm and let it analyse it. Based on this, the system starts to recognise patrons and tries to follow them when it has to make decisions on its own. This is nothing magical; it’s an algorithm and maybe some science. So, our first problem of automating test automation with AI is data. I can’t come up with a dataset that contains hundreds of thousands of automated test flows that are readable by an AI platform. And even if I could find it, it will most likely only cover one application or at most one kind. So, in order to use AI to replace us, we will need a big data set to train and test AI. And even then, it would be questionable whether it would be good enough to test every application. Every company’s business flows are different, they name things differently, and they expect different results.

So that means all those tools that claim to be AI test tools are bullshit. I mean, they don’t do my work for me, so why use them? I don’t think that’s the right thing to say. AI has come a long way in the last couple of years. Mainly because the power of our computers has grown in a way that it gets easier to process huge amounts of data, make predictions and use machine learning. Many test automation tools have started to use AI to make our lives easier. They don’t take over our jobs (yet) but they try to make it easier to automate and make the tests more stable.

For example, test automation tools with Artificial Intelligence are now able to make better image comparisons. Where in the past, a visual check would fail when a button shifted by just one pixel to the left or the right. Tools that use AI could be able to detect whether this is a major change or just a pixel shift that no one cares about. This will make it a lot easier to do visual comparisons of a UI.

Other tools will try to predict how likely it is that the newly appearing button is replacing the old, missing button. In the past, a missing element would fail the test, and a newly placed button would not be seen by the tool. An AI can create a link between the missing and the newly appearing buttons. And then if the AI predicts that it’s likely that the new button is just replacing the old one, it won’t fail the test like it used to do in the past, but it will just click the button.

So, as you can see, AI is already used in many different test tools, but with small steps, only one at a time. For now, AI isn’t here to replace us, but the AI is here to help us instead. AI has been proven to be useful in test automation, and it will be more and more used in different test automation solutions throughout the field. However, we won’t be replaced any time soon. Too bad, because we could spend our time with other testing activities, such as exploratory testing, which could feed the AI with useful information. So, we can spend more time with family, friends and hobbies.

Janus van Limpt
Test automation engineer

Would you like to know what possibilities we offer to become a Test automation engineer? Please click here.