Testim web automation review

Testim is a SaaS GUI-based test automation tool for web based applications. I've seen a wave of next-gen test automation tools and services come to life this year. Testim is in that category, along with others I've been introduced to, such as Virtuoso and dev tools ai. Today however, I'll be sharing a high level walk-through of Testim, it's free plan features and my curious thoughts along the way.

Sign up

I'm using the free plan today to get a feel of the power of Testim. In future I'd love to experiment with the premium features but, that's for another day.

Interestingly, only business emails can be used to sign up for Testim. Always handy to have one.

Tests

Following the sign up steps, I was asked to create 3 test cases before even landing on this Test Library page. For a newbie exploring this tool for the first time, I was confused why I had to do that. Moving on ... here are those 3 test cases created earlier. They have no test steps at this point, only instances of their creation.  

Within Testim, tests are created by using one of the following options:

• Record your web UI interactions in Google Chrome, or
• Using an editor to code the test

Unforuntately at the moment, I cannot access the code editor tool. I am unsure if it's due to the free plan or something else. There's a button I can click but nothing happens. I haven't looked into this too much, let's skip it for now.

Creating tests

Now that I'm 'in the system', I can select any of the test instances I created earlier. When I do, a Play action, amongst others, presents itself.

It seems the 3 test cases created earlier (which were mandatory by the way) were simply to start users off with. At this point, you can record steps to add to those existing test cases or simply create a new test case instead. I followed the latter approach.

Oh ok, that's not too bad. Let's install the extension anyway.

Oh ... ok then. Looks like I'll have to convert.

Upon installing the requested extension in Chrome, I began to think "Hmm, what if I want to edit tests using something other than Chrome?". A scenario for another day.

Moving to Chrome, everything works without errors. Previously when I was logged in on Safari, I would often get these errors when navigating pages.

Using Chrome now, I no longer encounter this randomness. The first time you create a test, it is done via the chrome extension. The extension is synced with your Testim account. At this point, using the extension starts a session using a provided base url to record all your test steps, including all UI interactions and collects them all as your test steps.

Once your test is complete, you can view the recording within seconds. This process is simple, intiuitive and most importantly, fast.  

Test steps

When you return to Testim dashboard, you'll see all the individual UI interactions for your recorded test listed as test steps.

This looks pretty handy, as you can edit or delete specific actions at whim. There is a settings button on the right (not visible here) to view properties related to each step. Some of the properties available to view & edit for each test step include:

• Step description
• element checks (e.g element must be visible check
• Step fail event
• Override timeout

This would prove useful to create custom configurations for each test step. The downside though (given my brief use of Testim so far), is that I don't see a way to share or save the applied property settings across other test steps within the test case. This would lead me to manually editing many test steps one by one. Not a great use of time. Would be interested to learn more about this scenario in future.

Network Log

Another helpful feature on this page is the Network Log, displaying a collection of captured logs recorded during the browser test. This layout resembles the typical browser dev tools Network tab, which could use a little TLC in terms of presenation.

Image comparison

Lastly, each test step also generates a new 'image' for the page to compare with a baseline image. This is awesome to have this capability in-built to the solution. I'm still working out how it functions and from where. Here is my baseline image comparison.

Test runs

This page looks as simplistic as it needs to be. It's concise, clean and sticks to its purpose.

Test run results

Another reporting page, this one here displays recent runs using graphs and charts. I do like how the large numbers (for runs, duration, etc) standout and are bold. This page presents the important data clearly to the user, using bold fonts and easy to understand colours (red = bad, green = passing). This form of messaging is common throughout the whole app, helping to deliver important information quickly and consistently.

Settings

Unfortunately there's not much to explore nor edit using the free plan here, with many additional features available only for paid users. Basic project + account edits are available though.

Test execution

Tests can be executed using one of the following options

• Locally with Chrome or Edge browsers
• Remotely using Testim Selenium Grid with additional browser support, running on AWS

Additional integrations for scaling web tests are available, including two heavy hitters in SauceLabs and BrowserStack.

Testim provides sample commands to run tests against CI or local machines. This feature is always helpful when experimenting new tools on a trial account.

Let's try it out!

Right, so we have an error now. Analysing the logs in my local cil, there's nothing immediately helpful to us to investigate the errors from that test un. Instead, all the test run logs are located within the Testim app.

Switching to Testim, failed test steps are coloured red and summarise the gist of their error.

We also have 2 error prompt options, Tag test failure and See error. Selecting See error takes me to the visual comparison feature, showing me that my recorded test step using the date 16/09/22 is now no longer clickable on 21/09/22.

To resolve, I'm manually editing the test steps containing date pickers to be dates further in the future. Not an elegant solution, but allows me to move forward nonetheless.

Success! 😎

Integration

Few integrations on offer here, with GitHub, Slack and TestRail the standouts for me. Having those 3 and Testim connected would be a Quality Engineers dream.

And that's it!

Final thoughts

Wow, this UI is amazing. Finally a web tool that looks and feels fresh and modern to use. The loading indicators, tool tips, button animations, colours, gradients 😁. The whole system is welcoming and easy to use.

There are enough opportunities and allowance of features in the free version to demonstrate how ROI could be achieved in the long term for testers and teams.

I would happily try out the paid version in future to explore the advanced features, including:

• scheduled runs, event hooks, test reporting
• integrations with GitHub, TestRail, BrowserStack
• SSO, account management, permissions
• export to code, export of data, code editing

Anywho, that's all for now. Testim looked, felt & performed amazing so far, with its friendly UI and feature filled platform showcasing what a modern, web automation framework can accomplish.

Stay tuned as I'll be keeping an eye out on new developments in the app and hope to try out the full feature set sometime soon.