Testing Talks Reunion Conference - Melbourne

Heading down to Melbourne for Testing Talks Reunion, a conference focused on - you guessed right - testing. It’s also my first trip down to Melbourne in ~6 years.

After everything with COVID these past few years, it felt 'normal' to head out for a conference again, socializing in close proximity was welcomed and the limited mask wearing allowed people to read and show expressions.

Let's get into it then shall we?

General Melbourne

During the flight, I was observing the Victorian landscape. Mountains, grassy fields, farmlands. Witnessing the Earth from above for the first time in years was a sight to behold. Looking forward to the space trip one day!

Just over an hour into the flight I hear "Beginning descent into Melbourne". Now for a brief few minutes, I honestly thought I was taken to the wrong Melbourne airport. Somewhere in the country side where I'd have to stay overnight in a cabin and hitchhike to Melbourne CBD. Thankfully, my worries were put to rest when the plane simply turned around and there it was, the city skyline.

Now ... Melbourne is amazing. There's multiple lanes, organized drivers, spacious streets, low building levels. This was not the purpose of my trip but I was fascinated by how different Melbourne CBD compares to Sydney CBD.

Below I have captured all the basic notes I took during the conference. It was a balance of listening, absorbing and note taking so I apologize in advance if context is missing from items or not making sense.

TL;DR

• great to be at a large conference again
• enjoyed chatting about different test frameworks
• low code + no-code tools are increasing in quantity & quality
• contract testing is a hot topic with tools such as PactumJS, Pact, Pactflow
• takeaway is expensive

Machine learning

• Repeated exposure
• Backward propagation

Design Driven Concepts for Automated Testing

Using the speaker's recent fridge breakdown and analogies to explain the different concepts of testing, integration and design.

• For us to understand a system, we need to break it down into its component parts
• System interactions should be defined through interfaces
• Contract Test: To check the expected structure of an interaction remains acceptable over time
• Collaboration Test: To check the behaviour of an interaction is carried out when requested, how we expect
• Designing “what” to test is critical, before you write a line of code

Contract testing is the way to go

• Check any contract changes against a known good version
• A provider can validate changes that may break consumers directly
• This is a good indicator of how changes will play out between system entities

Behaviour testing

• Inform the provider about the usage of their services
• Consumers don’t care about implementations, only behaviour

Automate Any App with Appium #AAAA

• Introductory talk on Mobile Automation using Appium, sponsored by Headspin,
• Appium is based on Selenium web driver http protocol
• Recommended to set implicit waits, allow appium time (e.g. 5 secs) to find an element

⚡️️A brief introduction to contract testing: the who, why, what and how ⚡️

High performance organizations:

• <1 day lead time for changes
• Able to deploy on demand
• Low change failure rate
• Restore services within 1 hour
• Only 20% of companies are elite performers

Contract testing

• Simpler > test a single integration at a time, without needing a deploy
• No dedicated test ends
• Get fast, reliable feedback
• Tests that scale linearly
• Deploy services independently

Suggested tool: Pactflow

• A product recently acquired by SmartBear
• Pactflow removes the need for complicated release coordination: we have static knowledge about systems
• Pactflow captures & records mocks, uses mocks as a static base to test apis

Elevate your testing: testing smarter, not harder, in the cloud

Hmm it seems I missed this session. I must have been daydreaming alongside the Yarra River.

What's new in Selenium 4?

The speaker ran through Selenium 4 release highlights and features.

• Removed a lot of code
• Deprecated desired capabilities

Relative locators

Friendly way of locating elements using terms that users normally use

• above, below, toLeftOf, toRightOf, near
• driver.switchTo().newWindow(WindowType.WINDOW)
• driver.switchTo().newWindow(WindowType.TAB)

Selenium Grid - Observability

• Debugging Grid is lot easier
• Distributed tracing via Open-Telemetry APIs
• End-to-End tracing: Client & Server
• Selenium server is instrumented with tracing using Open Telemetry
• Every request to the server is traced from start to end
• Each trace consists of a series of spans as a request is executed within the server

DevTools Protocol

• Native support for Chrome DevTools Protocol (referred to as CDP)
• Native support for chrome dev tools protocol through dev tools interface
• Helps retrieve chrome development properties
• Geo location
• Network Interception

Web Bidi

• HasAuthentication interface
• Able to automate login authentication
• NetworkInterceptor
• Capture network events
• Mock UI front-end (sounds interesting!)
• Webdriver = is the protocol (W3C), anyone can implement it
• Selenium = is the automation library
• WebDriver Bidi is the future of Selenium

Interactive Exploratory Testing - MobTesting, BugBashing, CrowdTesting, Oh, My!

• Completed mob testing, bug bashing + crowd testing activities
• Got people chatting with each other, involving the audience
• Each person defining their version of ‘bug vs incorrect feature’ & functionality assumptions

Techniques to Eradicate Flaky Tests

Challenges of e2e test automation

• Long sunning scenarios
• Slow feedback cycle
• Limitations of automation candidates
• Error/edge case handling from dependent systems
• Triggers from external systems
• Pain of cross-browser/cross-device execution
• Slow, special handling for devices, infrastructure cost, availability

Flaky tests

• Fails and passes that you have zero control over
• Retry test/functions hide the flaky tests

Why are they flaky?

• Network, performance, environment issues
• Data dependencies, UI/locator changes

How to reduce flaky tests?

• Reduce number of UI tests
• Remove external dependencies via Intelligent Virtualisation
• Using visual assertions instead of functional assertions
• Specmatic - Contract Driven Development
• Share your api contracts amongst test and features, if 1 changes the other, your test is likely to fail

Automation Testing Web Apps with Cypress

Eh I was pretty exhausted by this point. Didn't want to continue if I wasn't mentally present. I do recall though that the speaker for this talk primarily walked through creating Cypress tests for a few common, albeit simple, UI interactions. Seemed like a big Cypress fan too. Good to see the passion.

Summary

Wow, that's it. Lots of lists, notes and talks.

First of all, thanks to everyone who organised the conference, the speakers and the enthusiastic crowd.

Below are some pros & cons from my experience.

Pros

• Fantastic and encouraging to see a community of testers/quality engineers/qa (and more) gather from interstate
• Lots of products and sponsors showing off their tools & services, including new shirts to parade in the office
• Variety of games (mini golf, mini table tennis, nerf gun shooting). Had to resist the urge to retest and improve my performance

Cons

• I was hoping for more deep technical talks. I'm an Appium user and enthusiastic troubleshooter yet the Appium session was at an introductory level.
• Meanwhile the Contract Testing session unfortunately only lasted for 10 minutes, when I spoke with others afterwards they were longing for more
• Some booths were welcoming, others looked uninterested or unwilling to bring in a crowd nor engage their audience. Suppose it depends on the conference and who actually wants to host a booth vs who gets dragged out. I had mixed results
• 1 female speaker with overall ratio of attendees mostly male. Need to balance this in future (I have no immediate thoughts on how to improve this, would love to hear suggestions)
• Would have liked different session formats such as hands-on demos, forums for open discussions, group activities to automate a task? Some of these would have helped maintain interest and excitement throughout the day. Particularly those who may have wanted a break from the longer presentations. This would have helped to encourage and to facilitate interactions amongst attendees

Ending

That's all folks. For those of you who made it this far, thanks for reading my conference notes and summaries.

Thanks again to the organisers, sponsors and all those amazing QAs, QEs, testers, quality engineers, and on and on.

Till next time!