Automation Yoga: Stretching ROI with a Testing SDK - Brendan Connolly

Talk Description

Automation is judged by its return on investment (ROI), the time spent writing and maintaining compared the time savings. To ensure solid returns a great deal of effort is put forth creating automation frameworks and choosing the right tests to automate in order to optimize for successful results.

If automation is an investment, then this traditional approach to test automation is in effect creating a niche market. The problem is our potential ROI is limited by the size of this market, the fraction of completely automatable tests which are deemed worth the effort to automate and maintain.

Is your framework designed for writing automated tests, or for automated testing?

To truly maximize ROI, you need to stretch the total addressable market (TAM) your automation can serve. We call them automation frameworks but typically they are single purpose software full of assumptions around that purpose, tightly coupled to executing complete end to end test cases.

The focus needs to be around building a set of tools that can be composed into a platform to empower exploration as well test execution, a Testing SDK (Software Development Kit).

Inspired by the Richardson Maturity Model for REST API’s, I will share a model of automated testing maturity consisting of four levels:

Level 0: Tests exist but the framework does not. UI element locators inside tests or record play back tools.

Level 1: Framework exists, UI concerns encapsulated.

Level 2: Multi-Layer framework, Beyond UI, API or other components included for arranging or manipulating, only consumer is tests.

Level 3: Testing SDK, Separate libraries/packages, consumable outside of tests and multiple consumers. Mixing automation and manual testing. Open source potential.

At each level in the model, I will describe the benefits to the business and testers then provide steps to leveling up their automation.

Attendees will see that embracing a Testing SDK provides is a holistic strategy for test automation.

Takeaways:

• Common pitfalls in automation frameworks
• Automation is more than writing tests
• Enable and Instrument Exploratory Testing
• SDK's provide more options for contribution

What you’ll learn

By the end of this talk, you'll be able to: