Quality Coaching: encouraging testability

As an Agile Quality Coach, I focus on enabling teams to take ownership of the quality of their software. One of the ways I do this is by encouraging teams to consider the testability of their software. In this blogpost, I’ll share my experiences and tips and tricks for getting started with testability.

The Joy of a Testable System

When I originally started as a Test Analyst, I did not know how lucky I was. I had my own test environment to conduct my tests in. The software was built in a lot of different components that could be individually kicked off and observed. There was an activity log that listed all the jobs that were running on that environment and an error log that listed things that had gone wrong. The data in our database had metadata columns that listed status, originating source, mutation dates and more. When we implemented a change, we also considered what metadata to add in the activity log and all the metadata columns and tables. In short, though the system was thoroughly complex, it was observable, understandable, and testable.

The Frustrations of a Not-So-Testable System

Since then, I have been a tester and a quality coach in various teams and organizations, and this concept of a testable system is rarer than I’d like. Even if we are aware which quality characteristics are important to us and which risks threaten our product, we can struggle immensely with the test execution process. If we are not able to control our system or environments, have no metadata that allow us to observe our system and see under the hood what is happening, and lack a shared model of understanding as to how our system works – testing takes a lot of time and effort and often becomes a roadblock in the team’s progress.

Getting Started with Testability: What is it & How I Apply it as a Quality Coach?

Testability in short asks the question: how easy or difficult is it to test this software or product and determine its quality? And who in the team is concerning themselves with this question? The more formal definition that is used within TMAP is “the degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be performed to determine whether those criteria have been met.”

As an Agile Quality Coach, I encourage a whole team responsibility for quality. In my experience, quick wins in improving the quality of the testing can be found in improving the testability of our software first. I encourage teams to keep track of things that make testing difficult or impossible and look for ways in which these can be mitigated.

Rob Meaney’s 10 P’s of Testability

Rob Meaney introduced his 10 P’s of testability as a model of things that can show you where to look to improve testability:

1. People: their mindset, skills and knowledge to pursue quality
2. Philosophy: is whole team responsibility for quality embraced? How is the cross-functional collaboration within the team?
3. Product: is it designed to be testable? How is exploratory testing and automation facilitated?
4. Process: which size chunks does the process allow for? How is the accumulation of testing debt discouraged?
5. Project: which time, resources, space & autonomy is provided within the project for great testing?
6. Problem: which customer problem does the product solve for the customer and which risks need to be mitigated?
7. Pipeline: how fast is reliable, accessible, and comprehensive feedback provided on each change as it moves to production?
8. Productivity: how early are important problems unearthed? How is continuous feedback facilitated?
9. Production Issues: how many production issues occur (that impact customers) and how quickly can teams detect, debug, and remediate these issues?
10. Proactivity: how is the team seeking to improve their test approach, learn from their mistakes and experiment with new tools and techniques?

Asking the question as team: “how can we make this system easier to test?” often brings surprising results. Increasing the testability often increases productivity of the whole team and benefits the team atmosphere.

How easy or hard is it to test your product? And who in your team is currently concerning themselves with this question and feeling the pain if it’s hard?

Published: 8 May 2024
Author: Marianne Duijst