Quality Engineering at scale with SAFe®

Introduction

A rapidly increasing number of organizations feel the need to implement an Agile method at scale. Various Agile-at-scale models & frameworks are available, of which the Scaled Agile Framework (also known as SAFe®) is used by many large organizations around the globe. Although Quality Assurance (QA) is mentioned in SAFe®, in our opinion it is not elaborated in sufficient detail. Therefore, we elaborate on this. 

This section maps our vision on quality in Agile at scale with the QA components in SAFe®, and extracts key descriptors, characteristics and questions from the wealth of SAFe® documentation to make it easier to find the links between the two. Please refer to the SAFe framework and the www.scaledagileframework.com website for more details about SAFe®.

QA-related subjects within SAFe®

SAFe® comprises the following QA-related subjects in the context of Customer Centricity: 

• Built-in Quality: All Agile teams — software, hardware, business, or other — must create quality solutions and define their own built-in quality practices.
• Lean QMS: Compliance refers to a strategy and a set of activities and artifacts that allow teams to apply Lean-Agile development methods to build systems that have fit for purpose quality, while simultaneously assuring they meet any regulatory, industry, or other relevant standards.
• Relentless Improvement: Kaizen, or the relentless pursuit of perfection, has been one of the core tenets of Lean, ever since its inception in the Toyota Production System. Kaizen is illustrated in various “House of Lean” models including the SAFe® House of Lean.
• Design Thinking: Design thinking is integral to customer centricity. Design thinking has two main activities, understanding the problem and designing the solution.
• Release on Demand: Release on Demand captures the mechanisms and processes by which new functionality is deployed into production and released immediately or incrementally to customers based on demand.
• Culture of Shared Responsibility: Culture represents the philosophy of shared responsibility for fast value delivery across the entire Value Stream. It consists of everyone who helps create value, including Product Management, development, testing, security, compliance, operations, etc.
• Mindset and Principles: SAFe® is based on ten immutable, underlying principles for applying Lean and Agile at scale. These tenets and economic concepts inspire and inform the roles and practices of SAFe®, influencing leader behaviors and decision-making.
• Agile Testing (advanced topic on the SAFe® website): Agile Testing applies the principles of agile development to the practice of testing. Although traditional development has used a big-bang, deferred testing approach, agile testing develops and tests systems in small increments, often developing tests before writing the code, Story, or Feature.
• Test Driven Development (advanced topic on the SAFe® website): Test-Driven Development (TDD) is a philosophy and practice that recommends building and executing tests before implementing the code or a component of a system. By validating them against a series of agreed-to tests, TDD — an Agile Testing practice — improves system outcomes by assuring that the system implementation meets its requirements.

There is a relationship between the TMAP QA & testing topics and SAFe® quality related subjects. Our opinion is that the TMAP QA & testing topics are a Lean quality management system (QMS) to achieve built-in quality. 

The QA & testing topics are the parameters within which quality engineering activities should be run, setting standards and orchestrating QA across cross-functional teams. They provide a framework for scaled Agile QA and testing to be done the right way. 

Roles and responsibilities in SAFe®

In SAFe®, especially at the Teams and Program level, roles are pretty aligned with the ‘standard’ DevOps roles. One exception exists for the Release Train Engineer (RTE), which is an additional role. 

The RTE is a servant leader and coach for the Agile Release Train (ART). The RTE’s major responsibilities are to facilitate the ART events and processes and assist the teams in delivering value. He can be considered as the trains scrum master to ensure the train runs smoothly and keeps on track. 

At the ‘higher’ SAFe® levels (Solution, Portfolio) the mapping of roles cannot be made 1:1 due to their nature hence fades away.  

The most logical place according to SAFe® for organizing orchestration of end-to-end quality is a System team. This can be compared with the Support Team end-to-end Quality  (see the building block End-to-end QA at scale) that supports the process of one or more Agile Release Trains (ART). The system team can contribute considerably to achieve the required built-in quality and drive end-to-end testing. Note that Integration testing is an activity executed by, and between two Product Teams; the system team is only indirectly involved there by bringing integration under attention of the team and defining improvement for integration testing. 

Reasons to give this responsibility to the system team are, amongst others: 

• The system team supports one or more Release Trains. 
• The system team is automatically involved in a number of the focus areas as mentioned earlier. For example, it provides the infrastructure and tooling that the teams need to do their work. 
• It supports the various release moments where necessary. 
• SAFe® indicates that an End2End test can be organized from the system team.

The end-to-end Quality Orchestrator acts as the product owner (PO) of the system team. 

The organization of end-to-end quality in the Solution is, in SAFe®, done by the System Team and the PO orchestrates this. The execution of the work however is done via so called virtual teams. Instead of organizing a completely new team, including resources, skills and expertise we make use of what is already available in the organization. There are two types of virtual teams: one that works on the product quality via an end-to-end test and one that works on improving areas like people and infrastructure. 

In addition to SAFe®, the earlier mentioned Agile Quality Coach takes the role of Test-/QA manager who secures Release Train overarching activities in the organization, like drawing up and monitoring vision, policy and guidelines. But also leads the communities of practice in the field of QA and testing, that discuss and develop the needs in the field.