“For most organizations, half the time and half the money gets spent before a card ever gets to a development team’s backlog.”
So says a white paper from Planview, the company behind the Flow Framework.
The Flow Framework (and “flow” as a concept in software development) is an adaptation of decades-old concepts from lean manufacturing and value stream mapping. In manufacturing, the flow of products across the factory floor determines everything. In software, we still haven’t fully theorized the flow of code through our software development lifecycles.
Enter flow metrics.
Flow metrics, counter to the first assumption you might have, aren’t related to the “flow state” developers (and other professionals) enter when they’re feeling creative and focused. Flow metrics focus on product delivery rather than process efficiency or developer productivity, looking at the entire value stream so that enterprises can identify bottlenecks that lie outside the software team.
Flow provides a high-level framework that enterprises will find especially valuable – as long as they combine it with other metrics.
Flow metrics: An overview
Flow metrics are part of the Flow Framework, which was created by Dr. Mik Kersten, the Chief Technology Officer at Planview.
Kersten introduced the Flow Framework in 2018 when he published the book Project to Product. The methodology is recent, but the precedent is not.
The field of lean management, inspired by Toyota’s 1960s “lean manufacturing” approach, developed the idea of value stream mapping (VSM)—the core concept behind the Flow Framework. Managers use VSM to visualize and map the different ways value is delivered through the business. Built on concepts from VSM, The Flow Framework is responding to a “crisis” many enterprise IT organizations are facing. “Either they learn to operate as high-tech companies,” the Framework says, “or risk becoming digital relics.”
The Flow Framework provides enterprises with cross-functional metrics that “abstract away details like team structure, technical architecture, and tool implementation” to focus instead on business value. The Flow Framework then becomes something like a bridge between developers and business leaders. It uses software development artifacts, such as requirements, features, and stories, and connects those artifacts to metrics that show business leaders how value is being produced.
Some of the companies that use Flow metrics include TUI Group (61,000+ employees) and The Bank of New Zealand. Though Flow occupies a small piece of the pie (only 20% of companies measure Flow metrics, according to Planview research), it offers a perspective that other methodologies don’t.
Flow metrics vs. DORA metrics
When engineering leaders think about metrics, one acronym will likely come to mind: DORA. The DevOps Research and Development team promoted the four key metrics, now known as the DORA metrics, with the launch of the book Accelerate in 2018. These four key metrics were designed to help engineering teams measure and improve software delivery efficiency.
These metrics are quite different from Flow metrics, however, so teams should dig into the details to see what they can learn from each.
Flow metrics: High-level measurements of enterprise effectiveness
Flow metrics, according to Planview, “measure the rate of business value delivery for software products through the lens of your customers.”
If these metrics show that your enterprise’s ability to continuously improve how business value flows through your organization, then you’re succeeding. Once you know you’re succeeding along these metrics, you can double down on what works – vastly accelerating the pace of value of delivery.
There are five flow metrics, each of which asks a question that will help an enterprise better understand how it delivers value.
-
Flow Velocity: Is value delivery accelerating?
-
Flow Efficiency: Is upstream work holding up delivery?
-
Flow Time: Is time-to-market getting shorter?
-
Flow Load: Is demand overwhelming capacity?
-
Flow Distribution: Are priorities in line with business objectives?
Each metric has its own intricacies, but let’s take Flow Velocity as a representative example. Flow Velocity measures productivity by showing engineering leaders how many items the engineering team completed over a given span of time. The Flow Framework derives this definition of velocity from Agile and story points, emphasizing units of work instead of using overly strict task estimates.
As a bridge from engineers to engineering leaders to business leaders, this understanding of velocity is meant to be communicable in just a simple snapshot.
The emphasis on flow throughout the Flow metrics is intentional. The Framework argues that most enterprises have focused only on processes within siloed departments and specialization. Instead, enterprises need to build value stream networks that stretch across the organization.
If an enterprise is deploying new contact center software, for example, then an effective understanding of value needs to go beyond the IT department deploying the software so that the business can see how value flows through the contact center employees using the new software to the customers benefiting from it.
DORA metrics: Granular measurements of tech team performance
DORA metrics bear some similarities to Flow metrics but are designed to solve a different problem. Whereas Flow metrics are meant to help enterprises move as efficiently as startups and tech giants, DORA metrics are meant to help “technology-driven teams” implement DevOps better.
DORA emerged from research, including the first State of DevOps Report in 2014 and the book Accelerate: The Science of Lean Software and DevOps. Every year, the same group continues researching and produces another report with their findings.
From this research, four metrics capture the ability of especially effective software organizations to operate:
-
Deployment frequency
-
Lead time for changes
-
Mean time to recovery
-
Change failure rate
DORA only measures a small slice of the cycle, however, even though engineering is rarely the bottleneck. With DORA metrics, there’s a risk that software delivery becomes efficient, but the whole end-to-end process remains inefficient, slow, and wasteful.
Why Flow and DORA metrics are better together
Engineering leaders who take developer productivity and efficiency seriously are likely to feel overwhelmed with options – we haven’t even covered SPACE here, among others – but the choice isn’t Which framework is best for me? The choice is: What are the strengths of each framework, and how can we adapt each for the problems we face?
The Flow Framework itself takes this perspective, arguing that DORA metrics are necessary but not sufficient (at least for enterprises). “Without a doubt,” a whitepaper says, “You must become proficient at releasing code rapidly, securely, and confidently. But this has quickly become table stakes across the industry [...] Your organization will remain competitive only if it can deliver business value — not code changes — at an ever-increasing clip.”
This is a big claim, considering DORA research has been emphasizing the same four metrics for almost a decade now and, according to its 2023 report, 33% of respondents still have a change failure rate of 15%, and 17% still have a change failure rate of 64%.
The sentiment, however, is worth taking seriously: DORA metrics measure work on a granular, application-focused level, Flow metrics zoom out to measure work on a higher level, and business success requires thinking about business value as well as engineering effectiveness.
The same whitepaper compares Flow metrics to the vital signs a doctor takes to understand the human body. Before a doctor requests X-rays and MRIs of a single body part (the equivalent of DORA metrics), they need to understand the basic vital signs that show how well a patient’s body is working holistically. The first priority is understanding the overall health of the value streams that flow through the entire business and out through the engineering team.
With this in mind, the best companies don’t do one set of metrics or the other but combine both systems (and others) for their context-specific needs.
Take bottlenecks, for example. A bottleneck in a process can make all the work before it and after it very inefficient, but bottlenecks can be hard to identify.
DORA metrics alone can cause leaders to assume reviewers simply need to review faster, whereas Flow metrics alone might not demonstrate the severity of the issue. Together, engineering leaders can use DORA metrics to determine how slow lead times are and use Flow metrics to locate the process issue.
Why a holistic approach is necessary for improving software delivery
At a large enough scale, focusing on a single metric or bundle of metrics (we’re looking at you, developer productivity) will only provide limited benefits. Zoom out, and you'll see the complexity of creating value and the key part engineering plays in the value stream.
The authors of The Phoenix Project (a book about DevOps but inspired by, you guessed it, Lean Manufacturing) best capture the risks of losing a systemic perspective: “Any improvements made anywhere besides the bottleneck are an illusion.” Both methodologies, and many others, center around this issue: We all want to work better, but we know that the effectiveness of improving one process over another requires a systemic view of all processes.
With Flow metrics and DORA metrics together — as made visible through a holistic platform like Uplevel — enterprises can capture and maintain a systemic view of their processes that exceeds the view of any one methodology.