Let’s get the harsh reality out of the way first: Most organizational transformations fail (69%, according to McKinsey). In the same way that new startups have to contend with the fact that most startups fail, established organizations have to contend with the fact that most organizational change fails.
When an organization reaches a self-sustaining scale, it’s no longer an entity that leaders can push one way or another. It’s a complex, interdependent web of people, processes, and technologies that leaders can only address through sociotechnical strategies.
As Donnella Meadows, author of Thinking In Systems, writes, “A system isn't just any old collection of things. A system is an interconnected set of elements that is coherently organized in a way that achieves something.”
When a new CEO announces a sweeping culture change or a new technology (think AI) emerges and few see the writing on the wall, you’re seeing the results of a system reasserting itself. “A system generally goes on being itself, changing only slowly if at all, even with complete substitutions of its elements — as long as its interconnections and purposes remain intact,” Meadows writes.
This isn’t to say changing a system is impossible. In the same way that animal conservation requires understanding the ecosystem as well as the animal species itself, organizational change requires understanding the laws that underlie all organizations.
Luckily for us, three legendary thinkers – Melvin Conway, Charles Goodhart, and Fred Brooks – have laid out three laws that can help us understand why organizational change tends to fail and how we can beat the odds.
1. Conway’s Law: Organizational components are interdependent
Conway’s Law states, “Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.”
Melvin Conway is most well known for developing the concept of coroutines back in 1958, but in software development circles, you most likely hear his name via this law.
Martin Fowler, one of the signatories behind the Agile Manifesto, describes the law like this: “Conway's Law is essentially the observation that the architectures of software systems look remarkably similar to the organization of the development team that built it.”
In other words, software architecture resembles the structure of the team that built it – often known as “shipping the org chart.”
A single team that writes a compiler will write a one-pass compiler, Fowler explains as an example, but if the team is divided into two, it will write a two-pass compiler. Similarly, a company of six teams will inevitably build a system of six major subsystems.
Organizations tend to underestimate Conway’s Law. Why? When organizations are young and small, the Law doesn’t enforce itself.
“A dozen or two people can have deep and informal communications, so Conway’s Law indicates they will create a monolith,” Fowler writes. “That's fine - so Conway's Law doesn't impact our thinking for smaller teams. It's when the humans need organizing that Conway's Law should affect decision making.”
Fowler shows that Conway’s Law is most obvious when you hit its limits. If you organize your developers into frontend, backend, and database teams, for example, it will be hard to integrate those layers well because the teams building them are separate. And if different teams handle design, coding, and testing, there will inevitably be friction between the steps toward deployment.
Beating Conway’s Law starts with awareness. As Fowler writes, “The first step in dealing with Conway's Law is know not to fight it.”
Once you accept the gravitational pull of this technical/communication interdependence, you can start working with it. In your day-to-day work, this will look like asking seemingly obvious questions (and building processes and habits across your team that encourage them to ask them, too). The more your team can ask questions, the more they can resist the default tendency to replicate the organizational work in the architecture work.
Terry Crowley, former Corporate Vice President of Office Development at Microsoft, for example, found his organization hit the limits of Conway’s Law when it tried to organize shared teams (e.g., teams designing user experience elements like ribbons and task panes instead of teams developing Word or Excel). “Office faced this problem right from its inception and learned some valuable lessons about how to organize for these cross-cutting concerns,” Crowley writes.
According to Crowley, shared teams must have:
-
End-to-end accountability that allows them to own resource allocation decisions so that the team can prevent itself from becoming a bottleneck.
-
Real resources that allow them to execute instead of just coordinate other team’s resources.
-
Communication pathways that allow them to work across teams in different departments.
-
Support from leadership that legitimizes the shared team’s work.
Crowley writes, “It is the smart trade-offs in allocating resources against priorities that is the essence of the hard challenge in building a large product.”
Building a large product requires a large team that’s split in many different ways, and as Crowley shows, working with and against Conway’s Law requires deliberate strategies informed by the tradeoffs between your communication and technical systems.
2. Goodhart’s Law: Metrics are necessary but not sufficient
Goodhart’s Law states, "When a measure becomes a target, it ceases to be a good measure.”
Charles Goodhart was a British economist who worked at the Bank of England and the London School of Economics between the 1960s and early 2000s. Goodhart’s Law has remained a staple in development discussions because it regularly trips up even the most well-intentioned people.
Think about developer productivity. If a leader isn’t happy without the amount of work their team is doing, measuring the work and rewarding those who work the most makes sense. They might introduce a metric, such as lines of code committed, and announce that this metric will tie into discussions around promotions and raises.
The core problem here is the assumption that measurement is a neutral activity done from the outside in. With a systems thinking perspective, however, we can understand how metrics can change the system, especially its incentives.
In the prior example, measuring developers by lines of code produced might incentivize developers to lower quality standards to produce more code – a result that’s likely worse than the initially diagnosed problem.
Metrics inevitably introduce new incentives, and new incentives tend to motivate gamification. Without countermeasures, the intent behind the metrics can be lost.
This is why Kent Beck, the creator of extreme programming, argues, “Any conversation about metrics has to start with Goodhart's Law.” When organizations don’t start from this point, he writes the metrics can be “worse than inaccurate because people degrade the system to produce the number.”
That said, metrics are not toxic to organizational change. Quite the opposite: Will Larson, CTO at Carta, shows a counterexample by describing how using metrics led to reducing infrastructure costs at Stripe and Uber. His process involves diving deep into the relevant metrics, attributing them to the right teams, contextualizing them around performance, and nudging teams to follow the research results.
He writes, “For most company level metrics (cost, latency, development velocity, etc.), the first step of diving will uncover one team that is nominally accountable for the metric’s performance, but they are typically a cloak. When you pull that cloak aside, that team’s performance is actually driven by dozens of other teams.”
If you were to throw a goal at the team nominally in charge of a metric, they might be constrained by the dozens of teams contributing to that metric. You could end up very dissatisfied with the team or with the metric when the reality is that both are insufficient without the context that comes from research.
Metrics can help you measure and manage, but they aren’t enough to support change. With a holistic approach to your metrics informed by close research and countermeasures informed by potentially gamified incentives, you can measure and implement organizational change without falling prey to Goodhart’s Law.
3. Brooks’s Law: “More” is not a strategy
Brooks’s Law states, “Adding manpower to a late software project makes it later.”
Brooks’s Law – most well-known as “the mythical man month,” captured in an essay and book of the same name – sometimes invites the most skepticism because Brooks first argued it in 1975. When Brooks revisited it twenty years later, he found it held up.
Still, that was 1995 – before the introduction of Agile, DevOps, and microservices. And yet, when you look back at these efforts, they’re always at least partially reacting to Brooks’s law.
The Agile Manifesto, for example, decrees that teams should “welcome changing requirements, even late in development” and that they should “deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”
The former principle emphasizes change because, as Brooks warns, being reactive instead of proactive means throwing resources at the problem, which doesn’t work. The latter principle emphasizes delivering software frequently because Brooks shows that, as deadlines slip, increased resources can’t save you, which is especially consequential for massive projects with far-off deadlines.
Similarly, in The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win, by Gene Kim, Kevin Behr, and George Spafford, the authors write that “Any improvement made after the bottleneck is useless because it will always remain starved, waiting for work from the bottleneck. And any improvements made before the bottleneck merely results in more inventory piling up at the bottleneck.”
Without a systems perspective, bottlenecks can go unseen, and adding resources before or after will fail to help the project get delivered any faster.
Strategies to get past Brooks’s Law, like solutions to bottlenecks, tend to come before and after the project itself – either by estimating project duration better or reacting better to changes in requirements, scope, and timeline.
Changing your organization to avoid the fate of Brooks’s Law requires a holistic approach, one that encourages system thinking to see issues before and after a project, as well as the ability to see the system itself as something leaders can iterate on. One delayed project might show your estimation was off, the next might show late-in-the-game requirements delayed it, and the next might show that unseen bottlenecks elsewhere in your value stream were the cause of missed deadlines.
The key is to avoid reactivity: Throwing resources at the problem won’t fix it. Instead, think long-term and use each project as a lesson to change your organization and its processes in small, iterative steps.
Systemic change requires systems thinking
Ultimately, your organization is not a static thing that you can change from one version to another; it’s a complex assemblage of systems, processes, and incentives composed of people with widely varying interests and characteristics.
To make and sustain change, leaders must develop a continuous improvement mindset that emphasizes holistic thinking across numerous categories, including ways of working, alignment to value, velocity, and environment efficiency. Changes need to be in concert with each other and iterative over time.
As Meadows writes, “An important function of almost every system is to ensure its own perpetuation.” The system – through no one’s explicit effort and intention against you – will resist your efforts to change it. These three laws exemplify why.
As a result, hiring consultants and vendors to fix siloed problems will only increase the system-level challenges. Instead, build knowledge of your system, study the laws that undergird it, and develop a strategy for working with your system instead of against it.
Come for the platform. Stay for the solution.
The Uplevel Method empowers engineering leaders at all levels to use data to make change and keep their organizations improving.
