In 2024, BCG found that two-thirds of large-scale tech programs miss their targets on time, budget, and scope — and only 30% fully achieve their objectives. A more recent study found that 95% of enterprise companies are seeing zero return on investment from AI initiatives.
Volkswagen's software division Cariad illustrates this pattern at scale. Founded in 2020 to build a unified operating system across all VW Group brands, it accumulated over $7.5 billion in operating losses between 2022 and 2024. Insiders reported 17 status meetings per week. Developers made slide decks instead of shipping code. After approximately €14 billion invested, Volkswagen turned to Rivian for the software platform it couldn't build internally.
Many organizations struggle to meet targets and achieve sustainable change. After a series of failed initiatives, wasted resources, and missed opportunities, teams revert to familiar practices. Organizational inertia drags transformation out for years instead of months.
The problem: what to measure is different for every company.
Why is engineering transformation so hard to measure?
Software engineering transformation resists standardized metrics because context determines what success looks like, and no two organizations share the same starting point.
The idea of measuring progress with a universal framework is a little like measuring health by weight or BMI. (The Rock weighs 260 pounds and his BMI would categorize him as "morbidly obese," which we know is not the truth.) It paints an incomplete or inaccurate picture in a situation where context matters.
"Digital transformation means different things to different people," says Amy Carrillo Cotten, Uplevel’s Director of Client Transformation.
AI tools add a specific new layer to this problem. Knowing how many coding licenses are deployed tells you nothing about whether developers are producing better outcomes — the gap between AI adoption and AI impact is one of the most underexamined constraints in transformation work today.
Here’s what makes measuring success particularly difficult:
- One size metrics do not fit all. What success looks like is unique to your organization and depends on a number of factors like core competencies, architectures, components, and teams involved. Traditional organizations will have different measures of success than digitally native tech companies as they often have more inertia to overcome.
- Sustainable transformation goes beyond technology. It's a systemic, cultural shift in ways of working and adopting modern development practices that support cross-functional collaboration, transparency, agility, and value delivery. Many companies underestimate the investment these changes require and struggle to measure it.
- Enterprise systems are complex. Companies often underestimate the dependencies of their technical systems, hiring multiple consultants and vendors to advise and execute. These siloed efforts create inefficiencies that work against the goals of the transformation and harm visibility of progress as a whole.
- Transformation is never (really) done. The purpose of transformation is to be agile and responsive to market needs — to continually optimize people, systems, and processes. Defining and measuring success in that context requires a different approach than a project checklist.
What should you measure during a software engineering transformation?
Engineering transformation success can be measured. The key is to establish measurement practices before the initiative is underway — not after it stalls.
If "what gets measured gets managed," modern development practices are often left behind when measurement gets treated as a phase-two problem. Transformation becomes a list of projects to complete rather than a state change to track, and the longitudinal data that would reveal real progress never gets collected.
Starting to measure development practices before a transformation begins makes the transformation more likely to succeed — and how that sequencing plays out determines whether what follows sticks. Having the right engineering metrics helps you identify areas for improvement and monitor whether those practices are actually being adopted.
The sooner you can establish practices that ensure developers have enough time to do their work and are working on the right things, the sooner you can identify the major bottlenecks that are the biggest impediments to progress. Those bottlenecks are likely going to be a core component of your transformation roadmap.
Which engineering metrics actually drive transformation progress?
Digital transformation in engineering often suffers from trying to do too much at once, leading to inefficiency and lack of direction. While there’s no one way to measure success, it begins by identifying the biggest opportunities and prioritizing them. What technical priorities will help meet business objectives?
Matt Swann explains that “for digital businesses, technical priorities should be mapped clearly to key business priorities. It's critical to change the language from technical to business to ensure the business understands and cares about technical needs.”
Pre-transformation, technology priorities and business priorities might not have aligned. When the role of technology is supportive and not strategic, OKRs might look like achieving 99.999% server uptime or reducing IT support ticket resolution time — important goals, to be sure, but they don’t move the needle on a P&L.
Digital transformation changes that. Priorities now tie much more closely to what the business needs to achieve, and the metrics that become most important are the ones that reveal the sustainable progress of those priorities:
| Business Priority | Technology Priority | Engineering Metrics |
| Create a seamless omnichannel customer experience | Deliver frequent value in ecommerce and omnichannel features | Time spent vs. allocated to innovation and high-value initiatives |
| Quick response to market shifts | Replatform for scale and agility | Time spent vs. allocated to technical debt/KTLO work |
| Deliver X features this year without raising overhead | Adopt modern development practices | Delivery and quality (DORA) metrics |
Some metrics will be lagging indicators, but others can be early signals of where progress is happening — or isn't. Understanding actual vs. predicted performance lets engineering leaders measure progress, adjustbenchmarks, budget resources, and give executive teams the "why" behind the numbers.
No framework or set of metrics will be perfect. The right metrics reflect the priorities of your transformation and the directionality and velocity of the change you're trying to make.
How do engineering leaders get the visibility required?
Visibility into engineering activity is what makes transformation goals trackable. Without it, progress is estimated rather than measured — and estimates don't hold up in executive reviews or budget cycles.
Headcount is an engineering organization's largest budget item. Tracking how that investment is actually being spent — in terms of time allocation, focus, and sustainable ways of working — requires telemetry, the same way DevOps and cloud monitoring platforms provide telemetry for infrastructure.
The right engineering intelligence platform gives engineering leaders a way to align team activity directly to business outcomes. Combining standard productivity metrics with harder-to-surface indicators — how developers actually spend their time, where review complexity is building, where AI adoption is generating throughput and where it's hitting a ceiling — produces the picture that actually matters for your transformation goals.
What identifying the real bottlenecks looks like in practice
The pattern Uplevel sees across engineering organizations: teams know something is slowing them down, but the metrics they're tracking don't show where the constraint actually lives.
For example, at a distributed hardware company, AI coding licenses had been in place for over a year. Uplevel's GearUp sprint found that shallow enablement and quality gaps were capping how much teams could benefit from the tools already deployed. Test coverage was inconsistent across teams, with no sustainable program to build on. Based on those findings, the organization launched a structured enablement program — featuring dev-to-dev knowledge sharing, agentic PR reviews, AI-assisted test generation, and expanded coding standards. Test automation coverage increased 30–50% across teams, production incidents declined, and PR cycle time dropped 10%.
That outcome started with knowing what to measure.
.png)
