David Subar has spent a lot of time thinking about engineering effectiveness. As the founder and managing partner of Interna, he works with technology companies to improve their product management and engineering to be more effective. He keeps a strategic focus on a lean startup model. To some, that means efficiency, but David is quick to clarify that the goal is effectiveness.
In his words: “Efficiency is running fast. Effective is making change. You can run really fast in the wrong direction, and just get lost.”
His career started in research and development in machine learning and artificial intelligence, at a military think tank in DC. What sounded cool became frustrating. “You do a project, you write a paper, you present at a conference, 200 people listen, and then nothing happens. The world needs scientists, I’m just not one of them. Scientists provide foundations and materials. I want to build things with them.”
After years as CTO and CPO of many companies, he narrowed in on this passion. “I wanted to build stuff that made a difference. To be able to say “Hey, Mom, this is what we did. This is how we affected the world. Over time, I learned the best way to get leverage was to build great teams that could build things effectively.”
What has been one of your greatest successes or something you’re most proud of as an engineering leader?
I built a game company from nothing. We built games for Activision and IBM. It was building something that people used — I don’t know how important games are, but they’re fun. We were using technology to create leverage and to entertain people.
Now, one of the clients I am working with is Reify Health. It’s completely different — a B2B SaaS for clinical trials for pharmaceuticals. They help pharmaceutical companies who have a molecule that could be a drug, and clinicians who test it, run the whole process from finding clinicians to finding patients to drug testing to protocols to data collection, end to end. The company has grown from 40 to 160 people in two years during our partnership, and even stronger revenue growth. They’re working on COVID trials. We help Reify build things that really matter, that make a difference in people’s lives.
The game company and Reify Health are totally different, but they’re both ways to create teams that create value. They are both “Hey, Mom” things. I am glad to be a part of that.
What has been one of your greatest challenges as engineering leader over your career? Or, what keeps you up at night currently as an engineering leader?
Creating world-class teams is hard. Recruiting is really, really hard. Not just because there’s not enough great engineers — there’s just not — but getting the right people on the right team in the right place, and creating the environment where they can be successful. Aligning to value creation is a big issue. If an engineer thinks their job is to write code and a product manager thinks their job is to write user stories, they’re wrong. Those are necessary aspects, but not sufficient. They are only important if you can ship a product, and if that product changes a market. Getting the right people into place is a hard aspect of the job. Getting them aligned to the goal, creating value and shipping products that matter, and ultimately building companies, that is what is important. It’s what makes the job great.
Recruiting at high-growth companies is easier than recruiting at low-growth companies, because you have a story to tell. Engineers have choices of jobs. Paying is fungible. The next company can pay more, so you have to create value — not just for the market, but for the engineer: Why am I here? How do I contribute? Why does that have meaning? If you’re growing, you have a story to tell. If you have a high-growth company, or if you can tell how the engineers’ work will have an impact, you have something to sell to them. You have an intangible that is non-fungible.
How would you define engineering effectiveness?
Did the product we built have value for the market? That is the ultimate measure. Effectiveness is about making change. Revenue is often mistaken as the goal — it is a side effect of creating value for somebody else. If you create value and price your product or service with unit margin, you will generate revenue and profit.
You have to be smart about what you choose to do and why. My epics on the roadmap are making a bunch of bets. We have a thesis that we believe will create value, but we don’t expect every one to be right.
Snap CEO Evan Spiegel once told me, “We don’t get everything right. We just don’t market what we get wrong.” [laughs again] He was absolutely right. Build it, get it out there, see how the market reacts, then iterate.
What are some signals you use to know that your team is effective?
I think revenue is a secondary factor. You have to measure it, though. There is a name for companies without revenue for a long period of time. They are called a “hobby.” [laughs] The primary metrics resolve around what the market needs. By having [x] feature, it will have a [y] effect on the user, which we’ll see in [z] metric. That is how you determine if your team is effective – do they create market value.
Each item on the roadmap needs to answer four questions:
- What are we doing?
- For whom?
- How will it benefit them?
- How will we measure it?
After release, you return to the metrics:
- Did we achieve the goals?
- What did we learn?
- Was it a reasonable bet?
No battle plans survive first contact with the enemy. You can’t be good at knowing exactly what the market wants — but you can be directionally correct. The key is to execute and learn.
What’s your best advice for an engineer being promoted into their first management role?
Is this really what you want to do? [laughs] Most companies have two tracks, and one is more economically exciting than the others. Executive positions are paid better than engineers.
A lot of people go into management because it eventually has more prestige. The skills that made you a good engineer aren’t going to help you much — except for a couple. (1) Organizational design looks like software architecture and (2) You have to be a really good listener. Those engineering skills will be helpful. There are important skills you need that being an engineer does not train you for.
If you are considering going into management, think hard. Is this what you really want? Is this what you will be best at? If so, definitely do it. If not, be the best engineer you can be. The world needs you.
Conway’s Law says that you are destined to ship based on how you organize your people. Organizational pods around who you serve will create value — ship based on that. If you have a front-end team and back-end team, you won’t necessarily create user value. Just like you refactor your software, you need to refactor your organizational structure. A company’s needs change. Expect your org structure to change too.
Reflecting on your past experiences, what data have you found most valuable to help you manage an engineering organization? Are there any hard metrics you have found to be useful in managing your teams?
A manager’s job, at any level, is to set goals: here’s what we want to get to, and here’s why. If they’re good, they ask the team, please argue with me. Ask me why this is what we’re doing. If you find a hole in the logic, we will pivot. Next, it is the manager’s job to ask the team how they plan to accomplish the goal. Ask, don’t tell. When the team comes back with a method, ask hard questions; don’t tell what is right and wrong. Asking forces careful consideration and finds holes. Telling forces a direction. You want your team to have their ideas battle-tested and hardened, not dictated to them.
Within an engineering organization, the tenants of agile methodologies are true. Velocity and scrum, cycle time, and kanban are all good and important metrics. Even though those measure efficiency, the user stories talk about value creation.
On the other hand, your CEO probably doesn’t care about algorithms, velocity, or cycle time. Your CEO will care about value creation. The metrics you use to communicate to your team need to include value creation, but might include metrics that are not helpful to communicate with the CEO. Be wise about your audience and what is useful for them to hear.
Have you set goals or used data to manage your organization that turned out to be ineffective? Why?
Separating engineers from the rest of the organization—“protecting” them from outside distractions—is a dangerous thing. You want to increase communication so you can increase context. Engineers can do almost anything, but without the context, they won’t know which one to pick. Keeping the engineers in a black box also decreases the ability of the rest of the organization to know what Engineering is doing and decreases the possibility of redirecting with new information. Understanding the right data to use and the right integration to use is critical. You don’t want sales changing engineering just to hit a goal by the end of the month, but you also don’t want engineering cloistered like Benedictine monks.
Given a much greater WFH culture due to Covid, when you look forward over the next 5-10 years, what skills, tools, data, etc. will managers and leaders have to master to be effective leaders?
It comes back to the question: Why do I work at this company? Part of that is the mission, part of that is the people. Zoom is two-dimensional in every way—we lose the power of knowing each other. Breaking bread is a powerful tool. People start to know each other and talk about more than code. The challenge is to create that cohesion and culture when you don’t see people. At some point, you need to be face-to-face—not every day, but they need to come together to feel like they’re working with people and not just screens.
David Subar is the Founder and Managing Partner of Interna. To book a consultation, reach out here. For more of his insights, you can follow him on Twitter, LinkedIn, and Clubhouse.