# How to Set Up Technical KPIs When You're Not an Engineer
A founder I advise had a quarterly business review with her board last year. She had beautiful metrics on revenue, user growth, and churn. Then a board member asked: "How's engineering health?" She had no answer. She hadn't tracked anything about her engineering team's performance.
The board member wasn't being unfair. Engineering health directly predicts a startup's ability to execute. A team shipping quality code fast will outcompete a team that's slow and bug-prone, all else being equal. If you're not measuring it, you're flying blind on a significant portion of your operational risk.
The good news: you don't need to read code to track engineering health. These five metrics are observable without technical expertise, and together they tell you most of what you need to know.
Metric 1: Deployment Frequency
What it is: how many times per week your team ships changes to production.
Why it matters: teams that deploy frequently have broken their work into small, manageable pieces. Small deployments are easier to test, easier to roll back if something goes wrong, and surface problems faster. Teams that deploy infrequently are working in big batches - and when something goes wrong (and it will), the impact is larger and harder to diagnose.
How to track it: ask your engineering lead to give you a weekly number, or check your deployment tool (most platforms like Heroku, Vercel, or AWS CodeDeploy have deployment logs). This takes five minutes per week.
Benchmarks to aim for: world-class teams deploy multiple times per day. Most healthy startups should target three to five deployments per week. If you're deploying less than once a week, ask why.
I set this up for a client running a logistics platform. In the first month, deployment frequency was once every three weeks. That told us the team was working in big batches and had high deployment risk. Within two months of tracking and improving the process, they were at three deployments per week and their bug rate dropped by 60%.
Metric 2: Lead Time for Changes
What it is: how long it takes from when a developer starts working on something to when it's live in production.
Why it matters: long lead times mean your ideas take a long time to reach users. That slows your feedback loop and your ability to compete. It also often signals process problems - complicated code review, manual deployment steps, or organizational bottlenecks.
How to track it: most task management tools (Linear, Jira, GitHub Issues) track when tasks are created, started, and completed. "Lead time" is from start to live in production.
Target for early-stage startups: one to five days for a typical feature. If common features routinely take two to four weeks from start to deployed, you have a bottleneck worth investigating.
Metric 3: Mean Time to Recovery
What it is: when something breaks in production, how long before it's fixed.
Why it matters: every production incident costs you users and trust. But the question isn't whether incidents happen - they will. The question is how quickly your team recovers. Fast recovery means good observability (they know something is broken quickly) and good process (they can diagnose and deploy a fix quickly).
How to track it: keep a simple log of production incidents. When was it reported? When was it resolved? What was the user-visible impact?
Target: for most startup products, you want to resolve critical issues (product unusable, data at risk) within two to four hours. Non-critical issues within 24-48 hours.
A client building a marketplace had three major incidents in one quarter. The first took 18 hours to resolve. The second took 6 hours. The third took 2 hours. That trend - 18 to 6 to 2 - tells you the team is learning, investing in their tools, and building recovery capability. That's a healthy trend even though the incidents themselves weren't good.
Metric 4: Error Rate
What it is: what percentage of your product's operations result in errors.
Why it matters: high error rates mean users are having bad experiences. Even if they don't report them, errors cause frustration and churn. An error rate that's increasing tells you technical debt is accumulating or the team is shipping lower-quality code.
How to track it: every production product should have error tracking installed. Sentry and Bugsnag are the most common tools. They take less than a day to install and then automatically capture errors and report rates. This is not optional infrastructure - it's table stakes.
Ask your developer: "What's our current error rate, and is it going up or down?" If they can't answer, you don't have error tracking, and you should fix that this week.
Target: this varies by product type, but in general, your error rate should be below 1% of operations and trending down over time, not up.
Metric 5: Sprint Completion Rate
What it is: what percentage of work committed to in a given sprint actually gets completed within that sprint.
Why it matters: consistently completing 40% of committed work means your team is either over-committing (planning problem) or getting blocked by unexpected issues (process problem). Consistently completing 90% means either your team is sandbagging (under-committing to look good) or genuinely executing well.
The goal is accuracy, not 100%. A team that commits to 10 points and completes 9 consistently is more trustworthy than a team that commits to 20 points and completes 18 but occasionally commits to 20 and completes 8.
How to track it: any project management tool tracks this automatically. Review it at the end of each two-week sprint.
A client whose team had a 55% average completion rate had no idea there was a planning problem. When we started tracking it explicitly, the team became more conservative in their estimates, started breaking down large tasks, and hit 75% completion within six weeks. Knowing what you're measuring changes behavior.
How to Use These Five Metrics Together
Track all five in a simple spreadsheet or project management dashboard. Review them monthly with your engineering lead. Look for trends, not absolute numbers - most of these metrics will be poor at first. What you're watching for is direction.
If all five are improving over time, your engineering operation is healthy even if the absolute numbers are still modest. If two or three are consistently declining, you have a systemic problem worth investigating.
The board member who asked my client about engineering health? She implemented this dashboard in the following quarter. At the next board review, she had a clear answer - and it demonstrated operational maturity that her investors specifically cited when they approved her next raise.
You don't need to be technical to run an engineering organization. You need to measure the right things.
Book a 30-minute call: https://calendly.com/alpsf/zoom-with-aleksandr