Guides for engineering managers
How to show leadership what AI is doing for your team, without counting tokens
Token counts and AI-written lines mislead leadership and upset engineers. Here is what to report instead, on one page a month.
- Token counts, AI-generated lines of code and "percent of code written by AI" measure activity, not value. They also make engineers feel watched.
- Report what leadership actually cares about: what shipped, how fast work moves, whether quality held, and what you could do that you couldn't before.
- Keep it to one page a month, at team level, with two or three concrete examples.
- If your CEO is under cost pressure, put the AI bill next to the outcomes it bought, not next to headcount.
Sooner or later someone above you asks the question: "What is AI actually doing for the engineering team?" Sometimes it's curiosity. More often it's a board member, an investor, or a growing line on the tools budget.
The easy answer is a usage chart. Your AI tools will happily tell you how many tokens the team burned, how many suggestions were accepted, and how many lines were "AI-written". Those numbers are easy to get and easy to put in a slide. They are also close to useless, and they can do real damage to your team. This guide covers what to report instead, with a one-page template you can use this month.
Why token and line counts mislead
Usage numbers answer "how much did we use the tool?", not "what did we get for it?" A few specific problems:
- More is not better. An engineer who spends a day going back and forth with an agent on a confused prompt burns more tokens than one who thinks for ten minutes and gets it right. High usage can mean high leverage or a lot of thrashing.
- Lines of code were never a good measure. AI makes it cheap to produce lots of code, so counting it is even less meaningful than before. A 3,000-line change can be worse than a 30-line one.
- "Percent written by AI" is unknowable. Code gets generated, rewritten, partly kept and edited by hand. Any single number hides that.
- Whatever you count, people optimise. If leadership sees a token leaderboard, usage goes up whether or not anything useful happens.
The human cost matters just as much. Engineers notice when they are being measured on tool usage, and they don't like it. DX puts it plainly: research shows developers don't like metrics aimed at measuring individual engineers. Keep metrics at the team level and out of individual reviews.
So the goal is to give leadership a real answer without turning AI adoption into surveillance.
What to report instead
Leadership doesn't want to know how the sausage is made. They want to know whether the team is shipping more of what matters, whether it's getting faster, and whether anything is breaking. Four kinds of evidence answer that.
1. Outcomes shipped
Start with what reached customers or the business this month, in plain language. "Shipped self-serve SSO, which unblocks two enterprise deals" beats any chart. Where AI played a clear part, say so in a sentence, but the headline is the outcome.
2. Cycle time and review load trends
Two or three flow numbers, shown as a trend over several months, not a single snapshot:
- Cycle time: from first commit or ticket start to production. Is work moving faster?
- Review load: pull requests waiting for review, time to first review, or average PR size. AI makes writing code faster, which often shifts the bottleneck to review. If PRs are getting bigger and sitting longer, leadership should hear that from you before they see it as a slip.
- Throughput, carefully: features or meaningful tickets completed, not commits.
3. Quality signals
Speed that comes with more incidents isn't a win. Pick a few signals you already have: incidents and their severity, bugs reported by customers, rollbacks or hotfixes, flaky test counts. Report whether they held steady while speed changed. "Cycle time down, incidents flat" is a strong sentence.
4. Work AI made possible
This is the part numbers miss and the part executives remember. Collect two or three concrete examples a month of work that would not have happened, or would have taken much longer, without AI:
- A migration the team had postponed for a year, finished in two weeks.
- Test coverage added to a legacy module nobody wanted to touch.
- A designer or PM shipping a small fix themselves instead of waiting a sprint.
- An internal tool built in an afternoon that saves support an hour a day.
Ask the team for these in retro or standup. Engineers are usually happy to share what worked when it's framed as a team story rather than a personal score.
A one-page monthly AI update
Send this once a month, in the same format each time, so leadership learns to read it. Keep it at team level. No names next to numbers.
Monthly engineering and AI update template
- Headline (one or two sentences). The most important thing that shipped and what it means for the business.
- Shipped this month (3 to 5 bullets). Outcomes, in customer or business terms.
- Flow (a small table or three lines). Cycle time, PRs waiting for review, average PR size. This month, last month, three months ago.
- Quality (two or three lines). Incidents, customer-reported bugs, rollbacks. Note any change and why.
- What AI made possible (two or three examples). One sentence each: what we did, how long it took, what it would have taken before.
- Cost (one line). AI tool spend this month, and what it bought, pointing back to the sections above.
- What we're changing next (one to three bullets). For example, a review rotation to deal with larger PRs, or a tool trial you're ending.
Here is how the "made possible" section might read. The details are illustrative:
Maya moved our remaining 140 API endpoints to the new auth middleware in eight working days, with an agent doing the repetitive edits and Maya reviewing each batch. We had estimated six weeks and kept postponing it. It closes the last item from our security review.
The team used 41 million tokens this month, up 30%, and 62% of merged code was AI-assisted.
Framing it for a CEO under cost pressure
Many CEOs are now hearing two messages at once: become "AI-native", and spend less. Gergely Orosz describes this squeeze in The Pragmatic Engineer article "Headed for the Exit: the Great Engineering Leader Career Break" (August 2026), where CTOs are asked to cut engineering costs while the "CTO faces pressure on business results as AI coding bills rack up". If your AI bill is climbing, your CEO will ask about it whether or not you raise it first.
A few ways to make that conversation go better:
- Raise the cost yourself. One line in every monthly update, with the trend. Surprises are what make finance nervous.
- Tie spend to outcomes, not to people. "We spent this, and here is what shipped" is a better frame than "this is cheaper than a hire". The second invites a headcount conversation you may not want.
- Show where you're cutting waste. Seats nobody uses, tools that overlap, experiments you stopped. It shows the budget is managed, not just consumed.
- Be honest about the limits. If review load is rising or quality slipped, say so with your plan. Credibility on the bad months is what makes the good months believable.
- Don't promise a multiplier. "AI makes us 2x faster" will be quoted back to you at the next budget meeting. Show the trend and let it speak.
Gathering the evidence without extra work
Most of what goes in this update already exists: merged pull requests, closed tickets, release notes, incident logs and Slack threads where someone said "this would have taken a week". The work is pulling it together once a month. A short running note, updated weekly, makes the monthly update a 30-minute job rather than an afternoon. This is also the team-level brief Keel drafts from GitHub, Jira and Slack, without counting tokens or lines.
If you want the metrics side in more depth, read Measuring AI coding tools: what to track and what to ignore. For keeping a running record of individual work that feeds both reviews and updates like this one, see Brag docs for your whole team.