I lead a medium-sized team at a cybersecurity startup - fast-moving, high standards, low tolerance for bugs in prod.

I am in charge of technical direction, architecture, and defining/raising engineering standards - and AI has effectively thrown a wrench into my perfect world.

As revolutionary as AI is, it's also been the source of many pains, mainly: superficial planning, a reduction in code quality, increased test fragility, and an increase in bugs.

If you're facing something similar - whether you are leading the effort of AI adoption at your company or are a member of an evidently flailing team - then I hope you find something useful in here.

If you're already sold on AI and just want the process, skip to Enforcing Good Usage.

Note that this guide focuses on the most obvious symptom of AI abuse - code quality - over other important aspects of the job - planning, maintenance, and triage. I may do a follow up on each of these in the near future.

TL;DR

I wrote this for dev leads but the same process and principles could be applied to any team or industry looking to leverage AI:

1. AI is a good AND bad multiplier

2. The quality floor is set by the context you give AI

3. You can't mentor what you don't practice

4. Measure AI usage objectively

5. Put the AI cleanup burden back on each engineer, not the team

6. Let juniors build their own AI setup

7. Focus on the whole engineer, not just the output

8. Channel experimentation, don't suppress it

9. Frame everything around results, not AI

Understanding AI Impact

First things first, you have to realize that AI is a multiplier - it's not negative or positive - it is simply a multiplier, so if you put shit in, you get 10x shit; if you put goodness in, you might get 4x goodness.

AI is trained on billions to trillions of tokens of code - most of which stems from public repositories - this includes your average college project as well as that weekend project all of us gave up on. Of course it also includes well-established open source code, but most of it... is bad.

This might get better over time - with better frontier models, selective training data, or evolution in harnesses - but for now this is what we have.

AI also tends to bring out unexpected tendencies in people - you end up subconsciously delegating to AI, and may even stop reviewing the code. You have to actively stay engaged, which is hard to do with meaty problems as it is.

In addition to this, AI is your biggest hype-man; it can fool you into thinking you are Michelangelo when you're accidentally rolling your own version of (unscalable) Instagram on a Sunday night...

If you leave a dev to their own devices, they are MORE likely to yield bad bloated code which isn't suitable for production. This is why you need a process.

Ideal AI Setup

Now, if you're relatively new to AI-augmented programming, you might think - if AI is trained on "average" code then how can it ever produce "good" code?

The answer is context and harnesses:

  • Your existing codebase, style guides, and standards give it direction.
  • Lint, tests, and CI function as checks.
  • Harnesses (Claude Code, OpenAI Codex, etc.) give it access to powerful utility - web search, code interpreter, bash, file edit, git, Jira...

That combination is what makes it a Swiss Army knife - which can either help you or stab you.

Generally speaking, the more context you feed it about your codebase and standards, the higher the quality floor will be.

Create Measurable Metrics

If you aren't using AI effectively, then you can't expect your team to do so either. If you use AI well you can identify AI pitfalls in your team's usage.

You can't go off of feelings because AI is trained to give you dopamine hits. You have to use real metrics - see if there are positive or negative effects on your quality and speed; here are some common ones to start:

Quality Metrics

  • Git etiquette
  • PR review cycles (how many rounds before merge, how many comments)
  • Test coverage delta per PR
  • Number of red commits
  • Missed edge cases
  • Revert rate
  • Bugs in prod
  • KPIs
  • Customer reception

Speed Metrics

  • Time in planning
  • Time in dev
  • Time in code review
  • Time in QA
  • Deployment frequency
  • Estimation in story points vs actual
  • Week-over-week point completion
  • Turnaround time in triage

Establish a baseline from before AI adoption, then track. You're not looking for perfection - you're looking for direction. Is the trend positive?

It's not black and white either - if speed goes up a lot and quality goes down a bit then it might be perfect - it depends on your company, the software, and the business objectives.

If both your speed and quality have stayed the same or dropped, then you have to re-evaluate your usage.

Measurable speed and quality metrics give you true observability into your team's performance, as opposed to arbitrary feelings which are hard to act on.

Enforcing Good Usage by Process

Alright, that's all great and stuff, but how do you actually fix the AI-slop?

Treat every single dev as if AI doesn't exist - evaluate their work like you would have a couple of years ago.

And so if they send you code with:

  • 3 times as many tests riddled with patches
  • Odd roundabout logic
  • Code that doesn't match the style of the company or the repo
  • Mixed concerns
  • ...

You tell them like it is.

And if it's a repeat problem, then the conversation gets more serious and managers get involved - not because they are using AI, but because their speed or quality has suffered. This is a teaching moment, not a punishment for using AI.

Some people will need more help than others; this is where your expertise in AI-use will come in handy - work through a problem with them driving and utilizing AI - if you notice signs of over-reliance or AI-roulette, give them meaningful alternatives based on your own use. (You've been using AI effectively right?)

Now this is easier said than done; it can be overwhelming for you and the team. In order to make it feasible you need a checkpoint before anything gets into the codebase, and a mechanism to distribute the load.

Placing Systematic Checks

Put the onus of the PRs back onto the dev who wrote it.

Sure, AI made that dev 5x faster - now they can spend all their free time fixing the PR before another person ever lays eyes on it.

The minimum requirements I set in my team are:

  • The PR has to be ~500 lines max
    • Large PRs are where reviewers give up and bugs slip through
  • The PR should be covering a single concern
    • If you can't describe it in one sentence without "and", split it
  • The PR title should answer WHY and use some commit standard (conventional, etc.)
    • Good: fix: Handle transient upstream failures on payment service.
    • Bad: add retry logic
  • The PR should include tests or mutations to tests
    • No test changes means no proof of expected behavior
  • The PR should have a description
    • Must be short and to the point - no dev has time to read an (AI-hallucinated) novel
    • Should mention what changed, why, and anything the reviewer should watch for
  • The PR has to have a green build - coverage, lint, test, build reqs
    • No exceptions
  • The PR has to be atomic
    • Ensuring easy reverts
    • A prod incident is the wrong time to find out your fix was tangled with three other changes

Enforce minimal PR expectations and place the onus on the dev making the PR, and suddenly the number of AI-driven low-effort PRs is reduced.

Fixing AI-slop after the fact is not fun - this also gives each dev incentive to proactively improve their AI usage, so that the weight of the AI-generated issues doesn't land on them at the tail-end.

This generally solves the systematic issues with AI usage, now we can focus on the fun stuff.

Encouraging Experimentation

The goal is to channel experimentation, not suppress it.

You want to give the devs some leeway in their experimentation - they should be able to flex and mess with AI how they see fit - but experiments should not be found within production codebases, and to that end we need outlets.

Eng demos, brown bags, serendipity days, spike/research tickets, Slack channels are all great avenues for inspiring experimentation.

You want to create 'wow' moments for your team and company, inspire them to want to try AI in fruitful ways.

The best demos are the ones that can answer 'yes' to any of the following:

  • Did it meaningfully change how you work?
  • Did it save you real time?
  • Did it produce something you genuinely didn't expect?
  • Did it open you up to possibilities you didn't even imagine?

But what if a dev doesn't want to use AI?

Dealing with Holdouts

That's fine.

Remember, you're evaluating them the same way regardless - speed and quality. If they're hitting the bar, there's nothing to address.

If they're struggling, AI probably won't fix it anyway. AI is a good AND bad multiplier - a dev who can't design a clean abstraction or debug a race condition won't get better results by generating more code faster. They need more fundamental help first.

Encouraging Growth

One of the biggest AI victims by far are junior software engineers. Their avenues for learning are reduced - they don't run into the same hurdles you would have years ago. AI outputs "okay" code compared to a novice dev, and can solve a fair amount of issues - so they have fewer opportunities to learn.

To that end, refrain from giving less-experienced devs that set of perfectly curated markdown files that you have iterated over countless sessions late at night - instead, let each dev build their own.

AI output looks authoritative even when it's wrong, and it may compress planning, design, and implementation into a single run - so a junior never learns how those steps connect, and you end up unteaching bad habits instead of building good ones.

You might think an AI context file is just another tool config - it isn't. A linter or style guide enforces rules mechanically - a junior still has to think, make mistakes, and get feedback. An AI context file tells the model how to reason about a domain. In the hands of a novice who hasn't earned it, a heavy context file doesn't assist their thinking - it replaces it.

Ultimately, code throughput is not the goal - investment and growth are. If an engineer works on their fundamentals, they become a major asset in all aspects of the job, not just code throughput - planning, architecture, maintenance and triage to name a few. Improvement in one aspect also reinforces improvement in all other aspects. An engineer who is only good at outputting large amounts of code isn't going to be helpful with time-sensitive prod issues, and the code they output will likely not account for the scale and factors you need to consider long-term.

AI usage is personal, like a dev's IDE or terminal setup. Every dev needs to spend time figuring out how AI complements them. Guide them toward building their own context files - through their own trial and error. If your company is pushing AI broadly, keep it loose - build them a starter kit per the AI setup section above, but don't overprescribe tooling.

Invest in fundamentals over quick wins. Don't go all caveman - but don't hand them the answer.

Dealing with Resistance

The most common resistance you'll get is from senior devs - they'll tell you they move fast, that the PR checks slow them down, that their code doesn't need as many reviews. And sometimes they're right - but most of the time they're just protecting a workflow that AI has made lazy.

The key is to keep the conversation away from AI entirely. You're not telling them their AI usage is wrong, you're telling them their output doesn't meet the bar. That's a timeless conversation.

Leadership is a different challenge - they may push you to ship faster using AI and see process enforcement as obstruction. Used correctly, AI SHOULD increase velocity, but it needs to be approached with care. Show them that the systematic checks increase velocity and prevent regressions. Better yet, prove it - treat it like a science experiment and monitor the results. And hey, if your experiment fails (which I highly doubt) then eat your words.

Dial the process up or down based on the goal. If you're building a demo or a trial, open the gates - but set expectations upfront with Product and stakeholders: anything built that fast will need a proper rewrite before it goes anywhere near production.

So whether you're dealing with resistance in your team or from leadership, it comes down to removing AI from the equation and strictly focusing on results.

I don't know your reality or your company's culture, and many of these suggestions may seem far-fetched - that is completely expected. Accommodate and compromise so that you can find a happy medium - aim to find a solution that doesn't keep you up at night if you can help it.

Fin

A good dev is stable yet always learning, picking up any tools and addressing concerns - SLAs, scale, latency, stability, CI times, code quality, team health. AI is just another tool - a very powerful one - aim to use it, evaluate your usage, and adjust as needed.

If you are performing at the same level as a one-shot AI prompt, then how do you differentiate between yourself and the AI? What would make you or any of the devs on your team worth hiring - over spending a fraction of the price on a few Claude Max subscriptions?

It's your responsibility to help your devs grow and leverage AI effectively - not just to build a stronger team, but to invest in them as engineers. The devs who come out of this era knowing how to think, design, and execute will always have a place.

At the end of the day you have to ask yourself - are you and your team augmenting AI or is AI <negatively> augmenting you?