In this article
There's a race happening in AI-assisted software development, and I think most of it is pointed in the wrong direction. The race is toward autonomy: agents that plan alone, code alone, merge alone. The pitch is seductive — describe a feature, come back to a shipped product.
After building an agentic development framework for an enterprise frontend at a large shipping company — one where AI agents routinely take tickets from backlog to pull request — I've come to the opposite conclusion. The teams that win with AI agents won't be the ones that grant the most autonomy. They'll be the ones that design the best governance.
Capability is no longer the bottleneck
Today's coding agents can implement a mid-sized feature competently. That's precisely why the interesting questions have moved. When an agent can do almost anything in your repo, the engineering problem becomes: what should it do, what must it ask about, and what should it never touch?
In our framework, that answer is written down in a constitution — a small set of human-owned documents that agents can read but never edit. Three tiers: Always (run tests, update docs, follow conventions), Ask first (new dependencies, schema changes, anything irreversible), Never (push to protected branches, edit the rules that govern agents, touch credentials). Every AI tool the team uses — not just one vendor's — reads the same contract.
This sounds like bureaucracy. It's the opposite: once boundaries are explicit, you can grant agents far more freedom inside them. The constitution isn't a leash. It's the thing that makes a longer leash safe.
Human gates are a feature, not a compromise
Our pipeline has three hard stops. An agent triages a ticket and writes a specification — then halts until a human says, literally, "Spec approved." It plans and breaks down tasks — halts again. It implements, tests, self-reviews, passes a security audit — and halts one final time before any pull request exists.
Critics of human-in-the-loop call this a temporary crutch until models improve. I'd argue the gates are load-bearing, and not because the model might write bad code. They're where intent gets verified. A spec review catches the feature the business didn't actually want — no amount of model capability catches that, because the model faithfully built what the ticket said, and the ticket was wrong. The cheapest place to catch a misunderstanding is before implementation, and that's exactly where the gates sit.
The subtle design choice: approvals use exact phrases, not vibes. "Looks good, but..." doesn't open the gate. Consent is binary and auditable.
Enforce with machinery, not with prompts
Every team has rules that live in a wiki and die in practice. The agentic version is worse: teams write elaborate prompt instructions — "always update the documentation" — and then act surprised when the model, under context pressure, deprioritizes them.
The lesson that most changed how I build: anything that can be checked mechanically should never be enforced by prompt. In our framework, a runtime hook inspects the git diff after every agent turn; if code changed and documentation didn't, the system itself objects. Each specialized agent — the Jira analyst, the docs writer, the security auditor — is granted only the tools its job requires. The analyst cannot modify code; no write tool was ever wired to it. That's not the agent being obedient. That's the agent being incapable, in the same way least-privilege IAM makes a service incapable.
Prompts are advice. Harness configuration is law. Build accordingly.
Teach the system to remember — carefully
The most underrated failure mode of agentic development isn't bad code. It's amnesia. An agent makes a mistake, a reviewer corrects it, and next week a fresh session makes the identical mistake, because the correction evaporated with the context window.
Our answer is a knowledge lifecycle with three layers and a controlled promotion path. Corrections and PR feedback get harvested into an agent-writable memory file. Memory that proves durable gets promoted into living documentation. And the rules that actually govern agent behavior can only be changed by a human pull request. Knowledge flows upward through validation — the system learns from every review, but an agent can never launder its own conclusions into the rulebook.
That last constraint matters more than it looks. A self-improving system that can edit its own constraints isn't self-improving; it's self-unconstraining.
The framework is the product
Here's the reframe I'd offer any engineering leader: stop thinking about "using AI tools" and start thinking about operating an AI delivery system. Systems need maintenance loops. Ours includes commands that harvest lessons from PR feedback, consolidate and prune memory, and promote validated learnings into docs. The pipeline that ships features gets attention; the meta-loop that improves the pipeline is what compounds.
And accountability is part of the product. Every agent commit in our repo is tagged with the model that wrote it. Every PR carries the agent's reasoning — the approach chosen, the alternatives rejected. When something breaks months later, we know what wrote it, why, and which human approved it. Attribution isn't about blame; it's what makes trust auditable.
Where this goes
Models will keep getting better, and each improvement will make ungoverned autonomy look more tempting. Resist the framing. The mature question was never "how much can we let the agent do?" It's "what structure makes it safe to let the agent do more?"
Autonomy is cheap now. Governance is the craft.
