Even if you’re not an engineer, you know how ubiquitous AI-driven development is.
You’ve heard about Cursor’s meteoric rise in popularity. You’ve seen the way people geek out about what they’re building with Claude Code (so much faster than they could before) on Reddit, X, Substack, and pretty much every tech-adjacent podcast.
Which makes it easy to assume that you just ask these tools to do something and they do it. Straightforward as that.
But if you’re an engineer using these tools day in and day out, you know that getting LLMs to do exactly what you want them to do is no walk in the park. And like pretty much every other engineering org out there, we’re trying to figure out how to take maximum advantage of these tools without shipping AI slop.
Most of that figuring out happens in private, but we’d rather share what we’ve learned about spec-driven development and hear what’s working for everyone else.
Below are two strategies we’ve built over the past few months to bring AI into our software engineering lifecycle, so we ship features and fix bugs faster, without the slop.
Our First Big Spec-Driven Development Experiment: Spec Kit
When you start using LLMs to code, you quickly realize that there’s a lot of work that goes into getting them to behave the way you want them to.
Getting really good at being really prescriptive in your prompts, explaining every last detail, giving these models every piece of context they could possibly need.
Designing (and redesigning) checks at each stage of the development lifecycle to catch sloppy code that can slow down your app or platform.
Reminding LLMs of your standards and the way you do things, over and over and over.
It takes practice, it takes trial and error. So when we came across GitHub’s Spec Kit, basically a template for describing what you want to build to an LLM, we got excited about this head start.
How We Modified Spec Kit to Our Spec-Driven Development Workflow
Looking into the Kit, we decided to:
- Pare down some of the commands, so it was easier for our devs to adopt and work with
- Tweak others to fit our standards at OpsMill (for example, follow-ups that fire on events, like summary and handoff), to make sure outputs were even closer to what we wanted
- Add some extensions on top (extract, retrospect, summary), to handle more of the steps in our current workflow autonomously and close the loop
Once we got it to a good place, we introduced it to our engineers.
Our Spec-Driven Development Workflow
It took a minute to get used to the new workflow. Instead of diving straight into code, every new feature now must start with a markdown file, which explains:
- How the feature should work
- Who it’s for
- What the tradeoffs are
- What’s in scope
- What’s not in scope
Our engineers input a problem statement and a suggested solution, then one of our extensions takes that and turns it into something written for an LLM to read rather than a human. That spec goes up as a pull request and gets reviewed just like code would.
Once everyone agrees that, yes, this is what we’re building, and everything the new feature needs is captured in the specs, the feature moves to the “plan” phase, where the LLM uses the spec to outline what code changes need to happen to get to the desired outcome. Things like:
- Which files will get touched
- What functions need to be added or changed
- What the database schema looks like
Note: This plan is not code, it’s more of a “here’s what I (the LLM) would write if you give the ok.”
That’s what we want at this stage, because it’s way faster for our devs to review and adjust markdown files now than to review thousands of lines of code later. After the plan is approved, the coding starts.
As you can see, we’ve added a few additional commands to push a summary to Jira, making it even easier for developers to review the end result:
If you’re curious, you can check it out on GitHub.
The Results
At some point, we want spec-driven development to become the main way we develop new features. And since we started using this workflow in February, we’ve already shipped close to 115 pull requests.
Code review is still a bit of a bottleneck, but this way we speed up the process significantly by reviewing and reading the specs upfront, and we avoid “LGTM” reviews for large diffs, ensuring high-quality output.
We’re starting to put measures in place to track how this new workflow is impacting review time and merge rates, plus bugs (we’ll get to this next).
A couple other ancillary benefits we’ve seen:
- It helps catch edge cases. Being forced to go through a spec surfaces the edge cases we hadn’t thought of, helping us identify potential problems before code is even written, let alone pushed to prod.
- It helps with onboarding. A new engineer doesn’t have to absorb all of Infrahub before they can contribute, because the specs and our internal knowledge base carry a lot of that context for them. They still have to bring their own judgment, reading a spec and saying “No, that’s not how this should work” is its own skill, and this back and forth is a good way to learn the codebase.
What We Haven’t Figured Out
Real development isn’t a step 1, step 2, step 3 … type of workflow. You might write a spec, get that approved, have the LLM create a plan, get that approved, and then you talk to a colleague or customer and something changes.
That’s a problem for us because the spec, and the results from using that spec, are what we want to feed back to the LLM for learning. And if a spec is stale by the time a feature ships, then we’re feeding stale decisions back to our knowledge base.
Right now, closing that loop is manual. Once the PR is merged, we run the retrospect, extract the knowledge to our internal docs, and then archive the specs. We’re trying to use agents to fix that.
Care to experiment yourself? Here are our repository links for:
Our Second Experiment: An Agent-Driven Bug Analysis and Fix Workflow
Infrahub is open source, so anyone can submit an issue, one of our customers, someone on our internal team, or any engineer trying to build out their intent schema as a POC.
Which is great, because it helps us find and fix bugs faster. The flip side is triage. Normally, an engineer would read someone’s short bug description, then have to dig into the code to figure out what’s happening before deciding whether it’s worth fixing now or later.
To expedite this process, we built an agent-driven workflow.
How it Works
You point the agent at a GitHub issue, either by running it from the CLI with the issue number or by triggering it inline on GitHub with GitHub Agentic workflows. It reads the issue and the codebase, finds the root cause, reproduces the bug, and suggests a fix.
That fix is based on the same knowledge base our spec-driven workflow uses. It explains how our code works, the architectural decisions we’ve made, and the best practices the team has agreed on.
Once the analysis is finished running, it gets posted back to the GitHub issue. An engineer is then required to assess and review the analysis. This step intentionally requires supervision. That way, we can continuously improve the analysis step (fundamental in moving on to the next step).
The next command in sequence is the /bug-tdd, which writes a failing test that reproduces a confirmed bug. This step verifies that the incoming fix tackles the posted issue.
And finally comes the /bug-fix.
The Results
It’s still really new, so we’ve only merged and shipped around 15 fixes with it, but we’re already seeing how much faster it’s making our triage.
With our agent bug-fix workflow, the analysis – and the likely fix – is right there at our engineers’ fingertips, making it easy to spot the high-impact ones and get solutions shipped quickly.
What We Haven’t Figured Out
In a perfect world, we’d have one unified workflow, where, regardless of the type of work (new feature, bug fix, or any other improvement), we’d have:
- A spec automatically generated from an engineer’s inputs or a bug report
- A plan automatically created using test-driven development
- Implementation in line with our best practices
Of course, there will need to be a bunch of agents working in parallel to make this work well. And it’s going to need engineers in the loop for a while, but that’s the end goal.
What’s Next for Spec-Driven (and AI-Driven) Development at OpsMill
Further out, we’re curious whether spec-driven development works for the things nobody enjoys, like reducing tech debt. Our specs today are written around user stories. So a refactor of an old part of the codebase would need a much more technical spec. TBD on how we solve that.
If you’re working on similar AI-driven workflows, let us know in Discord – we’re eager to compare notes. And if you’d like to poke around any of this yourself, check out:
