It’s amazing how non-developers have recently been empowered to create their own apps that can even generate revenue. We’ve recently seen progress across the AI development field, from AI being successful in “greenfield code” (apps built from scratch) towards “brownfield code” (larger scale existing applications). The more recent models have become much better at using tools and have had tremendous success in implementing features inside larger applications, to the point that developers barely stop to review the code. They focus their time more on prompting requirements and functional testing instead, which empowers them to ship more.
At the same, building new features this fast risks increasing the technical debt to a point where it’s practically impossible to make progress anymore. Without oversight, teams can end up with piles or spaghetti code, so code quality standards ought to be front of mind before the problem spirals out of control.
Maintaining high code quality standards is easier said than done. Human review can’t keep up, and asking Claude or Cursor to review your code is like asking someone to review a dissertation without any context of the discipline or standards that should be followed.
What do teams do when AI code piles up
The cost of skipping reviews, especially on architectural changes, shows up later, usually a few months out. AI-generated changes stack on top of other AI-generated changes, and each layer assumes the one beneath it is sound. When it isn't, the model has weak ground to build on, and so do the developers prompting it. Bugs surface in strange places and even small changes start producing side effects that take a while to debug.
At some point, the velocity that was high in month one starts to reverse. The team ships slower because every PR now feels the weight of everything that came before it.
Then, when teams try to do cleanup, it’s a big task. Organizations are even trying to solve the problem with vibe coding cleanup specialists. You can find the people with this role on LinkedIn right now.
It seems contradictory to benefit from AI but then require more people to fix the issues that AI made possible. So teams should take steps to review the code early and efficiently.
How to keep code debt low
The best options people have found to lower code debt so far generally fall into one of the two categories:
- A tool to check code quality on pull requests to catch code debt early
- A tool to check code quality on repositories
Teams use them to:
- From a management perspective, check which teams could use more senior devs, or to see which teams need the code quality bar to be increased.
- Track individual findings. E.g., even when you have some legacy repository that you don’t touch because it “just works”, scrolling through the logic bug findings is still interesting, to understand if there might be unexpected side-effects with hidden impact.
Both of those use cases work only if the checks running underneath them are accurate, and accuracy depends on how tightly each check is scoped. Asking an LLM to review a whole repository in one pass runs into the same trouble as a loose prompt to Cursor, with too much in front of it to land on anything specific.
This is why Aikido’s Code Quality feature launches LLM calls on a per-rule basis. This greatly helps the LLM to focus on one specific question at a time. Additionally, it’s possible to fine-tune those rules with added context to make sure the findings match the code style. If there isn’t a rule that targets a specific requirement, custom rules can be added as well. That control layer helps to streamline the whole process across the team.
Moreover, this control layer applies to both PR checks and repository scanning at once, to ensure that the statistical numbers on repository scanning align with the feedback on pull requests.
Benchmarked prompts for code quality
A second advantage in using a dedicated code quality system is that the LLMs receive fine-tuned prompts that have been benchmarked (something we also do with AutoTriage). The process is simple. We collect code samples for a certain rule and manually label if they should be flagged or not with a confidence score.
Some code quality rules live in a grey zone, making it unclear whether or not to flag them. For example, we noticed big differences in how strictly teams would treat the “no obvious duplication” rule. Aikido’s own code style is to not apply this rule very strictly. Readability is often preferred over the maintainability advantage of deduplication. New hires would sometimes want it more ‘DRY’ and go a long way to add abstractions to make that happen. There is no right or wrong here, it’s just a different shade of grey. In such cases, a confidence label defines how many people we expect to flag something or not.
However, we need to make a black-or-white decision when flagging something on a PR. So how do we navigate the grey zone when applying confidence labels? First, we simply aim not to flag grey zone samples - we're more likely to frustrate developers with too many findings than to amaze them with spot-on findings. We also bake it into our prompt engineering system. After labeling the samples, we tune the prompts in a way that would maximize customer satisfaction.
Unfortunately, LLMs aren’t perfect and mistakes keep happening, so we need to choose our battles. Having grey zone samples helps us to choose the right battles. When we make the wrong decision on a grey zone sample, it’s penalized less compared to a wrong decision on an obvious sample. The result is that findings are quite close to the truth, significantly closer than what you get with vanilla prompting.
Focus on code quality over finding bugs
An interesting source of confusion between a code quality system and other PR review systems is that most systems tend to focus on finding bugs. That is of course an important aspect, but it serves a different purpose. The most convenient way to make progress is to quickly iterate over code style, and those systems should work fast. Aikido’s code quality system typically finishes in under 1 minute after pushing the commit, keeping the feedback loop short. That includes both code quality and security checks.
In the near future, it will also be possible to request a rigorous check on a commit. This check would then chase logic bugs and authorization issues in more depth. It behaves in an agentic way and it’s therefore slower and more expensive, but an ideal final check on a PR before deployment.
Conclusion
A generic prompt to Claude or Cursor checks code against whatever the model happens to prioritize that day, not the specific coding culture in a codebase. A dedicated code quality system fixes that because it runs the same tuned rules against pull requests and full repositories, so a team gets one consistent answer instead of two different ones depending on where the check runs. The next step makes that answer go deeper: an agentic check built to trace logic bugs and authorization issues, stable and safe enough to run right before deployment instead of on every commit.
