They overlook the original prompt used by the coding agent and fail to grasp the broader business context of the application.
Code review - whether conducted by a human or an LLM - doesn't actually verify whether the code accomplishes what was prompted. It cannot confirm that the code aligns with its intended spec, or that the agent's claim of "task completed" is checked in a deterministic way. LLMs notoriously defer hard, multi-step, or dependent tasks, which means that the intent itself has diverged by the time the PR is opened.
If we want to trust AI code in production, we have to stop verifying the diff and start verifying the claim.
Has anyone found a reliable way to bind the original Jira ticket / GitHub issue to the actual code change in CI? I've been working on solving this problem with Shipmoor.dev, but I'd love to hear how other teams are tackling the "intent gap."
https://github.com/BiggerPockets/.github/blob/main/.github/w...
Still a bit nicer to just do these yourself rather than reusing someone else’s project.