However, none of the dashboards actually analyzed how the engineers worked with the tools and if there were any areas for improvement! That's why I created https://www.promptster.ai.
Managers get aggregate level view of code quality and how that ties with team workflows (nothing on a per-engineer level). While engineers get personalized coaching on how they can save tokens while keeping output high.
We also have a tool built for individuals to test their local claude code setups. It's open-source and completely local. You can check that out here: https://github.com/pa-arth/cc-audit.
Would love feedback on either and reach out if you're interested in purchasing the teams product!
I do think, however, that holding out on it is a losing proposition. It's a very useful productivity tool if you know how to use it and those who don't will be left behind.
A lot of people brag about how much more code they write using an LLM, but I can't see how that's a good thing. The bottleneck is understanding the requirements and writing the minimum amount of code to meet them
The spec-development & testing phases before and after implementation can and should have human involvement, not the implementation. With that being said, there's so many places for this SDLC to be optimized / made more productive with agents that I think everyone should be using them. How well you use them, now that's another story... (check out the above^^)!
Usually in a day I have one agent going on whatever my large, main task is, generally feature work of some sort. Previously this is what most of my sprints would have been filled with.
But now, while that's going I have one or two others simultaneously working on crashes (I do mobile), other bugs, a small refactor, or maybe a small enhancement.
And finally I usually have another where I'm querying analytics or working on higher level questions to try and help guide what we should do next, either from a product or technical perspective.
A single one of these are all sped up by using LLMs but to be able to do several in parallel makes me much more effective than I was before. This is why I believe that those who don't use these tools properly will be left behind.
I believe it’s only possible to truly appreciate that after doing it for a while with trial and error, without external opinions
But honestly, it is extremely rare thing not using codex, or cluade code or any other agent.
What's the measure of code quality in this case?