I agree. The code despite detailed spec reveals bugs and edge cases upon inspection.

I'm talking Claude Opus 4.6 here.

For all we know, some important clients might just be getting better service out of Anthropic's/OpenAI's "black boxes".

The spec needs to be explicit about edge and corner cases.

At some point such a spec converges to the actual code you’d have written.

I've actually tried that and it helps. First I create a PRD type doc, then I have the AI break it down in a task doc, including code snippets where relevant. This helps it to think through edge cases before it starts implementing (oh we need X now, but that means we should have done task 3 differently to allow that).

> I use Claude Opus (4.5, 4.6) all the time and catch it making making subtle mistakes, all the time.

Didn't we make subtle mistakes without AI?

Why did we spend so much time debugging and doing code reviews?

> Are you really being more productive (let’s say 3x times more)

At least 2x more productive, and that's huge.

I think you’ve forgotten about the context of OP’s post. He said he uninstalled vscode and uses a dashboard for managing his agents. How are you going to be able to do code review well when you don’t even know what’s going on in your own project? I catch subtle bugs Claude emits because I know exactly what’s happening because I’m actively working with Claude, not letting Claude do everything.

The code is still visible if i want to review it.

But since I have a strong rule about always writing unit tests before code, my confidence is a lot higher.

https://simonwillison.net/2025/Dec/18/code-proven-to-work/

>The code is still visible if i want to review it.

I agree that the test harness is the most important part, which is only possible to create successfully if you are very familiar with exactly how your code works and how it should work. How would you reach this point using a dashboard and just reviewing PRs?

Are you getting paid 2x more?

Of course personal projects are much quicker because usually personal projects don't have high code standards... I'm talking about production code.

typical experience when only using one foundational model TBH. results are much better if you let different models review each other.

the bottleneck now is testing. that isn't going away anytime soon, it'll get much worse for a bit while models are good at churning code out that's slightly wrong or technically correct, but solving a different problem than intended; it's going to be a relatively short lived situation I'm afraid until the industry switches to most code being written for serving agents instead of humans.

The way LLMs work, different tokens can activate different parts of the network. I generally have 2-3 different agents review it from different perspectives. I give them identities, like Martin Fowler, or Uncle Bob, or whatever I think is relevant.

true - but the way LLMs are trained, google's RLVR is different from anthropic's is different from openai's. you'll get very good results sending the same 'review this change' prompt (literally) to different models.