In my own experience, I’m able to get a single prompt to output the correct code frequently. The LLM itself is wrong a lot between its first draft and the final output, but first corrects the syntax (compile errors) then corrects the behavior (functional errors) and outputs the correct code. It’s “one-shot” for me, many shots for the LLM. There’s enough tooling around the uncertainty to arrive at correctness without my intervention.
I suspect it's largely a function of the scope. Like with the value of Pi example, a small deviation isn’t noticeable with smaller calculations l, and becomes more noticeable with larger ones.
The larger/more complex the scope, rhe less effective one-shoting - curious if you're seeing that?
Also curious if your version of one-shot is literally a single prompt, of if you're following some flavor of SDD flow, which I don't qualify as one-prompt in the context of this discussion.
Tooling (we have an internal version of OpenClaw that is my go to) has gotten good enough to one shot even complex changes without a spec. It’s not perfect and I often follow up with a “simplify this” prompt, but it’s functionally correct code.
I fall back to a conversation (“ask me questions about this feature”) if the one shot fails, but that has become a rare occurrence lately.
In my own experience, I’m able to get a single prompt to output the correct code frequently. The LLM itself is wrong a lot between its first draft and the final output, but first corrects the syntax (compile errors) then corrects the behavior (functional errors) and outputs the correct code. It’s “one-shot” for me, many shots for the LLM. There’s enough tooling around the uncertainty to arrive at correctness without my intervention.
I suspect it's largely a function of the scope. Like with the value of Pi example, a small deviation isn’t noticeable with smaller calculations l, and becomes more noticeable with larger ones.
The larger/more complex the scope, rhe less effective one-shoting - curious if you're seeing that?
Also curious if your version of one-shot is literally a single prompt, of if you're following some flavor of SDD flow, which I don't qualify as one-prompt in the context of this discussion.
Tooling (we have an internal version of OpenClaw that is my go to) has gotten good enough to one shot even complex changes without a spec. It’s not perfect and I often follow up with a “simplify this” prompt, but it’s functionally correct code.
I fall back to a conversation (“ask me questions about this feature”) if the one shot fails, but that has become a rare occurrence lately.