Part of the solution is basic redesign. I'm thinking before the agent can mark a task complete it needs to pass a test suite, and instead of making local evaluations the evaluation stage should contribute to a namespace of tests for that part of the task. This inhibits confabulated completions.

But ultimately it's not clear to me that these models have a unified sense of the cyberspatial environment and this hobbles their local understanding and intelligence. Much like how if a parent of a blind-deaf child picks them up and carries them too much they never learn a connected environment.