Which sounds like it would be caused by a new system prompt optimized for chatbots rather than raw output. So, presumably, if we change the system prompt the models will start outputting raw code again? Who knows. 7/

4️⃣ Conclusion: If you're depending on closed models released as APIs you'll have to test your prompts for data drift each time a new version comes out, forever and ever. This sounds like a good argument for picking open, instead of closed models. 8/