The results are extremely clear. Having the LLM generate Python code to do math and then running the resulting code was both faster and more accurate than having the LLM do the math itself, and the difference in performance widens as the numbers get larger.

More detail (and Python code) in this Substack post: conspirator0.substack.com/p/chatbots-a...

Chatbots and basic arithmetic

Prompting an LLM to write Python code to do simple math is more efficient and more accurate than having the LLM attempt to do the math itself