This is the crux of it though there's only a difference in the experience if you believe you're actually leaving your body. If you don't believe astral travel outside of the body is possible then there is literally no difference it's just chemicals in your brain creating a similar experience.

Drugs allow you to experience the same feeling but don't allow you to leave your body. Historically witches used drugs to amplify magic. This is nothing more than modern day anti drug propaganda masquerading as a form of elitism. "I had to get there through effort you can't get there through drugs."