I'm definitely the best person to give advice on this, having used neither server-side Swift nor Go, but the general feeling I get from what people say about server Swift is… that it's still not all the way there yet (Linux support not being as good as on Mac etc.)

That said, it might not be a bad choice for you since you already know Swift, if you can have some patience for the things that won't work (but you already must have patience since you use Xcode, so… :p). There's this new thing that might be better than Vapor?

GitHub - hummingbird-project/hummingbird: Lightweight, flexible HTTP server framework written in Swift

Lightweight, flexible HTTP server framework written in Swift - hummingbird-project/hummingbird