What happens if the shape is symmetrical but the mean is undefined (fat tails etc). Does a shape that extends infinitely have a centre of mass?

I'd say no, but that's fine as it's not a thing that we could actually construct in physical form...