But then I won't get reply notifications and I most likely won't see them in my in-app notifications either 😔

They will still show up in the in-app notifications! I have all pop up notifications turned off and I still get them