Ahh I found what it is / the root of the bug. I have slightly bigger font size set in Android accessibility settings system-wide. All apps including Chrome handle it like a champ, so I even forgot I had set it, but in Edge it produces all these artifacts.

If I reset it, it renders fine but then, of course, everything is too small for me. I wonder where best to report this or who to tag.