It looks fungal to me too, but the closest damage match I could turn up for Asclepias spp was fusarium, which would collapse the interior structures of the stem, rather than looking like huitlacoche like this

If you live somewhere that experiences a hard freeze and winter dormancy for herbaceous species, cutting the affected plants below the infection should keep them from carrying the infection over to next year. If it doesn't freeze there, definitely need to ID exact pathogen