Both Shadows and Vorlons have recognized mankind as threat to them.
So they went and destroyed mankind before they could even become a threat.
Or so they thought...