Goo gone, adhesive remover, etc.
The finish under the badge may be different simply because it hasnt been exposed to the same conditions. May need to polish the whole panel to make it look right.
Or semen does the same job, possibly with less rubbing.