Well, they should obviously be as evil as it makes sense for them to be(Shocker becoming a serial killer would be ridiculous, IMO, unless there was a long series of sensical reasons leading up to it).
My rule is that however evil they are, a big supervillain should never, ever be aware of or relish in their own evil. They should either:
a) Think they're doing good in some big-picture, ends-justifying-means way(Magneto, Ra's Al Ghul, Lex Luthor outside of the movies, etc), or
b) Think that the concepts of good and evil are either illusions or moot(Voldemort, Norman Osborn, etc), seeking power and not taking pleasure from killing, but not caring about it either, or
c) Some combination of the two of those(Eric Cartman)
Even my favourite villain, the Joker, often gets thrown into the "evil for the sake of evil" category, but from his perspective, the concepts of good and evil are just a big joke. It doesn't matter.
What I hate is stuff like Superman Returns' Lex, who claims to want power, but then also relishes in the death of "billions" for no established reason. It's just quick-fix villainy. Boring in a character that big.
There are rare exceptions to this rule that still work, though, like Emperor Palpatine, who would fall into category b) if his powers didn't come from deliberately tapping into his own human potential for hatred. It's an original, cool and context-reasonable enough situation that it works.