I've seen different interpretations of the question of whether Thor needed the belt to lift and swing Mjolnir or not. Certainly having the belt (which is called Megingjord, or a variation thereof, in the myths) allows Thor to use the hammer much more effectively. Megingjord either doubles or triples Thor's strength, depending on whose version of the stories you read. If I recall correctly, Thor acquired Mjolnir before Megingjord, but I'll have to check that.
I think it worth noting, however, that Mark Millar doesn't seem to be drawing on Norse mythology so much as he's re-interpreting the Marvel version of Norse mytholgy, and both he and Marvel seem to play rather fast and loose with the subject matter. (For example, Volstagg doesn't appear in any of the myths of which I'm aware, and here in the Ultimate Universe he resembles mythological Thor much more closely than Ultimate Thor does.)
By the way, here's an interesting little tidbit people might enjoy:
The Staff of Jord. Note what Jord has to say about the magical items at the end of the story. Again, a different interpretation of the myths, so make of it what you wish.
Which reminds me: I've got a comment to add to the Issue #9 thread about Mjolnir when I get a minute....