The thing is ... it isn't actually clear what Voldy's motives are. Is it "take over the wooorld"; "reeeeee, muggles"; "immortality" or "must destroy Harry Potter"?
Yeah. Voldemort as shown in the books comes off as little more than a violent oaf. Not to mention the way he ultimately loses to an obscure bit of magical trivia which was never mentioned in any of the previous books.
The Tom Riddle we got glimpses of in CoS and HBP was way more intimidating than present-Voldemort. Prettyboy magical serial killer beats Skeletor knockoff anyday.
Most of all, your villain should be impressive. You can have a largely evil bad guy if you don't want your work to be morally complex, but said bad guy has to be intimidating with a strong presence unless you're writing a comedy.