What's not straight between the two? I'm not really following.
The first Dr. Strange doesn't really do anything with the multiverse. They utilize a different dimension, which seems to be consistent in the MCU of being something distinct from an alternate universe.
A more appropriate criticism would have been how Asgard and the nine realms were treated. In the first two Thors, it seems that the "realms" are dimensions. That's eventually just... ignored.
The Nine Realms, I think, are meant to be different dimensions, with key worlds (such as Earth/Midgard and Asgard) being the anchor stones for the Bifrost/Yggdrasill -- remember that before Odin went all peaceful, he was a conqueror along with Hela; it's likely he invaded those dimensions/universes
from those worlds as staging points, and the Asgardian armies plundered/conquered hundreds of worlds across each of the nine dimensions.
It'd be as if the Combine from Half Life 2 didn't just invade singular worlds via portal technology e.g. Earth, Xen, but also worlds like Mars, Venus, and others outside our solar system using Earth as the staging point.
Fuck knows where Olympus is meant to be... our (Midgard's) dimension, maybe?
Thor's concern in the Avengers, that "it signals that Midgard is ready for a higher form of war", wasn't just that the aliens of our dimension (e.g. Thanos) would see Earth as a threat, but the various aliens of the Nine Realms' dimensions, too.
As for Dormammu and his domain (which has said to "swallow worlds"?), I think that's another alternate dimension that's under the control of one overlord, but it could be an extra-dimensional realm (e.g. outside of the known universes/dimensions) -- basically the area outside of the "box" that all the dimensions, parallel universes, and timelines are contained in.
Then again, as you said, that's one of the few MCU elements that has been inconsistent throughout the films.