Those under equipped guards still had access to low level Elder Scrolls Magic. Something we don't have. Sure in time that can be obtained but in beginning Daedra are gonna be running amok in remote places.
The first person that opens a portal to the Oblivion realm will get a death worst than William Wallace got in RL from that nations government. If they aren't killed by Daedra first.