Well then, don't chicken out and continue a game longer than that...
If you don't need to take something from someone else, you don't need to be at war, it's not something you need to do, unless you have it planned out well you don't necessarily profit from having one.
Yes, colonies take a while...