The buildup, tactical/strategic planning and eventual conflict is far more fun than the inevitable cleanup phase of the game.
Oddly, this game was originally advertised as having a substantially shorter cleanup phase than other similar titles. Unfortunately, that was never born out in the actual game.
It's funny. The game has 4 ways of winning, but only 2 of them are basically the same. An influence victory almost always requires military dominance unless you're on a very tiny map.
If you own a majority of the population and planets, then the advantage is inevitable in most cases.
While it would be nice if there was a way to short-circuit the cleanup phase of the game, a simple majority isn't the way to do it. I've taken out civilizations that dwarfed my planet and population count.
Further, this game is already terrible at estimating the effectiveness of your military (I can take out someone with only 1/3rd to 1/2 their military rating). I'd hate to see it try to have some overall civilization effectiveness that actually determines victory.
From the sounds of it, it would seem that your playing on maps that take days to complete: Huge, Large, (TA Immense).
Trying playing a medium map!
As someone who plays primarily on medium-sized maps, the problem there is that the game is either pot-luck or just like Large+ map sizes. What I mean is this.
In order for any civilization to have a reasonable chance, you need a base number of tiles. Which means you need to have enough planets near you that you can reasonably colonize before others. I'd estimate that number to be 70-100 tiles. You also need at least one planet that can support large populations reasonably, as a money planet.
If you are playing with relatively few habitable worlds, on a Medium-sized map, whether you get this minimum level is sheer luck. Any civilization that doesn't reach this minimum level basically has lost. So if you're playing 5 civilizations and few worlds, then you're looking at maybe 2 of them are reasonably viable.
If you increase the number of planets so that a medium-sized map can support 5 civilizations, then you're basically playing on a large map except with less space, so speed and range aren't as important.