Most of my games are also like that at the beginning, but once in a while I get a really fun start. I had one game on a gigantic galaxy with all 9 civs where I started in corner next to the Drengin and almost isolated from the other races. I think it took over 100 turns before I met anyone else. I had to start building war ships right from the beginning to survive wave after wave of fighters and transports. I hadn't time to research much sensor (very slow tech setting) so that they were appearing from every where. I fought like that for countless turns. It was fun!
At some point the other races met us and i managed to research trade and stablish commercial routes with them (i needed a lot of life support on those ships), the extra money from trade gave the upper hand. When I finally defeated the Drengin and took over their planets, i was like the last monkey in the galaxy, way behind the other races. But then there was that UP resolution where every race must share two technologies, and i got a bunch of them, and after that the Thalans surrendered to me and i could build up my economy and become a power, that was my best game so far i think.
Sometimes i also start early wars myself, even if you have nothing to win, it's fun. You may prevent a race from destroying others and becoming too powerfull. It hurts your economy in the long run, and may make you loose the game, but who cares about winning? all you get when you win is a black screen telling you that.
If you need action and it doesn't come to you, go and get it. If the mountain doesn't go to mohammed, mohammed goes to the mountain or something like that.