I think you need to provide more info as to the game setup, and what you are doing in each early turn.

My first guess is that you may have left the "sliders" in the original 3-way division among research, ship, and planetary buildings settings, and have perhaps also not moved the tax and spend sliders from their initial positions.

Other contributing causes could be that you have not made use of your opportunity in the game setup to improve the abilities and or starting techs of your race.  That's why I suggested that you post your game setup of your race.

What about the taxation and spending sliders?  What do you do with the miner ship?  What race do you play?  Please, more details!

BTW, is this DL, DA, or TA? 

You can restart with Control-N until you get a HomeWorld with a couple bonus tiles that you want, and not ones you do NOT want.  A 3x research and/or a 3x production tile with a morale bonus tile is a good start.  A 7x tile is sweet.

The first thing I would suggest is to make sure your spending slider is at 100% and keep it there as long as you can.  100% morale helps build population, but is less important than other aspects.  The Super Diplomat can be helpful, but you might want to try the Industrialist or Techno super ability, or simply play with super abilities OFF.  Even Federalist can help more than Diplo if you cannot expand very well.

The second thing is to convert your miner to a colony ship on Turn 1.  Also, make sure to land your colony ship and then launch it again, to make sure it has max colonists (starts with 100K but can hold 250K).

Try building fewer buildings early and concentrate on research, instead.  There are several techs that give Empire-wide bonuses, and you want to get them early, since they help all game long once you get them.  Besides, the early buildings are not that good and, if you do build them, you are stuck having to pay to upgrade.  I would suggest buying one factory and/or research building on the bonus tile(s) and that's it, until you have done several research things.  You also want to build colony ships pretty soon.  The need to get the research done and get on with it is why the techno political party is so helpful early.

For example, I find it helpful to research directly to where I can build an Economic Capital and then build it on my Homeworld because it has the highest population.  Also, remember, that higher governments confer Empire-wide bonuses.

So, start spending down your cash to buy that one or two buildings and to convert your miner.  Colonize two decent planets fairly quick but wait to build much until their pop grows a bit for tax revenue.  Start building more colony ships, especially once you get decent engine tech and/or find a juicy high PQ planet that you covet.  You can always finish it by rush-buying it, if you want.

Try that, and, if you get that far, you should be well on your way!

Conversely, I try to keep in the 70s (I want my people to like me) and have never had a lot of trouble with the economy (this is on "Tough").

Regarding taxes: note that any planet with 76% or more morale gets a bonus to population growth, so it's often best to balance your taxes to get above that threshold early in the game.

You colonize 2 planets... but in which galaxy size? On tiny or small, 2 planets may be all you can grab, depending on your settings. By the way, don't colonize the low planet quality secondary planet next to your homeworld but try to grab one with better quality instead. Once your homeworld develops, the seconary planet will come under your cultural influence (and flip to your control) anyway.

To speed up your start, rush-buy 1 or 2 factories (see which payback period is the cheapest overall)

What research order do you follow? If you fail to defend yourself, is that due to not having enough military ships defending your planets, not having powerful enough ships, ...?