Yes. You have to spend, but you must control the rate of spending. Moderation in all things! If you spend too fast, or jerk around your tax rate too much, your civilization will suffer economic "rubberbanding" as your citizens try to cope with the changes. That affects their morale, their ability to produce, and the amount of money you can expect to get.
Of course, one easy way to help yourself to more money is to give your civ big financial bonuses for their abilities and pick a political party that gives you financial bonuses.
I find that it helps when designing your starting colonies to not have specialized planets -- each planet should have factories, food, entertainment, diplomacy, commerce, and research. If the planet can support 3 factories, then it can have a Starport, too. Later, after you have a few planets, you can re-build things to make production, research, and political worlds. Unbalanced, specialized plamets tend to drain other resources your civ needs in the early game.
For instance, I used to make my biggest planet my production planet, and fill it with factories. Unfortunately, the people were upset, the planet was not influential, and my civ went broke trying to build large, antiquated designs.
Especially if you have research, and make a bee-line for trade, once you start talking with other civs, you can trade older techs for money, which will greatly increase your treasury after the anomalies run out.
