If I am building up my infrastructure then I'm not training a unit. That is a trade off.
Why would I want to train a unit when my sovereign, her favorite champion, and their summoned elementals are killing every band of monsters and completing every quest by themselves? I don't need a crappy trio of spearmen weighing them down, and by the time my military tech has reached a point where they can even threaten the Wildlands my magic is so strong that I don't need them anymore. I guess if an enemy faction was breathing down my neck right at this particular moment I might bother spending upkeep on some soon-to-be-obsolete troops just in case, but I'm skeptical that they could handle my sovereign's group with how widely they seem to scatter their experience. Worst-case scenario my empire winds up short two or three schoolhouses in its smallest cities. It barely matters; I mostly just build them because I have nothing better to do. The population, and therefore research and economy and production, is still growing at almost exactly the same pace regardless of what I produce.
Whenever you gain a city (conquer or found) you are diluting your growth speed. Fewer cities grow faster than more cities.
Why would I care where my population goes? Percentage-based improvements aren't really common enough or powerful enough for it to make a huge difference where they live; as long as they exist I'm getting gold and research out of them. That's all that matters. And controlling more cities brings more total citizens into existence faster than fewer cities. The "cost", in the short term, of a quarter of a new citizen being diverted away from my well-developed capital into a smaller city is next to meaningless, because that's not a huge number and crappy cities aren't that much worse than big cities at making use of citizens anyway. Assuming my core cities haven't all been sitting at their population caps for dozens of turns already, that is. In the extremely likely case that that has happened or will happen soon, I have even LESS reason to care.
Another thing to bear in mind: the more cities I have, the less one additional city impacts the growth of each existing city. The difference between one and two is kind of noticeable, but between 6 and 7, when the seventh city is going to be increasing my total prestige by 1 anyway? The number in the panel doesn't even change. And don't try to tell me that a one-city empire is a sensible alternative to a seven-city empire. Even if concentrated population was more valuable than it is, my population is going to grow faster than I can research new food-related techs if I don't spread it out. Especially if I focus on the military techs you expect me to care about at this phase of the game.
Raising taxes lowers your research and production but provides capital to buy things sooner at the cost of slowing your natural progression.
In my current game, my taxes sat at 0 for the first 150 or so turns, then 1% for the next 150, and then I finally put it up to 2% when I started producing meaningful numbers of troops for the first time. Not because I NEEDED to; I had just gotten used to having 4000 gilden in the bank and didn't feel like seeing the number turn red. Besides, the most important aspect of my "natural progression" is probably prestige (population is the root of everything), and I'm pretty sure unrest doesn't even impact population growth.
Can you see what I'm getting at? There's a "choice" between producing soldiers and producing infrastructure, but is it really meaningful? If I am at peace, I have no reason to waste gildar on troops who will be sitting around doing nothing and waiting to become obsolete, so I build infrastructure and rely on my sovereign to do all the killy jobs. When my diplomatic information tells me that I am close to war, I build troops, because I might actually need them and the actual benefit of the constructed infrastructure barely matters anyway. My decisions during peacetime aren't likely to enhance or detract from my empire's potential as a war machine when the time comes. The only forms of infrastructure that DO matter in a big, obvious way--the existence of cities, total prestige, and control over resources via outposts--are maximized so cheaply and so quickly that doing so doesn't detract from any of my other capabilities (in fact, it directly supports all of them in the most efficient possible manner). Am I actually making meaningful strategic decisions and sacrifices, or am I making no-brainer decisions and reacting to stimuli?
Research comes from citizens. Income comes from citizens. Civic production comes from citizens. Military production comes from citizens. More citizens means more of all of these things, linearly, without drawback (actually that's not true; military production increases BETTER than linearly due to the base production values in the cities I build to keep citizens in). There's exactly one way to get more citizens and that's to expand. Expanding (via founding new cities, at least) is the least costly thing a player is capable of doing with their existing cities. The faster you expand, the more total prestige you have and the faster citizens build up. So of course I do it as much as possible. Maybe I wind up producing a lot of troops, maybe I wind up producing a lot of infrastructure. That's largely determined by the environment, not by my decisions earlier in the game. And can you guess what my best option is for preparing for either contingency? It's to maximize citizens, every single time. There is no visible difference between an empire designed for war and an empire designed for peace, except in the mid-to-high level techs you take. The only difference is what I have in the production queue at this exact second.
EDIT: Also, I have to ask: if you (somehow) feel that diluting your prestige matters so much, then why did you program the AI to expand so aggressively? I have yet to see terrain so arid and lifeless that the AI will not settle in it. They even take the lousy 2/3 land that I don't bother with (more out of laziness than sensible strategy, probably). That's part of why it pays to city-spam yourself: the AI will take all the land if you don't.