The other civs rate Military strength far, far too highly. Civs that rely on your trade for a significant portion of their income will attack you if your military is too low. Civs that are pursuing diplomatic or influence victories will behave likewise. Even civs at the other end of the map (i.e. that cannot even reach you due to range) will declare war for no apparent reason, even if you have been trading with them.
This is the experience I have had when I decided not to use my usual "Master diplomacy" racial trait, and to forgo building Diplomatic Translators. With these bonuses, no-one declares war on you unless you really push it. Without them, I would go so far as to say that diplomatic victory becomes impossible. Research and cultural victories are still possible, but no longer make any sense. If you have to have a huge military to remain on even decent terms with the other major civs, why even bother with anything other than a military victory? Why split your resources between the 'war' branch of the tech tree and the cultural/research branches when you could simply research war on its own?
The biggest problem with this is that it often causes an unstoppable chain reaction: You enter a war, become weakened, so the other civs start turning hostile and declare war... and so on. Before long you have an unwinnable scenario, especially since other civs will hate you for "Historic animosity" even for wars THEY started. idiocy.
"Military strength" needs to stop being a factor in diplomatic relations, and should be considered seperately. This is especially relevant to a human player, who can often defend him/herself with a smaller force. I know that my military is enough to defend my worlds from 1-2 civs. Since 3-4 civs disagree, my choices become strictly limited to either fighting off several civs at once, or copying the AI's pointless stacking of defender-type ships around my worlds
After all, surely a civ that is either weak militarily, or is engaged in another conflict, should be more comfortable with me having a smaller military than a large one?