Here's a trick... but promise you won't share it with anyone else(!

Build some or many transports (for later use) at maximum loading capacity, pack'hem all with people (these WON'T be happy/or/mad or interfere with growth or even pay direct taxes, eat food, etc) and park..send the ships in numerous orbit(s) of your planets nearest to your next targeted enemy.
While this lowers population, the effective invasion strength would already be in place. Eventually, you
WILL have to send to some troops (citizens, btw) around.
In as low as ten turns... your morale will peek, and the populations on the move AREN'T obstacles to natural planetary expansion levels.
When the opportunity strikes, lock & load the money you spared in producing necessary attacking ships, escort the troopers and USE them for a better purpose than wrecking havoc to morale for too long.
- Zyxpsilon.