If a planet has a counter espionage center and you build over it, you can't rebuild it. I don't know if the planet is still invulnerable to agents though. Occasionally I would like to be able to move it if I've built it on an approval bonus tile but later in the game have an advanced morale building and need the boost.
