Defections may happen much too soon.
There should be a "period of grace" during which the planet waits before switching again allegiances...
I have seen a planet culturally flip for me one turn, then flip again to the I-league the next turn! I even had spent a hefty 7000 to buy a virtual thing (unsufficient ; that case needed 3 of them...)