I get far too many colonies with excess population, requiring me to waste many tiles on entertainment buildings, so I have developed a set of rules:
No farms on tiles with a bonus greater than 100% unless I am trying to fast-grow a colony on a high-PQ world.
Replace farms on farming-bonus tiles with farms on no-bonus tiles (build the new one before destroying the old, so you don't lose a huge amount of population to starvation) if you have more than triple (or frequently more than double) the number of entertainment buildings as you do farms.
Never have more than one farm on any world with less than 15 PQ, in which case use farming-bonus tiles instead of non-bonus tiles unless it would result in excessive entertainment buildings (i.e.: never, ever use a 300% farming-bonus tile for a farm).
Some might advise you that the higher population is valuable due to the tax income, but realise this: The top players always have much more cash than they can reasonably use per turn, and the higher your morale, the higher you can jack up taxes. I currently have taxes at 60% with each and every planet at 100% morale, and not a single planet has more than 1 farm if any at all, or more than 2 entertainment buildings.
Also note that if you destroy a farm, the next turn your planet's population is reduced to the maximum that you can feed with your current food production.