As long as your colony is at 100% approval, you will be at the maximum possible rate of growth. Growth is a percentage of the people on the planet, so the more there are, the bigger the gain each turn.
At about 1 billion, there is no morale hit, but as population rises a morale penalty is charged which counters population growth. At about 3billion it's maybe -3%, and at 5billion it's about -10%, and at 10billion it's about 25%. As morale drops below 100%, population growth slows. It slows even more at around 75-80%, and again until it completely stalls, and then starts dropping away. Once enough people have died, it settles back to stalled, and equilibrium is established -- a miserable but stable population. However, that long process is easily altered by changing the tax setting, and by building morale improvements on the planet.
To answer your question more directly: A colony can support up to 5 billion without requiring a farm. So one can start building factories and research until nearing that cap. By then there will be about a -10% morale hit, which can be compensated for with one morale building. Moving extra people onto the planet will give not only the extra people, but the larger steps forward.
My earliest major people mover is with Impulse-II and Enhanced Miniturization (2nd level). With this I can build a double-colonizer with speed-5, that will start a planet out with 1 billion people.