Why are the AI so poor with their improvment choices?

What level of difficulty are you playing at. They won't do that in the higher levels. At least that I have seen.

at higher levels some of the ai planets are better than mine (know thats pathetic but im a military man not a social engineer) so i ususally like to take a couple of them now and then to support my huge mega crusade before i banish all the evil civs from the galaxy and force the good ones to ally with me for a nice diplomatic victory

Aside from the Technological and Manufacturing Capitals, the planets are much, much worse than mine. Well, some are alright, but as I said, most have too few factories. In my plan, between 33 and 50% of the terrain should be used for factories. I am playing it where the AI is at the hardest level where they still produce at the same rate as you, so the AI should be fine.

What level you playing at?

At lower levels, the AI pretty much randomly throws stuff on planets.

At higher levels, it gets down to business.

Gimpy AI = build wrong thing on bonus tiles, buy stuff we don't need. AI do better jobs at higher difficulty. So up the difficulty level next time!

I have 3 general types of planets:
Manufacturing
Research
Economy/Influence

Only the manufacturing will necessarily have many factories left as the game progresses.
On research or economy planets, I will generally create only a few factories, and after they are full i may convert some of those factories into labs or banks.

This isn't to say that the AI doesn't sometime misuse planets (esp tile bonuses). But having very few factories isn't necessarily horrible.

Realistically, the AI will sometimes build something on a food spot if it has something else in mind. I don't think you'll find the AI building stuff on research or production ones. I hope they aren't anyway.

So, Frogboy, at what difficulty level does the AI start to more efficiently used its tiles? At difficulty > normal?

Yes, sorry I couldn't remeber the exact difficulty before, but the computer AI were all set to 'Intelligent'
I should just post a screenshot of the most terrible planet, it's mostly like it was before since the structures cost so much to buy now...

Frogboy, in my last game with AI set to Inteligence. My arch nemesis Yor had constructed factory on +300% research tile and again builded a farm on +100% manufacturing tile

What is the difference between difficulty normal and the intellegience level of the AI civs?

On inteligent AI uses more algorithms to play better aka it counterstrikes your weapons by researching defence against em etc. Also on Inteligence AI does not get any economical bonuses or minuses.

Realistically, the AI will sometimes build something on a food spot if it has something else in mind.

As will I.

I'm afraid to say that in my duel vs an incredible level AI last night, I invaded a class 11 planet, which the AI had held since near the start, to find it only had 7 improvements built. Unfortunately I did not take a screenshot, but I will certainly keep an eye out for this in future.

There's usually a method to the madness. I took over a world recently that was filled with nothing but embassies; the AI was trying to defend it against my culture, but sadly not against my shock troopers.

A mega manufacturing bonus on a planet only does you good if you have the money to fund it. Etc.

My guess is that for some reason the computer picks what to build and then where to build it. If, in the above case, the computer built the farm first, then the factory, then the research center it is possible the land was just used up on the previous build. Still, I think the computer should use their bonuses for the appropriate buildings when possible, and if there is no bonus, build on an empty tile first.