Regardless of how well tweaked an AI is, it tends to die off quickly if it doesn't have many worlds nearby to colonize, and particularly if it's in a very tight pocket in between two or more other races.

In other words, the spawn locations, with respect to the planets generated, have more of an effect than almost anything else (exceptions: that bug where AIs failed to build planetary improvements, that bug where AIs failed to research/build fighters) in determining who your rivals will be, and who will get their brains splattered all over the floor.

As this planet generation is pseudo-random, your rivals should change somewhat from game to game.  While the planet generation and spawn locations could be better chosen, it's not precisely that the game has chosen AIs to die off sooner-it just makes it more likely.

Other than the geographical factors that Sole mentioned (astronomical factors?)  There's the fact that any given AI could simply follow a not-so-great strategy.  It may spend poorly.  For example, I've noticed that those races sending out an early constructor seem to have a big lag in putting out colony ships.  So, if they start near a resource, they may hamper expansion by buying a constructor, or something.  They could also get sidetracked in research and spend a long time researching techs that don't help early development.  So they go early for some expensive tech while others are getting all the early bonuses and basic improvements going.

Those kinds of things can be hard to come back from.  I've seen the AI make those mistakes and still manage to come out strong, but not often as a top contender.

It is worth noting that which races tend to do better or worse seems to depend at least somewhat on which races you play as. In almost a dozen games as the Terrans, both with and without tech tree fixes, I've never encountered strong Drath, yet in my very first game as the TA Drengin they crushed me overwhelmingly.