I play EU – perhaps the difficulty was tweaked for this version?
As far as I have heard, EU is the same as the individual games, just combined into one. Same as the whole ultimate bundle pack... of course, Sole Soul is the expert on all this type of stuff, his immense knowledge of EVERYTHING never ceases to amaze me. lol
When I first started playing, with DL at least I found the move from Challenging to Tough the most Challenging (no pun intended), if I can remember right. I had to change my tactics quite a bit. Same with when I went from Maso to Obscene (DRASTIC change there, although I absoultely refuse to ever use the all-factories or all-research stategy). I will attempt to beat the toughest difficutly without using those strategies one of these days.
EDIT: I forgot Suicidal is the toughest difficulty... it's actually not too difficult, bummer I thought there was one above it. I started a new game and am going to post an AAR for it soon.
It is said in the GalCiv WIKI:
"Brad has yet to fill us in on the details of what exactly the AI does at each of these levels."
This is probably old news, but it also states that from Tough difficulty and upward "from this level up to the highest AI expertly picks abilities and all known human tactics are searched and countered".
It is likely that the AI builds social improvements more aggresively on Painful compared to Tough, and the research improvements built are what is most likely to show. Since the 5% increase isn't enough to explain the huge difference, the Global difficulty probably cause this (see below).
ALSO: it is said:
"There are two different sorts of difficulty level - one that acts on a global level, and one acting on individual AI's."
It is likely that one of the difficulties is making the game more tough, and that is likely the Global difficulty level, not the AI difficulty level. The Global difficulties are: ...Challenging, Tough, Painful, Crippling, Maso, etc.... The AI difficulties are the corresponding: ...Bright, Intelligent, Gifted, Genius, Incredible, etc... These both can be adjusted in the game startup screens.
And even MORE interesting!:
"When you select a game each race has a varying degree of intelligence based on the game difficulty selected above. Exact details are not known, but AIs on higher levels (masochistics and higher) get significantly more "points" to spend in race creation and are not bound to the same point-spending rules the human player is (e.g. they can have 80% bonus in military production); and bonuses to economy, production, research, influence, diplomacy, range, sensors range and miniaturization. These bonuses vary to some degree in each game. Please note it's hard to distinguish invisible bonuses from AI abilities until a player gets first spying level on the AI"
This means the AI get's more spending points, and many invisible bonuses that players are incapable of getting once you go into Maso and higher.