Performance Issue (Here's what I noticed)

from Stardock Forums
I'm sure it's been reported here and there that performance gets to be an issue when you get toward the end of games on the larger maps. I noticed something rather striking when at one point, I adjusted my military slider to zero. When I did that, the performance got ALOT better. So this begs the question of why it is so much slower and what is it about military that's dragging it down.

Now based on a few things I'm seeing, I don't think the bottleneck is the AI. It seems to be doing okay on its own. But one other thing happened that made me wonder. Toward the end of this aforementioned game, I gave away something like 15 defenders to a minor civ. When that happened, my computer froze for 2 to 3 minutes.

I find this stunning since I'm on an Athlon XP 1700 with 512mb of RAM. (Galciv's memory footprint is only about 180mb, so it fits entirely in memory)

It seems to me that Galciv slows down when it is either inserting ships into your fleet, or moreso when moving ships from your fleet into the computers. Being a software engineer myself, I find it strange that it should take slow long to move a collection of say 15 objects and stick them into another collection of only maybe 5 objects.

The only thing I could think of that maybe slowing things down is that there's a sorting algorithm going on when these kind of transfers take place, but the Big O of moving ships into a fleet or transfering them shouldn't be that big, unless these ships are presorted in some way. The operation should be quick, even assuming a doubly linked list how the ships are organized, because these are mostly straight integer type operations.

I also noticed the same thing when giving away planets (which seems to cause worse problems). Again, you should only need to move objects from one place to another, which shouldn't be *that* expensive of an operation.

It would seem to me that something like this can be fixed, unless there's something else associated with these moves that I'm not privvy to. Is there a way this can be fixed?

SeanB
975 views 29 replies
Reply #1 from Stardock Forums Top
Hm that's interesting. Perhaps the game resaves all their stats over to the other civs', then looks through your records to make sure it deletes all the data from your ship list, then the AI has to reevaluate the situation based on those ships being added to the minor civ (5 times), adjust spending, etc. accordingly, also change its attitude toward you. Then all the economies might need recalc, so all the planets would need recalc as to how much advancement each of the projects would receive per turn, and the minor civ AI would have to recalc its strategy too.

All that's just a guess, though.
Reply #2 from Stardock Forums Top
I'm thinking that the game might suffer from internal memory fragmentation as a given gaming session lasts longer and longer. I've had gigantic maps get pretty sluggish after several hours of continuous play, then I save, exit the game, restart, and they were crisp again even though WinXP reported pretty much the same amount of overall memory being used.

I've had that '100 corvettes' event freeze up my computer for up to 30-40 seconds, that's a 2.4 gig P4 with 512 meg rambus memory, seems a lot of processing for such an event, but if memory was internally fragmented in some way then it becomes more understandable.

It also could be what Blue Ocean said, that certain things trigger a global 'What's up?' algo to run, and if a lot of those things happen at once...
Offworld Trading Company
Reply #3 from Stardock Forums Top
One thing that you may have over looked is the fact that giving ships to another race may drasticly change the balance of the game. The races military score may change and the reactions of all the other races to them will have to be recalculated, the same for that races reactions to other races.

Depending on how much they have optmized the giving/recieving of ships/planets/tech will gauge how fast each transaction is processed.

For example, say I give the Torians a 10 Dreadnaughts, which happens to give them a 100% increase to their military rating.

1 of 2 possibilites are implemented

1. Does the group of Dreadnaughts as a whole get processed? They all change posession and all relevant reactions/values calculated.

2. As each one changes possession are all the reactions calculated?

The latter is very timeconsuming. The former isn't as time consuming but takes some time to get "Just Right(TM)", depending on the interface they have designed and implemented. Frogboy, any comments on this?

Assuming they do a re-calc for each item given they may not have spent that much time optimizing it, rather spending the time fixing bugs, and adding featurs.

I've also seen this behaviour and would like it changed. But I would like to see some bugs fixed first and a few UI tweaks like waypoints and some type of constructer modification so they auto-goto specific points to build/upgrade starbases with a queued construction list.

Reply #4 from Stardock Forums Top
Infodragon, I would buy off on your conclusion, if only for the fact that

1) This behavior with ships being removed or added to your fleet from production/disbandment takes place

2) Remember when I mentioned that if military spending is zero'd now, no new ships are being introduced to the fleet, the time in between turns gets much shorter (we're talking the delay gets cut from about 15 to 20 seconds to maybe 1 or 2)

3) Assuming your statement to be true, say I'm giving someone 20 objects. I account for the value, who it came from, etc. It shouldn't take a computer *that* long to figure out how it's going to adjust a reaction to a gift from another empire. Giving technologies doesn't seem to produce this sideeffect either, and I would assume what you're describing should happen here as well, but it doesn't.

You see the delay doesn't necessarily manifest itself in trades, but also in between turns, giving me two choices, cut off military spending when I get tired of the long delay, or grin and bear it. But it does interfere with my enjoyment of the end game because for every 10 turns, I lose about 5 to 10 minutes of time waiting for the computer to do its thing

SeanB
Reply #5 from Stardock Forums Top
The production of ships within your empire may be something entirely different, or possibly the same thing. With the adition of each ship your military rating changes. Each races reaction to you needs to be calculated. So the question that begs asking is... If no ship is produced on that turn does the turn go quickly? Another question is If only 1 ship is produced does the turn go any faster, i.e. is the time taken directly proportonial to the number of ships produced?


With giving ships to another race... I have had this happen, and tested it a bit... There was a minor race that I was trading with and the Acreans were about to crush them, I cought a glimps of the massive fleet being sent in. I gave them an Excalibur(just 1), the Acreans had Avatars, the minor race Dreadnaughts, it took quite a while on my P4 3.06GHz box, about 5-10 seconds, didn't use a stop watch. So I tried again with a small ship, it was instantanious. My guess is that with each transaction GalCiv calcs the reaction of that race to every other race and every other race to that race. So if there are 5 minor civs and 5 major civs there are 10 calcs of the race you gave the ships to, 4 minor races 5 major races and you. Then there are 9 calcs, 4 minor races and 5 major races to the race you gave the ship. Now there are just as many calcs all over again because the ships you gave away have affected you too. So that is another 10 reaction calcs.

Now all these calculations may be very quick and what we are seeing is something entirely different. But due to the size of the game's footprint, which I doubt is more than 50% gfx, which gives about 90MB of game data. Each calculation may be going through about 9-12MB of Data on average, depending on the size of each race, and with 10 races with the above exampel 90MB/10 = 9MB. Now with 10 years of developing with c++ even trivial calculations of 9-12MB of data take a while, even on a shiny new 2GHz machine. A while being at least .5 seconds. So assuming that these are trivial calculations and there are 30 total re-calcs with the above scenario that is a 15 sec delay.

Am I makeing sence or am I talking out of my arse due to extreme burnout due to multiple 60hr weeks? :notsure:
Reply #6 from Stardock Forums Top
It seems that game lagging severely then it creates new ships. I tested this. Try to build something cheap on all your planets and you'll see. Better to test almost w/o comps.
Maybe same issue with up to 15m loading time (gigantic lategame).

~SDC~
Reply #7 from Stardock Forums Top
Infodragon....

Yeah you're actually making sense to me.... trust me I've done the 60hr burn before, so I know exactly how you feel bud.

SeanB.
Reply #8 from Stardock Forums Top
Well then i tested this on map was 1 minor civ and 1 major civ.

~SDC~
Reply #11 from Stardock Forums Top
I've had background loading ever since I installed it. It is *VERY* nice on the P4 3.06GHz with hyper threading! LOL :D :)
Reply #13 from Stardock Forums Top
Btw...

Za H, Concerning your question on background loading, here's what the readme.txt file says:

## Performance Options ##
If you have Windows XP and either a very fast machine or a machine with
Intel Hyperthreading(TM) you can increase the loading of the game by turning on
"background loading" in the main options. This allows the game to load itself while
you are viewing the intro movies and choosing game options.

There is an experimental feature in prefs.ini called HyperthreadEnhancement.
If you have an Intel Hyperthreaded machine and set this to 1 the game will
make use of this feature. Note that it is still being experimented on so
subsequent updates (and as we obtain more hyperthread machines for
testing) this feature will become more useful.

~SDC~
Reply #14 from Stardock Forums Top
EBZeroMatrix,

HyperThreading is only present in the P4 3.06 GHz or the newer Xeons, I think 2.2GHz+ but I'm not sure about that. Then you also need a mother board to support the feature.

BTW, I just turned on the HyperthredEnhancment for GalCiv in the prefs.ini file. I'll post here how it goes.

Reply #16 from Stardock Forums Top
I've also seen this issue (Athlon XP 1900, 512 Mb). I fooled around during one game and noticed that the freeze-up time after giving/buying the ships seemed to increase roughly linearly with the number of ships involved in the transaction (all the way up to a few minutes for large numbers of ships).

I like infodragon's explanation regarding this, but what I don't get is why the time would scale like this. Does it really take ten times longer to process an AI's reaction if I give away 20 ancient defenders instead of 2? Maybe it does; I'm a programming lightweight so correct me if I'm wrong ;)

Oh yeah, and this one time just to see what would happen, I gave away my whole fleet (a big one, since it was late-game gigantic). Almost instant CTD.

(BTW, a great way to disband a whole class of obsolete ships is to give them all at once to some minor race. Much faster than clicking on them each individually. You just slide through the list on the diplomacy screen... it would be nice to have a 'scrap entire class' button, though.)
Reply #17 from Stardock Forums Top
yeah it locks up for several minutes when i play huge/abundant and in late game use corvettes to boost my military standing.

and especially when i buy all the starbases i can get from the ai

~SDC~
Reply #18 from Stardock Forums Top
I am thinking that this would cause a linear increase in time in the number of ships given away:

1. ship removed from your military
2. human military recalc
3. ship added to AI (modify hp/att/def)
4. AI military recalc
5. All AI (minor and major) and human relationships recalced
6. all influences recalced

repeat for every single ship that is traded/bought/sold. :) This could take quite a bit of time.
Reply #19 from Stardock Forums Top
After some more experimentation, I noticed that the huge delay hits when you lose alot of ships in combat...which seems rather inexplicable. The delay involved the loss of just 6 starfighters.

SeanB
Reply #20 from Stardock Forums Top
Sounds like excessive memory gymnastics to me, from what is being described here.
Reply #21 from Stardock Forums Top
"I am thinking that this would cause a linear increase in time in the number of ships given away:

1. ship removed from your military
2. human military recalc
3. ship added to AI (modify hp/att/def)
4. AI military recalc
5. All AI (minor and major) and human relationships recalced
6. all influences recalced

repeat for every single ship that is traded/bought/sold. This could take quite a bit of time. "


That would do it. But wouldn't it be kind of retarded to recalculate for each and every ship in a batch rather than to transfer all the ships and then recalculate once?
Reply #22 from Stardock Forums Top
Madfizzicks,

Depending on how they decided to impelement the transfer of ships it may not be retarted.... The nature of C++ makes it very easy to define one impelementation of removing ships from you and recalcing. After the removal then the transaction. The transaction can be either a destroyed ship, or a traded ship or a dispanded ship...

Now, durring battle you can loose ships and so can the enemy. Every time one of your ships is lost their reactions would have to be recalculated. Because you can destroy a large portion of their fleet and then sue for peace. Now that same code can be used for trading, and disbanding.

So for the sake of development time, why develop 2 methods of ship transactions. Just use what you got and only have to debug one portion of code, rather than developing 2 portions of code and then having more than double the debug time. From experience debug time goes up exponentionaly rather than lineraly with the nubmer of lines of code.

After playing some more yesterday and paying more attention to this I have come to the conclusion that they recalculate everything on an individual ship basis. Frogboy, could you confirm this to sate my morbid developers curriosity?

From the point of view of early design/development this approach may have made sence. But things tend to grow and developers can't predict how long something is going to take when the product is finished. This may be a point in a later release that they can optimize, maybe not. But it is definalty not retarted to do this in such a way.
Reply #23 from Stardock Forums Top
infodragon,

Thanks for the explanation. It does make more sense now. Do programmers ever get sick of explaining this sort of thing to the ignorant masses? ;)
Reply #24 from Stardock Forums Top
Madfizzicks,

Nope, we get tired of explaining it to our ignornant project managers that shouldn't be ignorant! :sniff!:
Reply #25 from Stardock Forums Top
Excellent theory Infodragon. If you're correct, hopefully the developers can condition/separate the extra functions to occur only when needed.