Exploring effects of simultaneous combat
from 
GalCiv2 Forums
GalCiv2 Forums
Question: How will the simultaneous-firing system for combat (announced for 1.2) affect the ideal balance between weapons and armor?
This post attempts to answer this quantitatively using my combat simulator. I've started by writing a new version of the simulateCombat function that implements the simultaneous firing.
To show you the difference, I've uploaded two graphs. Here's how to read them: Combat is simulated a number of times (10000 here), and the "result" of each combat is saved. The result is defined as (Attacker hitpoints remaining - Defender hitpoints remaining), so a positive value means the attacker won, a negative value means the defender won, and (in the new system) a result of zero means both fleets were destroyed. The graphs show the number of times (frequency) that each result appears.
Attacking fleet: 3 ships, each 10 hitpoint, 6 beam attack, 4 beam defense
Defending fleet: same as attacking fleet
(Each ship has a small hull, Warp Drive II, 2 Phasors, 2 Shields II, miniaturization 25%, cost 252, size 32/32)
Current system:
Simultaneous firing:
The attacking fleet won 63.86% of the 10000 iterations in the current system, but only 47.74% of the 10000 iterations with simultaneous firing. Note also that the simultaneous system is quite symmetric around zero, which matches with our intuition: now that attackers and defenders are treated the same, we expect that when two identical fleets fight that it will be an even coin toss. More specifically, one expects that the true distribution is perfectly symmetric around zero. Also, note that while in the previous system there were no zero values, now they are common. In fact, there are (have always been) sharp spikes in frequency at the values that represent "whole" numbers of ships remaining, and zero is now one of these spikes.
So we set up the experiment as follows:
1. Pick two fleets, A and B, where A and B represent realistic options for a player at a certain tech level, and where A has only weapons and B has weapons and shields.
2. Using the sequential-firing simulateCombat function, estimate the probability that: (i) A wins attacking A, (ii) A wins attacking B, (iii) B wins attacking A, (iv) B wins attacking B.
3. Repeat (2) using the simultaneous-firing simulateCombat function
4. Interpret the data
1. Pick two fleets
We want to pick fleets that will give us meaningful results. In the interests of brevity, I'll skip to what I came up with.
Fleet A: Three ships, each with:
Small hull, Warp Drive II, 3 Phasors, miniaturization 25 =
10 hitpoints, 9 beam attack, 30/32 used space, 192 cost
Fleet B: Three ships, each with:
Small hull, Warp Drive II, 2 Phasors, 2 Shields II, miniaturization 25 =
10 hitpoints, 6 beam attack, 4 beam defense, 32/32 used space, 252 cost
The cost on the second ship is higher and it requires an additional technology (Shields II), but we're just going to ignore that. We're not looking for the best split between offense and defense, we're just looking to see how the effectiveness of each design type will change in the simultaneous firing system. Also, since the cost of researching Shields II isn't considered, note that while this may be useful in helping you decide which model to build if you have Shields II, it won't tell you if Shields II is worth researching in the first place. Despite these limitations, Fleet A and Fleet B are pretty representative, I think, and should be good enough for us to get something interesting.
2. Use sequential-firing model to estimate victory probabilities
Running sequential-firing simulateCombat with 5000 iterations,
(Probability attacker wins, median result, mean +/- standard deviation)
A attacks A: 86.88%, 15, 12.93 +/- 9.82
A attacks B: 62.06%, 9, 4.41 +/- 13.85
B attacks A: 86.70%, 16, 13.24 +/- 10.69
B attacks B: 64.04%, 9, 5.02 +/- 13.56
3. Use simultaneous-firing model to estimate victory probabilities
Running simultaneous-firing simulateCombat with 5000 iterations,
(Probability attacker wins, probability of tie, median result, mean +/- standard deviation)
A attacks A: 41.78%, 17.40%, 0, 0.04 +/- 8.25
A attacks B: 32.06%, 7.90%, -5, -3.99 +/- 10.37
B attacks A: 61.32%, 7.66%, 5, 4.17 +/- 10.37
B attacks B: 47.66%, 3.60%, 0, -0.07 +/- 12.28
4. Interpret the data
It looks like the ships chosen may have been... not so great. The results would have been clearer if, for example, B had been inferior in the first system but superior in the second. However, we see here that B is the superior ship is both: not surprising considering that it requires more research and has a higher production cost. Perhaps, in choosing ship design, Fleet A should have been given better weapons (thus more attack power and higher cost) in order to balance them. However, we can see some things worth noting in the data.
In the current system, Fleet A and Fleet B perform about the same as attackers. Fleet B, however, is much better at taking a punch. Fleet A's chance of surviving is about 13.2%, while Fleet B's chance of surviving is about 36.9%, about 2.8 times higher.
In the simultaneous-firing data, we see two things right away: The advantage to the attacker is gone now, and it was big. Also, Fleet B is superior to Fleet A in every way now: It will win about 2/3 of the time, regardless of if it attacks or defends.
In the current system,
When Attacking: Fleet A and B approximately the same
When Defending: Fleet B 2.8 times more likely to survive (36.9% vs 13.2%)
In the simultaneous-firing system,
When Attacking or Defending: Fleet B wins about 61%, ties 8%, and loses 32% of the time.
And it looks like that may be it for conclusions. We know a bit more about what's going on that we did when we started, but I don't feel that we can answer the original question based on the somewhat murky results of one experiment.
I feel that the fleets should have been chosen better at the start... it's hard to say exactly how the change in system shifts the ideal balance between weapons, but it would appear that it shifts it towards defense. This might just be my bias, though, since I expected it to increase the value of defense. (Based on the reasoning the all-weapon ships are very bad at taking a punch and derive much of their current power from being able to take out half the enemy fleet before it fires).
If you can think of a good set of fleets to test against each other in the two systems, describe them in a comment and I'll run them through. Maybe with more/better data points we'll start to see a pattern.
As an aside, we expect certain things from a symmetric system, and we see them here. This serves as a convenient reality-check that the simulation is working as desired. First, when two identical fleets fight, the probability of victory should equal the probability of loss, and since Pr(victory) + Pr(tie) + Pr(loss) = 100%, then Pr(victory) + Pr(tie)/2 should = 50%. (41.78% + 17.40%/2 = 50.48%, and 47.66% + 3.60%/2 = 49.46%). Second, Pr(Attacker wins, X vs Y) should equal Pr(Attacker loses, Y vs X). So 32.06% should equal 100%-61.32%-7.66%=31.02%, and 61.32% should equal 100%-32.06%-7.90%=60.04. All in all, this checks out quite nicely. Also, the mean, median, and standard deviation values all behave as we would expect with the symmetry of simultaneous firing.
This post attempts to answer this quantitatively using my combat simulator. I've started by writing a new version of the simulateCombat function that implements the simultaneous firing.
To show you the difference, I've uploaded two graphs. Here's how to read them: Combat is simulated a number of times (10000 here), and the "result" of each combat is saved. The result is defined as (Attacker hitpoints remaining - Defender hitpoints remaining), so a positive value means the attacker won, a negative value means the defender won, and (in the new system) a result of zero means both fleets were destroyed. The graphs show the number of times (frequency) that each result appears.
Attacking fleet: 3 ships, each 10 hitpoint, 6 beam attack, 4 beam defense
Defending fleet: same as attacking fleet
(Each ship has a small hull, Warp Drive II, 2 Phasors, 2 Shields II, miniaturization 25%, cost 252, size 32/32)
Current system:
Simultaneous firing:
The attacking fleet won 63.86% of the 10000 iterations in the current system, but only 47.74% of the 10000 iterations with simultaneous firing. Note also that the simultaneous system is quite symmetric around zero, which matches with our intuition: now that attackers and defenders are treated the same, we expect that when two identical fleets fight that it will be an even coin toss. More specifically, one expects that the true distribution is perfectly symmetric around zero. Also, note that while in the previous system there were no zero values, now they are common. In fact, there are (have always been) sharp spikes in frequency at the values that represent "whole" numbers of ships remaining, and zero is now one of these spikes.
So we set up the experiment as follows:
1. Pick two fleets, A and B, where A and B represent realistic options for a player at a certain tech level, and where A has only weapons and B has weapons and shields.
2. Using the sequential-firing simulateCombat function, estimate the probability that: (i) A wins attacking A, (ii) A wins attacking B, (iii) B wins attacking A, (iv) B wins attacking B.
3. Repeat (2) using the simultaneous-firing simulateCombat function
4. Interpret the data
1. Pick two fleets
We want to pick fleets that will give us meaningful results. In the interests of brevity, I'll skip to what I came up with.
Fleet A: Three ships, each with:
Small hull, Warp Drive II, 3 Phasors, miniaturization 25 =
10 hitpoints, 9 beam attack, 30/32 used space, 192 cost
Fleet B: Three ships, each with:
Small hull, Warp Drive II, 2 Phasors, 2 Shields II, miniaturization 25 =
10 hitpoints, 6 beam attack, 4 beam defense, 32/32 used space, 252 cost
The cost on the second ship is higher and it requires an additional technology (Shields II), but we're just going to ignore that. We're not looking for the best split between offense and defense, we're just looking to see how the effectiveness of each design type will change in the simultaneous firing system. Also, since the cost of researching Shields II isn't considered, note that while this may be useful in helping you decide which model to build if you have Shields II, it won't tell you if Shields II is worth researching in the first place. Despite these limitations, Fleet A and Fleet B are pretty representative, I think, and should be good enough for us to get something interesting.
2. Use sequential-firing model to estimate victory probabilities
Running sequential-firing simulateCombat with 5000 iterations,
(Probability attacker wins, median result, mean +/- standard deviation)
A attacks A: 86.88%, 15, 12.93 +/- 9.82
A attacks B: 62.06%, 9, 4.41 +/- 13.85
B attacks A: 86.70%, 16, 13.24 +/- 10.69
B attacks B: 64.04%, 9, 5.02 +/- 13.56
3. Use simultaneous-firing model to estimate victory probabilities
Running simultaneous-firing simulateCombat with 5000 iterations,
(Probability attacker wins, probability of tie, median result, mean +/- standard deviation)
A attacks A: 41.78%, 17.40%, 0, 0.04 +/- 8.25
A attacks B: 32.06%, 7.90%, -5, -3.99 +/- 10.37
B attacks A: 61.32%, 7.66%, 5, 4.17 +/- 10.37
B attacks B: 47.66%, 3.60%, 0, -0.07 +/- 12.28
4. Interpret the data
It looks like the ships chosen may have been... not so great. The results would have been clearer if, for example, B had been inferior in the first system but superior in the second. However, we see here that B is the superior ship is both: not surprising considering that it requires more research and has a higher production cost. Perhaps, in choosing ship design, Fleet A should have been given better weapons (thus more attack power and higher cost) in order to balance them. However, we can see some things worth noting in the data.
In the current system, Fleet A and Fleet B perform about the same as attackers. Fleet B, however, is much better at taking a punch. Fleet A's chance of surviving is about 13.2%, while Fleet B's chance of surviving is about 36.9%, about 2.8 times higher.
In the simultaneous-firing data, we see two things right away: The advantage to the attacker is gone now, and it was big. Also, Fleet B is superior to Fleet A in every way now: It will win about 2/3 of the time, regardless of if it attacks or defends.
In the current system,
When Attacking: Fleet A and B approximately the same
When Defending: Fleet B 2.8 times more likely to survive (36.9% vs 13.2%)
In the simultaneous-firing system,
When Attacking or Defending: Fleet B wins about 61%, ties 8%, and loses 32% of the time.
And it looks like that may be it for conclusions. We know a bit more about what's going on that we did when we started, but I don't feel that we can answer the original question based on the somewhat murky results of one experiment.
I feel that the fleets should have been chosen better at the start... it's hard to say exactly how the change in system shifts the ideal balance between weapons, but it would appear that it shifts it towards defense. This might just be my bias, though, since I expected it to increase the value of defense. (Based on the reasoning the all-weapon ships are very bad at taking a punch and derive much of their current power from being able to take out half the enemy fleet before it fires).
If you can think of a good set of fleets to test against each other in the two systems, describe them in a comment and I'll run them through. Maybe with more/better data points we'll start to see a pattern.
As an aside, we expect certain things from a symmetric system, and we see them here. This serves as a convenient reality-check that the simulation is working as desired. First, when two identical fleets fight, the probability of victory should equal the probability of loss, and since Pr(victory) + Pr(tie) + Pr(loss) = 100%, then Pr(victory) + Pr(tie)/2 should = 50%. (41.78% + 17.40%/2 = 50.48%, and 47.66% + 3.60%/2 = 49.46%). Second, Pr(Attacker wins, X vs Y) should equal Pr(Attacker loses, Y vs X). So 32.06% should equal 100%-61.32%-7.66%=31.02%, and 61.32% should equal 100%-32.06%-7.90%=60.04. All in all, this checks out quite nicely. Also, the mean, median, and standard deviation values all behave as we would expect with the symmetry of simultaneous firing.
