Quantitative analysis: A combat simulator in R

I’ve put together a little GalCiv2 combat simulator using RLink, an open source statistical programming language.

Basically, there’s a function simulateCombat(iterations, AttackingFleet, DefendingFleet). For each iteration, it simulates combat between the two fleets and stores the result in a vector (one element for each iteration). The result is calculated as the sum of the remaining hitpoints of the attacking fleet minus the sum of the remaining hitpoints of the defending fleet. Note that since one of these is at zero (by definition) when combat ends, this number will be positive if and only if the attacker wins and negative if and only if the defender wins; also, the absolute value of the result is the remaining hitpoints of the victor.

The user can then plot the results to visualize the probability distribution of the outcome of combat between any two fleets. Here’s what the graphs look like so far (they’re not very pretty yet).

Plot: Link
iterations = 1000
attacking fleet: 5 ships x 20 hitpoints, 8 beam attack, 3 beam defense
defending fleet: same as attacking fleet

Probability that attacking fleet wins: 72.1%
Probability that defending fleet wins: 27.9%
Median result: 29 hitpoints remaining for the attacker

And this is with identical fleets; it’s clear that attacking first is a huge advantage!

Anyway, I just got this up and running, and I thought it was neat.

It can also be used to answer some higher-level questions. For example, imagine the set of fleets with the following characteristics:
3 identical ships
40 hitpoints per ship
beam attack + beam defense = 20
beam attack > 5 (because 2 attack, 18 defense versus 2 attack, 18 defense simulations take a long time)

Using the combat simulator, we can construct a 15x15 matrix of the probability that any given configuration (there are 15, from attack=6, defense=14 to attack=20, defense=0) will win when attacking any other configuration. At the same time, we can construct a similar matrix of the median result.

Setting the iterations per simulation to 200 (for a total of 15 x 15 x 200 = 45000 iterations, a little under 8 minutes at 1.8GHz), we get tables composed of the victory probability and median result for each attack fleet / defense fleet pair of the 15 possible models of the ship. I won’t reproduce the tables here, but after a bit of analysis, the computed best ship (Nash equilibrium) to choose if you’re attacking is the attack = 20, defense = 0 model, and the best ship to choose if you’re defending is the attack = 15, defense = 5 model (there’s a 69.5% chance of the attacker winning, and the median result is 39 hitpoints remaining for the attacker… the data say that the attacker cannot choose better given the choice of the defender, and the defender cannot do better given the choice of the attacker, which is the definition of a Nash equilibrium). This is according to the median result data; there is no Nash equilibrium if victory probability is the payoff, although this is just a quick analysis and there’s some noise in the data. It’s hard to visualize what’s going on in this example… maybe in a later post.

Anyway, that’s just to demonstrate the sort of thing one can do with this combat simulator.

If anyone has requests for simulations I’d be happy to run them, if possible. I’m planning on making a follow-up post once I’ve had more time to play with this, let me know if there’s something you want to see.

If there’s interest I’ll post the source once I polish it some more.

Locked Post

Advanced Search

Watch this post

Do not email me updates for this post Email me updates for this post

Successfully updated karma reason!

Sorry, but there are already simulators that run either near-instant or do 10,000 simulations (or both)

Someone beat you to it. Link and Link

Nice to see people working on stuff like this though.

Successfully updated karma reason!

Veblen,

Nice! Your effort is appreciated and I look forward to your further developments of this area.

Thanks.

AR

Successfully updated karma reason!

Sorry, but there are already simulators that run either near-instant or do 10,000 simulations (or both)

AFAIK the other simulators dont include tools to calculate Nash equilibria. And it consumes time to do this manually. So Veblen's work is one step ahead. Brilliant idea, I would like to see the details.

Successfully updated karma reason!

Sorry, but there are already simulators that run either near-instant or do 10,000 simulations (or both)

Yup, this isn't the first. The Java simulator's cool, and you can't beat Darth Kryo's for easy of use. They're all pretty different, though.

Unlike the Java simulator, this one isn't a stand-alone program. Right now, it's a 150-line long R script that defines a function simulateCombat(), and is intended to be used interactively in the R environment.

So with the Java simulator, you download it, download and install the Java runtime environment, run it, and it opens up an application where you can run a simulation.

With the R simulator, you download the Windows (for example) R interpreter and IDE (free, good), open it up, and you have access to the simulate combat function inside of a powerful interactive programming environment. R has some great data visualization functions built in, too.

To give an example, say the Torians walk up with some crazy battleship with technology way beyond mine. I want to know how many of my old fighters have to be in a fleet to take out one of these. So in the R interactive window, I type:

source("simulateCombat.R")

median.result = vector(length=15)
victory.probability = vector(length=15)

torian.fleet = createFleet(1, 60, 39, 0, 0, 22, 0, 0)

for(i in 1:15) {
my.fleet = createFleet(i, 12, 8, 0, 0, 2, 1, 1)
results = simulateCombat(1000, my.fleet, torian.fleet)
median.result[i] = median(results)
victory.probability[i] = sum(results > 0) / length(results)
}

And that's it. Here's the breakdown, if you're curious:
ships, median result, victory probability
1, -60, 0.000
2, -58, 0.000
3, -54, 0.000
4, -50, 0.000
5, -44, 0.000
6, -38, 0.003
7, -30, 0.017
8, -21, 0.084
9, -11, 0.235
10, 12, 0.523
11, 51.5, 0.756
12, 83.0, 0.915
13, 96.0, 0.981
14, 120.0, 0.999
15, 132.0, 1.000

Typing
plot(victory.probability) pops up this:
Link

The way it is now takes some programming knowledge to use. It's much less user friendly than the other simulators. This is a tradeoff for the more powerful analysis that it makes possible.

Successfully updated karma reason!

Here's a somewhat rambling post about my most recent application of the simulator.

I'm interested in getting at the answer to the age-old (well, two month old) question, "Is it better to have a fleet of large ships or to have a fleet of small ships?" So the first step is to chisle this question down to something specific enough to tackle.

There are many variables involved here. Too many. So let's hold the ones we're not interested in constant at some reasonable value. Technology is at the top of the list.

Assume the following have been researched: Plasma weapons, Shields, Warp Drive, Medium Scale Building, Enhanced Miniaturization, Advanced Logistics
Yielding a Miniaturization bonus of +25% and a Logistics level of 21 (15 + racial bonus of 6)
Let's further constrain the ship types by saying that each one has exactly one warp drive.

The next step is to ask, "With the set of components available at given these technologies, what are the best ship designs for each hull size?"

Playing around in the ship editor and using my intuition, I find these candidates:

Tiny 1: hitpoints = 6, logistics = 2, a.beam = 3, cost=110
Tiny 2: hitpoints = 6, logistics = 2, a.beam = 2, d.beam = 1, cost = 115
Small 1: hitpoints = 10, logistics = 3, a.beam = 5, cost = 165
Small 2: hitpoints = 10, logistics = 3, a.beam = 4, d.beam = 2, cost = 190
Small 3: hitpoints = 10, logistics = 3, a.beam = 3, d.beam = 3, cost = 210
Medium 1: hitpoints = 16, logistics = 4, a.beam = 7, cost = 225
Medium 2: hitpoints = 16, logistics = 4, a.beam = 6, d.beam = 2, cost = 250
Medium 3: hitpoints = 16, logistics = 4, a.beam = 5, d.beam = 3, cost = 270
Medium 4:hitpoints = 16, logistics = 4, a.beam = 4, d.beam = 5, cost = 295

And what are the possible combinations of hull sizes that fill up 21 logistics?

5 mediums
4 medium, 1 small, 1 tiny
3 medium, 3 small
3 medium, 1 small, 3 tiny
2 medium, 4 small
2 medium, 3 small, 2 tiny
2 medium, 1 small, 5 tiny
1 medium, 5 small, 1 tiny
1 medium, 3 small, 4 tiny
1 medium, 1 small, 7 tiny
7 small
5 small, 3 tiny
3 small, 6 tiny
1 small, 9 tiny

We're going to have to prune this down. With multiple possible ship designs for each hull size, we have 217 possible fleets. To simulate 100 battles between each of these would take 4,708,900 iterations, or something like 11 hours on my computer.

We're most interested here in looking at the fleet composition, not weapon/defense balance, so I'm going to tame the size of the sample set by cutting the ship designs down to one per hull size, just using my intuition to pick the design that seems strongest. So that gives us:

Tiny: hitpoints = 6, logistics = 2, a.beam = 2, d.beam = 1, cost = 115
Small: hitpoints = 10, logistics = 3, a.beam = 4, d.beam = 2, cost = 190
Medium 1: hitpoints = 16, logistics = 4, a.beam = 6, d.beam = 2, cost = 250
Medium 2: hitpoints = 16, logistics = 4, a.beam = 5, d.beam = 3, cost = 270

Ok, we'll leave two designs for medium hulls since I'm curious as to which performs better. Now we have 24 possible fleets, and simulating 100 battles between each takes 57600 iterations... somewhere around 8 minutes. That will work. Let's write up some R code to run these 24x24=576 simulations.

ship.tiny = createShip(6, 2, 0, 0, 1, 0, 0)
ship.small = createShip(10, 4, 0, 0, 2, 0, 0)
ship.medium.1 = createShip(16, 6, 0, 0, 2, 0, 0)
ship.medium.2 = createShip(16, 5, 0, 0, 3, 0, 0)

fleet.composition = matrix(nrow=24, ncol=4)
fleet.composition[1,] = c(5, 0, 0, 0)
fleet.composition[2,] = c(4, 0, 1, 1)
fleet.composition[3,] = c(3, 0, 3, 0)
fleet.composition[4,] = c(3, 0, 1, 3)
fleet.composition[5,] = c(2, 0, 4, 0)
fleet.composition[6,] = c(2, 0, 3, 2)
fleet.composition[7,] = c(2, 0, 1, 5)
fleet.composition[8,] = c(1, 0, 5, 1)
fleet.composition[9,] = c(1, 0, 3, 4)
fleet.composition[10,] = c(1, 0, 1, 7)
fleet.composition[11,] = c(0, 5, 0, 0)
fleet.composition[12,] = c(0, 4, 1, 1)
fleet.composition[13,] = c(0, 3, 3, 0)
fleet.composition[14,] = c(0, 3, 1, 3)
fleet.composition[15,] = c(0, 2, 4, 0)
fleet.composition[16,] = c(0, 2, 3, 2)
fleet.composition[17,] = c(0, 2, 1, 5)
fleet.composition[18,] = c(0, 1, 5, 1)
fleet.composition[19,] = c(0, 1, 3, 4)
fleet.composition[20,] = c(0, 1, 1, 7)
fleet.composition[21,] = c(0, 0, 7, 0)
fleet.composition[22,] = c(0, 0, 5, 3)
fleet.composition[23,] = c(0, 0, 3, 6)
fleet.composition[24,] = c(0, 0, 1, 9)

median.result = matrix(nrow=24, ncol=24)
victory.probability = matrix(nrow=24, ncol=24)

for(i in 1:24) {
# this is a rather clunky way of creating the fleets, but it works...
attacker = createFleet()
attacker.ships = 0

if(fleet.composition[i, 1]) {
attacker[(attacker.ships+1)

attacker.ships+fleet.composition[i, 1]),] = ship.medium.1
attacker.ships = attacker.ships + fleet.composition[i, 1]
}
if(fleet.composition[i, 2]) {
attacker[(attacker.ships+1)

attacker.ships+fleet.composition[i, 2]),] = ship.medium.2
attacker.ships = attacker.ships + fleet.composition[i, 2]
}
if(fleet.composition[i, 3]) {
attacker[(attacker.ships+1)

attacker.ships+fleet.composition[i, 3]),] = ship.small
attacker.ships = attacker.ships + fleet.composition[i, 3]
}
if(fleet.composition[i, 4]) {
attacker[(attacker.ships+1)

attacker.ships+fleet.composition[i, 4]),] = ship.tiny
attacker.ships = attacker.ships + fleet.composition[i, 4]
}

for(j in 1:24) {
defender = createFleet()
defender.ships = 0

if(fleet.composition[j, 1]) {
defender[(defender.ships+1)

defender.ships+fleet.composition[j, 1]),] = ship.medium.1
defender.ships = defender.ships + fleet.composition[j, 1]
}
if(fleet.composition[j, 2]) {
defender[(defender.ships+1)

defender.ships+fleet.composition[j, 2]),] = ship.medium.2
defender.ships = defender.ships + fleet.composition[j, 2]
}
if(fleet.composition[j, 3]) {
defender[(defender.ships+1)

defender.ships+fleet.composition[j, 3]),] = ship.small
defender.ships = defender.ships + fleet.composition[j, 3]
}
if(fleet.composition[j, 4]) {
defender[(defender.ships+1)

defender.ships+fleet.composition[j, 4]),] = ship.tiny
defender.ships = defender.ships + fleet.composition[j, 4]
}

results = simulateCombat(100, attacker, defender)
median.result[i,j] = median(results)
victory.probability[i,j] = sum(results>0) / length(results)

}
}

That took longer than expected, around 15 minutes. There's some overhead when we make that many separate calls to simulateCombat(). Well, here's what we got:
median.result: Link
victory.probability: Link

Now to make sense of it. First, let's restate what we did so that we can interpret the results
.
There are 4 models of ships, one tiny hull, one small hull, and two medium hulls. The models were picked to represent the best ships that can be built when these technologies have been researched: Plasma weapons, Shields, Warp Drive, Medium Scale Building, Enhanced Miniaturization, Advanced Logistics (Miniaturization: +25%, Logistics: 21). Each ship has one warp drive. The stats for the ships are:

Tiny: HP=6, a.beam=2, d.beam=1
Small: HP=10, a.beam=4, d.beam=2
Medium 1: HP=16, a.beam=6, d.beam=2
Medium 2: HP=16, a.beam=5, d.beam=3

24 fleets are constructed containing permutations of these ships such that the sum of the logistics in each fleet is as close to 21 as possible.
The compositions of the fleets are:

Fleet index = (Medium 1, Medium 2, Small, Tiny)
1 = (5, 0, 0, 0)
2 = (4, 0, 1, 1)
3 = (3, 0, 3, 0)
4 = (3, 0, 1, 3)
5 = (2, 0, 4, 0)
6 = (2, 0, 3, 2)
7 = (2, 0, 1, 5)
8 = (1, 0, 5, 1)
9 = (1, 0, 3, 4)
10 = (1, 0, 1, 7)
11 = (0, 5, 0, 0)
12 = (0, 4, 1, 1)
13 = (0, 3, 3, 0)
14 = (0, 3, 1, 3)
15 = (0, 2, 4, 0)
16 = (0, 2, 3, 2)
17 = (0, 2, 1, 5)
18 = (0, 1, 5, 1)
19 = (0, 1, 3, 4)
20 = (0, 1, 1, 7)
21 = (0, 0, 7, 0)
22 = (0, 0, 5, 3)
23 = (0, 0, 3, 6)
24 = (0, 0, 1, 9)

The result of one iteration of the simulation is the total remaining hitpoints for the attacking fleet. The matrix mean.result contains in each element mean.result[i, j] the median result from 100 simulated combats with the fleet corresponding to index i attacking the fleet corresponding to index j. The matrix victory.probability contains the percentage of times that the attacking fleet was victorious.

Here's what the victory.probability looks like when it's shaded. Light squares indicate a high probability of the attacker winning.
Link

I'm getting tired so I'll have to let that image speak for itself, except to say that, at least at this early point in the game, the best fleets are those represented by indices 1, 2, 3, 11, 12, 13, 18, and 21. The medium ships are quite powerful, while the tiny ships get owned (but we wouldn't expect them to shine with early techs and no starbase). Fleet 11 is the best choice of defender 23/24 times and the best attacker 15/24 times (the other indices that can claim best attacker are 1, 2, and 3), according to median.result, and (1, 11) is the Nash equilibrium according to victory probability (no NE exists in the median.result data, but (1, 11) is very close.

So for the specific techs and ship designs we assumed at the beginning, it looks like a fleet of five medium hulls is the winner.

Successfully updated karma reason!