Space Empires had a system of fleet orders. It could be added to the ship / fleet info screen (where you set Auto-Explore etc).
Fleet Orders define how a battle will be fought, without requiring tactical control of the action, e.g.:
Commitment:
1. Fight to the death (current default)
2. Fight to 75% damage
3. Fight to 50% damage
4. Fight to 25% damage
5. Run at first damage
Focus:
1. Prioritise most offensive (current default)
2. Prioritise most dangerous (i.e. most offensive against this fleet's defences)
3. Prioritise fastest
4. Prioritise non-combatants
Then, the auto-combat resolves using these settings. A user who never accesses that window won't see any difference from the current behaviour, if the defaults are left as the current settings.
And I agree that retreat should be a percentage, based on relative speed (absolute and remaining) and possibly sensors too. Not sure about the added complexity of setting retreat policy as well, though.