Well, not that boneheaded perhaps. Suppose you could change the mod data mid-game. Then you as part of your changes, you'd remove the data for one type of planetary improvement for example. Now, when you load your game where you have those built, the game can't find the data ... and it might just crash. Or otherwise do something unexpected. Or some other data might become corrupted. So probably SD didn't want the headache of designing sane behaviour for those situations, and just went with the simple and sane (from software engineering PoV) solution.

I don't know how mod developers get around this limitation - some tech and items typically become available only later in the game, so you can't just start a quick game to test them out in the typical setting. Is it possible to edit saved games? (I hope regular games are different from metaverse games in this respect.)

SD simply took the lazy way out with this design decision. I tend to mod the hell out of every (non-FPS) game I play, and I've never encountered an implementation like this. If planetary improvements or ships are suddenly missing data from a mod, they should simply disappear. This isn't perfect, but it saves the player from losing 15 hours of game time because they realize a mod isn't working as intended.

Civ IV can gracefully handle the removal of mod components at any time during the course of a campaign. On the occasions that TW can't, it does crash but logs a very clear indication of exactly what missing element caused the crash. The TES games are the best here - not only do they gracefully handle the addition and removal of data, they also notify the player when they load a saved game that used mod elements that are no longer present.

Anyway, as pissed off as I was when I first ran into this, it did turn out to be a serendipitous thing: being forced to start a new game sucked, but the new campaign is shaping up to be a whole lot more interesting than the first.