How We Balance a Simulation

The easiest mistake to make in a business simulation is a game where every reasonable plan wins. The screens are full, the numbers climb, the player is content — and there is no decision anywhere in it. If you win whatever you do, you are not choosing.

The second easiest is the opposite: a game with one right answer. You add seven investment products, one of them beats the other six on every axis, and that screen has stopped asking a question.

You cannot tell without measuring

Neither is visible from the design table. Looking at the constants and saying "this seems balanced" does not work, because nobody can hold in their head how twelve variables chase each other across a hundred and twenty months.

So every game has a harness beside it that plays the game without a human. The harness takes a handful of hand-written households or companies — the frugal one, the extravagant one, the one that borrows, the one that does everything right — and plays each of them start to finish, dozens of times, from a fixed seed. The fixed seed matters: if the same seed does not produce the same run, what you are measuring is noise rather than balance.

A contract, written down

Then we write down what those runs are supposed to produce. In a file, sentence by sentence. The first clause in the e-commerce game’s contract, for instance, is this: doing nothing must never turn a profit. If a player who opens the shop and makes no decisions is making money, what is paying them is not the game but a bug in it.

Another: a plan deliberately designed to be bad must not out-earn a competent one. Another: at least three plausible plans have to lose money. Each is tied to a test, and the tests run on every change. When a balance tweak breaks one of those clauses, we find out rather than the players.

A game in which every plausible line wins is a game with no decisions in it.

A search that plays better than the designer

There is one more problem: hand-written strategies are limited by the imagination of whoever wrote them. So we also run a search that tries settings at random and hill-climbs towards the best it can find. The aim is not to find the best strategy — the aim is to find out whether an exploit nobody thought of is sitting there.

In the e-commerce game that search found a plan meaningfully better than the best one the designer had written. When it does, there are two options: close the gap, or accept what it found as the ceiling of the game and tune everything else against it. In that case we did the second, because what it had found was not a cheat but genuinely good play.

What it does not measure

None of this tells you whether a game is fun. It does not measure whether a screen reads clearly, whether a sentence is well written, whether a decision is exciting. It answers exactly one question: in this game, does your decision change the outcome? If the answer is no, nothing else about it matters.

Why the seed has to be fixed

A simulation contains chance: the market moves, events arrive, some months go well and some badly. Play the same strategy twice and you get two different results. That is good for the game and a disaster for measurement.

So the tests do not remove the chance, they pin it. The random number generator is handed a starting value, and the same value always produces the same sequence. Then if a result changed after a balance tweak, the only thing that could have changed it is the tweak. The first thing this turned up in one of the games was that the same seed produced two different runs — which is to say the measuring instrument itself was broken.

What an imbalance looks like

A concrete example: on a screen with seven investment products we found that the deposit account offered both the highest return and zero volatility. The other six had no remaining reason to exist. You do not see that by reading the table; you see it by playing the strategies and comparing what they end with.

Another from the same game: the mortgage rate was lower than the deposit rate. Borrowing and putting the money on deposit was risk-free profit. No player ever reports that as a bug — whoever finds it simply wins, and concludes the game is easy.