A scoring model is a prediction. The only honest test is whether the accounts it ranked highest went on to reply more, and that test needs real volume before it means anything.
Stopping is a successful run
Positive replies are rare, so a comparison on a handful of them is noise that happens to look like a table. A tier is compared only once it clears 200 sends and 5 positives; at least two tiers must clear.
A tier that falls short is reported as not enough data and left out. It is never merged into a neighbour to reach the floor. If fewer than two tiers clear, the run says how much more volume is needed and changes nothing. Re-weighting a model off three replies would reorder thousands of future accounts.
Sorting is different from predicting
If A beats B and B beats C, the model is separating. If the tiers come out flat, the model sorts accounts without predicting anything, and the per-dimension read shows where. If the order is inverted, we check the obvious causes first: whether the top tier got different copy, a different channel or a different week.
Tiers get different treatment by design, so tier and treatment are tangled together in the data. The retro says so on every read. It produces evidence, and treating that evidence as proof is how a retro does damage.
No conclusion is a valid finding
Each dimension is split into accounts that scored high and low on it, and the two groups are compared on positives per send. A dimension with real lift has earned its weight. One with no lift on a good sample becomes a candidate to down-weight, though not to delete on one read.
A dimension with no lift on a thin sample gets no conclusion at all, and the report says exactly that. Email and LinkedIn are read separately, and so are tiers scored under different versions of the model.
Some things are never re-fit from data
Hard disqualifiers are policy, so they stay out of the retro. A competitor replying well is no reason to start emailing competitors. Weights move in steps, so one retro never re-tiers most of a list, and every accepted change goes into a dated new version with a changelog.
Lists already scored under the old version keep their old tiers until they are re-scored. The retro names which ones, so nobody works from a superseded model without knowing it.
Runs well with
Skills compound. These are the ones we usually install alongside it.