Parallel testing during a migration is supposed to confirm that the new environment produces the same answer as the old one. Usually a mismatch points to a defect in the new logic. Occasionally it reveals that the old result was wrong in a way that had never become visible.
In one migration, a margin-by-location report had reconciled cleanly with finance for years. Rebuilding its logic produced a number that was close, but not identical.
The legacy report was not wrong in one place. It was wrong in two places that happened to offset each other: a discount type was misclassified as a cost-of-goods adjustment, while a freight-allocation rule spread shared shipping costs unevenly across locations. For that business and that period, the two errors landed close enough to the expected total that nobody had reason to investigate.
Why silently fixing it would create a new trust problem
The tempting response is to correct both errors in the new model and let the improved result speak for itself. That would be a mistake.
The errors would not continue offsetting by the same amount. Discount mix and shipping distribution change over time. If the logic were corrected without explanation, the legacy and new reports would drift apart in ways that looked like migration defects just as users were being asked to trust the replacement environment.
Instead, the correction became a documented and communicated change: what the old report actually did, how the new definition differed, the size of the recent variance, and why that variance would change. Finance validated the explanation before the corrected measure reached an executive report.
A mismatch has more than one possible meaning
When parallel results differ, the investigation should consider at least five possibilities:
- A defect in the new implementation
- A defect in the legacy implementation
- An intentional definition change
- A timing or allocation difference
- Multiple legacy errors that previously offset each other
The same discipline applies when two totals match. Agreement may mean both implementations are correct. It may also mean they reproduce the same defect—or that different defects happen to cancel each other out.
Reconciliation proves that two answers agree. It does not prove that either answer is true.