Migrating 40 million comments without losing one

The comments on a high-traffic publisher are not a side feature: they are part of why readers come back. Migrating that means any mistake is visible immediately.

The data

The conversion pipeline carried more than 40 million comments, tens of gigabytes, across four schema versions. During the re-import an index-bloat problem was fixed that cut index size by more than half: import first and rebuild indexes afterwards, rather than paying the write-amplification cost.

The rollout was designed behind a feature flag, so the new system could be enabled gradually and rolled back instantly. The payment and auth libraries the platform depends on were upgraded along the way.

Around it

Ephemeral per-pull-request preview environments were proposed and built: each one gets its own namespace and a live URL posted automatically, cleaned up on merge.

The caching for a live pricing widget was also solved, with a reverse cache in front of the API: one upstream fetch every 15 seconds serves every reader, instead of letting browser requests reach the application layer.

On confidentiality

The client and the delivery channel are covered by a non-disclosure agreement, so this project is published without naming them. We can go into detail on scope and technical decisions in a meeting.