Data migration: move systems without losing records

Key takeaways

A data migration moves records from an old system to a new one, and most problems come from skipping the repetitive groundwork: mapping fields, cleansing duplicates, test loads and counting records in and out. Treat it as a project with a finish line, clean the data before moving it, and reconcile every table before switching off the old system.

• Map every field from old to new before moving anything.
• Cleanse duplicates and errors first, not afterwards.
• Run at least one test load and check samples.
• Reconcile record counts before switching over.

Why do data migrations go wrong?

Migrations go wrong when they are treated as an export and an import. Fields do not match, formats differ, duplicates multiply, attachments are left behind and nobody counts what arrived. The fix is not cleverness but a long list of routine checks done consistently, which is ideal work for a dedicated team.

• How UK SMEs can use outsourcing to handle repetitive tasks
• Running a business on spreadsheets: when it breaks

What are the steps in a migration?

Most small-business migrations follow the same sequence. Each step produces something you can check, so problems are caught before go-live rather than discovered by customers.

• Inventory the data: tables, fields, attachments, history.
• Map each old field to a new one, or decide to drop it.
• Cleanse: remove duplicates, fix formats, fill gaps.
• Test load a sample and check it field by field.
• Full load, then reconcile counts and totals.
• Freeze the old system as read-only.

• Data enrichment: fill the gaps in your customer records
• Single source of truth: one place for each business fact

How big is the job?

In an illustrative example, moving 8,000 customer records with notes and 2,000 open invoices to a new system takes about 60 to 90 hours of mapping, cleansing, testing and checking. These are example figures. The cleansing share is usually the largest and the most repetitive, which is why it is often outsourced.

• Recurring tasks vs one-off projects: what to outsource
• Double data entry: what rekeying costs and how to stop it

What about data protection?

A migration is a good moment to apply UK GDPR's storage limitation and accuracy principles: delete what you no longer need and correct what is wrong. Anyone helping with the migration who can see personal data is a processor and needs a written contract. Keep a backup of the old data until you are sure the new system is complete.

• Data processing agreements for outsourcing
• Data security with an outsourced team

What does this look like in practice?

A pattern we see: a business migrates, then spends months fixing duplicates the old system had hidden. Cleansing before the move is slower in week one and much faster over the year.

• See client work
• BPO Solutions for UK businesses

Migration checklist

Do not switch over until these are done.

• Field map signed off.
• Duplicates and errors cleansed.
• Test load checked field by field.
• Counts and totals reconciled.
• Old system set to read-only and backed up.

• Parallel running: test an outsourced task safely

Next step

Tell us the old and new systems and roughly how many records you have. We will outline the steps and say plainly which repetitive parts a team could take on.