Here our data migration team share their insights to support organisations planning a move to a new data handling process and to help you avoid some of the stumbling blocks that can occur when moving your existing data to a new system.
What Is Data Migration?
Data migration is the process of taking your existing data and transferring it into a new system. Your data will need to be translated from its original format to a format that your new system recognises.
It’s A Bit Like Moving House
Data migration’s a bit like moving house. You have all the things that need to be transferred from one place to another, and it feels like it should be simple, but when it comes down to it things aren’t always straightforward. Everything needs to be prepared and wrapped up, ready to go.
Some things Are Easy...
You have a fitted wardrobe in your old home, and a new one that you know all your shirts will fit into fine, so you can pack them up in a case with no problem. Others aren’t so straightforward. You have three sofas in your current house, but you’re going to get a couple of armchairs for the new one so you’ll need to decide what you’re planning to keep. And then there’s the cutlery. You’ve always just kept it loose in a drawer, but now you’d like to use dividers and keep different pieces separate, so that will need sorting out. You also have far more forks than any human could ever reasonably need, so you want to pare those down.
Then It gets Really Tricky...
You have a super-sized bed which was put together in the master bedroom, and you can’t get it back out of the door. That will need dismantling carefully – it looks a bit fragile – then putting back together again at the other end. However, if you take that before the chest of drawers then there won’t be space to get the drawers through the bedroom door in your new house, so you’ll need to think about what order to take things in.
Finally, That Bit Everyone's Been Avoiding...
The cupboard under the stairs, the shed, the garage or the loft (or maybe all of the above). No-one’s quite sure what’s in there, but whatever it is will need carefully sifting through and untangling. Most of it may well be discarded, it’s been there forever, after all, but you’ll need to see what’s in there first. It will take time, but in order to make sure you keep everything you want, it’s a job that will need doing.
Once You’re In Your New House...
A few things may still need sorting. You’ll probably find that you’ve come across duplicate kitchen utensils (there was a whole box of them in the loft, who knew?), or things aren’t quite where you expected them to be.
Sorting out these kinds of questions is what data migration is about – configuring your existing data to make it ready for the move, sorting out parts which have been heaped together for a while and untangling the knotty bits. The following practical tips can help make this as smooth as possible.
Things To Bear In Mind When Deciding What Data To Upload
It’s tempting to decide to keep all your old data in your new system. However, as well as being potentially costly, you may find that transferring all your old data clutters up the new system with information you no longer need.
If you’d like to keep costs down and make sure that your new system doesn’t hold unnecessary information, these pointers can help you:
Will you be able to get to your original data after you’ve started using your new system?
If you currently use Excel spreadsheets, for example, you can keep those to refer to. If there is old information there that you won’t need to report on or use day to day, you probably don’t need to put it into your new system. You can keep it in its current format (e.g. spreadsheets), then consult it if necessary.
Can you transfer it to Excel spreadsheets?
If your original data is not going to be available after you move to your new system (for example, it’s in a bespoke system that will be switched off), you can consider transferring it to Excel spreadsheets. This way, as above, you’ll be able to go back to the data if necessary, but won’t need to store it all in your new system.
Do you plan to stop collecting some of the information that you’ve asked for previously?
For example, you may have stored details for a project you no longer run. If so, you might not have made provision (set up the necessary fields) to store this data in your new system. If it’s data you no longer plan to collect, there’s a good chance you don’t need to transfer this old information into your new system either.
Do you need to take all your historic activity records?
Or perhaps just the ones for the last 3 to 5 years, for example. You might have information going back 20 years, but do you really need to keep it all at your fingertips? You must also consider how long you should be keeping hold of personal data for and removing any that goes back further than this.
Do you need to transfer the records for all your clients?
Or perhaps just the ones who’ve been active in the last 3 to 5 years for example. It could well be that you won’t need to know the details of people who haven’t engaged with you recently.
Do you trust your data?
If your information hasn’t been reliably recorded previously, you may decide that you want to start your new system fresh. However, if you’re wondering whether to leave all your previous data out of your new system and start from scratch, you’ll need to consider whether this will this affect any reporting that you will need to do at year end. Might you need to know what services you’ve given to current service users? Knowing what has happened in the recent past can be a benefit when you’re looking at service provision.
What Factors Can Increase The Cost Of Migration If You Pay For It?
Usually, the more records you migrate, the higher the cost. Beside limiting the number of records (i.e. information about the people you work with and the activities you’re involved in), there are other things which can help minimise the charge for data migration.
Information In Excel Spreadsheets
1. Prune columns: after saving a copy for the archive, delete all the columns you don’t need or trust for accuracy.
2. Prune rows: delete all the older data you won’t need in the new system. You’ve got a copy in the archive if you do need to refer to it.
3. Try to make sure that you only have one type of data in each cell.
Example A: If you have a number field, make sure there is only one number in each one.
Example B: Where there are different types of information in one field (in the example below, an amount and a note), if they’re going into separate fields in your new system then it will help to separate them out. (If they will be in a single text field then this is not an issue).
Example C: Where possible, separate out mobile and landline numbers.
Example D: Where you would like addresses to be in separate lines in the contact details tab, make sure that each line is in its own cell in your spreadsheet. (If the whole address is in one cell, then it could end up in ‘Address Line 1’ in your new system).
4. Give clear lists of the people and organisations that you would like to add to your system.
5. Look for any duplication and see if you can merge it first. You know your data and you’ll know best what to keep and what to discard.
Is This Preparation Worthwhile?
The more complex your data (i.e. having different information in the same cell) the longer it will take a data migration team to sort out, so the more it is likely to cost.
And, as with moving house, migrating data takes time. For many small charities, even thinking about taking the time (and money) to invest in new systems can seem like a luxury, when what you really want to be doing is getting out there and getting on with your passion and your mission. But, with the right support, the benefits of streamlining and centralising your data into the system that is right for you can make a significant difference to your organisation in the long term.
‘Obviously, all the reporting is much easier… that goes without saying. But I think it’s given us, as well, access to really quick at a glance numbers’ – HomeStart HOST
‘[Having a system] based on the delivery of our services and reporting requirements has resulted in the organisation working more efficiently.’ – SAFE Foundation
