Data migration tips to save you time!
So we are almost ready to go. In this data migration series, we have already looked at how to migrate data and some tools to help with the migration. But now for the fun part, I want to share with you some data migration tips to help make your migration into Salesforce as smooth as possible.
Let’s jump right on in, and please share any of your own tips in the comments section below.
Data Migration Tip #1: External IDs & Upserts
Let’s walk through a common scenario in a data migration…
The Data Migration nightmare:
You are about to migrate 12 different objects into Salesforce, everything from Accounts, Contacts and Tasks through to Opportunities, Cases and a number of custom objects you have created in your org.
How do you create these records in Salesforce, and still have all related records end up linked together?
Do you insert the first object (let’s say Accounts). Because you need to link the Contacts to those Accounts, in the second file do you use Excel to create the link? Do you take the import file from Dataloader, run a VLOOKUP in the Contacts file and then pull in the related Account IDs? YUCK! Now imagine having to repeat that process for all 12 objects…
I have been there myself. And when it is a very large migration it can become extremely difficult to track and manage all the spreadsheets. Especially when you are working with live data, that is always changing and is never static.
This scenario can be the nightmare of any data migration but it doesn’t need to be. #NoMoreVLOOKUPS
Salesforce External IDs:
As you know, each Salesforce record is given a unique 15/18 character ID. That is great, if you are only dealing with data in a single system or Salesforce org.
But what happens when you are need to store the ID of a record, that is actually sourced from a system outside of your Salesforce? For example an ID from an invoicing system like SAP or Oracle.
This is where External IDs come into the picture.
By configuring a field to be an External ID you can then use it to store the original reference/ID from the other system. Additionally, you can also use it to prevent duplication within Salesforce by setting the External ID field to unique.
It is this ‘uniqueness’ that helps us out when migrating bulk data into Salesforce. As Salesforce will automatically index the External IDs, allowing you to use this as a reference when doing an Upsert within Dataloader.
#NoMoreVLOOKUPS: Upserts with External IDs:
Now let’s take a quick look at Upsert within Dataloader. Upsert combines the ability to either Insert/Update records while using a single file. However, using Upsert in Dataloader also gives you access to a few extra features.
After you click Upsert and start going through the Import Wizard, you will get the ability to select either the Salesforce ID or an External ID (if available on the object).
Additionally, for any relationship you have on the object, Dataloader will also prompt you to select the applicable ID fields which is also referenced in our import file.
This is how you avoid using VLOOKUPs!
By populating an External ID, on each record you load into Salesforce you can then use this to get Salesforce to do the lookup for you. Even if you don’t actually have an External ID from another system, you can create your own. Just make sure it is unique for each record.
I know this is a little confronting to start with. It took me a little while to wrap my head around just how exactly to do this when I ran my first data migration. However Upsert & External IDs can save you so much time, I strongly recommend you practice and learn how to use this ‘double act’ to your advantage 🙂
There is of course more resources available on this topic. Salesforce Help has an article which gives you a step by step process you can follow. I also found a video series on YouTube by Doug Ayers, which really breaks this down over an informative video tutorial.
Data Migration Tip #2: Update System Audit Fields (Salesforce’s ‘hidden’ trick)
In Salesforce, every record has a number of system date fields which typically can’t be edited. Fields like ‘Created Date’, ‘Created By’, ‘Last Modified Date’ and ‘Last Modified By’ are locked down (and for good reason!)
These are your standard audit fields and are very important when tracking a record’s history. While planning a data migration, you might come to realise that you need to retain this information from the previous data source. And that is where the permission ‘Update System Audit Fields’ comes into the story.
Imagine a sales person’s activity report, which is displays only activities created in this quarter. As such after a data migration is complete, you still want your users to run these reports. So what can you do?!
In the past, being able to edit these fields was part of what I called the ‘hidden Salesforce toolkit’. You know the list of things that Salesforce can do, but they don’t really publicise? Like increasing the cheeky workflow limit here or there… But all part of the Salesforce Support toolkit, but you had to ask for it to be enabled in your org first. The problem was, you had to know them in the first place you could even ask, so that you could get it requested to be turned on!
Winter ’16 Release & Update System Audit Fields
In Winter ’16 Salesforce made a number of these ‘hidden’ features publicly available. And with a few clicks you can now enable it easily for your data migration.
There are a few considerations to look at before you enable the ability to update the System Audit Fields, so check them out here.
And then head over to here, for instructions on how to setup and enable this feature in your org!
The biggest point I will highlight about accessing these fields, is that they are only available on record creation via Dataloader. Meaning you can’t insert a record and then try to update the created date on it separately. If you did try to edit/update these fields after a record has been created you will get an error.
Data Migration Tip #3: Automation Anchors
Automation with Salesforce is fantastic. Admins and Developers have access to a wide array of automation tools within most Salesforce orgs, including Process Builder, Triggers, Workflows, Sharing Rules. But when you are migrating data, they can quickly become an anchor.
What do I mean by that? Do you know what automation will fire off based on the data work you are doing? For example, is there a workflow that will send an email alert to another team, or even the customer directly? Is this what you expect to happen, if so that is OK. But you do need to make sure you know what any data you load or update into the system will do to the existing automated processes in your org.
In addition, automation in Salesforce can end up adding extra time to any data insert, update or upserts. Remember even deletion of records can fire off a trigger! When you are moving or manipulating a bulk volume of records, this time can easily add up. Complex sharing rules might really slow down an insert/update of records. And depending on the volume of data you are working with, you of may end up hitting some of Salesforce processing limits.
So before doing any data migration, test and make sure the automation you expect to work does. Otherwise see if you can deactivate the workflow while you are doing the migration.
Share your tips
This is by no means a definitive list, and there is plenty more to come. But in the meantime, please share your own experiences or tips in the comments section below!