Salesforce Data Deployments Tutorial

Salesforce data deployment involves the transfer of data from one Salesforce org to one or multiple other Salesforce orgs. After testing and approving data changes in a Sandbox or Development Org, they can be moved to Test and UAT sandboxes, and ultimately to the production environment using FlexDeploy, with appropriate approvals and quality gates.

It is possible to migrate new data, as well as update existing data, using FlexDeploy.
It is possible to take a backup of all the data.
Multiple objects can be included in a single migration, enabling simultaneous data migration for multiple objects.
By creating a set of Data file Types, also known as templates, it is possible to migrate data together while preserving the parent-child relationships between records.
Code, or metadata files, and data can be migrated together in a single deployment package, streamlining the deployment process and ensuring consistency between the code and data in the target environment.
To provide an additional layer of protection, an automatic backup of the existing data can be optionally taken before initiating the migration process. This enables organizations to have a safety net in place in case of any issues that may arise during or after the migration, while also providing flexibility to choose whether or not to take the backup.

The initial project setup in FlexDeploy is the same for code and data deployments. Create a new project for Salesforce or use an existing Salesforce project to start migrating your Salesforce data. Check our Salesforce Tutorial if you did not create a Salesforce project to start with.

Setup Required in Salesforce Before using FlexDeploy

To ensure that each record in an object can be uniquely identified, it's necessary to have a column/field that serves as an identifier. Ideally, this field should already exist in the object and be indexed. By using External IDs, data migration can be simplified. With an External ID field, records can be easily updated, deleted, or upserted. In cases where there is no existing indexed field available, a custom field can be created to serve as the identifier using the below steps.

Create a custom field as an External ID field in each object that is going to use for data migration with data populated. Using the same field label for all the objects makes the setup easy.
1. Populating unique data into a new field can be a challenging task. To simplify the process, creating an Auto Number field first and populating initial data into all existing rows can help. After that, the data type can be updated to Text, making it easier to populate unique data into the field.

After setting the External ID in all the relevant objects in the Production Org, it is important to refresh the data to all the partial and full sandboxes, as well as the Development Org, to ensure synchronization of the External ID field across all environments.

Setup in FlexDeploy

Create Salesforce package-based project or you can use an existing project if you already have one. Check our Salesforce Tutorial if you did not create a Salesforce project in FlexDeploy to start with.

Project Configuration

Go to Project Configuration → Properties and enter the External ID

If you are maintaining the same External ID field name for all the objects you are trying to populate, you can enter the External ID directly as below.

if you have different External ID fields, you can enter key=value pair with separated by a comma in this property. The key is your Object and the value is your external field id for that object. You can use the key All for default for all other objects.

All=FD_External_ID__c,User=Username,Contact=Email

Package Creation (Data Template Setup) in FlexDeploy

You can create this once and migrate data whenever needed.

Create a Package in FlexDeploy as you would normally. You can have metadata files and data files in the same package.
Make a list of all the objects you would like to migrate
Use Create button to Create a new Data object template file to migrate (build and deploy). Start creating with the top-level object of your objects and repeat the step for all the objects in the sequence. You can adjust sequencing if got created in the wrong order.

a. Select Operation Type - Insert to create new records, Upsert to update existing records or insert if they don't exist, Update to modify existing records, and Export Only for no deployment, only export to CSV for backup purposes, which allows you to select non-updatable fields.

b. Select the Object from the list; FlexDeploy retrieves all Object names from the Source Org.

c. Select all the fields (columns) you wish to migrate, including Lookup and Master-Detail fields. If you are only backing up and need to include non-updatable columns, select the Export Only operation type, which will allow you to choose these columns.

d. Enter the Filter if you would like. Here are some examples,

Name LIKE 'A%'
CALENDAR_YEAR(CreatedDate) = 2011

e. Select the External ID. Must be the same as what is configured in the project properties and should be included in the Column Names selection in #c.

f. The Retry Count in Data Deployment automatically retries failed records during data migration, enhancing the success rate by improving the handling of errors.

g. File Path defaults based on the Object selected. You can change it to be more appropriate. The need to be unique in your project.

h. Click Save

That’s it. The package is ready for build and deployment when needed. You can repeat the build to get new data updates for the same set of files and fields.

Retry Count in Data Deployment

The Retry Count feature in Data Deployment provides an enhanced mechanism for handling failed records during the data migration process. This feature enables the system to automatically retry the deployment of failed records, improving the overall success rate of data migrations.

Key Benefits

Enhanced Reliability: The Retry Count feature enhances the reliability of data deployments by automating the retry process for records that initially fail to migrate.
Improved Dependency Resolution: The system recreates a CSV file containing only the failed records and processes it at the end of all completed files in the deployment package. This ensures that any dependencies are addressed, allowing for a smoother resolution of inter-record dependencies.
Iterative Process: The retry mechanism follows an iterative process, attempting to deploy failed records until either success is achieved or a predefined maximum retry count is reached.
Detailed Reporting: In cases where records continue to fail after the specified retry count, the system adds the CSV file of failed data records to the reports folder. This detailed reporting functionality aids in comprehensive post-migration analysis and issue resolution.

Build / Download Data

Build the package to connect to your Salesforce Source org and download data as per your package files setup. Click on the Execution tab to submit the build request for that package. Click on Build.

The Build Request Form is displayed. Select the Environment that you need to download the data from. Click on the “Submit” button for the build to execute. This process will download the CSV files and add them to the FlexDeploy artifact repository. This artifact (set of CSV files) is ready to deploy.

Deploy / Upload Data

Click on the Deploy icon to submit a request to upload to the target Org. You can also use the Release Management pipeline to migrate data.

FlexDeploy enables you to package your code, low-code solutions, configurations, and data into a single bundle, facilitating seamless migration across various environments.