Salesforce to Database Account Migration

Moves a large set of accounts from Salesforce to a database. You can trigger this manually or programmatically with an HTTP call. Accounts are upserted so that the migration can be run multiple times without worrying about creating duplicates.

This template uses batch to efficiently process many records at a time. Parent accounts of the contacts are created if they do not exist in the destination system, or can be set to be a specific account for all contacts that are migrated. A database table schema is included to make testing this template easier.

License Agreement

This template is subject to the conditions of the MuleSoft License Agreement. Review the terms of the license before downloading and using this template. You can use this template for free with the Mule Enterprise Edition, CloudHub, or as a trial in Anypoint Studio.

Use Case

I want to migrate accounts from Salesforce to Database.

This template serves as a foundation for the process of migrating accounts from Salesforce to Database, being able to specify filtering criteria and desired behavior when an account already exists in the Database.

As implemented, this template leverages the Mule batch module.

The batch job is divided in Process and On Complete stages.

Migration process starts from fetching all the existing Accounts that match the filter criteria from Salesforce.

On Process stage matched accounts are grouped and inserted or updated in Database.

Finally during the On Complete stage, the template outputs statistics data to the console and sends a notification email with the results of the batch execution.

The template is covered by the integration tests using the MUnit. To be able to run the tests, see the example configuration of the test property file.

Considerations

Note: This template illustrates the migration use case between Salesforce and a Database, thus it requires a Database instance to work.

The template comes packaged with a SQL script to create the database table that it uses.

Use that script to create the table in an available schema and change the configuration accordingly.

The SQL script file can be found in src/main/resources/account.sql.

This template is customized for MySQL. To use it with different SQL implementation, some changes are necessary:

• Update SQL script dialect to desired one
• Replace MySQL driver library dependency to desired one in pom.xml file
• Update Database Config to suitable connection instead of db:my-sql-connection in global elements in config.xml in /src/main/mule/
• Update database properties in mule.*.properties file

Database Considerations

This template uses date time or timestamp fields from the database to do comparisons and take further actions. While the template handles the time zone by sending all such fields in a neutral time zone, it cannot handle time offsets. (Time offsets are time differences that may surface between date time and timestamp fields from different systems due to a differences in each system's internal clock.)

Take this in consideration and take the actions needed to avoid the time offset.

As a Data Destination

There are no considerations with using a database as a data destination.

Salesforce Considerations

Here's what you need to know about Salesforce to get this template to work:

• Where can I check that the field configuration for my Salesforce instance is the right one? See: Salesforce: Checking Field Accessibility for a Particular Field.
• How can I modify the Field Access Settings? See: Salesforce: Modifying Field Access Settings.

As a Data Source

If the user who configured the template for the source system does not have at least read only permissions for the fields that are fetched, then an InvalidFieldFault API fault displays.

java.lang.RuntimeException: [InvalidFieldFault [ApiQueryFault
[ApiFault  exceptionCode='INVALID_FIELD'
exceptionMessage='Account.Phone, Account.Rating, Account.RecordTypeId,
Account.ShippingCity
^
ERROR at Row:1:Column:486
No such column 'RecordTypeId' on entity 'Account'. If you are
attempting to use a custom field, be sure to append the '__c' after the
custom field name. Reference your WSDL or the describe call for the
appropriate names.'
]
row='1'
column='486'
]
]

Run it!

Simple steps to get this template running.

Running On Premises

In this section we help you run this template on your computer.

Where to Download Anypoint Studio and the Mule Runtime

If you are new to Mule, download this software:

• Download Anypoint Studio
• Download Mule runtime

Note: Anypoint Studio requires JDK 8.

Importing a Template into Studio

In Studio, click the Exchange X icon in the upper left of the taskbar, log in with your Anypoint Platform credentials, search for the template, and click Open.

Running on Studio

After you import your template into Anypoint Studio, follow these steps to run it:

• Locate the properties file mule.dev.properties, in src/main/resources.
• Complete all the properties required as per the examples in the "Properties to Configure" section.
• Right click the template project folder.
• Hover your mouse over Run as.
• Click Mule Application (configure).
• Inside the dialog, select Environment and set the variable mule.env to the value dev.
• Click Run.

Running on Mule Standalone

Update the properties in one of the property files, for example in mule.prod.properties, and run your app with a corresponding environment variable. In this example, use mule.env=prod.

After this, to trigger the use case you just need to browse to the local http endpoint with the port you configured in your file. If this is, for instance, 9090 then you should browse to: http://localhost:9090/migrateaccounts and this creates a CSV report and sends it to the mails set.

Running on CloudHub

When creating your application in CloudHub, go to Runtime Manager > Manage Application > Properties to set the environment variables listed in "Properties to Configure" as well as the mule.env value.

Once your app is all set and started, if you choose as domain name sfdc2dbaccountmigration to trigger the use case, you just need to browse to http://sfdc2dbaccountmigration.cloudhub.io/migrateaccounts and report is sent to the emails configured.

Deploying a Template in CloudHub

In Studio, right click your project name in Package Explorer and select Anypoint Platform > Deploy on CloudHub.

Properties to Configure

To use this template, configure properties such as credentials, configurations, etc.) in the properties file or in CloudHub from Runtime Manager > Manage Application > Properties. The sections that follow list example values.

Application Configuration

HTTP Connector Configuration

• http.port 9090

Batch Aggregator Configuration

• page.size 1000

Salesforce Connector Configuration

• sfdc.username bob.dylan@org
• sfdc.password DylanPassword123
• sfdc.securityToken avsfwCUl7apQs56Xq2AKi3X

Database Connector Configuration

• db.host localhost
• db.port 3306
• db.user joan.baez
• db.password JoanBaez456
• db.databasename template-sfdc2db-account-migration

SMTP Services Configuration

• smtp.host smtp.gmail.com
• smtp.port 587
• smtp.user email%40example.com
• smtp.password password

Email Details

• mail.from batch.migrateaccounts.migration%40mulesoft.com
• mail.to your.email@gmail.com
• mail.subject Batch Job Finished Report

API Calls

Salesforce imposes limits on the number of API Calls that can be made. Therefore calculating this amount may be an important factor to consider. The template calls to the API can be calculated using the formula:

X + X / ${page.size}

X is the number of Accounts to be synchronized on each run.

Divide by ${page.size} because, by default, Accounts are gathered in groups of ${page.size} for each Upsert API Call in the commit step.

For instance if 10 records are fetched from origin instance, then 20 API calls will be made (10 + 10).

Customize It!

This brief guide provides a high level understanding of how this template is built and how you can change it according to your needs. As Mule applications are based on XML files, this page describes the XML files used with this template. More files are available such as test classes and Mule application files, but to keep it simple, we focus on these XML files:

• config.xml
• businessLogic.xml
• endpoints.xml
• errorHandling.xml

config.xml

This file provides the configuration for connectors and configuration properties. Only change this file to make core changes to the connector processing logic. Otherwise, all parameters that can be modified should instead be in a properties file, which is the recommended place to make changes.

businessLogic.xml

Functional aspect of the Template is implemented on this XML, directed by one flow responsible of excecuting the logic.

For the purpose of this particular Template the mainFlow uses a batch job, which handles all the logic of it.

endpoints.xml

This file provides the inbound and outbound sides of your integration app.

This Template has only an HTTP Listener as the way to trigger the use case.

Inbound Flow

HTTP Listener Connector - Start Report Generation

• ${http.port} is set as a property to be defined either in a property file or in CloudHub environment variables.
• The path configured by default is migrateaccounts and you are free to change for the one you prefer.
• The host name for all endpoints in your CloudHub configuration should be defined as localhost. CloudHub will then route requests from your application domain URL to the endpoint.
• The endpoint is configured as a request-response since as a result of calling it the response will be the total of Accounts synced and filtered by the criteria specified.

errorHandling.xml

This file handles how your integration reacts depending on the different exceptions. This file provides error handling that is referenced by the main flow in the business logic.