Salesforce to Salesforce and Database Account Broadcast

(0 reviews)

home

Synchronizes online accounts from a Salesforce instance to many destination systems using the publish-subscribe pattern. Each time a new account or a change occurs, the integration polls for changes in the Salesforce source organization, publishes the changes to a JMS topic, and each subscriber is then responsible for updating the accounts in the target systems.

The application has two batch jobs consuming the JMS topic, one for migrating the changes to the second Salesforce org and the other for migrating changes to the database. During the Process stage, each Salesforce account is matched with an existing account in the Salesforce org B or the database by name. The last step of the Process stage groups the accounts and creates updates in Salesforce org B.

During the On Complete stage, the template logs output statistics data to the console.

Read more about the publish-subscribe pattern in this blog post.

d0feeebc-4129-4e1a-9e25-5884cec38bd0-image.png

License Agreement

This template is subject to the conditions of the MuleSoft License Agreement. Review the terms of the license before downloading and using this template. You can use this template for free with the Mule Enterprise Edition, CloudHub, or as a trial in Anypoint Studio.

Use Case

As a Salesforce admin I want to synchronize accounts from a Salesforce Org to another Salesforce Org and a Database.

This template serves as a foundation for setting an online sync of accounts from a Salesforce instance to many destination systems, using the publish-subscribe pattern. Every time there is new account or a change in an already existing account, the integration polls for changes in the Salesforce source organization, publishes the changes to a topic and each subscriber is responsible for updating the accounts in the target systems.

Requirements have been set not only to be used as example, but also to establish a starting point to adapt the integration to your requirements.

As implemented, this template leverages the Mule batch module. The batch job is divided in Process and On Complete stages. The integration can be triggered in two different ways: by a scheduler defined in the endpoints file or by an outbound notification generated by Salesforce. Either way, the application queries or receives newest Salesforce updates or creates matching a filter criteria and adds them to a JMS topic.

The application has two different batch jobs consuming this JMS topic, one for migrating the changes to the second Salesforce organization and the other one for migrating the changes to the Database. During the Process stage, each Salesforce account is matched with an existing account in the Salesforce organization B or the database by name. The last step of the Process stage groups the accounts and creates or updates them in Salesforce organization B.

Finally during the On Complete stage the template logs output statistics data in the console.

Considerations

To make this template run, there are certain preconditions that must be considered. All of them deal with the preparations in both source and destination systems, that must be made for the template to run smoothly. Failing to do so can lead to unexpected behavior of the template.

Database Considerations

This template uses date time or timestamp fields from the database to do comparisons and take further actions. While the template handles the time zone by sending all such fields in a neutral time zone, it cannot handle time offsets. (Time offsets are time differences that may surface between date time and timestamp fields from different systems due to a differences in each system's internal clock.) Take this in consideration and take the actions needed to avoid the time offset.

As a Data Destination

There are no considerations with using a database as a data destination.

Salesforce Considerations

As a Data Source

If a user who configures the template for the source system does not have at least read only permissions for the fields that are fetched, then an InvalidFieldFault API fault displays.

java.lang.RuntimeException: [InvalidFieldFault [ApiQueryFault 
[ApiFault  exceptionCode='INVALID_FIELD'
exceptionMessage='Account.Phone, Account.Rating, Account.RecordTypeId, 
Account.ShippingCity
^
ERROR at Row:1:Column:486
No such column 'RecordTypeId' on entity 'Account'. If you are attempting to 
use a custom field, be sure to append the '__c' after the custom field name. 
Reference your WSDL or the describe call for the appropriate names.'
]
row='1'
column='486'
]
]

As a Data Destination

There are no considerations with using Salesforce as a data destination.

Run it!

Simple steps to get this template running.

See below.

Run On Premises

In this section we detail the way you have to run your template on your computer.

After your app is all set and started, there is no need to do anything else. The application polls accounts to see if there are any newly created or updated objects and synchronizes them.

Download Anypoint Studio and the Mule Runtime

If you are new to Mule, download this software:

Note: Anypoint Studio requires JDK 8.

Import a Template into Studio

In Studio, click the Exchange X icon in the upper left of the taskbar, log in with your Anypoint Platform credentials, search for the template, and click Open.

Run on Studio

After you import your template into Anypoint Studio, follow these steps to run it:

  1. Locate the properties file mule.dev.properties, in src/main/resources.
  2. Complete all the properties required per the examples in the "Properties to Configure" section.
  3. Right click the template project folder.
  4. Hover your mouse over Run as.
  5. Click Mule Application (configure).
  6. Inside the dialog, select Environment and set the variable mule.env to the value dev.
  7. Click Run.

Run on Mule Standalone

Update the properties in one of the property files, for example in mule.prod.properties, and run your app with a corresponding environment variable. In this example, use mule.env=prod.

Run on CloudHub

While creating your application on CloudHub (or you can do it later as a next step), you need to go to Deployment > Advanced to set all environment variables detailed in "Properties to Configure" as well as the mule.env.

Once your app is all set and started, there is no need to do anything else. Every time an account is created or modified, it will be automatically synchronized to Salesforce Org B and the Database as long as it matches the filtering criteria.

Deploy a Template in CloudHub

In Studio, right click your project name in Package Explorer and select Anypoint Platform > Deploy on CloudHub.

Properties to Configure

To use this template, configure properties such as credentials, configurations, etc.) in the properties file or in CloudHub from Runtime Manager > Manage Application > Properties. The sections that follow list example values.

Application Configuration

HTTP Connector Configuration
  • http.port 9090
Batch Aggregator Configuration
  • page.size 200
Scheduler Configuration
  • scheduler.frequency 10000
  • scheduler.startDelay 100
Watermarking Default Last Query Timestamp Such As 2019-12-13T03:00:59Z
  • watermark.default.expression YESTERDAY
Trigger policy(push, poll)
  • trigger.policy push | poll

Note: The property trigger.policy can assume only one of the values above as either push or poll.

JMS Connector Configuration
  • jms.username admin
  • jms.password admin
  • jms.url tcp://localhost:61616
Database Connector Configuration
  • db.host localhost
  • db.port 3306
  • db.user user-name1
  • db.password user-password1
  • db.databasename dbname1

If it is required to connect to a different database, provide the JAR for the library and change the value of that field in the connector.

Salesforce Connector Configuration for Company A
  • sfdc.a.username bob.dylan@orga
  • sfdc.a.password DylanPassword123
  • sfdc.a.securityToken avsfwCUl7apQs56Xq2AKi3X
Salesforce Connector Configuration for Company B
  • sfdc.b.username joan.baez@orgb
  • sfdc.b.password JoanBaez456
  • sfdc.b.securityToken ces56arl7apQs56XTddf34X

API Calls

Salesforce imposes limits on the number of API calls that can be made. Therefore calculating this amount may be an important factor to consider. The template calls to the API can be calculated using the formula:

  • 1 + X + X / ${page.size} -- Where X is the number of accounts to synchronize on each run.
  • Divide by ${page.size} because by default, accounts are gathered in groups of ${page.size} for each upsert API call in the commit step. Also consider that this calls are executed repeatedly every polling cycle.

For instance if 10 records are fetched from origin instance, then 12 API calls are made (1 + 10 + 1).

Customize It!

This brief guide provides a high level understanding of how this template is built and how you can change it according to your needs. As Mule applications are based on XML files, this page describes the XML files used with this template. More files are available such as test classes and Mule application files, but to keep it simple, we focus on these XML files:

  • config.xml
  • businessLogic.xml
  • endpoints.xml
  • errorHandling.xml

config.xml

This file provides the configuration for connectors and configuration properties. Only change this file to make core changes to the connector processing logic. Otherwise, all parameters that can be modified should instead be in a properties file, which is the recommended place to make changes.

businessLogic.xml

The functional aspect of this template is implemented in this XML file, directed by two different batch jobs consuming this JMS topic, one for migrating the changes to the second Salesforce organization and the other one for migrating the changes to the database. The several message processors constitute four high level actions that fully implement the logic of this template:

  1. The template queries Salesforce for all the existing accounts that match the filtering criteria.
  2. During the Process stage, each Salesforce account is matched with an existing account in the Salesforce organization B or the database by name.
  3. The last step of the Process stage groups the accounts and creates or updates them in Salesforce organization B.
  4. Finally during the On Complete stage the template logs output statistics data to the console.

endpoints.xml

This file contains three Flows:

  1. push flow - Contains an HTTP endpoint that listens for Salesforce outbound messages. Each is processed and thus updates or create accounts, and then executing the batch job process.
  2. scheduler flow - Contains the Scheduler endpoint that periodically triggers the sfdcQuery flow and then executes the batch job process.
  3. sfdcQuery flow - Contains watermarking logic for querying Salesforce for updated or created accounts that meet the defined criteria in the query since the last polling. The last invocation timestamp is stored by using Object Store component and updated after each Salesforce query.

The property trigger.policy defines from which endpoint the template receives its data. When the push policy is selected, the HTTP Listener is used for the Salesforce outbound messaging and the polling mechanism is ignored. The property can only assume one of two values push or poll any other value results in the template ignoring all messages.

errorHandling.xml

This file handles how your integration reacts depending on the different exceptions. This file provides error handling that is referenced by the main flow in the business logic.


Reviews

TypeTemplate
OrganizationMuleSoft
Published by
MuleSoft Organization
Published onJun 25, 2019
Asset overview

Asset versions for 2.1.x

Asset versions
VersionActions
2.1.3
2.1.2

Categories

Functions
Enterprise ITNo values left to add
Pattern
BroadcastPublish-SubscribeNo values left to add
Products
SalesforceNo values left to add
Object
AccountOrganizationNo values left to add