Database to Database Account Bi-Directional Sync

Bidirectionally synchronizes account data between two database systems using MySQL or a JDBC protocol supporting a database system.

This template enables quick configuration of the fields to synchronize, how they map, and the criteria for when to trigger the synchronization. This template is triggered by using the Mule polling mechanism.

This template leverages watermarking to select only the most recent items to synchronize and uses batch to process many records at a time. A database table schema is included to make testing this template easier.

License Agreement

This template is subject to the conditions of the MuleSoft License Agreement. Review the terms of the license before downloading and using this template. You can use this template for free with the Mule Enterprise Edition, CloudHub, or as a trial in Anypoint Studio.

Use Case

This template serves as a foundation for setting an online bidirectional sync of Accounts between two database instances with ability to specify filtering criteria.

This template fetches data for changes to new or modified accounts that have occurred in any of the database instances during a period of time. For those accounts that were identified as not present in the target instance, the integration triggers an insert or update operation on the existence of the object in the target instance taking the last modification of the object as the one that should be applied.

Considerations

Note: This template illustrates the migration use case between two databases, thus it requires database instances to work.

The template comes packaged with SQL scripts to create the database tables that it uses.

It is the user's responsibility to use provided script to create the table in an available schema and change the configuration accordingly.

The SQL script files can be found in the src/main/resources/ directory.

This template is customized for MySQL. To use it with different SQL implementation, some changes are necessary:

• Update the SQL script dialect as needed.
• Replace the MySQL driver library (or add another) dependency in your project's pom.xml file.
• Update the database configuration to a suitable connection instead of db:my-sql-connection in the global elements (config.xml)
• Update connection configurations in mule.*.properties file

To make this template run, there are certain preconditions that must be considered. All of them deal with the preparations in both,

that must be made for the template to run smoothly.

Failing to do so could lead to unexpected behavior of the template.

DB Considerations

This template uses date time or timestamp fields from the database to do comparisons and take further actions. While the template handles the time zone by sending all such fields in a neutral time zone, it cannot handle time offsets. (Time offsets are time differences that may surface between date time and timestamp fields from different systems due to a differences in each system's internal clock.)

Take this in consideration and take the actions needed to avoid the time offset.

As a Data Source

There are no considerations with using a database as a data origin.

As a Data Destination

There are no considerations with using a database as a data destination.

Run it!

Simple steps to get this template running.

To have your application up and running, you just need to complete two simple steps:

1. Configure the application properties.

2. Run it! on premise or in Cloudhub as describe in the sections that follow.

Running On Premises

In this section we help you run this template on your computer.

Where to Download Anypoint Studio and the Mule Runtime

If you are new to Mule, download this software:

• Download Anypoint Studio
• Download Mule runtime

Note: Anypoint Studio requires JDK 8.

Importing a Template into Studio

In Studio, click the Exchange X icon in the upper left of the taskbar, log in with your Anypoint Platform credentials, search for the template, and click Open.

Running on Studio

After you import your template into Anypoint Studio, follow these steps to run it:

• Locate the properties file mule.dev.properties, in src/main/resources.
• Complete all the properties required as per the examples in the "Properties to Configure" section.
• Right click the template project folder.
• Hover your mouse over Run as.
• Click Mule Application (configure).
• Inside the dialog, select Environment and set the variable mule.env to the value dev.
• Click Run.

Running on Mule Standalone

Fill in all properties in one of the property files, for example in mule.dev.properties and run your app with the corresponding environment variable to use it. To follow the example, this will be mule.env=dev.

Running on CloudHub

When creating your application in CloudHub, go to Runtime Manager > Manage Application > Properties to set the environment variables listed in "Properties to Configure" as well as the mule.env value.

To create your application on CloudHub go to Deployment > Advanced to set all environment variables detailed in "Properties to Configure" as well as to set the mule.env value.

Deploying a Template in CloudHub

In Studio, right click your project name in Package Explorer and select Anypoint Platform > Deploy on CloudHub.

Properties to Configure

To use this template, configure properties such as credentials, configurations, etc. in the properties file or in CloudHub from Runtime Manager > Manage Application > Properties. The sections that follow list example values.

Application Configuration

Application configuration

• scheduler.frequency 10000 The milliseconds to run between two different checks for updates in either database instance.
• scheduler.startDelay 0
• watermark.default.expression 2018-02-25T11:00:00.000Z This property is an important one, as it configures what should be the start point of the synchronization. If the use case includes synchronization of every account created from the beginning of time, you should use a date previous to any account creation (perhaps 1900-01-01T08:00:00.000Z is a good choice). If you want to synchronize the accounts created from now on, use a default value accordingly. For example, if today is April 21, 2018 at 11 AM in London, use 2018-04-21T11:00:00.000Z.
• page.size 1000

Database Connector Configuration A

• db.a.host localhost
• db.a.port 3306
• db.a.databasename dbnameA
• db.a.user user-nameA
• db.a.password user-passwordA
• db.a.integration.user.id user-idA

Database Connector Configuration B

• db.b.host localhost
• db.b.port 3306
• db.b.databasename dbnameB
• db.b.user user-nameB
• db.b.password user-passwordB
• db.b.integration.user.id user-idB

API Calls

Not relevant for this use case.

Customize It!

This brief guide provides a high level understanding of how this template is built and how you can change it according to your needs. As Mule applications are based on XML files, this page describes the XML files used with this template. More files are available such as test classes and Mule application files, but to keep it simple, we focus on these XML files:

• config.xml
• businessLogic.xml
• endpoints.xml
• errorHandling.xml

config.xml

This file provides the configuration for connectors and configuration properties. Only change this file to make core changes to the connector processing logic. Otherwise, all parameters that can be modified should instead be in a properties file, which is the recommended place to make changes.

businessLogic.xml

This file holds the functional aspect of the template. Its main component is a batch job, and it includes steps for both executing the synchronization from Database A to Database B, and the other way around.

endpoints.xml

This file should contain every inbound and outbound endpoint of your integration app.

In this particular template, this file contains a scheduler endpoint that query Database A and Database B for updates using watermark.

errorHandling.xml

This is the right place to handle how your integration will react depending on the different exceptions.

This file provides error handling referenced by the scheduler flow in the endpoints XML file.