Extract, Transform and Load using Batch Processing

This example polls for order records from a MySQL DB table, and transforms the data by merging the order record with more details from two other tables (SKU_DETAILS and ORDER_STATUS). The resultant record is then inserted into a new table called STATUS_REPORT.
This example is complementary to a developer blog series that gives step by step instructions to build out this example.

Example

In this example, we poll a database table for new orders every 10 seconds using the watermarking and polling capabilities of the Anypoint Platform.

We use the batch processing module to process each record by matching the order entry with records from the two other tables SKU and Order_Status. Once the record is transformed by joining information from the two tables, the record is loaded into the "Status_Report" table.

Set Up and Run the Example

1. Download and setup the MySQL driver in its default location.

2. Start the MySQL server from System Preferences.

3. Open the Example project in AnypointExchange and click Open.

4. Open src/main/resources/order.sql and run it in your mysql instance. This script creates a database called "ETL" and a number of tables load them with mock data.

5. Edit the src/main/app/mule-app.properties file to put the required MySQL configuration details.

6. Run the example application in Anypoint Studio or Standalone

7. Check whether the table "STATUS_REPORT" got populated with data.

Documentation

Read full documentation in GitHub