Salesforce Data Cloud Ingestion from SharePoint

The comprehensive instructions on this page show how to configure data sources and deploy applications to MuleSoft Anypoint Platform. Users are guided through the essential steps needed for successful deployments in two different scenarios:

• Deploying via MuleSoft Direct
• Deploying via Anypoint Platform

Data source configurations

The Microsoft Graph API is used for interacting with SharePoint. Some configuration is required on the Microsoft side in order for the ingestion application to function correctly. This includes creating an application in Microsoft Entra ID, creating indexes for SharePoint libraries, and obtaining the SharePoint Site ID required during application deployment.

The following sections provide some brief instructions for these. For more detailed information, refer to the following:

• Microsoft Entra ID Client Application
• Available permissions reference
• Create an Index
• Postman collection for testing Graph

Microsoft Entra ID

Microsoft Entra ID is the new name for Azure Active Directory and is Microsoft’s identity management platform. In Entra you need to create an application registration (connected app) and apply the appropriate API permissions (scopes) to access Microsoft Graph services.

1. Go to https://portal.azure.com
2. Either search for or click the icon for Entra ID
3. Click App registrations (not Enterprise Applications)
4. Click New Registration and provide a useful name for the app, such as MuleSoft Integration Client
5. Leave the selection on Accounts in this organizational directory only (first radio button)
6. Do not supply a redirect URI, since it is not necessary for this application
7. Click Register

The wizard should take you to the newly created application registration. On this screen, copy the Tenant ID (called Directory ID) and the Client ID (called Application ID), which you'll use to establish a connection to Microsoft Graph

1. On the sidebar menu, click Certificates & secrets
2. Click New Client Secret and define a name and expiration period
3. Click the copy button next to Value (NOT Secret ID) - this is your client secret

Finally, you'll add the appropriate API Permissions. At a minimum, the following permissions should be added; more may be needed depending on your requirements.

1. Click Add a permission -> Microsoft Graph-> Application Permissions
2. Add the following permissions (at a minimum):
   • Files.Read.All
   • Sites.Read.All
   • User.Read
   • User.Read.All
3. After applying the permissions, be sure to click the Grant Admin consent button

Your Entra App Registration should now be ready to connect with MS Graph. Use the Microsoft-supplied Postman collection to verify that your app has the appropriate permissions to access SharePoint data.

SharePoint Index

In order to query SharePoint using filters on select fields (mostly Created date and Modified date), you'll need to add a column index. Column indexes are applied to each and every document library (or list generically). The following steps must be completed for each library in the Site Collection, including the out-of-the-box libraries Documents and Shared Documents. Failure to add these indexes may cause the Graph API to return a warning or failure regarding non-deterministic results.

1. From the library in question, click the gear icon
2. Click Library Settings -> More Library Settings
3. Click Indexed Columns in the Columns group
4. Create a new index (do this step twice)
   • Primary column = Created
   • Click Create
   • Primary column = Modified
   • Click Create
5. Do not configure a Secondary column - simply create two indexes, one for each column

Once complete, you should see a confirmation message that indicates the number of indexes you have created on the list, with the Created and Modified column indexes listed.

SharePoint Site ID

In order to use this integration application, you'll need to supply the SharePoint Site ID as the source container identifier. The Site ID can be found by visiting the legacy SharePoint REST API in a browser window.

Go to the desired Site. The URL will be something like https://<subdomain>.sharepoint.com/sites/<some-site>. Append the path /_api/site/id to the URL, resulting in a URL like https://muleconnector.sharepoint.com/sites/datacloud-integration-testing/_api/site/id. This will return the shorthand version of the Site ID (there is a longer version available as well, but Graph will accept the short version).

Deploying via MuleSoft Direct

To enable the ingestion of unstructured data into Data Cloud this application must first be deployed using MuleSoft Direct. When enabling the integration, the following properties must be configured to connect with the data source and target system.

Additional configuration steps

After the application has been deployed there are some additional configuration steps that should be done within Anypoint Runtime Manager. The following instructions are for CloudHub 2.0 deployments but the steps are similar for CloudHub deployments.

• Find the deployed application by entering the name in the Search Applications field.
• Click on the application name or select the entry and click the Manage Application button on the right. For CloudHub 2.0 deployments, a screen appears with multiple tabs to configure the application (for CloudHub deployments you will need to click the Settings item on the left navigation bar).
• Update the Runtime Version to the latest patch release and select the Use Object Store V2 checkbox under the Runtime Options section.
• Consider increasing the number and size of the replicas (workers) if the source container being monitored has a large number of resources and/or a high volume of changes.
• Click the Apply Changes to deploy the new configuration

Deploying via Anypoint Platform

To support the ingestion of unstructured data into Data Cloud the application must first be deployed from MuleSoft Direct in order to create the required connector (in Data Cloud); once deployed the application can be updated in Anypoint Runtime Manager as needed.

Getting started

The Getting Started with MuleSoft Accelerators guide provides general information on getting started with the accelerator components. This includes instructions on setting up your local workstation for configuring and deploying the applications.

Deployment

Each Accelerator implementation template in Exchange includes Bash and Windows scripts for building and deploying the APIs to CloudHub. These scripts depend on repositories, global settings, deployment profiles and associated properties configured in the Maven settings.xml file. In particular, make sure the common properties for your environment have been provided in the CloudHub-DEV profile, like Anypoint Platform client ID and secret).

For additional details, please refer to the Application Deployment section of the Getting Started guide.

Required property overrides

Many templates can also be run from Anypoint Studio without having to customize the Run/Debug profiles. However, some templates make use of hidden deployment properties to protect sensitive information, like passwords and secret keys. These properties must be supplied to the runtime by updating the configuration profile and adding them as VM arguments. At a minimum, the following properties must be customized to reflect the target deployment environment.