Salesforce Bulk API Integration using Mule ESB
Salesforce CRM has been widely used in
organizations as part of managing their customer interactions. However,
Salesforce with the Cloud Delivery model, has become difficult and
expensive for the organizations to custom code their integration of
Salesforce with their existing on-premise systems.
Many organizations have a need for this
integration in a cost and time effective way to automate their business
processes. As a solution to this problem, WHISHWORKS has a way to
integrate the company’s existing systems with Salesforce using a modern,
lightweight and low cost Mule Enterprise Service Bus.
WHISHWORKS and Salesforce Bulk API
WHISHWORKS has extensive experience
utilizing the Mule ESB Anypoint Salesforce Connector to connect directly
with Salesforce APIs. This connector enables users, access to full
Salesforce functionality with seamless Salesforce integration.
In a business scenario, where there were
huge volumes of data to be migrated to Salesforce from the company’s
multiple existing systems, WHISHWORKS has implemented an effective way
of integrating with the Salesforce Bulk API.
Architecture Diagram
Mule ESB flows have been designed in a
way that can be reused for both initial and operational loads.
Transformation of data has also been provided in the process of data
import to give a standardized and consolidated form of data in
Salesforce.
How did we tune Salesforce Bulk API
Bearing in mind the different
constraints Salesforce Bulk API has, WHISHWORKS has tuned the batches
uploading to Salesforce in an effective manner enabling seamless
business automation between Salesforce and the existing Database
systems. Here is how:
- Threading Profile Settings: Salesforce allows a maximum of 25 concurrent threads at a time. To restrict the concurrent calls to not more than 25, threading profiles have been created at the flow and the VM endpoint level in which the Salesforce calls reside.
- Salesforce Batch Tuning: Each Salesforce batch being created is tuned such that the data size of the batch does not exceed 10MB. Tuning parameters have been configured to change the number of records each batch holds depending on the size of the entity.
- Time Delay between each Salesforce Call: Loading huge volumes of data to Salesforce in batches running concurrently can cause Salesforce to take longer time for processing the batches. To avoid this, a time delay has been provided between each concurrent call to avoid overloading Salesforce.
- Parallel Garbage Collection: To utilise the JVM memory efficiently while importing the data, Parallel Garbage Collection has been used to clean the Java objects that are not anymore strongly referenced.
All this was done on Mule ESB!
Benefits to the Customer
The Salesforce Integration with the organization’s multiple systems has provided the following benefits to the customer:
- This has enabled the customer to be able to streamline and fully automate their business processes.
- Scalability: Integration through Mule ESB has enabled adaptation to any new SOA infrastructure that needs to be defined as part of the company’s changing infrastructure.
- Speed of Integration: With an underlying platform that contains a single development environment and a reliable multi-tenant architecture, integration to Salesforce has been quickly and efficiently built.
- This has provided them with an ability to integrate more systems and to aggregate data for a consistent and accurate overview of business.
- Significant cost savings by using low cost Mule ESB Enterprise.
Author: Harika Guniganti is a Masters of Computer Science graduate having 4 years of experience as a Technical Specialist with WHISHWORKS. An experienced hand at integration and Mule ESB, Harika loves her cooking, crafts and travelling.
