can be also accomplished as two, 1:1 broadcast applications combined
and pointed in opposite directions, I would recommend reading my last
post on the broadcast pattern before digging into this one since I will omit a lot of the same content.
Pattern 3: Bi-Directional Sync
What is it?
Bi-directional sync is the act of unioning two datasets in twodifferent systems to behave as one while respecting their need to exist
as different datasets. The main driver for this type of integration need
comes from having different tools or different systems for
accomplishing different functions on the same data set. For example, you
may have a system for taking and managing orders and a different system
for customer support. For one reason or another, you find that these
two systems are best of breed and it is important to use them rather
than a suite which supports both functions and has a shared database.
Using bi-directional sync to share the dataset will enable you to use
both systems while maintaining a consistent real time view of the data
in both systems.
Why is it valuable?
Bi-directional sync can be both an enabler and a savior depending onthe circumstances that justify its need. If your situation has led you
to a situation where you have two or more independent, isolated,
representations of the same reality, you can use bi-directional sync to
optimize your processes, have the data representations be much closer to
reality in both systems, and reduce the compound cost of having to
manually address the inconsistencies or lack of data or the impact to
your business from letting the inconsistencies exist. On the other hand
you can use bi-directional sync to take you from a suite of products
that work well together but may not be the best at their own individual
function, to a suite that you hand pick and integration together using
an integration platform like our Anypoint Platform.
The reality of the situation is that no single suite, no matter how
large or perfect will enable you to do everything you want, but
unfortunately without the use of a proper integration platform, with
best practices, a suite may get you further than a compendium of best of
breed applications. With bi-directional sync you are able to let each
system do what its best at while still maintaining data integrity across
both of them, and if done right can buy you the flexibility to
modularly add and remove systems, sometimes even multiple systems that
subspecialize inside a domain like storage, CRM, or ERP. Our initial templates are for Salesforce to Salesforce integration but can be modified to be Salesforce to X or X to Salesforce and still save you considerable time.
When is it useful?
The need, or demand, for a bi-directional sync integrationapplication is synonymous with wanting object representations of reality
to be comprehensive and consistent. For example, if you want one view
of your customer, you can solve that manually by giving everyone access
to all the systems that have a representation of the notion of a
customer. With that solution you can achieve one view of the customer at
the expense of productivity, training, security, and cost. A more
elegant solution to the same problem is to list out which fields need to
be visible for that customer object in which systems, and which systems
are the owners of those. Most enterprise systems have a way to extend
objects such that you can modify the customer object data structure to
include those fields. Then, you can create integration applications
either as point to point applications (using a common integration
platform) if its a simple solution, or a more advanced routing system
like a pub/sub or queue routing model if there are multiple systems at
play. With this approach you can achieve a state where each person who
deals with customers can have their own view of the customer with only
the information that is relevant to them. For example a salesperson
should know the status of a delivery, but they don’t need to know which
warehouse the delivery is at. Similarly the delivery person needs to
know the name of the customer that the delivery is for without needing
to know how much the customer paid for it. Bi-directional
synchronization allows both of those people to have a realtime view of
the same customer within the perspective that they care about.
What are the key things to keep in mind when building applications using the bi-directional sync pattern?
Filtering and Object Mapping
Bi-directional sync is very similar to a pair of broadcastapplications meaning that you have the ability to scope and map the data
in both directions. In our Anypoint Templates that are built on the
broadcast pattern, we give you the flexibility to set the filters based
on the query (since we use a pull method) and each direction had its own data mapping
expression (or data mapper if we are mapping different systems). This
gives you flexibility to create applications that are contextual. For
illustration purposes here is an example: if you have two Salesforce
instances, one which is for the California Sales team, and one for the
rest of the US. If you implemented a bidirectional sync, instead of
always moving all contacts from one to the other, you could set a filter
like “From CA->US move all contacts that are in US but not CA” and
“From US->CA move all contacts that are CA”. With these kinds of
filters you will make it so that you don’t synchronize contacts which
make sense in your instance, but have no value in the other instance.
Similarly, if you wanted only specific fields to flow one way or
another, you can accomplish that by only mapping the fields that you
want to flow over. This again gives you more flexibility in managing the
information that you want to synchronize.
3+ Systems
Initially with our templates we tackled a point to point, two systemssolution which is of value if there are two systems that need to union
their dataset. If you have multiple systems, you will need to change to a
different architecture and start introducing the notion of master.
Typical arrangements include something like a hub and spoke architecture
where one system will be the system of record for all of the
information that needs to be synchronized. For example Workday can play
that role for the employee objects, or Salesforce in a B2B sales model
for CRM data. Another way to do it is to decouple the systems using
queues and a pub-sub model where every system will broadcast its change
to a queue where every other system will look for changes that they care
about. In the case of the pub-sub model you would probably want to
separate your integration applications too so that the creators of those
integration applications can modify and add additional applications to
the pub-sub system without having to touch the ones that are already
running. Hence, based on the way we came up naming convention, those
would really be independent broadcast applications that amalgamate to a
multi system bi-directional sync equivalent. This is one of those topics
that I cannot wait to produce templates for given the need for such
integration amongst our customer base looking to do integrations the
right way. As soon as we have a portfolio of simpler type interaction
templates we are going to start tackling some of these harder cases.
Other Considerations
There are many other considerations with bi-directional sync whichapply to the migration pattern as well. I would highly encourage you to
read the previous post if you have not already here.
In summary, the bi-directional sync pattern is useful when you are
building integration applications that union two relevant datasets for
two applications. It can enable you to use best of breed applications or
fill in the gap if you have multiple systems that serve the same
function but for two different sets of users. Check out our current bi-directional sync templates and feel free to leave feedback below!
