When to use shared libraries in Microservices architecture


One of the advantages of Microservices architecture is that it enables components to have deployment independence. Based on my consulting and software development experience deployment independence is often overlooked and very few teams achieve it. Deployment independence is important since it brings true agility and reduces communication overhead between different teams and services. 

Shared libraries make Microservices tightly coupled and introduce hard dependencies. Since, now a team making a change has to ensure that it does not break another service that depends on the shared library. This requires communication between multiple teams. Also, change in a shared library leads to all the services that depend on it to be redeployed.  This leads to long build, release, and deployment times. We might have to consider the deployment order of services as well. All this leads to more synchronization and communication between teams. So, it is recommended that in Microservices architecture teams should avoid using shared libraries. 

I also principally agree with the above mentioned thought process. There are a few exceptions where I still prefer to use shared libraries. These are:

  • You want to reuse a critical technical logic/concern that can’t have its own independent life cycle. They typically don’t change depending on the business need. Let’s take a couple of examples to understand this point.
    • an HTTP client that you use to talk to core services. Http library knows which operations are idempotent, adds idempotency key when required, adds retry logic with exponential backoff and jitter for idempotent operations, adds sensible timeouts, does exception translation, etc. I don’t want all my services to duplicate this logic. It is better to abstract this logic in a shared library which is maintained by the platform or shared libraries team. 
    • I prefer to use a library that all services use to return success and error response in a consistent format and structure. During the initial phase of the software development you might see changes happening to such libraries often but after the initial phase they stabilize and changes do not happen that often.
    • An extensible validation library. 
  • You want to write an eloquent API over a complex/confusing/poorly written/boilerplate friendly/low level library.  In the last assignment I wrote an abstraction over Apache POI to make it easy for my team to use it for our use cases. 
  • You want the freedom to change third party API in future but don’t want to pay the cost of pass-through service. These days the systems we built use many third party APIs. One of the neobank I worked at used a third party accounts and payments infrastructure API. One way to abstract your system from the third party API is to use service wrapper or pass through service. There is a network and performance cost to adding a pass-through API as you are adding a network hop. Another way is to wrap the third party API client in your custom library. This way you still have the freedom to tomorrow change the underlying third party API. It is like applying a strangler pattern. 

Conclusion

Software design is all about giving you an option to change and evolve your system in future. Shared libraries if done well and for the right reasons do not introduce hard dependencies that we want to avoid in Microservices architecture. Instead, extracting out such common code into shared libraries can speed up development of new services. 

One thought on “When to use shared libraries in Microservices architecture”

  1. “Shared libraries make Microservices tightly coupled and introduce hard dependencies. Since, now a team making a change has to ensure that it does not break another service that depends on the shared library. This requires communication between multiple teams. Also, change in a shared library leads to all the services that depend on it to be redeployed. This leads to long build, release, and deployment times. We might have to consider the deployment order of services as well. All this leads to more synchronization and communication between teams. So, it is recommended that in Microservices architecture teams should avoid using shared libraries. ”

    I started researching this issue over a year ago and it lead the following innovation, which i call federated microservices or federated applications.

    What I discoved was that DRY vs decoupled is a false dilemma. Solve using federation and a pattern close to the one you mention above, which i call the “Federated Client” pattern. You can read about this pattern and federartion in general here: trmidboe.medium.com, blog.federated-microservices.com, github.com/module-federation/aegis-host

    In a federated application, software is not installed on the server but streamed over the network at runtime as needed, typically from the source repo. Federated libraries are not a compile-time dependency. They can be managed by separate teams in separate repos from the applications that consume them. Moreover, federated libraries can be deployed independently into the application instance without restarting the process. Furthermore, federated applications can control the version of the library that they consume. They can even use multiple versions at once and decide programmatically at runtime whether or not to upgrade after running an A/B test.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: