These days we all are told to build stateless applications. Stateless apps are those that don’t store any state in the application process and fetch any state from a centralised datastore (it could be a global cache or a database). The sixth factor in 12 factor app also talk about the same principle.
Execute the app as one or more stateless processes
Twelve-factor processes are stateless and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.
There are advantages in building stateless applications primary being ability to scale horizontally with ease. When we build stateless applications we push the scalability problem to the database. We expect our database to scale horizontally. This usually is solved by sticking a global cache (Redis or Memcached) in between. Scaling cache is relatively easy and solved problem. Keeping cache updated with updates is a hard problem. We will discuss it some other time.
There are only two hard things in Computer Science: cache invalidation and naming things.
— Phil Karlton
In my 15 years of software development experience there have been situations where stateless apps do not cut. You have to choose stateful architecture.
Stateful services are those that store the state in the server memory or local disk so that they can serve request from the local storage instead of making expensive network calls.
In this blog I will share 4 reasons why you might need to build stateful applications.
1. Very low latency application (< 5 ms)
There are times you can’t pay the network cost. Both global cache and shared database involve network latency. This usually happens in highly interactive applications like games where you can’t hit the network service on each user action. The best way to achieve low latency in such situations is by batching user actions in memory and making one request network request per batch. There is a durability issue. You might loose all the state if our stateful app crashes. The durability problem can be managed by using some sort of local persistence.
2. Compute near to storage
Most commonly when we build applications we send the compute query to the database. Database executes those queries and return the result back. This works fine when you have relatively simple queries and data that each query touches is less. In one of the applications that I built we had to perform pricing computations and the worst query had to work with most of the data in the database. We are talking about database size close to 40GB. We didn’t want to write the complex pricing logic as stored procedures. Also, we didn’t want to build a batch application. So, we made use of data grid architecture to keep data and compute together. We were able to run Java jobs in our in-memory data grid cluster. The data grid was patched in real time so we were give real time pricing.
Also each request was routed to the node that had the data to process it. This gives better data locality.
3. Stronger consistency
By using session stickiness along with stateful services we can improve the consistency of the distributed system. By stickiness, we mean that the client request is routed to the same server which served the previous request. Many applications these days uses embedded databases like RocksDB to achieve stronger consistency.
There are obvious problems with sticky connections like unequal distribution of load in the backend servers. But it makes architecture simple and depending on the characteristics of your application it can be a good fit.
4. Simple architecture
Stateful services limit the need of external persistence store. Hence, making them simple and more available.
Conclusion
There is not much written about stateful services. So, I recommend you watch this talk by Caitie McCaffrey where she discussed her experience building scalable stateful services
Shekhar Gulati 