Rate limiting is great withdistributed systems but not without some complications. We already read aboutrate limiting and distributed systems before and now we will discuss the problems with them coming together.

1. Inconsistencies

When you have a distributed system withclusters of multiple nodes, a global rate limit policy is preferred since tracking of rate limit by each node might lead a consumer to exceed the global limit of sending requests to different nodes altogether. This is even riskier when there are even more nodes.

To solve this, we can use sticky sessions in load balancers which will result in each user being sent to exactly one node. However, this means there will be no fault tolerance and more scaling problems. Likewise, we can use a centralized data store likeRedis as well. Then, it has more latency and causes race conditions.

2. Race Conditions

Further, race conditions occur when a naïve “get-then-set” approach is used. Under the operation, the current rate limit counter is retrieved, followed by an increment and push to the data store. The issue here is that the new requests can come through in the time it takes to perform a full cycle of read-increment-store, each trying to store the increment counter with an invalid (lower) counter value. This way, the user can send more requests to bypass the rate-limiting controls.

To solve this problem, implementing distributed locking mechanism around the key might be a good way. It prevents other processes from gaining the access to the operations to write on the counter. However, the lock here transforms into a bottleneck and doesn’t scale well.

In addition, you can also use the alternative “set-then-get” approach that increments and checks the counter values quickly without letting the atomic operations get in the way.

Conclusion

Rate limiting can be a great deal withdistributed systems if the complications can be handled. However, it might be difficult if those complications increase repeatedly.

I hope this article was helpful to you.

Please don’t forget to follow me!!!

Any kind of feedback or comment is welcome!!!

Thank you for your time and support!!!!

Keep Reading!! Keep Learning!!!