Modularity and Event-Driven Microservice Architecture
Posted on May 16, 2022 — 4 Minutes Read
REST API, while tried and true, does not however scale well in a system with hundreds or even thousands of microservices. For a system of n services, there exists up to n x (n - 1) / 2 of possible communication paths and the number grows quadratically at an order of approximation of O(n^2^ / 2). Based on the HTTP request-response model, API calls are by nature synchronous and will as such block the operation until a respond is received, idling resources in the process when in most cases it could proceed without needing to wait. Recurring API calls to an upstream service to check for any change in state is compute and network resource -intensive, and in the case of a system with infrequent state changes, compute and network resource -wasteful indeed; while webhooks that issue API call to a downstream function should a state change emerge may not be supported by other microservices. Each API call is also made specific to a downstream service by IP or URL and, to an endpoint with a specific set of parameters for the request and the response. An update to the IP or URL, or the endpoint requires as such changes to all the other microservices that issue API calls to the host and is, as the number of microservices scales, a nightmare to maintain.
All of these drawbacks of scalability, flexibility and maintainability are in fact rather reminiscent of the issues of with a monolithic system that a microservice architecture was set out to solve, a new event-driven paradigm that, in a way, applies the microservice principle in system design to the communication architecture within a microservice system is as such conceived to address these problems. Instead of having each microservice issuing blocking API call directly to the downstream function, a change in state that is of interest to the other microservices are published, by the function that surfaces the change, in the form an event, to a middleware, which in turn filters, routes, and distributes as appropriate the event to the subscribers. Similar to disentangling a monolithic system into a swarm of microservices, this event-driven and middleware-centric architecture decouples the event subscribers from the publishers. The number of possible communication paths is reduced from a quadratic magnitude of O(n^2^ / 2) to a linear function of O(n) since each microservice will only have to interface with the event platform. Events in JSON format are designed to be asynchronous and non-blocking by nature for there exists now an event broker to handle the subsequent delivery to the subscribers. With the subscribers abstracted away from the publishers, new upstream service can be developed and integrated seamlessly to the system without having to consider which of the downstream services need to be notified of a state change as in the API-driven request-response model, and vice versa.
From software engineering, to system architecture and communication design, the modularity principle has time and again come to the rescue. One cannot help but wonder in what field will it be adapted next.