Modularity and Event-Driven Microservice Architecture
Posted on May 16, 2022 — 4 Minutes Read
Advancing the modularity principle from software development to system design is the microservice architecture. Features of a monolithic system are refactored into an array of microservices, in units of immutable, ephemeral and disposable container, that may be created and duplicated on demand as the workload requires. Application Programming Interface (API) based on the Representational State Transfer (REST) framework, being the de facto standard for connecting applications in the age of web services, was adapted as the protocol for coordination and communication between the now array of microservices to complete an activity. Endpoints that are available for an API call, along with the parameters of the request and of the response in JavaScript Object Notation (JSON) are well defined in the respective specifications. With the inner workings and intricacies of each microservice abstracted away by the API, new services that respond to emerging business needs can be rapidly developed and seamlessly integrated to the existing architecture by continuous integration/continuous delivery (CI/CD) for a system that is altogether flexible, scalable and maintainable.
REST API, while tried and true, does not however scale well in a system with hundreds or even thousands of microservices. For a system of n services, there exists up to n x (n - 1) / 2 of possible communication paths and the number grows quadratically at an order of approximation of O(n^2^ / 2). Based on the HTTP request-response model, API calls are by nature synchronous and will as such block the operation until a respond is received, idling resources in the process when in most cases it could proceed without needing to wait. Recurring API calls to an upstream service to check for any change in state is compute and network resource -intensive, and in the case of a system with infrequent state changes, compute and network resource -wasteful indeed; while webhooks that issue API call to a downstream function should a state change emerge may not be supported by other microservices. Each API call is also made specific to a downstream service by IP or URL and, to an endpoint with a specific set of parameters for the request and the response. An update to the IP or URL, or the endpoint requires as such changes to all the other microservices that issue API calls to the host and is, as the number of microservices scales, a nightmare to maintain.
All of these drawbacks of scalability, flexibility and maintainability are in fact rather reminiscent of the issues of with a monolithic system that a microservice architecture was set out to solve, a new event-driven paradigm that, in a way, applies the microservice principle in system design to the communication architecture within a microservice system is as such conceived to address these problems. Instead of having each microservice issuing blocking API call directly to the downstream function, a change in state that is of interest to the other microservices are published, by the function that surfaces the change, in the form an event, to a middleware, which in turn filters, routes, and distributes as appropriate the event to the subscribers. Similar to disentangling a monolithic system into a swarm of microservices, this event-driven and middleware-centric architecture decouples the event subscribers from the publishers. The number of possible communication paths is reduced from a quadratic magnitude of O(n^2^ / 2) to a linear function of O(n) since each microservice will only have to interface with the event platform. Events in JSON format are designed to be asynchronous and non-blocking by nature for there exists now an event broker to handle the subsequent delivery to the subscribers. With the subscribers abstracted away from the publishers, new upstream service can be developed and integrated seamlessly to the system without having to consider which of the downstream services need to be notified of a state change as in the API-driven request-response model, and vice versa.
From software engineering, to system architecture and communication design, the modularity principle has time and again come to the rescue. One cannot help but wonder in what field will it be adapted next.