In this article, author of the book "High Performance in-memory computing with Apache Ignite", discussing the design of modern application architecture with Apache Ignite. Part of this article taken from the book.
Let’s take a quick look at an architecture of a traditional system. The traditional application architecture uses data stores which have synchronous read-write operations. This is useful for data consistency and data durability, but it is very easy to get bottleneck if there are a lot of transactions waiting in the queue. Consider the following traditional architecture as shown below.
High-volume transaction processing.
In-memory data grid adds an additional layer within an environment, which uses the Random-Access Memory (RAM) of the server to store most of all data required by the applications. In-memory data grid sits between the application servers and the data store. In-memory data grid uses a cache of frequently accessed data by the client in the active memory and then can access the persistence store whenever needed and even asynchronously send and receive updates from the persistence store. An application architecture with in-memory data grid is shown below.
Resilient web acceleration.
With in-memory data grid like Apache Ignite, you can provide fault tolerance of your web application and accelerate your web application performance. Without changing any code, you can share session states between web applications through caches.
Event processing & real-time analysis.
Data tells the story of what's happing with your business on the background right now. With the IoT as a continuous data source, the opportunities to take advantage of the hot data is greater than ever. Traditional data management system cannot process big data fast enough to notify the business of important events as they occur: such as online credit card fraud detection or risk calculation. Apache Ignite allows processing continuous never-ending streams of data in scalable and fault-tolerant fashion in-memory, rather than analyzing data after it's reached the database.
Microservices in distributed fashion.
Microservice architecture has a number of benefits and enforces a level of modularity that in practice is extremely difficult to achieve with a monolithic code base. In-memory data grid like Apache Ignite can provide independent cache nodes to corresponding microservices in the same distributed cluster and gives you a few advantages over traditional approaches.
Hadoop has been widely used for its ability to economically store and analyze large data sets and has long passed the point of being nascent technology. However, it’s batch scheduling overhead and disk-based data storage have made it unsuitable for use in analyzing live, real-time data in the production environment. One of the main factors that limit performance scaling of Hadoop and Map/Reduce is the fact that Hadoop relies on a file system that generates a lot of input/output (I/O) files. An alternative is to store the needed distributed data within the memory. Placing Map/Reduce in- memory with the data it needs eliminates file I/O latency.
Cache as a Service.
Data-driven applications that take too long to load are boring and frustrating to use. Four out of five online users will click away if a page stalls while loading. In-memory data grid can provide a common caching layer across the organization, which can allow multiple applications to access managed in-memory cache.
These are some of the ways in-memory grids like Apache Ignite have served as an essential, architectural component for transforming the way businesses use their data to do business. But that’s not all folks! We will cover a lot of in-memory data grid use cases and application architecture in more depth through the book.