In-Memory Data Grid with Apache Ignite

The Apache Ignite® in-memory data grid accelerates and scales your databases, services, and APIs. It supports key-value and ANSI SQL APIs, ACID transactions, co-located processing, and machine learning libraries. As an in-memory data grid, Ignite is frequently used to increase the performance and scalability of existing real-time applications, for high-performance computing, or for caching of data scattered across databases.

In-Memory Data Grid with Apache Ignite

What is an In-Memory Data Grid?

An in-memory data grid is a read-through/write-through caching strategy in which the application layer treats the data grid as the primary data store. Applications write to and read from the in-memory cluster that propagates all the changes to any underlying databases automatically keeping them updated and consistent with the in-memory data.

Ignite as an in-memory data grid provides all essential APIs needed to simplify its adoption. The APIs include distributed key-value and ANSI SQL queries, ACID transactions, co-located processing, and machine learning models. While key-value and SQL calls let you request, join, and group distributed data sets, the compute and machine learning components help to eliminate data shuffling over the network, thus, boosting compute and data-intensive calculations.

Ignite can store data both in memory and on disk with two options for data persistence. You can persist changes in an external database or have Ignite keep data in its native persistence.

IGNITE as an in-memory data grid AND EXTERNAL DATABASES

Ignite can improve the performance and scalability of any external database such as RDBMS, NoSQL or Hadoop, by sliding in as an in-memory data grid between the application and the database layer. When an application writes data to the cache, Ignite automatically writes-through or writes-behind all data modifications to the underlying external store. Ignite also performs ACID transactions where it coordinates and commits a transaction across the cluster as well as the database.

Additionally, Ignite can be deployed as a shared and unified in-memory layer that stores data sets originating from disjointed databases. Your applications can consume all the data from Ignite as a single store while Ignite can keep the original databases in sync whenever in-memory data gets updated.

However, there are some limitations if an external database is used as a persistence layer for Ignite in-memory data grid deployments. For instance, if you run Ignite SQL or scan queries, you need to ensure that all data is preloaded to the in-memory cluster. Note that Ignite SQL or scan queries can read data from disk only if the data is stored in the Ignite native persistence.

IGNITE NATIVE PERSISTENCE

Ignite native persistence is a distributed ACID and SQL-compliant disk store that transparently integrates with the Ignite in-memory layer. With native persistence enabled, Ignite stores both data and indexes on disk and eliminates the time-consuming cache warm-up step. The native persistence always keeps a full copy of data on disk so you are free to cache a subset of records in memory. If a required data record is missing in memory, Ignite reads it from the disk automatically regardless of the API, whether you use SQL, key-value, or scan queries.