Native Persistence in Apache Ignite

Even though Apache Ignite® is broadly used as a caching layer on top of external databases, it comes with its native persistence - a distributed, ACID, and SQL-compliant disk-based store. The native persistence integrates into the Ignite multi-tier storage as a disk tier that can be turned on to let Ignite store more data on disk than it can cache in memory and to enable fast cluster restarts.

Apache Ignite Native Persistence

When the native persistence enabled, Ignite stores a superset of data on disk and caches as much as it can in memory. For example, if your application needs to store 200 records in an Ignite cluster and the memory capacity allows caching only 150 records, then all 200 will be stored on disk, out of which 150 will be served from memory while the rest 50 from disk whenever the application requests them.

Ignite Persistence vs. External Databases

The native persistence has the following advantages over external databases:

Write-Ahead Logging and Checkpointing

If Ignite native persistence is selected as a disk tier, then every time a record is updated in memory, the change is added to the write-ahead log (WAL). The purpose of the WAL is to propagate updates to disk in the fastest way possible and provide a consistent recovery mechanism that supports full cluster failures. As the WAL grows, it periodically gets checkpointed to the main storage. Checkpointing is the process of copying dirty pages from the memory tier to the partition files on disk. A dirty page is a page that was updated in memory, was appended to the WAL, but was not written to the respective partition file on disk yet.

Durability

Ignite native persistence provides the following ACID guarantees across the cluster: