Apache Ignite Blog

As of March 15, 2021, Apache Ignite 2.10 has been released. You can directly check the full list of resolved Important JIRA's but here let’s briefly overview some valuable improvements.

Thin Clients

Thin clients now support several important features which, previously were available only on the thick clients. Thin clients are always backward and forward compatible with the server nodes of the cluster, so the cluster upgrade process will be more convenient if the lack of these features prevented you from doing that.

See the list of what is changed for thin clients below:

• Transactions
• Service invocations
• Continuous Queries
• SQL API
• Cluster API
• Cache Async API
• Kubernetes Discovery (ThinClientKubernetesAddressFinder)

You may check the List of Thin Client Features that supported by platforms you are interested in or see the What's new in Apache Ignite.NET 2.10.

As of October 23, 2020, Apache Ignite 2.9 is available. Like every other Ignite release, release 2.9 includes many changes. Let's take a look at the major features of release 2.9.

Cluster Snapshots

Ignite 2.9 provides the ability to create full cluster snapshots for deployments that use Ignite Persistence. Snapshots can be taken online, when the cluster is active and accessible to users. An Ignite snapshot includes a cluster-wide copy of all data records that exist at the moment the snapshot is started. All snapshots are consistent — in terms of concurrent, cluster-wide operations as well as in terms of ongoing changes in Ignite Persistence data, index, schema, binary metadata, marshaller, and other files on nodes. See Ignite documentation to learn about this feature.

With thousands of changes contributed to Apache Ignite 2.8 that enhanced almost all the components of the platform, it’s possible to overlook some of the improvements that can convince you to upgrade to this version sooner than later. While a quick check of the release notes will help to discover anticipated bug fixes, this article aims to guide through enhancements every Ignite developer should be aware of.

New Subsystem for Production Monitoring and Tracing

Several months of constant work on IEP-35: Monitoring & Profiling has resulted in the creation of a robust and elastic subsystem for production monitoring and diagnostic (aka. profiling). This was influenced by the needs of many developers who deployed Ignite in critical environments and were asking for a foundation that can be integrated with many external monitoring tools and be expanded easily.

The new subsystem consists of several registries that group individual metrics related to a specific Ignite component. For instance, you will find registries for cache, compute, or service grid APIs. Since the registries are designed to be generic, specific exporters can observe the state of Ignite via a myriad of tools supporting various protocols. By default, Ignite 2.8 introduces exporters for monitoring interfaces such as log files, JMX and SQL views, and contemporary ones such as OpenCensus.

Deep Learning With TensorFlow

Even though it was natural to provide machine learning algorithms in Ignite out of the box, another direction was taken for deep learning capabilities. Primarily because machine learning approaches have already been adopted in businesses from big to small -- while deep learning is still being used for narrow and specific use cases.

Thus, Ignite 2.7 can boast about an official integration with TensorFlow deep learning framework that gives a way to use Ignite as a distributed storage for TensorFlow calculations. With Ignite, data scientists can store unlimited data sets across a cluster, gain performance improvements and rely on fault-tolerance of both products if an algorithm fails in the middle of an execution.

Apache Ignite was always appreciated by its users for two primary things it delivers - scalability and performance. Throughout the lifetime many distributed systems tend to do performance optimizations from a release to release while making scalability related improvements just a couple of times. It's not because the scalability is of no interest. Usually, scalability requirements are set and solved once by a distributed system and don't require significant additional interventions by engineers.

However, Apache Ignite grew to the point when the community decided to revisit its discovery subsystem that influences how well and far Ignite scales out. The goal was pretty clear - Ignite has to scale to 1000s of nodes as good as it scales to 100s now.

It took many months to get the task implemented. So, please join me in welcoming Apache Ignite 2.5 that now can be scaled easily to 1000s of nodes and goes with other exciting capabilities. Let's check out the most prominent ones.

Usually, Ignite community rolls out a new version once in 3 months, but we had to make an exception for Apache Ignite 2.4 that consumed five months in total. We could easily blame Thanksgiving, Christmas and New Year holidays for the delay and would be forgiven, but, in fact, we were forging the release you can't simply pass by.

Let's dive in and search for a big fish.

Machine Learning General Availability

Eight months ago, at the time of Apache Ignite 2.0, we put out the first APIs that formed the foundation of the Ignite's machine learning component of today. Since that time, Ignite machine learning experts and enthusiasts have been moving the liary to the general availability condition meticulously. And Ignite 2.4 became a milestone that let us consider the ML Grid to be production ready.

As promised in my initial blog post on this matter, Apache Ignite community applied security patches against the notorious Meltdown Spectre vulnerabilities and completed performance testing of general operations and workloads that are typical for Ignite deployments.

The security patches were applied only for CVE-2017-5754 (Meltdown) and CVE-2017-5753 (Spectre Variant 1) vulnerabilities. The patches for CVE-2017-5715 (Spectre Variant 2) for the hardware the community used for testing are not stable yet an can cause system reboot issues or another unpredictable behavior. 

The applied patches have shown that the performance implications are negligible - the performance drop is just in the 0 - 7% range as the figure shows:

The world was rocked after the recent disclosure of the Meltdown and Spectre vulnerabilities that literally affect almost all software ever developed. Both issues are related to the way all modern CPUs are designed and this is why they have opened unprecedented security breaches -- making the software, including Apache Ignite, vulnerable to hacker attacks.

The vulnerabilities are registered in the National Vulnerability Database under the following CVEs:

• CVE-2017-5753 — Spectre variant 1
• CVE-2017-5715 — Spectre variant 2
• CVE-2017-5754 — Meltdown

We finally made this happen! I’m happy to invite all of the software architects and engineers out there to a series of webinars that will introduce you to the fundamental capabilities of in-memory computing platforms such as Apache Ignite.

There will also be a mix of theory and practice. A lot of code examples are waiting to be shown so that you can apply the theory in practice right away.

The series consists of two parts.

Part 1: Tuesday, November 21, 2017, 11:00am PT / 2:00pm ET

To be covered:

• Cluster configuration and deployment.
• Distributed database internals (partitioning, replication).
• Data processing with key-value APIs.
• Affinity Collocation.
• Data processing with SQL.

Putting aside the regular bug fixes and performance optimizations, the Apache Ignite 2.3 release brings new SQL capabilities and Ignite persistence improvements that are worth mentioning.

SQL

Let's start with SQL first.

Apache Ignite users have consistently told us that despite all of Ignite’s SQL capabilities, it’s been at times challenging trying to figure out how to start using Ignite as an SQL database.

This was mostly caused by scattered documentation pages, lack of “getting started” guides and tutorials. We’ve remedied this oversight! All related SQL knowledge has been curated in a single documentation domain.