Apache Ignite Blog

Apache Ignite 2.5: Scaling to 1000s Nodes Clusters

May 31, 2018 by Denis Magda. Share in Facebook, Twitter

Apache Ignite was always appreciated by its users for two primary things it delivers - scalability and performance. Throughout the lifetime many distributed systems tend to do performance optimizations from a release to release while making scalability related improvements just a couple of times. It's not because the scalability is of no interest. Usually, scalability requirements are set and solved once by a distributed system and don't require significant additional interventions by engineers.

However, Apache Ignite grew to the point when the community decided to revisit its discovery subsystem that influences how well and far Ignite scales out. The goal was pretty clear - Ignite has to scale to 1000s of nodes as good as it scales to 100s now.

It took many months to get the task implemented. So, please join me in welcoming Apache Ignite 2.5 that now can be scaled easily to 1000s of nodes and goes with other exciting capabilities. Let's check out the most prominent ones.

Apache Ignite 2.4 ings Advanced Machine Learning and Spark DataFrames Capabilities

March 15, 2018 by Denis Magda. Share in Facebook, Twitter

Usually, Ignite community rolls out a new version once in 3 months, but we had to make an exception for Apache Ignite 2.4 that consumed five months in total. We could easily blame Thanksgiving, Christmas and New Year holidays for the delay and would be forgiven, but, in fact, we were forging the release you can't simply pass by.

Let's dive in and search for a big fish.

Machine Learning General Availability

Eight months ago, at the time of Apache Ignite 2.0, we put out the first APIs that formed the foundation of the Ignite's machine learning component of today. Since that time, Ignite machine learning experts and enthusiasts have been moving the liary to the general availability condition meticulously. And Ignite 2.4 became a milestone that let us consider the ML Grid to be production ready.

Meltdown and Spectre patches show negligible impact to Apache Ignite performance

January 30, 2018 by Denis Magda. Share in Facebook, Twitter

As promised in my initial blog post on this matter, Apache Ignite community applied security patches against the notorious Meltdown Spectre vulnerabilities and completed performance testing of general operations and workloads that are typical for Ignite deployments.

The security patches were applied only for CVE-2017-5754 (Meltdown) and CVE-2017-5753 (Spectre Variant 1) vulnerabilities. The patches for CVE-2017-5715 (Spectre Variant 2) for the hardware the community used for testing are not stable yet an can cause system reboot issues or another unpredictable behavior

The applied patches have shown that the performance implications are negligible - the performance drop is just in the 0 - 7% range as the figure shows:

Protecting Apache Ignite from 'Meltdown' and 'Spectre' vulnerabilities

January 8, 2018 by Denis Magda. Share in Facebook, Twitter

The world was rocked after the recent disclosure of the Meltdown and Spectre vulnerabilities that literally affect almost all software ever developed. Both issues are related to the way all modern CPUs are designed and this is why they have opened unprecedented security breaches -- making the software, including Apache Ignite, vulnerable to hacker attacks.

The vulnerabilities are registered in the National Vulnerability Database under the following CVEs:

Apache Ignite Essentials: 2-part Webinar Series for Architects and Java Developers

November 17, 2017 by Denis Magda. Share in Facebook, Twitter

We finally made this happen! I’m happy to invite all of the software architects and engineers out there to a series of webinars that will introduce you to the fundamental capabilities of in-memory computing platforms such as Apache Ignite.

There will also be a mix of theory and practice. A lot of code examples are waiting to be shown so that you can apply the theory in practice right away.

The series consists of two parts.

Part 1: Tuesday, November 21, 2017, 11:00am PT / 2:00pm ET

To be covered:
  • Cluster configuration and deployment.
  • Distributed database internals (partitioning, replication).
  • Data processing with key-value APIs.
  • Affinity Collocation.
  • Data processing with SQL.

Apache Ignite 2.3 - More SQL and Persistence Capabilities

November 1, 2017 by Denis Magda. Share in Facebook, Twitter

Putting aside the regular bug fixes and performance optimizations, the Apache Ignite 2.3 release brings new SQL capabilities and Ignite persistence improvements that are worth mentioning.

SQL

Let's start with SQL first.

Apache Ignite users have consistently told us that despite all of Ignite’s SQL capabilities, it’s been at times challenging trying to figure out how to start using Ignite as an SQL database.

This was mostly caused by scattered documentation pages, lack of “getting started” guides and tutorials. We’ve remedied this oversight! All related SQL knowledge has been curated in a single documentation domain.

Apache Ignite Community News (Issue 3)

September 15, 2017 by Denis Magda. Share in Facebook, Twitter

by Tom Diederich

This is our third community update – there’s a lot going on, so let's get started.

Apache Ignite experts have already spoken at two meetups this month, both in Silicon Valley, but there are several more scheduled this month around the world.

On Sept. 9 Apache Ignite PMC chair Denis Magda was the featured presenter at the Big Data and Cloud Meetup in Santa Clara, Calif. His talk, titled "Apache Spark and Apache Ignite: Where Fast Data Meets the IoT," was highly rated and we’re planning a hands-on workshop with meetup organizers for November.

Apache Ignite Community Update (August 2017 Issue)

August 30, 2017 by Denis Magda. Share in Facebook, Twitter

by Tom Diederich

Igniters, here are some community highlights from the last couple week. If I missed anything, please share it here. Meetups! Did you know that Apache Ignite experts are available to speak at your meetup? And we also have spots open for YOU to speak at the following meetups that some of us co-organize:

Apache Ignite 2.1 - A Leap from In-Memory to Memory-Centric Architecture

July 27, 2017 by Denis Magda. Share in Facebook, Twitter

The power and beauty of in-memory computing projects are that they truly do what they state -- deliver outstanding performance improvements by moving data closer to the CPU, using RAM as a storage and spreading the data sets out across a cluster of machines relying on horizontal scalability.

However, there is an unspoken side of the story. No matter how fast a platform is, we do not want to lose the data and encounter cluster restarts or other outages. To guarantee this we need to somehow make data persistent on the disk.

Most in-memory computing projects address the persistence dilemma by giving a way to sync data back to a relational database (RDBMS). That sounds reasonable and undoubtedly works pretty well in practice, but if we dig deeper, you’ll likely encounter the following limitations:

Apache Ignite 2.0: Redesigned Off-heap Memory, DDL and Machine Learning

May 5, 2017 by Denis Magda. Share in Facebook, Twitter

We released the long-awaited Apache Ignite version 2.0 on May 5. The community spent almost a year incorporating tremendous changes to the legacy Apache Ignite 1.x architecture. And all of that effort paid off. Our collective blood, sweat (and perhaps even a few tears) opened up new and exciting opportunities for the Apache Ignite project.

Have I piqued your interest about this new release yet? Let's walk through some of the main new features that have appeared under the hood of Apache Ignite 2.0.