Apache Ignite Blog Posts
Introduction to Machine Learning with Apache® Ignite™By Akmal Chaudhri | June 06, 2018
In a previous article, we discussed the Apache® Ignite™ Machine Learning Grid. At that time, a beta release was available. Subsequently, in version 2.4, Machine Learning became Generally Available. Since the 2.4 release, more improvements and developments have been added, including support for Partitioned-Based Datasets and Genetic Algorithms. Many of the Machine Learning examples that are provided with Apache Ignite can work standalone, making it very easy to get started. However, later in this series of articles, we will perform some analysis on several freely available datasets using some of the algorithms that are supported by Ignite.
Apache Ignite 2.5: Scaling to 1000s Nodes ClustersBy Denis Magda | May 31, 2018
Apache Ignite was always appreciated by its users for two primary things it delivers - scalability and performance. Throughout the lifetime many distributed systems tend to do performance optimizations from a release to release while making scalability related improvements just a couple of times. It's not because the scalability is of no interest. Usually, scalability requirements are set and solved once by a distributed system and don't require significant additional interventions by engineers.
However, Apache Ignite grew to the point when the community decided to revisit its discovery subsystem that influences how well and far Ignite scales out. The goal was pretty clear - Ignite has to scale to 1000s of nodes as good as it scales to 100s now.
Apache Ignite: More than a simple cacheBy Gastón Lucero | May 16, 2018
Let us suppose that we start to develop a webserver for our IOT App with a few endpoints, like POST for receive events, GET devicesBySensorType, GET all, and PUT for update device metadata, etc.
At first, a cache for common data could seem like a secondary issue, but if we start to think long term and if we want to improve performance and/or to decrease response time (e.g when a service retrieve data from a database), we realize that a cache is a mandatory requirement.
Apache Ignite Transactions Architecture: Transaction Handling at the Level of Third-Party PersistenceBy Akmal Chaudhri | May 07, 2018
Handling transaction failure with third-party persistence is actually easier than you may think, as updates and changes are applied to the third-party store first.
Apache Ignite Transactions Architecture: Ignite Persistence Transaction HandlingBy Akmal Chaudhri | May 03, 2018
Learn about transaction handling at the level of Apache Ignite persistence (for example, WAL, checkpointing, and more).
Apache Cassandra and Apache Ignite: Selecting the Right Distributed Database SolutionBy Nikita Ivanov | April 10, 2018
Apache® Cassandra™ is a popular open-source, distributed, key-value store columnar NoSQL database used by companies such as Netflix, eBay, and Expedia for key parts of their business. For Apache Cassandra users that need ad-hoc SQL query capabilities but are otherwise happy with their database choice, Apache® Ignite™ can enhance the performance of Apache Cassandra. For Apache Cassandra users or companies considering Cassandra that find it does not have the read speed or SQL capabilities they need in a distributed key-value store database, Apache Ignite can be a powerful alternative to Apache Cassandra.
Using Apache Ignite to Prevent Duplicate REST RequestsBy David Khanaferov | March 19, 2018
Read on to find out how David Khanaferov used Apache Ignite to prevent duplicate REST requests.
Apache Ignite Persistence, Expiration Policies and Monitoring with Prometheus + GrafanaBy Sven Goly | March 17, 2018
Streaming data from Kafka to Ignite and processing it with horizontal and vertical scalability has been covered in the first four parts. At this point, we got our processed data in the cache – so we want to either persist that data or set up some expiration policy so we don’t run out of space. Since we want to make sure everything works as intended, we’ll also start setting up a monitoring.
Apache Ignite: In-Memory Option for Apache Cassandra DeploymentsBy Artem Schitow | February 22, 2018
In this blog, Artem Schitow shows how to deploy Apache Ignite as an in-memory layer above Cassandra. Read on to find out how it is possible to preserve existing deployments of Cassandra and address its limitations with Ignite
Meltdown and Spectre patches show negligible impact to Apache Ignite performanceBy Denis Magda | January 30, 2018
Apache Ignite community applied security patches against the notorious Meltdown Spectre vulnerabilities and completed performance testing of general operations and workloads that are typical for Ignite deployments.
Apache Ignite Transactions Architecture: Concurrency Modes and Isolation LevelsBy Akmal Chaudhri | January 25, 2017
Most modern multi-user applications allow concurrent data access and modification. To manage this capability and ensure that the system moves from one consistent state to another, the concept of transactions is used. Transactions rely upon locks, which can be acquired at the beginning of a transaction (pessimistic locking) or at the end of a transaction (optimistic locking) before work is committed.
Apache Ignite Transactions Architecture: 2-phase commit protocolBy Akmal Chaudhri | January 12, 2017
Apache Ignite supports a range of different Application Programming Interfaces (APIs). In this multi-part article series, we will take a more detailed look at how Apache Ignite manages transactions in its key-value API and some of the mechanisms and protocols it supports.
In this first part, we will begin with a discussion of the two-phase commit (2PC) protocol and then look at how this works with various types of cluster nodes.
Protecting Apache Ignite from 'Meltdown' and 'Spectre' vulnerabilitiesBy Denis Magda | January 08, 2018
The world was rocked after the recent disclosure of the Meltdown and Spectre vulnerabilities that literally affect almost all software ever developed. Both issues are related to the way all modern CPUs are designed and this is why they have opened unprecedented security breaches -- making the software, including Apache Ignite, vulnerable to hacker attacks.
In-Memory Technologies: Meeting Healthcare's Fast Data Challenges (Part 2)By Akmal Chaudhri | December 28, 2017
Learn about a healthcare case study from a company called e-Therapeutics, which specializes in drug discovery and development, and see how they used Apache Ignite.
In-Memory Technologies: Meeting Healthcare's Fast Data Challenges (Part 1)By Akmal Chaudhri | December 19, 2017
Modern healthcare has many demanding requirements for the storage and querying of data. Ignite provides a range of capabilities that can meet these needs perfectly.
Intel Optane™ SSDs Outperforms Regular SSDs - Verified on Apache IgniteBy Denis Magda | October 31, 2017
This article shows how you can improve performance in random write workloads of a database by replacing regular SSDs with Intel Optane SSDs.
Kubernetes and Apache Ignite Deployment on AWSBy Akmal Chaudhri | September 28, 2017
Bring K8s and Apache Ignite to the cloud (AWS, at least) with this guide, which touches on Kubernetes' installation, configuration, and connecting to Ignite clusters.
SSD Performance Tips - Checked on Apache IgniteBy Denis Magda | September 26, 2017
In this blog post, Denis explains how the cache coherence protocol can help you use Apache Ignite to increase the speed and performance of your platform.
Analyzing Ignite.NET Code With NDependBy Pavel Tupitsyn | September 19, 2017
See how NDepend works and how you can use it for code analysis to make sure your Ignite.NET code is clean and providing the best performance possible.
What is Apache Ignite?By Dmitriy Setrakyan | September 13, 2017
If you ever looked at Apache Ignite, you have probably noticed that it is a fairly rich platform with lots of components. However, despite the extensive feature set, Ignite community aims to make the platform easy to use and understand. In this blog, author Dmitriy Setrakyan defines Ignite by answering several "Is Ignite a ...?" questions.
Using Java and .NET Apps to Connect to an Apache Ignite ClusterBy Akmal Chaudhri | September 08, 2017
In many organizations, different departments and teams work with different programming languages and frameworks. However, there may be a need for a common storage format to allow various tools to access the same data. Apache Ignite provides the flexibility for development teams to continue working with their favorite programming languages and tools and have the ability to work with the same data in the cluster.
Apache Ignite - In Memory Performance with Durability of DiskBy Dmitriy Setrakyan | September 07, 2017
Starting with 2.1 release, Apache Ignite has become one of a very few in-memory computing systems that provides its own distributed persistence layer. Essentially, users do not have to integrate Ignite with any type of 3rd party databases (although such integration is still supported), and start using Ignite as a primary storage of their data on disk and in memory. So, what makes Ignite data storage unique?
Kubernetes and Apache® Ignite™ Deployment on AWSBy Akmal Chaudhri | September 07, 2017
This article will walk through the steps required to get Kubernetes and Apache Ignite deployed on Amazon Web Services (AWS). The local computer operating system used was Ubuntu and if you follow the steps in this article, you may need to adjust the commands as appropriate for your operating system and environment.
Apache Ignite Tip: Peer Class Loading Deployment MagicBy Akmal Chaudhri | August 29, 2017
Apache Ignite is great for modern data processing. It has a feature called Zero Deployment which is very useful for P2P class loading.
What's New in Apache Ignite .NET 2.1By Pavel Tupitsyn | August 02, 2017
Take a look at what's new in the latest release of Apache Ignite 2.1 from a .NET perspective. With this relase, Ignite is a complete database with a unique feature set.
Apache Ignite 2.1 - A Leap from In-Memory to Memory-Centric ArchitectureBy Denis Magda | May 12, 2017
The power and beauty of in-memory computing projects is that they truly do what they state -- deliver outstanding performance improvements by moving data closer to the CPU, using RAM as a storage and spreading the data sets out across a cluster of machines relying on horizontal scalability.
However, there is an unspoken side of the story. No matter how fast a platform is, we do not want to lose the data and encounter cluster restarts or other outages. To guarantee this we need to somehow make data persistent on the disk.
Implementing the Ignite.NET Plugin: Distributed SemaphoreBy Pavel Tupitsyn | July 23, 2017
See how the Apache Ignite.NET 2.0 plugin system can make Ignite and third party Java APIs like Ignite Semaphore available in .NET.
Getting Started With Apache Ignite (Part 6)By Akmal Chaudhri | July 16, 2017
A major use case for the Service Grid is to deploy a Singleton. There are a number of different types of Singletons that Apache Ignite supports but two common types are — Node Singleton and Cluster Singleton.
Getting Started With Apache Ignite (Part 5)By Akmal Chaudhri | July 10, 2017
Apache Ignite is a true Swiss Army Knife, providing scalability and performance for many varied and demanding workloads.
Apache Ignite with Spring DataBy Shamim Bhuyian | July 09, 2017
Apache Ignite IgniteRepository implements Spring Data CrudRepository interface and extends basic capabilities of the CrudRepository, which in turns supports:
- Basic CRUD operations on a repository for a specific type.
- Access to the Apache Ignite SQL grid via Spring Data API.
Getting Started With Apache Ignite (Part 4)By Akmal Chaudhri | July 03, 2017
Take a look at the Apache Ignite Streaming Grid component and see how it can help you ingest, manage, and query streaming data.
Getting Started With Apache Ignite (Part 3)By Akmal Chaudhri | June 18, 2017
Get a brief look at the Apache Ignite Compute Grid component and learn how it ensures that tasks can be executed in parallel within the grid.
Getting Started With Apache Ignite (Part 2)By Akmal Chaudhri | June 04, 2017
In Apache Ignite, a data grid can be thought of as a distributed Key-Value (K-V) store or a distributed HashMap. Learn more about it in this article.
ADO.NET as Ignite.NET Cache StoreBy Pavel Tupitsyn | May 31, 2017
Learn about implementing an efficient Ignite.NET persistent store with ADO.NET and SQL Server, continuing from a previous article on the entity framework cache store.
Getting Started With Apache Ignite (Part 1)By Akmal Chaudhri | May 29, 2017
In this series of articles, I will share what I learned about Apache Ignite as a beginner, starting with clustering and deployment.
How to Monitor Mulitple Apache Ignite ClustersBy Prachi Garg | May 24, 2017
With its latest release, Apache Ignite 2.0 introduced support for DDL, a redesigned off-heap memory architecture, distributed algebra, Spring data integration, support for Hibernate 5, Rocket MQ Integration, as well as plenty of improvements to the currently existing Apache Ignite feature set to enhance speed and performance. Another key feature rolled in is multi-cluster support for Apache Ignite Web Console, which means you can monitor and manage multiple Ignite clusters in parallel from a single instance of Ignite Web console deployed on your system.
This tutorial shows how to start two separate clusters on your system and connect it to an instance of Ignite Web Console deployed locally.
Benchmarking: Apache Ignite Still Keeps Ahead Of HazelcastBy Denis Magda | May 12, 2017
There's an ad saying that Hazelcast is up to 50% faster than Apache Ignite, but that may not be true anymore. Check out this benchmark to get the true story.
What's new in Apache Ignite.NET 2.0By Pavel Tupitsyn | May 12, 2017
Apache Ignite 2.0 has been released last week. Changes on Java side are tremendous, but Ignite.NET has some cool things to offer as well. Read on to to find out more.
An impatient start with Apache Ignite machine learning gridBy Shamim Bhuyian | May 12, 2017
Recently Apache Ignite 2.0 introduced a beta version of the in-memory machine learning grid, which is a distributed machine learning library built on top of the Apache IMDG. This beta release of ML library can perform local and distributed vector, decompositions and matrix algebra operations. The data structure can be stored in Java heap, off-heap or distributed Ignite caches. In this short post, we are going to download the new Apache Ignite 2.0 release, build the example and run them.
Apache Ignite 2.0: Redesigned Off-heap Memory, DDL and Machine LearningBy Denis Magda | May 05, 2017
This major release was under the development for a long time. The community spent almost a year incorporating tremendous changes to the legacy Apache Ignite 1.x architecture. Curious why are we so boastful about this? Some of the main features of Apache Ignite 2.0 are:
- Re-engineered Off-Heap Memory Architecture
- Data Definition Language
- Machine Learning Grid Beta - Distributed Algebra
- Integration with Spring Data, Rocket MQ, Hibernate 5
- Enchanced Inite.Net and Ignite C++ APIs
See release notes for a full list of the changes.
Apache Ignite: Build Cloud Ready Applications Today!By Turik Campbell | May 02, 2017
All applications fundamentally are comprised of computing instructions and data the instructions utilize to solve a problem. These applications are high performant when computing instructions and data are distributed among available computing resources. A ‘cloud ready’ application should be able to:
- Massively parallelize compute instructions.
- Massively parallelize data.
- Scale automatically as hardware resources are introduced into the network.
Light a fire under Cassandra with Apache IgniteBy Nikita Ivanov | April 27, 2017
Over time as business requirements evolve and Cassandra deployments scale, many organizations find themselves constrained by some of Cassandra’s limitations, which in turn restrict what they can do with their data. Apache Ignite, an in-memory computing platform, provides these organizations with a new way to access and manage their Cassandra infrastructure, allowing them to make Cassandra data available to new OLTP and OLAP use cases while delivering extremely high performance.
Microservices on Top of an In-Memory Data Grid: Part IIIBy Denis Magda | April 26, 2017
This is the last blog post in a series recommending how to design and implement microservices-based architecture on top of Apache Ignite In-Memory Data Fabric. The first two posts in the series can be found here:
- Part I - Overview of the proposed solution.
- Part II - Various coding templates needed to implement the solution in a live environment.
This final post describes how to integrate the cluster with a persistent store and send requests to the microservices from external applications -- apps that know nothing about the cluster and don't rely on its APIs.
LINQ vs. SQL in Ignite.NET: PerformanceBy Pavel Tupitsyn | March 29, 2017
Ignite.NET offers a LINQ provider which translates C# expressions to SQL queries. LINQ has many benefits over SQL — but at what cost? Read on to find out.
Getting Started with Apache Ignite - Part 1By Dani Traphagen | March 29, 2017
My best definition of Apache Ignite is that it's a distributed in-memory cache, query and compute engine built to work with large-scale data sets in real-time. A cluster of Ignite nodes (which is simply a combination of server and client nodes) will slide between the application and data layers.
Deploying Apache Ignite in Kubernetes on Microsoft AzureBy Denis Magda | March 21, 2017
Apache Ignite's most recent release includes a Kubernetes integration. See it in action as you learn to run a cluster on Microsoft Azure.
Continuous Queries in Apache Ignite C++ 1.9By Igor Sapego | March 21, 2017
Apache Ignite 1.9 was released last week and it brings some cool features. One of them is Continuous Queries for Apache Ignite C++ that allows you to track data modifications on caches.
Modern Application Design With In-Memory Data FabricsBy Shamim Bhuiyan | March 16, 2017
In-memory grids like Apache Ignite have served as an essential, architectural component for transforming the way businesses use their data to do business.
What's New in Apache Ignite.NET 1.9By Pavel Tupitsyn | March 14, 2017
The newest version of Apache Ignite includes TransactionScope API, Distributed DML, and LINQ improvements. Read on to find out how these functionalities can provide enhanced transactional and SQL capabilities.
Apache Ignite 1.9 Release HighlightsBy Denis Magda | March 06, 2017
Apache Ignite community released a new version of Apache Ignite In-Memory Data Fabric. Learn more about improvements available in version 1.9.
Book Review: High Performance In-Memory Computing With Apache IgniteBy Shamim Bhuiyan | February 16, 2017
The Apache Ignite platform is very big and growing day by day. This book focuses on features of Apache Ignite that help improve application performance.
The ASF asks: Have you met Apache Ignite?By Sally Khudairi | January 18, 2017
Did you know that numerous Fortune 500 enterprises depend on Apache Ignite's in-memory data platform to process large-scale data sets in real-time, at orders of magnitude faster than traditional technologies?
Running Microservices on Top of In-Memory Data Grid: Part IIBy Denis Magda | January 18, 2017
Let's look into the Apache Ignite Cluster Layer, a GitHub project that includes the basic building blocks needed to implement a proposed microservices-based architecture.
Book: High performance in-memory computing with Apache Ignite has been publishedBy Shamim Bhuiyan | January 09, 2017
This book wraps all the topics like in-memory data grid, highly available service grid, streaming and in-memory computing use cases from high-performance computing to get the performance gain.
Enabling Access to Apache Ignite via Redis ProtocolBy Roman Shtykh | January 09, 2017
The Apache Ignite versions have the ability to store and retrieve data in the grid using any Redis client. Let's make connections to an Ignite cluster and do Redis string operations.
Apache Ignite Enables Full-fledged SQL Support for PHPBy Denis Magda | December 27, 2016
It's time to get your SQL statements and queries up and running on Apache Ignite's PHP offerings. You'll need a driver and some setup, but it's quick and easy.
Using the GridGain Web Console for Automatic RDBMS Integration With Apache IgniteBy Prachi Garg | December 19, 2016
Apache Ignite can import database schemas and automatically generate all the required XML OR-mapping configurations and Java domain model POJOs that you can easily download and copy into your Apache Ignite project.
Geospatial Queries With Apache IgniteBy Denis Magda | December 16, 2016
Storing and querying location data can be useful for any number of apps for projects. Apache Ignite has a geospatial component made just for that.
What's New in Apache Ignite.NET 1.8By Pavel Tupitsyn | December 14, 2016
The newest version of Apache Ignite includes an entity framework second-level cache, ASP.NET session state cache, custom logging, and LINQ improvements.
Apache Ignite With JPA: A Missing ElementBy Shamim Bhuiyan | December 07, 2016
Learn how to persist your entities with Apache Ignite and JPA. This tutorial will guide you through the setup of execution of that handy ability.
Entity Framework As Ignite.NET Cache StoreBy Pavel Tupitsyn | October 27, 2016
Learn how to implement Ignite.NET persistent store with Entity Framework and SQL Server.
Running Microservices on Top of In-Memory Data Grid: Part IBy Denis Magda | October 26, 2016
With this post, we start a series that will provide a guide on building a fault-tolerant, scalable, microservice-based solution with Apache Ignite In-Memory Data Fabric.
Ignite.NET Serialization PerformanceBy Pavel Tupitsyn | October 04, 2016
How fast are different Ignite serialization modes? How do they compare to other popular serializers? Find out in this blog by Pavel Tupitsin.
Deadlock-Free Transactions with Apache IgniteBy Prachi Garg | September 21, 2016
Deadlocks can kill services, so see how Apache Ignite avoids it by assigning numbers to transactions in order to compare and utilize them in a fluid manner.
ASP.NET Distributed Output Cache with Apache IgniteBy Pavel Tupitsyn | September 19, 2016
You can speed up your ASP.NET web farm with Apache Ignite distributed caching. Read on to learn more.
Building a Multi-Platform Ignite Cluster: Java + .NETBy Pavel Tupitsyn | September 09, 2016
Ignite cluster can consist of nodes on any supported platform: Java, .NET, and C++. This example shows you how to run a .NET/Java cluster with NuGet and Maven.
Apache Ignite 1.7: Welcome Non-Collocated Distributed Joins!By Denis Magda | August 24, 2016
Apache Ignite 1.7.0 has been recently rolled out, and among the new changes, you can find a killer one that was awaited by many Apache Ignite users and customers for a long time — Non-Collocated Distributed Join support for SQL queries.
Using Apache Ignite.NET in LINQPadBy Pavel Tupitsyn | August 19, 2016
Here is a quick how-to for using Apache Ignite.NET in LINQPad.
What's New in Apache Ignite.NET 1.7By Pavel Tupitsyn | August 09, 2016
Apache Ignite.NET 1.7 brings some pretty cool new features. Read on to find out more!
Getting Started with Apache Ignite.NET Part 3: Cache QueriesBy Pavel Tupitsyn | July 28, 2016
In Part 3 of Pavel Tupitsyn's series, he covers the cache queries: Scan, SQL, LINQ, and Text.
Getting Started with Apache Ignite.NET Part 2: Distributed CacheBy Pavel Tupitsyn | July 16, 2016
Learn the cache operations and object serialization of Apache Ignite.NET.
Getting Started with Apache Ignite.NET Part 1By Pavel Tupitsyn | June 25, 2016
Learn the basics of using Apache Ignite.NET, from an explanation about the terminology to helpful code snippets illustrating the instructions.
Real-time In-memory OLTP and Analytics with Apache Ignite on AWSBy Babu Elumalai | June 06, 2016
This post shows you how to build a Lambda architecture using Apache Ignite, and provides some examples explaining how to perform ANSI SQL on real-time data and how to use it as a cache for OLTP reads.
Apache Ignite: How to Read Data from Persistent StoreBy Prachi Garg | June 03, 2016
A tutorial on how to load data from a MySQL database into an Ignite distributed cache.
Pitfalls of the MyBatis Caches with Apache IgniteBy Shamim Bhuiyan | March 09, 2016
A tutorial on how to look at cache entries in Apache Ignite.
A Universal Streamer for Apache Ignite based on Apache CamelBy Raúl Kripalani | January 28, 2016
Apache Ignite has the concept of Data Streamers: components to ingest fast data in a streaming fashion into an Ignite cache from a variety of protocols, technologies or platforms, such as JMS, MQTT, Twitter, Flume, Kafka, etc. However, with Apache Ignite 1.5.0 we released the jack of all trades: an Apache Camel streamer.
Apache Ignite: Distributed In-Memory Key-Value StoreBy Prachi Garg | January 27, 2016
For systems where low latency is critical, there is nothing better than caching the data in memory in a distributed cluster. While storing data in memory provides fast data access, distributing it on a cluster of nodes increases application performance and scalability. And Apache Ignite helps you achieve exactly that.
Getting Started with Apache IgniteBy Prachi Garg | December 19, 2015
This tutorial shows you how to create a simple "Hello World" example in Apache Ignite.
Apache Ignite for Database CachingBy Prachi Garg | Septmeber 23, 2015
A tutorial on how to use Apache Ignite for caching RDBMS, NoSQL, or HDFS databases.
How Apache Ignite Processes Geographically Distributed TransactionsBy Yakov Zhdanov | July 22, 2015
Imagine a bank offering variety of services to its customers. The customers of the bank are located in different geo-zones (regions), and most of the operations performed by a customer are zone-local, like ATM withdrawals or bill payments... However, some operations, such as wire transfers for example, may affect customers across different zones. Cross-zone operations are not as frequent, but nevertheless need to be supported in a transactional fashion as well.