Benchmarking Ignite With Yardstick Framework
Yardstick Ignite Benchmarks
Apache Ignite benchmarks are written on top of the Yardstick Framework, allowing you to measure the performance of
various Apache Ignite components and modules. The documentation below describes how to execute and configure pre-assembled
benchmarks. If you need to add new benchmarks
or build existing one, then refer to the instructions from Ignite’s
DEVNOTES.txt file in the source directory.
Visit the Yardstick Repository for more details on the resulting graphs generation and how the framework works.
Running Ignite Benchmarks Locally
The simplest way to start with benchmarking is to use one of the executable scripts available under the
The command above will benchmark the cache
put operations for a distributed atomic cache. The results of the benchmark
will be added to an auto-generated
./bin/benchmark-run-all.sh command is executed as-is without any parameters and modifications in the configuration
file, then all the available benchmarks will be executed on a local machine using the
configuration. In case of any issues, refer to the logs that are added to an auto-generated
For more information about available benchmarks and configuration parameters, refer to the Existing Benchmarks and Properties And Command Line Arguments sections below.
Running Ignite Benchmarks Remotely
To benchmark Apache Ignite across several remote hosts:
<value>127.0.0.1:47500..47509</value>with actual IPs of all the remote hosts. Refer to the documentation section below if you prefer to use another kind of IP finder: Cluster Configuration
localhostwith actual IPs of the remote hosts in the following places:
DRIVERis a host (usually an Ignite client node) that executes benchmarking logic.
SERVERSare Ignite nodes that are benchmarked. Replace the
localhostoccurrences in the same places in the
config/benchmark-remote.propertiesfile if you plan to execute a full set of benchmarks available.
Upload Ignite Yardstick Benchmarks to one of your
DRIVERShost in its own working directory.
Log in on the remote host that will be the
DRIVER, and execute the following command:
By default, all the necessary files will be automatically uploaded from the host in which you run the command above to
every other host to the same path. If you prefer to do it manually set the
AUTO_COPY variable in property file to
The command above will benchmark the cache put operation for a distributed atomic cache. The results of the benchmark will
be added to an auto-generated
If you want to execute all the available benchmarks across the remote hosts, then execute the following command on the
The following benchmarks are provided by default:
GetBenchmark- benchmarks atomic distributed cache get operation.
PutBenchmark- benchmarks atomic distributed cache put operation.
PutGetBenchmark- benchmarks atomic distributed cache put and get operations together.
PutTxBenchmark- benchmarks transactional distributed cache put operation.
PutGetTxBenchmark- benchmarks transactional distributed cache put and get operations together.
SqlQueryBenchmark- benchmarks distributed SQL query over cached data.
SqlQueryJoinBenchmark- benchmarks distributed SQL query with a Join over cached data.
SqlQueryPutBenchmark- benchmarks distributed SQL query with simultaneous cache updates.
AffinityCallBenchmark- benchmarks affinity call operation.
ApplyBenchmark- benchmarks apply operation.
BroadcastBenchmark- benchmarks broadcast operations.
ExecuteBenchmark- benchmarks execute operations.
RunBenchmark- benchmarks running task operations.
PutGetOffHeapBenchmark- benchmarks atomic distributed cache put and get operations together off-heap.
PutGetOffHeapValuesBenchmark- benchmarks atomic distributed cache put value operations off-heap.
PutOffHeapBenchmark- benchmarks atomic distributed cache put operations off-heap.
PutOffHeapValuesBenchmark- benchmarks atomic distributed cache get value operations off-heap.
PutTxOffHeapBenchmark- benchmarks transactional distributed cache put operation off-heap.
PutTxOffHeapValuesBenchmark- benchmarks transactional distributed cache put value operation off-heap.
SqlQueryOffHeapBenchmark-benchmarks distributed SQL query over cached data off-heap.
SqlQueryJoinOffHeapBenchmark- benchmarks distributed SQL query with a Join over cached data off-heap.
SqlQueryPutOffHeapBenchmark- benchmarks distributed SQL query with simultaneous cache updates off-heap.
PutAllBenchmark- benchmarks atomic distributed cache batch put operation.
PutAllTxBenchmark- benchmarks transactional distributed cache batch put operation.
Properties And Command Line Arguments
Note that this section only describes configuration parameters specific to Ignite benchmarks, and not for Yardstick framework.
To run Ignite benchmarks and generate graphs, you will need to run them using the Yardstick framework scripts in the
Refer to the Yardstick Documentation for common Yardstick properties and command line arguments for running Yardstick scripts.
The following Ignite benchmark properties can be defined in the benchmark configuration:
--backups <num>- Number of backups for every key.
--Config <path>- Path to Ignite configuration file.
--cacheStore- Enable or disable cache store readThrough, writeThrough.
--client- Client flag. Use this flag if you running more than one
DRIVER, otherwise additional drivers would behave like a
--nearCache- Near cache flag.
--nodeNumber <num>- Number of nodes (automatically set in
benchmark.properties); used to wait for the specified number of nodes to start.
-syncMode <mode>- Synchronization mode (defined in CacheWriteSynchronizationMode`).
--range- Range of keys that are randomly generated for cache operations.
-rd or --restartdelay- Restart delay in seconds.
-rs or --restartsleep- Restart sleep in seconds.
--restHost <host>- REST TCP host.
--restPort <num>- REST TCP port, indicates that a Ignite node is ready to process Ignite Clients.
--syncSend- Flag indicating whether synchronous send is used in
--txConcurrency <value>- Cache transaction concurrency control, either
--txIsolation <value>- Cache transaction isolation (defined in
--writeBehind- Enable or disable writeBehind for cache store.
For example, if you want to run 2
IgniteNode servers on localhost with the
PutBenchmark benchmark, number of
backups set to
1, and synchronization mode set to
PRIMARY_SYNC, then the following configuration should be specified
SERVER_HOSTS=localhost,localhost ... # Note that -dn and -sn, which stand for data node and server node, # are native Yardstick parameters and are documented in # Yardstick framework. CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark`IgniteNode"
Building From Sources
./mvnw clean package -Pyardstick -pl modules/yardstick -am -DskipTests in the Apache Ignite root directory.
This command will compile the project and also unpack the scripts from
yardstick-resources.zip file to
Artifacts can be found in the
Custom Ignite Benchmarks
All benchmarks extend the
AbstractBenchmark class. A new benchmark should also extend this abstract class and
test method (this is the method that actually tests performance).
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.