Edit

Thread Pools Tuning

Ignite creates and maintains a variety of thread pools that are used for different purposes. In this section, we list some of the more common internal pools and show how you can create a custom one.

System Pool

The system pool handles all the cache related operations except for SQL and some other types of queries that go to the queries pool. Also, this pool is responsible for processing compute tasks' cancellation operations.

The default pool size is max(8, total number of cores). Use IgniteConfiguration.setSystemThreadPoolSize(…) or a similar API from your programming language to change the pool size.

Queries Pool

The queries pool takes care of all SQL, Scan, and SPI queries being sent and executed across the cluster.

The default pool size is max(8, total number of cores). Use IgniteConfiguration.setQueryThreadPoolSize(…) or a similar API from your programming language to change the pool size.

Public Pool

Public pool is the work-horse of the Compute Grid. All computations are received and processed by this pool.

The default pool size is max(8, total number of cores). Use IgniteConfiguration.setPublicThreadPoolSize(…) or a similar API from your programming language to change the pool size.

Service Pool

Service Grid calls go to the services' thread pool. Having dedicated pools for the Service and Compute components allows us to avoid threads starvation and deadlocks when a service implementation wants to call a computation or vice versa.

The default pool size is max(8, total number of cores). Use IgniteConfiguration.setServiceThreadPoolSize(…) or a similar API from your programming language to change the pool size.

Striped Pool

The striped pool helps accelerate basic cache operations and transactions by spreading operations execution across multiple stripes that don’t contend with each other for resources.

The default pool size is max(8, total number of cores). Use IgniteConfiguration.setStripedPoolSize(…) or a similar API from your programming language to change the pool size.

Data Streamer Pool

The data streamer pool processes all messages and requests coming from IgniteDataStreamer and a variety of streaming adapters that use IgniteDataStreamer internally.

The default pool size is max(8, total number of cores). Use IgniteConfiguration.setDataStreamerThreadPoolSize(…) or a similar API from your programming language to change the pool size.

Snapshot Pool

The snapshot pool is used for processing all the cluster operations related to taking or restoring Apache Ignite snapshots.

The default pool size is 4 (see the IgniteConfiguration.DFLT_SNAPSHOT_THREAD_POOL_SIZE). Use IgniteConfiguration.setSnapshotThreadPoolSize(…) to change the pool size.

Creating Custom Thread Pool

It is possible to configure a custom thread pool for compute tasks. This is useful if you want to execute one compute task from another synchronously avoiding deadlocks. To guarantee this, you need to make sure that a nested task is executed in a thread pool separate from the parent’s tasks thread pool.

A custom pool is defined in IgniteConfiguration and must have a unique name:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">

    <property name="executorConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.ExecutorConfiguration">
                <property name="name" value="myPool"/>
                <property name="size" value="16"/>
            </bean>
        </list>
    </property>

</bean>

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setExecutorConfiguration(new ExecutorConfiguration("myPool").setSize(16));

Now, let’s assume that you want to execute the following compute task in a thread from the myPool defined above:

public class InnerRunnable implements IgniteRunnable {
    @Override
    public void run() {
        System.out.println("Hello from inner runnable!");
    }
}

To do that, use IgniteCompute.withExecutor(), which will execute the task immediately from the parent task, as shown below:

public class OuterRunnable implements IgniteRunnable {
    @IgniteInstanceResource
    private Ignite ignite;

    @Override
    public void run() {
        // Synchronously execute InnerRunnable in a custom executor.
        ignite.compute().withExecutor("myPool").run(new InnerRunnable());
        System.out.println("outer runnable is executed");
    }
}

The parent task’s execution might be triggered the following way and, in this scenario, it will be executed by the public pool:

ignite.compute().run(new OuterRunnable());

Warning

Undefined Thread Pool

If an application attempts to execute a compute task in a custom pool which is not defined in the configuration of the node, then a special warning message will be printed to the logs, and the task will be picked up by the public pool for execution.

© 2024 The Apache Software Foundation.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.