New Metrics System | Ignite Documentation

Ignite Summit — June 6, 2023 — Join virtually!

Edit

New Metrics System

Overview

Ignite 2.8 introduced a new mechanism for collecting metrics, which is intended to replace the legacy metrics system. This section explains the new system and how you can use it to monitor your cluster.

Let’s explore the basic concepts of the new metrics system in Ignite. First, there are different metrics. Each metric has a name and a return value. The return value can be a simple value like String, long, or double, or can represent a Java object. Some metrics represent Histograms.

And then there are different ways to export the metrics — what we call exporters. To put it another way, the exporter are different ways you can access the metrics. Each exporter always gives access to all available metrics.

Ignite includes the following exporters:

  • JMX (default)

  • SQL Views

  • Log files

  • OpenCensus

You can create a custom exporter by implementing the MetricExporterSpi interface.

Metric Registers

Metrics are grouped into categories (called registers). Each register has a name. The full name of a specific metric within the register consists of the register name followed by a dot, followed by the name of the metric: <register_name>.<metric_name>. For example, the register for data storage metrics is called io.datastorage. The metric that return the storage size is called io.datastorage.StorageSize.

The list of all registers and the metrics they contain are described here.

Metric Exporters

If you want to enable metrics, configure one or multiple metric exporters in the node configuration. This is a node-specific configuration, which means it enables metrics only on the node where it is specified.

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="metricExporterSpi">
        <list>
            <bean class="org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi"/>
            <bean class="org.apache.ignite.spi.metric.log.LogExporterSpi"/>
            <bean class="org.apache.ignite.spi.metric.opencensus.OpenCensusMetricExporterSpi"/>
        </list>
    </property>
</bean>
IgniteConfiguration cfg = new IgniteConfiguration();

// Change metric exporter.
cfg.setMetricExporterSpi(new LogExporterSpi());

Ignite ignite = Ignition.start(cfg);
This API is not presently available for C++. You can use XML configuration.

The following sections describe the exporters available in Ignite by default.

JMX

org.apache.ignite.spi.metric.jmx.JmxMetricExporterSpi exposes metrics via JMX beans.

IgniteConfiguration cfg = new IgniteConfiguration();

// Create configured JMX metrics exporter.
JmxMetricExporterSpi jmxExporter = new JmxMetricExporterSpi();

//export cache metrics only
jmxExporter.setExportFilter(mreg -> mreg.name().startsWith("cache."));

cfg.setMetricExporterSpi(jmxExporter);
This API is not presently available for C++.
Note

This exporter is enabled by default if nothing is set with IgniteConfiguration.setMetricExporterSpi(…​).

IgniteConfiguration cfg = new IgniteConfiguration();

// Disable default JMX metrics exporter. Also could be disabled with the system property
// '-DIGNITE_MBEANS_DISABLED=true' or by setting other metrics exporter.
cfg.setMetricExporterSpi(new NoopMetricExporterSpi());
This API is not presently available for C++.

SQL View

SqlViewMetricExporterSpi is enabled by default, SqlViewMetricExporterSpi exposes metrics via the SYS.METRICS view. Each metric is displayed as a single record. You can use any supported SQL tool to view the metrics:

> select name, value from SYS.METRICS where name LIKE 'cache.myCache.%';
+-----------------------------------+--------------------------------+
|                NAME               |             VALUE              |
+-----------------------------------+--------------------------------+
| cache.myCache.CacheTxRollbacks    | 0                              |
| cache.myCache.OffHeapRemovals     | 0                              |
| cache.myCache.QueryCompleted      | 0                              |
| cache.myCache.QueryFailed         | 0                              |
| cache.myCache.EstimatedRebalancingKeys | 0                         |
| cache.myCache.CacheEvictions      | 0                              |
| cache.myCache.CommitTime          | [J@2eb66498                    |
....

Log

org.apache.ignite.spi.metric.log.LogExporterSpi prints the metrics to the log file at regular intervals (1 min by default) at INFO level.

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:util="http://www.springframework.org/schema/util" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="         http://www.springframework.org/schema/beans         http://www.springframework.org/schema/beans/spring-beans.xsd         http://www.springframework.org/schema/util         http://www.springframework.org/schema/util/spring-util.xsd">
    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="metricExporterSpi">
            <list>
                <bean class="org.apache.ignite.spi.metric.log.LogExporterSpi"/>
            </list>
        </property>
    </bean>
</beans>

If you use programmatic configuration, you can change the print frequency as follows:

IgniteConfiguration cfg = new IgniteConfiguration();

LogExporterSpi logExporter = new LogExporterSpi();
logExporter.setPeriod(600_000);

//export cache metrics only
logExporter.setExportFilter(mreg -> mreg.name().startsWith("cache."));

cfg.setMetricExporterSpi(logExporter);

Ignite ignite = Ignition.start(cfg);

OpenCensus

org.apache.ignite.spi.metric.opencensus.OpenCensusMetricExporterSpi adds integration with the OpenCensus library.

To use the OpenCensus exporter:

  1. Enable the 'ignite-opencensus' module.

  2. Add org.apache.ignite.spi.metric.opencensus.OpenCensusMetricExporterSpi to the list of exporters in the node configuration.

  3. Configure OpenCensus StatsCollector to export to a specific system. See OpenCensusMetricsExporterExample.java for an example and OpenCensus documentation for additional information.

Configuration parameters:

  • filter - predicate that filters metrics.

  • period - export period.

  • sendInstanceName - if enabled, a tag with the Ignite instance name is added to each metric.

  • sendNodeId - if enabled, a tag with the Ignite node id is added to each metric.

  • sendConsistentId - if enabled, a tag with the Ignite node consistent id is added to each metric.

Histograms

The metrics that represent histograms are available in the JMX exporter only. Histogram metrics are exported as a set of values where each value corresponds to a specific bucket and is available through a separate JMX bean attribute. The attribute names of a histogram metric have the following format:

{metric_name}_{low_bound}_{high_bound}

where

  • {metric_name} - the name of the metric.

  • {low_bound} - start of the bound. 0 for the first bound.

  • {high_bound} - end of the bound. inf for the last bound.

Example of the metric names if the bounds are [10,100]:

  • histogram_0_10 - less than 10.

  • histogram_10_100 - between 10 and 100.

  • histogram_100_inf - more than 100.