Performance Statistics
Overview
What Is Collected
Performance statistics includes the following records:
| Category | Records |
|---|---|
Cache operations |
Cache API and cache store operations: |
Cache metadata |
Cache start records with cache IDs and cache names |
Transactions |
Transaction commit and rollback records with affected cache IDs, start time, and duration |
Queries |
SQL, SCAN, and INDEX query records with query text or cache/index details, query ID, start time, duration, and success flag |
Query details |
Query logical and physical reads, processed rows, and custom query properties |
Tasks and jobs |
Compute task and job records with task names, session IDs, execution time, queue time, affinity partition, and timeout flag |
Persistence |
Checkpoint timing records and page write throttling records |
System views |
System view schemas and rows captured when statistics collection starts (some views are skipped, see the list below) |
The following system views are skipped because they are large or duplicate data that is already collected as performance records:
-
baseline.node.attributes -
node.attributes -
metrics -
caches -
sql.queries -
nodes -
cacheGroupPageLists -
dataRegionPageLists -
partitionStates -
statisticsPartitionData -
metastorage -
distributed.metastorage
Collecting Statistics
JMX interface and Control Script are used to manage statistics collecting.
Each node collects performance statistics in binary files. These files are placed under
the Ignite_work_directory/perf_stat/ directory. Regular performance events are written to
node-{nodeId}.prf. If a file with the same name already exists, Ignite adds an index to the file name:
node-{nodeId}-{index}.prf.
When statistics collection starts, each node also writes a snapshot of system views
(System Views) to node-{nodeId}-system-views.prf.
If a file with the same name already exists, Ignite adds an index:
node-{nodeId}-system-views-{index}.prf. The rotate operation rotates only regular performance event files.
|
Warning
|
Sensitive InformationPerformance statistics files can contain sensitive information, including query text, cache names, task names, node attributes, and values exposed by system views. Store and share these files according to your security policies. |
Performance statistics files are used to build the report offline.
For regular performance events, nodes use off-heap cyclic buffer to temporarily store serialized statistics. The writer thread flushes buffer to the file when the flush size is reached. Some statistics are skipped if the buffer overflows due to a slow disk. See the properties section for customization.
Each statistics collection process creates new files on nodes. Each next file has the same name with the corresponding index. See the examples below:
-
node-faedc6c9-3542-4610-ae10-4ff7e0600000.prf -
node-faedc6c9-3542-4610-ae10-4ff7e0600000-1.prf -
node-faedc6c9-3542-4610-ae10-4ff7e0600000-2.prf -
node-faedc6c9-3542-4610-ae10-4ff7e0600000-system-views.prf -
node-faedc6c9-3542-4610-ae10-4ff7e0600000-system-views-1.prf
Building the Report
Ignite provides a tool to generate the report from performance statistics files. The tool is published in the ignite-extensions repository as performance-statistics-ext extension.
Follow these steps to build the performance report:
-
Stop collecting statistics and place files from all nodes under an empty directory. For example:
/path_to_files/ ├── node-162c7147-fef8-4ea2-bd25-8653c41fc7fa.prf ├── node-162c7147-fef8-4ea2-bd25-8653c41fc7fa-system-views.prf ├── node-7b8a7c5c-f3b7-46c3-90da-e66103c00001.prf ├── node-7b8a7c5c-f3b7-46c3-90da-e66103c00001-system-views.prf ├── node-faedc6c9-3542-4610-ae10-4ff7e0600000.prf └── node-faedc6c9-3542-4610-ae10-4ff7e0600000-system-views.prf -
Run the script from the release package of the tool:
performance-statistics-tool/build-report.sh path_to_files
The performance report is created in the new directory under the performance statistics files
path: path_to_files/report_yyyy-MM-dd_HH-mm-ss/. Open report_yyyy-MM-dd_HH-mm-ss/index.html in the browser to see
the report.
You can also print raw performance statistics records to the console or to a file by using print-statistics.sh from
the performance-statistics-ext release package. For example, to print checkpoint timing and page write throttling
records:
performance-statistics-tool/print-statistics.sh path_to_files --ops CHECKPOINT,PAGES_WRITE_THROTTLE
To write the output to a file, use the --out parameter:
performance-statistics-tool/print-statistics.sh path_to_files --ops CHECKPOINT,PAGES_WRITE_THROTTLE --out checkpoints.json
Management
The following section provides information on JMX, Control Script and system properties.
JMX
You can manage the performance statistics collection via the PerformanceStatisticsMBean interface:
| Method | Description |
|---|---|
start() |
Start collecting performance statistics in the cluster. |
stop() |
Stop collecting performance statistics in the cluster. |
rotate() |
Rotate collecting performance statistics in the cluster. |
started() |
True if performance statistics collection is started. |
Control Script
You can manage the performance statistics collection via the Control Script.
control.sh --performance-statistics [start|stop|rotate|status]
control.bat --performance-statistics [start|stop|rotate|status]
Parameters:
| Parameter | Description |
|---|---|
|
Start collecting performance statistics in the cluster. |
|
Stop collecting performance statistics in the cluster. |
|
Rotate collecting performance statistics in the cluster. |
|
Get status of collecting performance statistics in the cluster. |
System properties
| Property | Type | Default Value | Description |
|---|---|---|---|
IGNITE_PERF_STAT_FILE_MAX_SIZE |
Long |
32 Gb |
Maximum performance statistics file size in bytes. Performance statistics collection is stopped when the file size is exceeded. |
IGNITE_PERF_STAT_BUFFER_SIZE |
Integer |
32 Mb |
Performance statistics off heap buffer size in bytes. |
IGNITE_PERF_STAT_FLUSH_SIZE |
Integer |
8 Mb |
Minimal performance statistics batch size to be flushed in bytes. |
IGNITE_PERF_STAT_CACHED_STRINGS_THRESHOLD |
Integer |
10240 |
Maximum performance statistics cached strings threshold. String caching is stopped when the threshold is exceeded. |
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.