Package org.apache.ignite.spi.checkpoint.sharedfs

Class SharedFsCheckpointSpi

  • All Implemented Interfaces:
    CheckpointSpi, IgniteSpi
    @IgniteSpiMultipleInstancesSupport(true)
@IgniteSpiConsistencyChecked(optional=false)
public class SharedFsCheckpointSpi
extends IgniteSpiAdapter
implements CheckpointSpi
    This class defines shared file system CheckpointSpi implementation for checkpoint SPI. All checkpoints are stored on shared storage and available for all nodes in the grid. Note that every node must have access to the shared directory. The reason the directory needs to be shared is because a job state can be saved on one node and loaded on another (e.g. if a job gets preempted on a different node after node failure). When started, this SPI tracks all checkpoints saved by localhost for expiration. Note that this SPI does not cache data stored in checkpoints - all the data is loaded from file system on demand.

    Directory paths for shared checkpoints should either be empty or contain previously stored checkpoint files.

    Configuration

    Mandatory

    This SPI has no mandatory configuration parameters.

    Optional

    This SPI has following optional configuration parameters:

    Java Example

    SharedFsCheckpointSpi can be configured as follows: 
     IgniteConfiguration cfg = new IgniteConfiguration();

 SharedFsCheckpointSpi checkpointSpi = new SharedFsCheckpointSpi();

 // List of checkpoint directories where all files are stored.
 Collection dirPaths = new ArrayList();

 dirPaths.add("/my/directory/path");
 dirPaths.add("/other/directory/path");

 // Override default directory path.
 checkpointSpi.setDirectoryPaths(dirPaths);

 // Override default checkpoint SPI.
 cfg.setCheckpointSpi(checkpointSpi);

 // Starts grid.
 G.start(cfg);

    Spring Example

    SharedFsCheckpointSpi can be configured from Spring XML configuration file: 
     <bean id="grid.custom.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" singleton="true">
     ...
     <property name="checkpointSpi">
         <bean class="org.apache.ignite.spi.checkpoint.sharedfs.GridSharedFsCheckpointSpi">
             <!-- Change to shared directory path in your environment. -->
             <property name="directoryPaths">
                 <list>
                     <value>/my/directory/path</value>
                     <value>/other/directory/path</value>
                 </list>
             </property>
         </bean>
     </property>
     ...
 </bean>


    For information about Spring framework visit www.springframework.org

    See Also:
    CheckpointSpi

    • Field Detail

      • DFLT_DIR_PATH

        public static final String DFLT_DIR_PATH
        Default checkpoint directory. Note that this path is relative to IGNITE_HOME/work folder if IGNITE_HOME system or environment variable specified, otherwise it is relative to work folder under system java.io.tmpdir folder.
        See Also:
        IgniteConfiguration.getWorkDirectory(), Constant Field Values

    • Constructor Detail

      • SharedFsCheckpointSpi

        public SharedFsCheckpointSpi()
        Initializes default directory paths.

    • Method Detail

      • getDirectoryPaths

        public Collection<String> getDirectoryPaths()
        Gets collection of all configured paths where checkpoints can be saved.
        Returns:
        Collection of all configured paths.

      • getCurrentDirectoryPath

        public String getCurrentDirectoryPath()
        Gets path to the directory where all checkpoints are saved.
        Returns:
        Path to the checkpoints directory.

      • setDirectoryPaths

        @IgniteSpiConfiguration(optional=true)
public SharedFsCheckpointSpi setDirectoryPaths​(Collection<String> dirPaths)
        Sets path to a shared directory where checkpoints will be stored. The path can either be absolute or relative to IGNITE_HOME system or environment variable.

        If not provided, default value is DFLT_DIR_PATH.

        Parameters:
        dirPaths - Absolute or Ignite installation home folder relative path where checkpoints will be stored.
        Returns:
        this for chaining.

      • spiStart

        public void spiStart​(String igniteInstanceName)
              throws IgniteSpiException
        This method is called to start SPI. After this method returns successfully kernel assumes that SPI is fully operational.
        Specified by:
        spiStart in interface IgniteSpi
        Parameters:
        igniteInstanceName - Name of Ignite instance this SPI is being started for (null for default Ignite instance).
        Throws:
        IgniteSpiException - Throws in case of any error during SPI start.

      • spiStop

        public void spiStop()
             throws IgniteSpiException
        This method is called to stop SPI. After this method returns kernel assumes that this SPI is finished and all resources acquired by it are released.

        Note that this method can be called at any point including during recovery of failed start. It should make no assumptions on what state SPI will be in when this method is called.

        Specified by:
        spiStop in interface IgniteSpi
        Throws:
        IgniteSpiException - Thrown in case of any error during SPI stop.

      • loadCheckpoint

        public byte[] loadCheckpoint​(String key)
                      throws IgniteSpiException
        Loads checkpoint from storage by its unique key.
        Specified by:
        loadCheckpoint in interface CheckpointSpi
        Parameters:
        key - Checkpoint key.
        Returns:
        Loaded data or null if there is no data for a given key.
        Throws:
        IgniteSpiException - Thrown in case of any error while loading checkpoint data. Note that in case when given key is not found this method will return null.

      • saveCheckpoint

        public boolean saveCheckpoint​(String key,
                              byte[] state,
                              long timeout,
                              boolean overwrite)
                       throws IgniteSpiException
        Saves checkpoint to the storage.
        Specified by:
        saveCheckpoint in interface CheckpointSpi
        Parameters:
        key - Checkpoint unique key.
        state - Saved data.
        timeout - Every intermediate data stored by checkpoint provider should have a timeout. Timeout allows for effective resource management by checkpoint provider by cleaning saved data that are not needed anymore. Generally, the user should choose the minimum possible timeout to avoid long-term resource acquisition by checkpoint provider. Value 0 means that timeout will never expire.
        overwrite - Whether or not overwrite checkpoint if it already exists.
        Returns:
        true if checkpoint has been actually saved, false otherwise.
        Throws:
        IgniteSpiException - Thrown in case of any error while saving checkpoint data.

      • removeCheckpoint

        public boolean removeCheckpoint​(String key)
        This method instructs the checkpoint provider to clean saved data for a given key.
        Specified by:
        removeCheckpoint in interface CheckpointSpi
        Parameters:
        key - Key for the checkpoint to remove.
        Returns:
        true if data has been actually removed, false otherwise.