Package org.apache.ignite.spi.deployment.uri

Class UriDeploymentSpi

  • All Implemented Interfaces:
    DeploymentSpi, IgniteSpi
    @IgniteSpiMultipleInstancesSupport(true)
@IgniteSpiConsistencyChecked(optional=false)
public class UriDeploymentSpi
extends IgniteSpiAdapter
implements DeploymentSpi
    Implementation of DeploymentSpi which can deploy tasks from different sources like file system folders, email and HTTP. There are different ways to deploy tasks in grid and every deploy method depends on selected source protocol. This SPI is configured to work with a list of URI's. Every URI contains all data about protocol/transport plus configuration parameters like credentials, scan frequency, and others.

    When SPI establishes a connection with an URI, it downloads deployable units to the temporary directory in order to prevent it from any changes while scanning. Use method setTemporaryDirectoryPath(String)) to set custom temporary folder for downloaded deployment units. SPI will create folder under the path with name identical to local node ID.

    SPI tracks all changes of every given URI. This means that if any file is changed or deleted, SPI will re-deploy or delete corresponding tasks. Note that the very first apply to findResource(String) is blocked until SPI finishes scanning all URI's at least once.

    There are several deployable unit types supported:

    • JAR file.
    • GAR file.
    • Local disk folder with structure of an unpacked deployment archive.
    • Local disk folder containing only compiled Java classes.

    Deployment package

    Deployment package can be represented as a regular JAR file with a specific structure similar to WAR format. Package files can have '.jar' or '.gar' extension.

    Package structure: 

          META-INF/
              |
               - ignite.xml
               - ...
      lib/
         |
          -some-lib.jar
          - ...
      xyz.class
      ...
    • META-INF/ entry may contain ignite.xml file which is a task descriptor file. The purpose of task descriptor XML file is to specify all tasks to be deployed. This file is a regular Spring XML definition file. META-INF/ entry may also contain any other file specified by JAR format.
    • lib/ entry contains all library dependencies.
    • Compiled Java classes must be placed in the root of a package file.
    A package may be deployed without a descriptor file. If there is no descriptor file, SPI will scan all classes in archive and instantiate those that implement ComputeTask interface. In that case, all grid task classes must have a public no-argument constructor. Use ComputeTaskAdapter adapter for convenience when creating grid tasks.

    By default, all downloaded packages that have digital signature in META-INF folder will be verified and deployed only if signature is valid.

    URI

    This SPI uses a hierarchical URI definition. For more information about standard URI syntax refer to java.net.URI documentation.
    [scheme:][//authority][path][?query] [#fragment]

    Every URI defines its own deployment repository which will be scanned for any changes. URI itself has all information about protocol, connectivity, scan intervals and other parameters.

    URI's may contain special characters, like spaces. If encodeUri flag is set to true (see setEncodeUri(boolean)), then URI 'path' field will be automatically encoded. By default this flag is set to true.

    Code Example

    The following example demonstrates how the deployment SPI can be used. It expects that you have a package file in 'home/username/ignite/work/my_deployment/file' folder which contains 'myproject.HelloWorldTask' class. 
     IgniteConfiguration cfg = new IgniteConfiguration();

 DeploymentSpi deploymentSpi = new UriDeploymentSpi();

 deploymentSpi.setUriList(Arrays.asList("file:///home/username/ignite/work/my_deployment/file"));

 cfg.setDeploymentSpi(deploymentSpi);

 try (Ignite ignite = Ignition.start(cfg)) {
     ignite.compute().execute("myproject.HelloWorldTask", "my args");
 }

    Configuration

    UriDeploymentSpi has the following optional configuration parameters (there are no mandatory parameters):

    Protocols

    Following protocols are supported by this SPI out of the box: Custom Protocols.

    You can add support for additional protocols if needed. To do this implement UriDeploymentScanner interface and plug your implementation into the SPI via setScanners(UriDeploymentScanner...) method.

    In addition to SPI configuration parameters, all necessary configuration parameters for selected URI should be defined in URI. Different protocols have different configuration parameters described below. Parameters are separated by ';' character.

    File

    For this protocol SPI will scan folder specified by URI on file system and download any deplloyment files or directories that end with .jar or .gar from source directory defined in URI. For file system URI must have scheme equal to file.

    Following parameters are supported:

    Parameter Description Optional Default
    freq Scanning frequency in milliseconds. Yes 5000 ms specified in UriDeploymentFileScanner.DFLT_SCAN_FREQ.

    File URI Example

    The following example will scan 'c:/Program files/ignite/deployment' folder on local box every '1000' milliseconds. Note that since path has spaces, setEncodeUri(boolean) parameter must be set to true (which is default behavior).
    file://freq=2000@localhost/c:/Program files/ignite/deployment

    HTTP/HTTPS

    URI deployment scanner tries to read DOM of the html it points to and parses out href attributes of all <a> tags - this becomes the collection of URLs to deployment package files that should be deployed. It's important that HTTP scanner uses URLConnection.getLastModified() method to check if there were any changes since last iteration for each deployment file before redeploying.

    Following parameters are supported:

    Parameter Description Optional Default
    freq Scanning frequency in milliseconds. Yes 300000 ms specified in UriDeploymentHttpScanner.DFLT_SCAN_FREQ.

    HTTP URI Example

    The following example will download the page `www.mysite.com/ignite/deployment`, parse it and download and deploy all deployment packages specified by href attributes of <a> elements on the page using authentication 'username:password' every '10000' milliseconds (only new/updated packages).
    http://username:password;freq=10000@www.mysite.com:110/ignite/deployment

    Java Example

    UriDeploymentSpi needs to be explicitly configured to override default local deployment SPI. 
     UriDeploymentSpi deploySpi = new UriDeploymentSpi();

 IgniteConfiguration cfg = new IgniteConfiguration();

 // Set URIs.
 deploySpi.setUriList(Arrays.asList("http://www.site.com/tasks",
     "file://freq=20000@localhost/c:/Program files/gg-deployment"));

 // Override temporary directory path.
 deploySpi.setTemporaryDirectoryPath("c:/tmp/grid");

 //  Override default deployment SPI.
 cfg.setDeploymentSpi(deploySpi);

 //  Start grid.
 Ignition.start(cfg);

    Spring Example

    UriDeploymentSpi can be configured from Spring XML configuration file: 
     <bean id="grid.custom.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" singleton="true">
         ...
         <property name="deploymentSpi">
             <bean class="org.apache.ignite.grid.spi.deployment.uri.UriDeploymentSpi">
                 <property name="temporaryDirectoryPath" value="c:/tmp/grid"/>
                 <property name="uriList">
                     <list>
                         <value>http://www.site.com/tasks</value>
                         <value>file://freq=20000@localhost/c:/Program files/gg-deployment</value>
                     </list>
                 </property>
             </bean>
         </property>
         ...
 </bean>


    For information about Spring framework visit www.springframework.org

    See Also:
    DeploymentSpi

    • Field Detail

      • DFLT_DEPLOY_DIR

        public static final String DFLT_DEPLOY_DIR
        Default deployment directory where SPI will pick up packages. Note that this path is relative to IGNITE_HOME/work folder if IGNITE_HOME system or environment variable specified, otherwise it is relative to work folder under system java.io.tmpdir folder.
        See Also:
        IgniteConfiguration.getWorkDirectory(), Constant Field Values

      • XML_DESCRIPTOR_PATH

        public static final String XML_DESCRIPTOR_PATH
        Default task description file path and name (value is META-INF/ignite.xml).
        See Also:
        Constant Field Values

    • Constructor Detail

      • UriDeploymentSpi

        public UriDeploymentSpi()

    • Method Detail

      • setTemporaryDirectoryPath

        @IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setTemporaryDirectoryPath​(String tmpDirPath)
        Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.

        If not provided, default value is java.io.tmpdir system property value.

        Parameters:
        tmpDirPath - Temporary directory path.
        Returns:
        this for chaining.

      • setUriList

        @IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setUriList​(List<String> uriList)
        Sets list of URI which point to a directory with packages and which should be scanned by SPI for the new tasks.

        If not provided, default value is list with file://${IGNITE_HOME}/work/deployment/file element. Note that system property IGNITE_HOME must be set. For unknown IGNITE_HOME list of URI must be provided explicitly.

        Parameters:
        uriList - Package file URIs.
        Returns:
        this for chaining.

      • setCheckMd5

        @IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setCheckMd5​(boolean checkMd5)
        If set to true then SPI should exclude files with same md5s from deployment. Otherwise it should try to load new unit regardless to possible file duplication.
        Parameters:
        checkMd5 - new value for the property
        Returns:
        this for chaining.

      • isCheckMd5

        public boolean isCheckMd5()
        Gets checkMd5 property.
        Returns:
        value of the checkMd5 property.

      • setEncodeUri

        @IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setEncodeUri​(boolean encodeUri)
        Indicates that URI must be encoded before usage. Encoding means replacing all occurrences of space with '%20', percent sign with '%25' and semicolon with '%3B'.

        If not provided, default value is true.

        Parameters:
        encodeUri - true if every URI should be encoded and false otherwise.
        Returns:
        this for chaining.

      • getTemporaryDirectoryPath

        public String getTemporaryDirectoryPath()
        Gets temporary directory path.
        Returns:
        Temporary directory path.

      • getUriList

        public List<String> getUriList()
        Gets list of URIs that are processed by SPI.
        Returns:
        List of URIs.

      • setListener

        public void setListener​(@Nullable
                        @Nullable DeploymentListener lsnr)
        Sets or unsets deployment event listener. Grid implementation will use this listener to properly add and remove various deployments.
        Specified by:
        setListener in interface DeploymentSpi
        Parameters:
        lsnr - Listener for deployment events. null to unset the listener.

      • getScanners

        public org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner[] getScanners()
        Gets scanners.
        Returns:
        Scanners.

      • setScanners

        @IgniteSpiConfiguration(optional=true)
public UriDeploymentSpi setScanners​(org.apache.ignite.spi.deployment.uri.scanners.UriDeploymentScanner... scanners)
        Sets scanners.
        Parameters:
        scanners - Scanners.
        Returns:
        this for chaining.

      • spiStop

        public void spiStop()
             throws IgniteSpiException
        This method is called to stop SPI. After this method returns kernel assumes that this SPI is finished and all resources acquired by it are released.

        Note that this method can be called at any point including during recovery of failed start. It should make no assumptions on what state SPI will be in when this method is called.

        Specified by:
        spiStop in interface IgniteSpi
        Throws:
        IgniteSpiException - Thrown in case of any error during SPI stop.

      • spiStart

        public void spiStart​(String igniteInstanceName)
              throws IgniteSpiException
        This method is called to start SPI. After this method returns successfully kernel assumes that SPI is fully operational.
        Specified by:
        spiStart in interface IgniteSpi
        Parameters:
        igniteInstanceName - Name of Ignite instance this SPI is being started for (null for default Ignite instance).
        Throws:
        IgniteSpiException - Throws in case of any error during SPI start.

      • findResource

        @Nullable
public @Nullable DeploymentResource findResource​(String rsrcName)
        Finds class loader for the given class.
        Specified by:
        findResource in interface DeploymentSpi
        Parameters:
        rsrcName - Class name or class alias to find class loader for.
        Returns:
        Deployed class loader, or null if not deployed.

      • register

        public boolean register​(ClassLoader ldr,
                        Class<?> rsrc)
                 throws IgniteSpiException
        Registers a class loader with this SPI. This method exists to be able to add external class loaders to deployment SPI. Deployment SPI may also have its own class loaders. For example, in case of GAR deployment, every GAR file is loaded and deployed with a separate class loader maintained internally by the SPI.

        The array of classes passed in should be checked for presence of ComputeTaskName annotations. The classes that have this annotation should be accessible by this name from DeploymentSpi.findResource(String) method.

        Specified by:
        register in interface DeploymentSpi
        Parameters:
        ldr - Class loader to register.
        rsrc - Class that should be checked for aliases. Currently the only alias in the system is ComputeTaskName for task classes; in future, there may be others.
        Returns:
        True if resource was registered.
        Throws:
        IgniteSpiException - If registration failed.

      • unregister

        public boolean unregister​(String rsrcName)
        Unregisters all class loaders that have a class with given name or have a class with give ComputeTaskName value.
        Specified by:
        unregister in interface DeploymentSpi
        Parameters:
        rsrcName - Either class name or ComputeTaskName value for a class whose class loader needs to be unregistered.
        Returns:
        True if resource was unregistered.