Smart Executor Service Documentation¶

The Smart Executor is a gCube service that runs tasks on behalf of the infrastructure: the management, monitoring and harvesting activities that must run once, after a delay or on a recurring schedule. Each kind of task is a plugin, a jar deployed with the service; the service exposes the plugins it holds and lets a client launch them, follow their executions and stop them, through a REST API.

The REST API is described by the OpenAPI document of the service, shown by ReDoc and Swagger UI from the menu of these pages.

Concepts¶

Plugins¶

A plugin is identified by its name and version, and declares the inputs it accepts and its capabilities. GET /plugins lists the plugins the instance holds.

Launching an Execution¶

A client launches a plugin with POST /plugins/{name}/executions, sending a launch parameter:

Field	Meaning
`pluginName`	The plugin to run; it must be the one named in the URL
`pluginVersion`	The version of the plugin; absent means any version
`pluginCapabilities`	The capabilities the plugin must declare, or it is not launched; absent means no constraint
`inputs`	The inputs passed to the plugin
`pluginStateNotifications`	The notifiers to call at every change of state
`scheduling`	Absent for a single immediate run; otherwise the scheduling below

The service answers with the UUID of the scheduled task, which identifies its executions in every later request.

Scheduling¶

Field	Meaning
`cronExpression`	A cron-like expression for a recurring task; not used together with `delay`
`delay`	The seconds between subsequent executions; not used together with `cronExpression`
`schedulingTimes`	How many times the pattern is applied; `0` means indefinitely
`firstStartTime`	The first instant the scheduling can start; absent or `0` means immediately
`endTime`	The instant after which no execution starts; absent or `0` means never
`previuosExecutionsMustBeCompleted`	With a cron expression, an execution fired while the previous one is still running is discarded
`global`	The task is taken over by another Smart Executor instance if the one running it stops

States of an Execution¶

State	Final	Meaning
`CREATED`	No	The execution is created and not yet running
`RUNNING`	No	The execution is running
`STOPPED`	Yes	The execution was stopped
`DONE`	Yes	The execution terminated successfully
`FAILED`	Yes	The execution failed
`DISCARDED`	Yes	The scheduler discarded the execution because the previous one was still running

GET /plugins/{name}/executions/{uuid} returns the state of the last execution of a task, or of the one given by iteration.

Stopping and Unscheduling¶

DELETE /plugins/{name}/executions/{uuid} stops the running execution of a task and keeps its scheduling; with unschedule=true, or with the PURGE method on the same URL, it also removes the scheduling, the global one included.

Orphan Tasks¶

GET /plugins/@orphan/executions lists the scheduled tasks whose Smart Executor instance is no longer running: it does not know the task, does not answer as a Smart Executor, its host name does not resolve or it refuses the connection. A Smart Executor that starts takes in charge the orphan tasks of its plugins. Any other failure, e.g. a timeout, leaves the task to its instance, so that a task never runs twice.

Statistics¶

The node gives the public statistics of the service, as totals only, in the group executor at /smartgears/smart-executor/statistics, with no token. They are computed every 15 minutes:

Figure	What it counts
`plugins`	The plugins installed on the node
`scheduledTasks`	The scheduled tasks
`scheduledTasksByKind`	The scheduled tasks, recurrent (cron expression or delay) or one-time
`executions`	The executions in the database by final state (`DONE`, `FAILED`, `STOPPED`, `DISCARDED`); purged executions are not counted
`executionsLast24h`	The executions that reached their final state in the last 24 hours, by final state

Authorization¶

D4Science uses OpenID Connect for authentication and UMA 2 for authorization. Every request carries a JWT access token in the Authorization: Bearer header; the Context of the token is the Context the task runs in. A Bearer token can be obtained following How to access resources.

The task keeps the Context of the request and the identifier of its caller, never its token. Every execution, immediate or scheduled, runs in that Context: the plugin runs with the token of its own client, and the service records the states of the execution with the token of the node.

Configuration of a Plugin¶

Every plugin has its configuration in WEB-INF/lib/plugins/<plugin name>/<plugin name>.yaml, written by the provisioning. SmartGears reads it when the service starts, and the Smart Executor gives it to the plugin at every execution (Plugin.getConfiguration()): a plugin never looks for its configuration in the classpath, which the plugins of a node share. The file holds at least the client of the plugin:

client_id: "smart-executor-plugin-hello-world"
client_secret: "..."

and then the keys of the plugin itself. An execution of a plugin whose file is missing, or holds no client, ends FAILED.

HTTP Statuses¶

Status	When
`200 OK`	The request succeeded and the response carries content
`204 No Content`	The execution was stopped
`400 Bad Request`	Missing or invalid inputs, e.g. a launch parameter naming another plugin
`401 Unauthorized`	No token, or a token that is not valid
`404 Not Found`	The plugin or the execution does not exist
`500 Internal Server Error`	A failure of the service

A 500 Internal Server Error that persists after retrying the request is reported in the gCube ticketing system, with the request, how to reproduce it and the time it happened.

Service Discovery¶

The service is published in the Information System as an EService. It is found with this JSON query:

{
  "type": "EService",
  "consistsOf": [
    {
      "type": "IsIdentifiedBy",
      "target": {
        "type": "SoftwareFacet",
        "group": "org.gcube.vremanagement",
        "name": "smart-executor"
      }
    }
  ]
}

SmartGears publishes every plugin the instance holds as a RunningPlugin related to the EService by Activates, with a stable identifier and the name of the plugin (its Maven artifactId). /smartgears/smart-executor/plugins shows the plugins SmartGears found.

The service keeps its scheduled tasks and the states of their executions in an OrientDB database of its own. It finds the address and the credentials of the database in the Information System, in the Context of the task: the AccessPointFacet of the EService identified by the SoftwareFacet org.orientdb / orientdb and classified by the ClassificationFacet Database / SmartExecutor.

Maven Coordinates¶

<groupId>org.gcube.vremanagement</groupId>
<artifactId>smart-executor</artifactId>