flowserv.controller.worker.base module

Base class for workers that execute workflow steps in different environments. Implementations of the base class may execute workflow commands using the Docker engine or the Python subprocess package.

class flowserv.controller.worker.base.ContainerWorker(variables: Optional[Dict] = None, env: Optional[Dict] = None, identifier: Optional[str] = None, volume: Optional[str] = None)

Bases: flowserv.controller.worker.base.Worker

Execution engine for container steps in a serial workflow. Provides the functionality to expand arguments in the individual command statements. Implementations may differ in the run method that executes the expanded commands.

exec(step: flowserv.model.workflow.step.ContainerStep, context: Dict, store: flowserv.volume.fs.FileSystemStorage) → flowserv.controller.serial.workflow.result.ExecResult

Execute a given list of commands that are represented by template strings.

Substitutes parameter and template placeholder occurrences first. Then calls the implementation-specific run method to execute the individual commands.

Note that the container worker expects a file system storage volume.

Parameters

  • step (flowserv.controller.serial.workflow.ContainerStep) – Step in a serial workflow.

  • context (dict) – Dictionary of argument values for parameters in the template.

  • store (flowserv.volume.fs.FileSystemStorage) – Storage volume that contains the workflow run files.

Return type

flowserv.controller.serial.workflow.result.ExecResult

abstract run(step: flowserv.model.workflow.step.ContainerStep, env: Dict, rundir: str) → flowserv.controller.serial.workflow.result.ExecResult

Execute a list of commands in a workflow step.

Parameters

  • step (flowserv.controller.serial.workflow.ContainerStep) – Step in a serial workflow.

  • env (dict, default=None) – Default settings for environment variables when executing workflow steps. May be None.

  • rundir (string) – Path to the working directory of the workflow run.

Return type

flowserv.controller.serial.workflow.result.ExecResult

class flowserv.controller.worker.base.Worker(identifier: Optional[str] = None, volume: Optional[str] = None)

Bases: object

Worker to execute steps in a serial workflow. For each class of workflow steps a separate worker can be implemented to execute instances of that particular step type.

abstract exec(step: flowserv.model.workflow.step.WorkflowStep, context: Dict, store: flowserv.volume.base.StorageVolume) → flowserv.controller.serial.workflow.result.ExecResult

Execute a given workflow step in the current workflow context.

Parameters

  • step (flowserv.model.workflow.step.WorkflowStep) – Step in a serial workflow.

  • context (dict) – Dictionary of variables that represent the current workflow state.

  • store (flowserv.volume.base.StorageVolume) – Storage volume that contains the workflow run files.

Return type

flowserv.controller.serial.workflow.result.ExecResult