flowserv.model.files module

Classes for files that are part of workflow run inputs and outputs. Information about these files that are maintained on storage volumes is also stored in the database.

This module also defines the folder structure on the storage volume for workflows and their associated resources.

The folder structure is currently as follows:

/workflows/                  : Base directory
    {workflow_id}            : Folder for individual workflow
        groups/              : Folder for workflow groups
            {group_id}       : Folder for individual group
                files/       : Uploaded files for workflow group
                    {file_id}: Folder for uploaded file
        runs/                : Folder for all workflow runs
            {run_id}         : Result files for individual runs
        static/

class flowserv.model.files.FileHandle(name: str, mime_type: str, fileobj: flowserv.volume.base.IOHandle)

Bases: flowserv.volume.base.IOHandle

Handle for a file that is stored in the database. Extends the file object with the base file name and the mime type.

The implementation is a wrapper around a file object to make the handle agnostic to the underlying storage mechanism.

open() → IO

Get an BytesIO buffer containing the file content. If the associated file object is a path to a file on disk the file is being read.

Return type

io.BytesIO

Raises

flowserv.error.UnknownFileError –

size() → int

Get size of the file in the number of bytes.

Return type

int

flowserv.model.files.group_uploaddir(workflow_id: str, group_id: str) → str

Get base directory for files that are uploaded to a workflow group.

Parameters

• workflow_id (string) – Unique workflow identifier

• group_id (string) – Unique workflow group identifier

Return type

string

flowserv.model.files.io_file(data: Union[List, Dict], format: Optional[str] = None) → flowserv.volume.base.IOBuffer

Write simple text to given bytes buffer.

Parameters

• data (list or dict) – List of strings or dictionary.

• format (string, default=None) – Format identifier.

Return type

flowserv.volume.base.IOBuffer

flowserv.model.files.run_basedir(workflow_id: str, run_id: str) → str

Get path to the base directory for all files that are maintained for a workflow run.

Parameters

• workflow_id (string) – Unique workflow identifier

• run_id (string) – Unique run identifier

Return type

string

flowserv.model.files.run_tmpdir() → str

Get path to a temporary workflow run directory.

Return type

string

flowserv.model.files.workflow_basedir(workflow_id: str) → str

Get base directory containing associated files for the workflow with the given identifier.

Parameters

workflow_id (string) – Unique workflow identifier

Return type

string

flowserv.model.files.workflow_groupdir(workflow_id: str, group_id: str) → str

Get base directory containing files that are associated with a workflow group.

Parameters

• workflow_id (string) – Unique workflow identifier

• group_id (string) – Unique workflow group identifier

Return type

string

flowserv.model.files.workflow_staticdir(workflow_id: str) → str

Get base directory containing static files that are associated with a workflow template.

Parameters

workflow_id (string) – Unique workflow identifier

Return type

string