🚨 Deprecation 🚨
This repository was merged into the CloWM backend and will NOT be developed further.
Everything is in read-only mode.
CloWM Workflow Service
Description
This is the Workflow service of the CloWM service.
Configuration
General
| Env variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
CLOWM_CONFIG_FILE_YAML |
- | config.yaml |
Filepath | /path/to/config.yaml |
Path to a YAML file to read the config. See example-config/example-config.yaml. |
CLOWM_CONFIG_FILE_TOML |
- | config.toml |
Filepath | /path/to/config.toml |
Path to a TOML file to read the config. See example-config/example-config.toml. |
CLOWM_CONFIG_FILE_JSON |
- | config.json |
Filepath | /path/to/config.json |
Path to a JSON file to read the config. See example-config/example-config.json. |
CLOWM_API_PREFIX |
api_prefix |
unset | URI path | /api |
Prefix before every URL path |
DEV_SYSTEM |
dev_system |
false |
Boolean | false |
Activates an endpoint that allows execution of an workflow from an arbitrary Git Repository. HAS TO BE False in PRODUCTION! |
* CLOWM_UI_URI
|
ui_uri |
unset | HTTP URL | https://localhost |
HTTP URL of the CloWM website |
Database
| Env variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
CLOWM_DB__HOST |
db.host |
localhost |
<db hostname / IP> | localhost |
IP or Hostname Address of DB |
CLOWM_DB__PORT |
db.port |
3306 | Integer | 3306 | Port of the database |
* CLOWM_DB__USER
|
db.user |
unset | String | db-user |
Username of the database user |
* CLOWM_DB__PASSWORD
|
db.password |
unset | String | db-password |
Password of the database user |
* CLOWM_DB__NAME
|
db.name |
unset | String | db-name |
Name of the database |
CLOWM_DB__VERBOSE |
db.verbose |
false |
Boolean | false |
Enables verbose SQL output. Should be false in production |
| Variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
CLOWM_SMTP__SERVER |
smtp.server |
unset | SMTP domain / console
|
localhost |
Hostname of SMTP server. If console, emails are printed to the console. If not set, emails are not sent. |
CLOWM_SMTP__PORT |
smtp.port |
587 | Integer | 587 | Port of the SMTP server |
CLOWM_SMTP__SENDER_EMAIL |
smtp.sender_email |
no-reply@clowm.com |
no-reply@clowm.com |
Email address from which the emails are sent. | |
CLOWM_SMTP__REPLY_EMAIL |
smtp.reply_email |
unset | clowm@example.org |
Email address in the Reply-To header. |
|
CLOWM_SMTP__CONNECTION_SECURITY |
smtp.connection_security |
unset |
starttls / ssl
|
starttls |
Connection security to the SMTP server. |
CLOWM_SMTP__LOCAL_HOSTNAME |
smtp.local_hostname |
unset | String | clowm.local |
Overwrite the local hostname from which the emails are sent. |
CLOWM_SMTP__CA_PATH |
smtp.ca_path |
unset | Filepath | /path/to/ca.pem |
Path to a custom CA certificate. |
CLOWM_SMTP__KEY_PATH |
smtp.key_path |
unset | Filepath | /path/to/key.pem |
Path to the CA key. |
CLOWM_SMTP__USER |
smtp.user |
unset | String | smtp-user |
Username to use for SMTP login. |
CLOWM_SMTP__PASSWORD |
smtp.password |
unset | String | smtp-password |
Password to use for SMTP login. |
S3
| Env variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
* CLOWM_S3__URI
|
s3.uri |
unset | HTTP URL | http://localhost |
URI of the S3 Object Storage |
* CLOWM_S3__ACCESS_KEY
|
s3.acess_key |
unset | String | ZR7U56KMK20VW |
Access key for the S3 that owns the buckets |
* CLOWM_S3__SECRET_KEY
|
s3.secret_key |
unset | String | 9KRUU41EGSCB3H9ODECNHW |
Secret key for the S3 that owns the buckets |
* CLOWM_S3__ADMIN_ACCESS_KEY
|
s3.admin_acess_key |
unset | String | ZR7U56KMK20VW |
Access key for the Ceph Object Gateway user with user=*,bucket=* capabilities. |
* CLOWM_S3__ADMIN_SECRET_KEY
|
s3.admin_secret_key |
unset | String | 9KRUU41EGSCB3H9ODECNHW |
Secret key for the Ceph Object Gateway user with user=*,bucket=* capabilities. |
CLOWM_S3__PARAMS_BUCKET |
s3.params_bucket |
clowm-workflow-params |
String | clowm-workflow-params |
Bucket where the workflow execution parameters should be saved |
CLOWM_S3__WORKFLOW_BUCKET |
s3.workflow_bucket |
clowm-workflows |
String | clowm-workflows |
Bucket where to cache workflow files |
CLOWM_S3__ICON_BUCKET |
s3.icon_bucket |
clowm-icons |
String | clowm-icons |
Bucket where to save workflow icons |
Security
| Env variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
* CLOWM_PUBLIC_KEY / CLOWM_PUBLIC_KEY_FILE
|
public_key / public_key_file
|
unset | Public Key / Path to Public Key | /path/to/key.pub |
Public part of RSA Key in PEM format to verify JWTs |
* CLOWM_OPA__URI
|
opa.uri |
unset | HTTP URL | http://localhost |
URI of the OPA Service |
Cluster
| Env variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
* CLOWM_CLUSTER__SLURM__URI
|
cluster.slurm.uri |
unset | HHTP Url | http://localhost |
HTTP URL to communicate with the Slurm cluster |
* CLOWM_CLUSTER__SLURM__TOKEN
|
cluster.slurm.token |
unset | String | - | JWT for communication with the Slurm REST API. |
* CLOWM_CLUSTER__SLURM__USER
|
cluster.slurm.user |
slurm |
String | slurm |
User on the slurm cluster who should run the job. Should be the user of the cluster.slurm.token
|
* CLOWM_CLUSTER__NXF_CONFIG
|
cluster.nxf_config |
unset | Path on slurm cluster | /path/to/nextflow.config |
Configuration file on the slurm cluster that is the same for every nextflow run |
* CLOWM_CLUSTER__NXF_BIN
|
cluster.nxf_bin |
nextflow |
Path on slurm cluster | /path/to/nextflow |
Path to the nextflow executable. Default it is in the PATH
|
* CLOWM_CLUSTER__WORKING_DIRECTORY
|
cluster.working_directory |
/tmp |
Path on slurm cluster | /path/to/directory |
Working directory for the slurm job with the nextflow command |
* CLOWM_CLUSTER__ACTIVE_WORKFLOW_EXECUTION_LIMIT
|
cluster.active_workflow_execution_limit |
3 |
Integer | 3 |
Limit of active workflow execution a user is allowed to have. -1 means infinite |
* CLOWM_CLUSTER__JOB_MONITORING
|
cluster.job_monitoring |
EXPONENTIAL |
EXPONENTIAL,LINEAR,CONSTANT,NOMONITORING |
EXPONENTIAL |
Strategy for polling the slurm job status for monitoring the workflow execution |
* CLOWM_CLUSTER__EXECUTION_CLEANUP
|
cluster.execution_cleanup |
true |
Boolean | true |
Enable/Disable the cleanup after a workflow execution |
Monitoring
| Env variable | Config file key | Default | Value | Example | Description |
|---|---|---|---|---|---|
CLOWM_OTLP__GRPC_ENDPOINT |
otlp.grpc_endpoint |
unset | String | localhost |
OTLP compatible endpoint to send traces via gRPC, e.g. Jaeger. If unset, no traces are sent. |
CLOWM_OTLP__SECURE |
otlp.secure |
false |
Boolean | false |
Connection type |
Nextflow Variables
All environment variables with the prefix WORKFLOW_ENV_ are included in the environment of the nextflow job of the
cluster with the prefix cut off.
Example:
Pin the nextflow version
WORKFLOW_ENV_NXF_VER=23.04.0 -> NXF_VER=23.04.0
License
The API is licensed under the Apache 2.0 license. See the License file for more information.