Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

License

NotificationsYou must be signed in to change notification settings

miniwdl-ext/miniwdl-slurm

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

73 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Extends miniwdl to run workflows on SLURM clusters in singularity containers.

ThisSLURM backend plugin forminiwdl runs WDL task containersby creating a job script that is submitted to a SLURM cluster. In case the jobdescription has a container, singularity will be used as container runtime.

Installation

For the latest stable version:

pip install miniwdl-slurm

For the development version:

pip install git+https://github.com/miniwdl-ext/miniwdl-slurm.git

Configuration

The followingminiwdl configurationexample can be used to use miniwdl on a SLURM cluster:

[scheduler]container_backend=slurm_singularity# task_concurrency defaults to the number of processors on the system.# since we submit the jobs to SLURM this is not necessary.# higher numbers means miniwdl has to monitor more processes simultaneously# which might impact performance.task_concurrency=200# This setting allows running tasks to continue, even if one other tasks fails.# Useful in combination with call caching. Prevents wasting resources by# cancelling jobs half-way that would probably succeed.fail_fast = false[call_cache]# The following settings create a call cache under the current directory.# This prevents wasting unnecessary resources on the cluster by rerunning# jobs that have already succeeded.put = trueget = truedir ="$PWD/miniwdl_call_cache"[task_runtime]# Setting a 'maxRetries' default allows jobs that fail due to intermittent# errors on the cluster to be retried.defaults = {"maxRetries": 2,"docker":"ubuntu:20.04"    }[singularity]# This plugin wraps the singularity backend. Make sure the settings are# appropriate for your cluster.exe = ["singularity"]# the miniwdl default options contain options to run as a fake root, which# is not available on most clusters.run_options = ["--containall"    ]# Location of the singularity images (optional). The miniwdl-slurm plugin# will set it to a directory inside $PWD. This location must be reachable# for the submit nodes.image_cache ="$PWD/miniwdl_singularity_cache"[slurm]# extra arguments passed to the srun command (optional).extra_args="--partition heavy_users,gpu --comment 'run with miniwdl'"

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages


[8]ページ先頭

©2009-2025 Movatter.jp