Skip to content

feat: add Snakemake profiles

BIOPZ-Bak Maciej requested to merge profiles into dev

Replace --cluster-config CL snakemake parameter with new a mechanism: profiles.

Snakemake profiles are, in structure, directories which encapsulate all the information required for snakemake call (all CL params as well as cluster-specific cluster configuration files, job submission and job status query wrappers).

More info:

Having a set of defined profiles snakemake call for zarp is reduced to (e.g.):

snakemake \
    --profile="../../profiles/slurm-conda" \
    --configfile="../input_files/config.yaml"

Notable mentions:

  • ${PWD} is not properly parsed from a profile YAML in one of the rules, we need to use relative paths within singularity-args: --bind arguments.
  • cleaned unnecessary mapping keywords between sbatch params in the snakemake CL params and the content of cluster configuration file; now the cluster config JSON is dedicated to slurm and thus contains exact specific slurm keywords.
  • Running the workflow on your own samples section in the README changed significantly since we now provide the profiles instead of the user having to create them on their own.

TODO

  • Wait for the last test of mine to finish: conda+slurm
  • Update manual test commands in the README
Edited by BIOPZ-Bak Maciej

Merge request reports