- clean up command line interface
  - improve descriptions
  - add consistent structure
  - remove or merge superfluous CLI arguments
  - set defaults
  - update test calls
  - update docs
  - when importing data from LabKey, table is saved to 'samples.tsv.labkey' in same directory as Snakemake sample table
- allow user to specify environment variables and relative paths in input table and on CLI
  - relative paths in the input table are interpreted with respect to the directory containing the input table
  - relative paths will are interpreted with respect to the current working directory; this is to achieve portability with respect to tests but is discouraged in production because its behavior is not very predictable from the user's perspective; consequently a warning is thrown
- set STAR index size to read length - 1
- remove `gtf_filtered` and `tr_fasta_filtered` and update Snakefiles and test sample tables accordingly
- rename some MultiQC report-related parameters and update Snakefiles and test config files accordingly
- add logging
- add docstrings to module and all functions
- add typing definitions to all functions
- restructure and comment code to improve readability
- linters `flake8` and `mypy` pass

updated pipeline documentation

described seqmode specific rules

adapted to updates in Snakefile

moved star_rpm and samtools index to main snakefile description

linked README in documentation and vice versa

- log and, if workflow is executed on cluster, cluster log directories are explicitly created in `Snakefile`
- location of main log directory can be configured in `config.yaml` (field `log_dir`, previously: `local_log`; requires change in script `labkey_to_snakemake.py` as well as subworkflows as field name is hard-coded there)
- location of cluster log directory can be configured in `cluster.json` (in field `__default__` -> `out`)
- `config.yaml` and `cluster.json` in `tests/input_files` are set such that a directory `logs/` is created in the directory where Snakemake is run (i.e., the directory of each test); cluster logs are stored in a subdirectory `logs/cluster`
- removes instructions to explicitly create log directories from docs and all test scripts
- cleans up main `Snakefile` (apart from Snakemake-specific syntax, now passes `flake8` linter test)

- add 3 separate Conda environment files for installing
  - essential dependencies without Singularity (most users)
  - essential dependencies including Singularity (root users on Linux only)
  - non-essential dependencies (required for tests etc.)
- install Python dependencies via `pip` _within_ Conda environment to avoid #56
- update Conda-based installation instructions
  - describe usage of different environment files
  - add supported OS versions (macOS not yet added, because only installation works)
- update CI config to use same setup (based on Miniconda Docker image)

BIOPZ-Katsantoni Maria authored 5 years ago and Alex Kanitz committed 5 years ago

- add script that prepares Snakemake input files 'samples.tsv' and 'config.yaml' from LabKey table
- script either connects to API directly (with '--remote' and related options) or processes a tab-separated LabKey dump file
- add tests for both use cases
- common input files for tests now in 'tests/input_files'
- update all other tests to account for new file locations
- update documentation

CJHerrmann authored 5 years ago and Alex Kanitz committed 5 years ago

- describe installation of Singularity inside `conda` environment when user has root access
- put clone repo section before install section

CJHerrmann authored 5 years ago and Alex Kanitz committed 5 years ago

- add script `tests/test_rule_graph/test.sh` to generate a rule graph in `images/rule_graph.svg`
- display rule graph created in `README.md` instead of specific workflow DAG
- add test script to GitLab CI config
- renamed test to create workflow DAG from `test_create_dag_chart` to `test_create_dag_image` (also output file is renamed from `images/workflow_dag.svg` to `images/dag_test_workflow.svg`

- remove log files and add '.snakemake' directories to '.gitignore'
- update wrong link in 'README.md'
- delete superfluous script documentation 'scripts/labkey_api.md'
- add Snakemake-specific file extension '.smk' to subworkflows
- remove non-deterministic workflow output from md5 sums

`README.md` file describes
- aim and background of the project (including the workflow DAG representation)
- how to install requirements (including setting up a `conda` environment for the project)
- how to execute the workflow run integration test
- how to run the workflow on your own samples (including how to auto-generate required params from LabKey metadata)

Additional minor changes:
- minor changes in various test and related files, including updates of paths
- root directory now includes subdirectory `runs/` for a user's workflow runs (contents not version-controlled)