PCA analysis on genes and transcripts TPM for kallisto and salmon

See merge request !77

- Delete unused files scripts/fg_extract_transcripts.py, scripts/heatmap_and_clustermap.py, scripts/perform_PCA.py
- Add rules (pca_kallisto, pca_salmon) that run zpca (https://github.com/zavolanlab/zpca) on genes and transcripts TPM tables from kallisto and salmon.
- The output is wired to multiqc_report but the plots are not visualized to multiqc. Update documentation.
- Update dag and rulegraph.
Fixes #140 #142

docs: revised terminology for non-experts

See merge request !73

Merge kallisto

See merge request !75

BIOPZ-Iborra de Toledo Paula authored 4 years ago and BIOPZ-Gypas Foivos committed 4 years ago

The rules rely on https://github.com/zavolanlab/merge_kallisto
Update info in pipeline_documentation.md

Cluster.json was updated

See merge request !74

feat: add support for multi-lane samples

See merge request !71

BIOPZ-Katsantoni Maria authored 4 years ago and Alex Kanitz committed 4 years ago

fix: Renamed samples_concat.tsv to samples.multiple_lanes.tsv. Renamed rows with split with the same name as the other test samples, so that I do not change the tests (md5 and sunch). Removed the one lane samples. Created config that uses this tsv file

refactor: rename LabKey/input table column

See merge request !70

Add Snakemake report

See merge request !64

docs: rename project/workflow

See merge request !69

fix(prepare_inputs): support relative paths

See merge request !68

Salmon argument fix

See merge request !67

BIOPZ-Katsantoni Maria authored 4 years ago and BIOPZ-Gypas Foivos committed 4 years ago

fix: added arguments for fragment size mean and sd required for the salmon quantification approach in single-end mode

Gtf2bed update

See merge request !65

Extend workflow description

See merge request zavolan_group/pipelines/rhea!63

- add linked ToC
- add table of third party software
- add rule graph
- clean up sample table description
- formalize structure for each rule description
- cross-reference rules via inputs/outputs
- clean list of references

Major refactoring before release

Closes #117, #112, #111, #108, #107, #101, and #74

See merge request zavolan_group/pipelines/rhea!62

- clean up command line interface
  - improve descriptions
  - add consistent structure
  - remove or merge superfluous CLI arguments
  - set defaults
  - update test calls
  - update docs
  - when importing data from LabKey, table is saved to 'samples.tsv.labkey' in same directory as Snakemake sample table
- allow user to specify environment variables and relative paths in input table and on CLI
  - relative paths in the input table are interpreted with respect to the directory containing the input table
  - relative paths will are interpreted with respect to the current working directory; this is to achieve portability with respect to tests but is discouraged in production because its behavior is not very predictable from the user's perspective; consequently a warning is thrown
- set STAR index size to read length - 1
- remove `gtf_filtered` and `tr_fasta_filtered` and update Snakefiles and test sample tables accordingly
- rename some MultiQC report-related parameters and update Snakefiles and test config files accordingly
- add logging
- add docstrings to module and all functions
- add typing definitions to all functions
- restructure and comment code to improve readability
- linters `flake8` and `mypy` pass

BIOPZ-Katsantoni Maria authored 4 years ago and Alex Kanitz committed 4 years ago

* Sequencing mode-related changes:
  * allowed sequencing modes in Snakemake input table changed from `paired_end` and `single_end` to `pe` and `se`, respectively
  * remove sequencing mode from output paths for each rule
  * corresponding wild cards removed entirely from all rules that do not depend on sequencing mode (currently all rules that are defined in the main `Snakefile` in the project root directory)
  * where absolutely necessary, sequencing mode is added as part of output file or directory instead
  * remove dependency of sequencing mode for rule for `FastQC`; now runs separately for each strand
* Changes related to MultiQC and output file/directory structure
  * moving and renaming outputs for MultiQC is no longer required
  * code to create MultiQC custom config externalized into script `scripts/rhea_multiqc_config.py`
  * add MultiQC output files with deterministic output to md5 sum checks performed during execution of `tests/test_integration_workflow/test.{local,slurm}.sh`
  * output filenames for each rule now follow this general structure: `samples/{sample_name}/{rule}/{output_file}`
  * change log directory structure matches results directory structure
* Miscellaneous changes
  * consistent, PEP8-compliant formatting in most parts, including Snakemake files, where allowed
  * remove rule `extract_decoys_salmon`; equivalent file `chrName.txt` produced by `star_index` is used instead
  * add rule `start` which copies sample data to the results directory and enforces uniform naming
  * refactoring of ALFA rules and modification of the CI/CD test to ensure compatibility

Update salmon transcriptome index generation

See merge request zavolan_group/pipelines/rhea!59

Second fix in results restructuring

See merge request zavolan_group/pipelines/rhea!57

fix absolute / relative path issue in fastqc results parsing

See merge request zavolan_group/pipelines/rhea!56

update documentation

See merge request zavolan_group/pipelines/rhea!54

Use minified container images

Closes #105

See merge request zavolan_group/pipelines/rhea!55