Use biocontainers images for star, gffread, salmon, kallisto, cutadapt, samtools, fastqc, alfa, bedtools, bedgraphtobigwig. Change container from bash to ubuntu. Fixes #149

BIOPZ-Katsantoni Maria authored 4 years ago and BIOPZ-Gypas Foivos committed 4 years ago

and are therefore not required in the rules.
- Snakefile
 star_rpm: --outWigNorm (default RPM was used)
 star_rpm: --outWigStrand (default Stranded was used)
 rename_star_rpm_for_alfa: orientation in params is redundant (Fixes #152)
- single_end.snakefile.smk
 map_genome_star: outFilterMismatchNoverLmax
 map_genome_star: outFilterScoreMinOverLread
 map_genome_star: outFilterMatchNminOverLread
 quantification_salmon: --writeUnmappedNames
- paired_end.snakefile.smk
  pe_map_genome_star: outFilterMismatchNoverLmax
  pe_map_genome_star: outFilterScoreMinOverLread
  pe_map_genome_star: outFilterMatchNminOverLread
  quantification_salmon: --writeUnmappedNames

- Delete unused files scripts/fg_extract_transcripts.py, scripts/heatmap_and_clustermap.py, scripts/perform_PCA.py
- Add rules (pca_kallisto, pca_salmon) that run zpca (https://github.com/zavolanlab/zpca) on genes and transcripts TPM tables from kallisto and salmon.
- The output is wired to multiqc_report but the plots are not visualized to multiqc. Update documentation.
- Update dag and rulegraph.
Fixes #140 #142

BIOPZ-Iborra de Toledo Paula authored 4 years ago and BIOPZ-Gypas Foivos committed 4 years ago

The rules rely on https://github.com/zavolanlab/merge_kallisto
Update info in pipeline_documentation.md

BIOPZ-Katsantoni Maria authored 4 years ago and Alex Kanitz committed 4 years ago

fix: Renamed samples_concat.tsv to samples.multiple_lanes.tsv. Renamed rows with split with the same name as the other test samples, so that I do not change the tests (md5 and sunch). Removed the one lane samples. Created config that uses this tsv file

- clean up command line interface
  - improve descriptions
  - add consistent structure
  - remove or merge superfluous CLI arguments
  - set defaults
  - update test calls
  - update docs
  - when importing data from LabKey, table is saved to 'samples.tsv.labkey' in same directory as Snakemake sample table
- allow user to specify environment variables and relative paths in input table and on CLI
  - relative paths in the input table are interpreted with respect to the directory containing the input table
  - relative paths will are interpreted with respect to the current working directory; this is to achieve portability with respect to tests but is discouraged in production because its behavior is not very predictable from the user's perspective; consequently a warning is thrown
- set STAR index size to read length - 1
- remove `gtf_filtered` and `tr_fasta_filtered` and update Snakefiles and test sample tables accordingly
- rename some MultiQC report-related parameters and update Snakefiles and test config files accordingly
- add logging
- add docstrings to module and all functions
- add typing definitions to all functions
- restructure and comment code to improve readability
- linters `flake8` and `mypy` pass

BIOPZ-Katsantoni Maria authored 5 years ago and Alex Kanitz committed 4 years ago

* Sequencing mode-related changes:
  * allowed sequencing modes in Snakemake input table changed from `paired_end` and `single_end` to `pe` and `se`, respectively
  * remove sequencing mode from output paths for each rule
  * corresponding wild cards removed entirely from all rules that do not depend on sequencing mode (currently all rules that are defined in the main `Snakefile` in the project root directory)
  * where absolutely necessary, sequencing mode is added as part of output file or directory instead
  * remove dependency of sequencing mode for rule for `FastQC`; now runs separately for each strand
* Changes related to MultiQC and output file/directory structure
  * moving and renaming outputs for MultiQC is no longer required
  * code to create MultiQC custom config externalized into script `scripts/rhea_multiqc_config.py`
  * add MultiQC output files with deterministic output to md5 sum checks performed during execution of `tests/test_integration_workflow/test.{local,slurm}.sh`
  * output filenames for each rule now follow this general structure: `samples/{sample_name}/{rule}/{output_file}`
  * change log directory structure matches results directory structure
* Miscellaneous changes
  * consistent, PEP8-compliant formatting in most parts, including Snakemake files, where allowed
  * remove rule `extract_decoys_salmon`; equivalent file `chrName.txt` produced by `star_index` is used instead
  * add rule `start` which copies sample data to the results directory and enforces uniform naming
  * refactoring of ALFA rules and modification of the CI/CD test to ensure compatibility

Dominik Burri authored 5 years ago and Alex Kanitz committed 5 years ago

- generate nucleotide distribution for unique reads only
- new rule to generate PNG image for MultiQC

corrected md5sum for config.yaml

remove unnecessary file

- renaming bedgraph
- creating ALFA qc plots

removed conda dependence, moved import statement.

included ALFA in finish rule, corrected annotation.gtf and config.yaml, created new .svg

Merged paired end and single end rules for star_rpm and index_genomic_alignment_samtools. Fixed wiring of calculate tin score: bam should be input and not params.