fix various small issues

- remove log files and add '.snakemake' directories to '.gitignore'
- update wrong link in 'README.md'
- delete superfluous script documentation 'scripts/labkey_api.md'
- add Snakemake-specific file extension '.smk' to subworkflows
- remove non-deterministic workflow output from md5 sums

.gitignore

@@ -329,3 +329,4 @@ pip-selfcheck.json
 snakemake/.*
 runs/.*
 !runs/PUT_YOUR_WORKFLOW_RUN_CONFIGS_HERE
+.snakemake/

README.md

@@ -117,7 +117,7 @@ configuration files:
 values. Have a look at the examples in the `tests/` directory to see what the
 files should look like, specifically:
 
-    - [samples.tsv](tests/test_integration_workflow/input_files/samples.tsv)
+    - [samples.tsv](tests/test_integration_workflow/samples.tsv)
     - [config.yaml](tests/test_integration_workflow/config.yaml)
     - [cluster.json](tests/test_integration_workflow/cluster.json)
 

scripts/labkey_api.md

-> **NOTE**: Include this info in the main doc in the root directory once 
-> available; add information on how to get credentials for the LabKey server 
-> (i.e., how to obatain a password for the `.netrc` file)
-
-In order to connect to the LabKey through API, you will first need to create a
-file `.netrc` in your home directory:
-
-```bash
-touch ${HOME}/.netrc
-```
-
-Add the following lines to the file:
-
-```console
-machine <remote-instance-of-labkey-server>  
-login <user-email>
-password <user-password>  
-```
-
-To secure the file, set permissions in a way that only you can see the content
-of the file: 
-
-```bash
-chmod 400 .netrc
-```
-
-Install the `labkey` and `pandas` packages, ideally from a virtual environment
-(e.g., `virtualenv` or `conda`):
-
-```bash
-pip install labkey pandas
-```
-
-Run the LabKey API client script:
-
-```bash
-python labkey_api.py project_name labkey_table_nane  
-```
-
-Example:
-
-```bash
-python labkey_api.py TEST_ABOERSCH RNA_Seq_data_template  
-```
-
-Right now the script prints a representation of a `pandas` data frame
-containing the requested LabKey table the the screen. For further processing
-the current script could be included in another script, or it could be modified
-to write out the data in a desired file format (e.g., TSV).

snakemake/Snakefile

@@ -18,8 +18,8 @@ localrules: finish
 # Execution dependend on sequencing mode
 ##################################################################################
 
-include: 'paired_end.snakefile'
-include: 'single_end.snakefile'
+include: 'paired_end.snakefile.smk'
+include: 'single_end.snakefile.smk'
 
 #################################################################################
 ### Final rule

tests/test_create_dag_chart/.snakemake/log/2020-02-04T153123.241756.snakemake.log

-FileNotFoundError in line 13 of /scicore/home/zavolan/kanitz/SANDBOX/rnaseqpipeline/snakemake/Snakefile:
-[Errno 2] No such file or directory: 'input_files/samples.tsv'
-  File "/scicore/home/zavolan/kanitz/SANDBOX/rnaseqpipeline/snakemake/Snakefile", line 13, in <module>
-  File "/scicore/home/zavolan/kanitz/REMOVE_FROM_HOME/soft/miniconda3/envs/rnaseq_pipeline/lib/python3.6/site-packages/pandas/io/parsers.py", line 676, in parser_f
-  File "/scicore/home/zavolan/kanitz/REMOVE_FROM_HOME/soft/miniconda3/envs/rnaseq_pipeline/lib/python3.6/site-packages/pandas/io/parsers.py", line 448, in _read
-  File "/scicore/home/zavolan/kanitz/REMOVE_FROM_HOME/soft/miniconda3/envs/rnaseq_pipeline/lib/python3.6/site-packages/pandas/io/parsers.py", line 880, in __init__
-  File "/scicore/home/zavolan/kanitz/REMOVE_FROM_HOME/soft/miniconda3/envs/rnaseq_pipeline/lib/python3.6/site-packages/pandas/io/parsers.py", line 1126, in _make_engine
-  File "/scicore/home/zavolan/kanitz/REMOVE_FROM_HOME/soft/miniconda3/envs/rnaseq_pipeline/lib/python3.6/site-packages/pandas/io/parsers.py", line 2269, in __init__
-  File "/scicore/home/zavolan/kanitz/REMOVE_FROM_HOME/soft/miniconda3/envs/rnaseq_pipeline/lib/python3.6/site-packages/pandas/io/common.py", line 431, in get_handle

tests/test_create_dag_chart/.snakemake/log/2020-02-04T153204.024195.snakemake.log

-Building DAG of jobs...
-MissingInputException in line 2 of /scicore/home/zavolan/kanitz/SANDBOX/rnaseqpipeline/snakemake/paired_end.snakefile:
-Missing input files for rule pe_fastqc:
-input_files/GSM1502498_1.fastq.gz
-input_files/GSM1502498_2.fastq.gz

tests/test_create_dag_chart/.snakemake/log/2020-02-04T153253.153466.snakemake.log

-Building DAG of jobs...
-MissingInputException in line 2 of /scicore/home/zavolan/kanitz/SANDBOX/rnaseqpipeline/snakemake/paired_end.snakefile:
-Missing input files for rule pe_fastqc:
-input_files/GSM1502500_1.fastq.gz
-input_files/GSM1502500_2.fastq.gz

tests/test_create_dag_chart/.snakemake/log/2020-02-04T153305.441930.snakemake.log

-Building DAG of jobs...

tests/test_create_dag_chart/.snakemake/log/2020-02-04T153343.033762.snakemake.log

-Building DAG of jobs...

tests/test_create_dag_chart/.snakemake/log/2020-02-04T153536.300877.snakemake.log

-Building DAG of jobs...

tests/test_create_dag_chart/.snakemake/log/2020-02-04T154438.496145.snakemake.log

-Building DAG of jobs...

tests/test_integration_workflow/expected_output.md5

 c45be0333e4d84285d530855342763e0  results/paired_end/HNRNPC_control_rep1/HNRNPC_control_rep1.remove_adapters_mate1.fastq
 e1e0d16add8db1a314c780d497e863c9  results/paired_end/HNRNPC_control_rep1/HNRNPC_control_rep1.remove_polya_mate2.fastq
-7c63754abeff36d37b4d32c15d603e93  results/paired_end/HNRNPC_control_rep1/mate1_fastqc/GSM1502498_1_fastqc.html
 a46359f784d3eca4268c788a74789bf2  results/paired_end/HNRNPC_control_rep1/HNRNPC_control_rep1.remove_polya_mate1.fastq
 58ef63f61c82e050f05b871eea8d4b25  results/paired_end/HNRNPC_control_rep1/HNRNPC_control_rep1.remove_adapters_mate2.fastq
-794e0ed5b9abdeba17627ca76a298001  results/paired_end/HNRNPC_control_rep1/mate2_fastqc/GSM1502498_2_fastqc.html
 d3898e6e03d98db65b61172f8275f319  results/paired_end/HNRNPC_KD_rep1/HNRNPC_KD_rep1.remove_adapters_mate2.fastq
-e4eb9fe5f364c6f3395fc678b80853f9  results/paired_end/HNRNPC_KD_rep1/mate1_fastqc/GSM1502500_1_fastqc.html
 489e5a5bd92fafe9a79e164fb334e036  results/paired_end/HNRNPC_KD_rep1/HNRNPC_KD_rep1.remove_polya_mate2.fastq
 8a01f7aa476992c2bc32d3457f703865  results/paired_end/HNRNPC_KD_rep1/HNRNPC_KD_rep1.remove_adapters_mate1.fastq
-befd24fe0de9a24c853eb3c29d175168  results/paired_end/HNRNPC_KD_rep1/mate2_fastqc/GSM1502500_2_fastqc.html
 f048f50b6695c80e2c9d4ecf10c0d0a9  results/paired_end/HNRNPC_KD_rep1/HNRNPC_KD_rep1.remove_polya_mate1.fastq