Skip to content
Snippets Groups Projects
Commit dafbdbc5 authored by BIOPZ-Bak Maciej's avatar BIOPZ-Bak Maciej
Browse files

fixed rel/abs path issue

parent a3f01399
No related branches found
No related tags found
1 merge request!56fix absolute / relative path issue in fastqc results parsing
Pipeline #10545 passed
This commit is part of merge request !56. Comments created here will be created in the context of that merge request.
...@@ -970,7 +970,7 @@ rule prepare_files_for_report: ...@@ -970,7 +970,7 @@ rule prepare_files_for_report:
output: output:
samples_dir = directory(os.path.join( samples_dir = directory(os.path.join(
"{output_dir}", config["output_dir"],
"samples")) "samples"))
params: params:
results_dir = config["output_dir"], results_dir = config["output_dir"],
...@@ -979,9 +979,12 @@ rule prepare_files_for_report: ...@@ -979,9 +979,12 @@ rule prepare_files_for_report:
config["log_dir"], config["log_dir"],
"samples") "samples")
log: log:
LOG_local_log = \ stderr = os.path.join(
os.path.join("{output_dir}", "local_log", \ config["log_dir"],
"prepare_files_for_report.log") "prepare_files_for_report.stderr.log"),
stdout = os.path.join(
config["log_dir"],
"prepare_files_for_report.stdout.log")
run: run:
# remove "single/paired end" from the results directories # remove "single/paired end" from the results directories
...@@ -1111,6 +1114,7 @@ rule prepare_files_for_report: ...@@ -1111,6 +1114,7 @@ rule prepare_files_for_report:
sample_name = zipfile.split("/")[-3] sample_name = zipfile.split("/")[-3]
  • Maintainer

    Not maintainable. This needs improvements.

  • Author Maintainer

    Again, this is not an issue of this branch/ merge request - this is already in the master.
    But regardless - just above the line you mention there is:

            fastq_zip_list = glob.glob(
                os.path.join(
                    params.results_dir,
                    "samples",
                    "*",
                    "*_fastqc",
                    "*_fastqc.zip"))

    therefore hardcoding [-3] (counting from the back) brings us directly below "samples" i.e. where all the directories with sample names are. I guess we might want to change that to get sample names from the design table by a function inside the Snakefile in 0.2. But not here - this merge request is designed to address the bug which Anastasia has found, not code style.

    Edited by BIOPZ-Bak Maciej
  • Owner

    Agree with both of you: yes, it certainly looks dangerous, and no, it's not the point of this merge request and can wait till 0.2.0.

  • Maintainer

    Opened #107 (closed) #108 (closed) for the next version.

  • Please register or sign in to reply
zipfile_path_chunks = zipfile.split("/") zipfile_path_chunks = zipfile.split("/")
new_path = os.path.join(*(zipfile_path_chunks[:-1])) new_path = os.path.join(*(zipfile_path_chunks[:-1]))
if params.results_dir.startswith("/"): new_path = "/" + new_path
  • Maintainer

    This is not clean. How can we maintain this in the future?

  • Author Maintainer

    Why is this not clean? params.results_dir hold whatever the user provides in the config file. If it starts with / then we know we are dealing with absolute path and we have to append / as a prefix. In all other cases we deal with relative paths and there is no need for that prefix. How would you propose to re-write this?

  • Owner

    Agree with Foivos.

    1. Using bare / should never be used as it's not portable. Better to use os.path.sep which expands to the correct path separator for the current system.
    2. This is overly complicated. There is a method isabs() in the standard os.path module, which is written exactly for the purpose of checking whether a path is absolute.
    3. You may not even need to do this check, and instead simply return an absolute path for your path, regardless of whether it is relative or absolute. Probably the easiest and certainly the most Pythonic way. You can use os.path.abspath() for that.
    Edited by Alex Kanitz
  • changed this line in version 2 of the diff

    Toggle commit list
  • Author Maintainer

    Fixed in the new commit with os.path.abspath()

  • Please register or sign in to reply
with ZipFile(zipfile, 'r') as zip_f: with ZipFile(zipfile, 'r') as zip_f:
zip_f.extractall(new_path) zip_f.extractall(new_path)
fastqc_data_f = os.path.join( fastqc_data_f = os.path.join(
...@@ -1180,11 +1184,11 @@ rule prepare_MultiQC_config: ...@@ -1180,11 +1184,11 @@ rule prepare_MultiQC_config:
''' '''
input: input:
multiqc_input_dir = os.path.join( multiqc_input_dir = os.path.join(
"{output_dir}", config["output_dir"],
"samples") "samples")
output: output:
multiqc_config = os.path.join( multiqc_config = os.path.join(
"{output_dir}", config["output_dir"],
"MultiQC_config.yaml") "MultiQC_config.yaml")
params: params:
logo_path = os.path.join( logo_path = os.path.join(
...@@ -1194,9 +1198,12 @@ rule prepare_MultiQC_config: ...@@ -1194,9 +1198,12 @@ rule prepare_MultiQC_config:
"logo.128px.png"), "logo.128px.png"),
results_dir = config["output_dir"] results_dir = config["output_dir"]
log: log:
LOG_local_log = \ stderr = os.path.join(
os.path.join("{output_dir}", "local_log", \ config["log_dir"],
"prepare_MultiQC_config.log") "prepare_MultiQC_config.stderr.log"),
stdout = os.path.join(
config["log_dir"],
"prepare_MultiQC_config.stdout.log")
run: run:
with open(output.multiqc_config, "w") as YAML: with open(output.multiqc_config, "w") as YAML:
YAML.write("---\n\n") YAML.write("---\n\n")
...@@ -1259,15 +1266,19 @@ rule MULTIQC_report: ...@@ -1259,15 +1266,19 @@ rule MULTIQC_report:
config["output_dir"], config["output_dir"],
"MultiQC_config.yaml") "MultiQC_config.yaml")
output: output:
MultiQC_report = \ MultiQC_report = directory(os.path.join(
directory(os.path.join("{output_dir}", "multiqc_summary")) config["output_dir"],
"multiqc_summary"))
params: params:
results_dir = config["output_dir"], results_dir = config["output_dir"],
log_dir = config["log_dir"] log_dir = config["log_dir"]
log: log:
LOG_local_log = \ stderr = os.path.join(
os.path.join("{output_dir}", "local_log", \ config["log_dir"],
"MULTIQC_report.log") "MULTIQC_report.stderr.log"),
stdout = os.path.join(
config["log_dir"],
"MULTIQC_report.stdout.log")
singularity: singularity:
"docker://ewels/multiqc:1.7" "docker://ewels/multiqc:1.7"
shell: shell:
...@@ -1277,5 +1288,5 @@ rule MULTIQC_report: ...@@ -1277,5 +1288,5 @@ rule MULTIQC_report:
--config {input.multiqc_config} \ --config {input.multiqc_config} \
{params.results_dir} \ {params.results_dir} \
{params.log_dir} \ {params.log_dir} \
&> {log.LOG_local_log}; 1> {log.stdout} 2> {log.stderr}
""" """
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment