Pipeline crashes on OS X

Moving part of #55 (closed) here:

When I run the pipeline with snakemake (5.10.0) on a mac os x (10.14.6) with singularity (3.3.0-rc.1.658.g7427b73f1.dirty) the test tests/test_integration_workflow/test.local.sh fails. I get an error in the first rule (STAR index). Below is the log:

[Tue Feb 18 13:42:51 2020] rule create_index_star: input: ../input_files/homo_sapiens/genome.fa, ../input_files/homo_sapiens/annotation.gtf output: results/star_indexes/homo_sapiens/75/STAR_index/chrNameLength.txt, results/star_indexes/homo_sapiens/75/STAR_index/chrName.txt log: local_log/homo_sapiens_75_create_index_star.log jobid: 20 wildcards: organism=homo_sapiens, index_size=75 threads: 4

(mkdir -p results/star_indexes/homo_sapiens/75/STAR_index; chmod -R 777 results/star_indexes/homo_sapiens/75/STAR_index; STAR --runMode genomeGenerate --sjdbOverhang 75 --genomeDir results/star_indexes/homo_sapiens/75/STAR_index --genomeFastaFiles ../input_files/homo_sapiens/genome.fa --runThreadN 4 --outFileNamePrefix results/star_indexes/homo_sapiens/75/STAR_index/STAR_ --sjdbGTFfile ../input_files/homo_sapiens/annotation.gtf) &> local_log/homo_sapiens_75_create_index_star.log Activating singularity image /Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow/.snakemake/singularity/8c1d141c868101b1f368c91b2ba06a8e.simg BASH=/bin/bash BASHOPTS=cmdhist:complete_fullquote:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath BASH_ALIASES=() BASH_ARGC=([0]="1") BASH_ARGV=([0]="pipefail") BASH_CMDS=() BASH_EXECUTION_STRING=set BASH_LINENO=() BASH_SOURCE=() BASH_VERSINFO=([0]="4" [1]="4" [2]="12" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu") BASH_VERSION='4.4.12(1)-release' DIRSTACK=() DISPLAY=192.168.64.1:0.0 EUID=502 GROUPS=() HOME=/Users/gypafoiv HOSTNAME='(none)' HOSTTYPE=x86_64 IFS=$' \t\n' IMG=/dev/sda LD_LIBRARY_PATH=/.singularity.d/libs LOGNAME=gypafoiv MACHTYPE=x86_64-pc-linux-gnu MYROUTE=192.168.64.1 NETIP=192.168.64.11 OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PPID=1112 PS4='+ ' PWD=/Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow SHELL=/bin/bash SHELLOPTS=braceexpand:hashall:interactive-comments SHLVL=4 SINGACT=exec SINGARG=$'bash -c set -euo pipefail; (mkdir -p results/star_indexes/homo_sapiens/75/STAR_index; \t\tchmod -R 777 results/star_indexes/homo_sapiens/75/STAR_index; \t\tSTAR \t\t--runMode genomeGenerate \t\t--sjdbOverhang 75 \t\t--genomeDir results/star_indexes/homo_sapiens/75/STAR_index \t\t--genomeFastaFiles ../input_files/homo_sapiens/genome.fa \t\t--runThreadN 4 \t\t--outFileNamePrefix results/star_indexes/homo_sapiens/75/STAR_index/STAR_ \t\t--sjdbGTFfile ../input_files/homo_sapiens/annotation.gtf) &> local_log/homo_sapiens_75_create_index_star.log' SINGCWD=/Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow SINGIP=dhcp SINGULARITY_APPNAME= SINGULARITY_BINDPATH=/host/Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow/../input_files:/Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow/../input_files,/host/mnt/snakemake:/mnt/snakemake SINGULARITY_CONTAINER=/dev/sda SINGULARITY_MESSAGELEVEL=1 SINGULARITY_NAME=8c1d141c868101b1f368c91b2ba06a8e.simg SOFTWARE_VERSION=2.6.0a TERM=linux UID=502 USER=gypafoiv USERNAME=gypafoiv USER_PATH=:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin =/usr/bin/singularity ipv4=dhcp msglvl=1 sing_cwd=/Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow sing_img_name=8c1d141c868101b1f368c91b2ba06a8e.simg sing_user=gypafoiv:502:20 singularity_action=exec singularity_arguments=$'bash -c set -euo pipefail; (mkdir -p results/star_indexes/homo_sapiens/75/STAR_index; \t\tchmod -R 777 results/star_indexes/homo_sapiens/75/STAR_index; \t\tSTAR \t\t--runMode genomeGenerate \t\t--sjdbOverhang 75 \t\t--genomeDir results/star_indexes/homo_sapiens/75/STAR_index \t\t--genomeFastaFiles ../input_files/homo_sapiens/genome.fa \t\t--runThreadN 4 \t\t--outFileNamePrefix results/star_indexes/homo_sapiens/75/STAR_index/STAR \t\t--sjdbGTFfile ../input_files/homo_sapiens/annotation.gtf) &> local_log/homo_sapiens_75_create_index_star.log' singularity_binds='|cyQpNT:/Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow/../input_files|xqqMgT:/mnt/snakemake|home:/Users/gypafoiv' bash: local_log/homo_sapiens_75_create_index_star.log: No such file or directory [ 4.211568] reboot: Power down Waiting at most 5 seconds for missing files. MissingOutputException in line 46 of /Users/gypafoiv/Desktop/rnaseqpipeline/Snakefile: Missing files after 5 seconds: results/star_indexes/homo_sapiens/75/STAR_index/chrNameLength.txt results/star_indexes/homo_sapiens/75/STAR_index/chrName.txt This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait. Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow/.snakemake/log/2020-02-18T134250.867224.snakemake.log

rm -rf logs/ results/ .snakemake/ .java/ local_log/

cd /Users/gypafoiv/Desktop/rnaseqpipeline/tests/test_integration_workflow

Edited Feb 19, 2020 by BIOPZ-Gypas Foivos

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information