Pulling of Singularity containers times out
Observed behavior
Occasionally, Snakemake fails with the following error during pulling of container images:
Pulling singularity image docker://[IMAGE]
WorkflowError:
Failed to pull singularity image from docker://[IMAGE]:
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
FATAL: While making image from oci registry: while building SIF from layers: conveyor failed to get: Error initializing source [...]: Error reading blob sha256:[...]: Get https://production.cloudflare.docker.com/[...]: dial tcp [IP]: i/o timeout
This happened both on my local machine and on the testing VM. Rerunning the command again (multiple times if necessary) resolves the issue. Issue occurred for multiple images, at least for:
zavolab/cutadapt:1.16
zavolab/salmon:0.11.0
zavolab/kallisto:0.9
Expected behavior
Snakemake should successfully pull (Docker) images from Docker Hub unless there is a real connection issue.
Possible solutions
- Check whether Snakemake provides an option that increases the time it attempts to pull images. If there is no such option, the issue should be reported to the Snakemake developers.
- Irrespective of this apparent bug, attempts should be made to reduce the size of container images, as this would also considerably shorten the time that tests require to run.