堆栈内存溢出
  繁体   English   中英

如何在 Snakemake 中使用奇点和 conda 包装器

[英]How to use singularity and conda wrappers in Snakemake

 原文  2020-09-21 22:24:50  1   1    python/ conda/ snakemake/ singularity-container

问题描述

TLDR我收到以下错误：

“conda”命令在您的奇点容器映像中不可用。 Snakemake 将您的 conda 安装安装到奇异点。 有时，由于 shell 限制，这可能会失败。 它已经过测试，可以与 docker://ubuntu 一起使用，但它例如无法与 docker://bash 一起使用

我创建了一个Snakemake工作流程，并将shell:命令转换为基于规则的 package 管理，通过Snakemake wrappers: 。

但是，我在 HPC 上运行时遇到了问题，一位 HPC 支持人员强烈建议不要在任何 HPC 系统上使用conda ，因为：

“如果 [包装器] 的构建器不是超级小心，conda 环境中存在的依赖主机库的动态库（总是有几个存在，因为构建器大部分时间都是无忧无虑的）将会崩溃。我认为依赖管道的Singularity性将使系统更强大。” -匿名

我周末做了一些阅读，根据这篇文档，可以将容器与基于 conda 的 package 管理结合起来； 通过定义全局conda docker 容器和每个规则yaml文件。

注意：与上面链接中的示例（图 5.4 ）相反，它使用预定义的yaml和shell:命令，这里我使用 conda 包装器将这些yaml文件下载到Singularity容器中（如果我的想法正确的话）所以我认为function 应该相同 - 请参阅注意：最后......

Snakefile , config.yaml和samples.txt

Snakefile

# Directories------------------------------------------------------------------
configfile: "config.yaml"

# Setting the names of all directories
dir_list = ["REF_DIR", "LOG_DIR", "BENCHMARK_DIR", "QC_DIR", "TRIM_DIR", "ALIGN_DIR", "MARKDUP_DIR", "CALLING_DIR", "ANNOT_DIR"]
dir_names = ["refs", "logs", "benchmarks", "qc", "trimming", "alignment", "mark_duplicates", "variant_calling", "annotation"]
dirs_dict = dict(zip(dir_list, dir_names))

import os
import pandas as pd
# getting the samples information (names, path to r1 & r2) from samples.txt
samples_information = pd.read_csv("samples.txt", sep='\t', index_col=False)
# get a list of the sample names
sample_names = list(samples_information['sample'])
sample_locations = list(samples_information['location'])
samples_dict = dict(zip(sample_names, sample_locations))
# get number of samples
len_samples = len(sample_names)


# Singularity with conda wrappers

singularity: "docker://continuumio/miniconda3:4.5.11"

# Rules -----------------------------------------------------------------------

rule all:
    input:
        "resources/vep/plugins",
        "resources/vep/cache"

rule download_vep_plugins:
    output:
        directory("resources/vep/plugins")
    params:
        release=100
    resources:
        mem=1000,
        time=30
    wrapper:
        "0.66.0/bio/vep/plugins"

rule get_vep_cache:
    output:
        directory("resources/vep/cache")
    params:
        species="caenorhabditis_elegans",
        build="WBcel235",
        release="100"
    resources:
        mem=1000,
        time=30
    log:
        "logs/vep/cache.log"
    cache: True  # save space and time with between workflow caching (see docs)
    wrapper:
        "0.66.0/bio/vep/cache"

config.yaml

# Files
REF_GENOME: "c_elegans.PRJNA13758.WS265.genomic.fa"
GENOME_ANNOTATION: "c_elegans.PRJNA13758.WS265.annotations.gff3"

# Tools
QC_TOOL: "fastQC"
TRIM_TOOL: "trimmomatic"
ALIGN_TOOL: "bwa"
MARKDUP_TOOL: "picard"
CALLING_TOOL: "varscan"
ANNOT_TOOL: "vep"

samples.txt

sample  location
MTG324  /home/moldach/wrappers/SUBSET/MTG324_SUBSET

提交

snakemake --profile slurm --use-singularity --use-conda --jobs 2

日志

Workflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
        1   get_vep_cache
        1

[Mon Sep 21 15:35:50 2020]
rule get_vep_cache:
    output: resources/vep/cache
    log: logs/vep/cache.log
    jobid: 0
    resources: mem=1000, time=30

Activating singularity image /home/moldach/wrappers/SUBSET/VEP/.snakemake/singularity/d7617773b315c3abcb29e0484085ed06.simg
Activating conda environment: /home/moldach/wrappers/SUBSET/VEP/.snakemake/conda/774ea575
[Mon Sep 21 15:36:38 2020]
Finished job 0.
1 of 1 steps (100%) done

注意：将--use-conda留在工作流的提交之外将导致get_vep_cache: - /bin/bash: vep_install: command not found错误

Workflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
        1   download_vep_plugins
        1

[Mon Sep 21 15:35:50 2020]
rule download_vep_plugins:
    output: resources/vep/plugins
    jobid: 0
    resources: mem=1000, time=30

Activating singularity image /home/moldach/wrappers/SUBSET/VEP/.snakemake/singularity/d7617773b315c3abcb29e0484085ed06.simg
Activating conda environment: /home/moldach/wrappers/SUBSET/VEP/.snakemake/conda/9f602d9a
[Mon Sep 21 15:35:56 2020]
Finished job 0.
1 of 1 steps (100%) done

添加第三条规则fastq时出现问题：

更新Snakefile

# Directories------------------------------------------------------------------
configfile: "config.yaml"

# Setting the names of all directories
dir_list = ["REF_DIR", "LOG_DIR", "BENCHMARK_DIR", "QC_DIR", "TRIM_DIR", "ALIGN_DIR", "MARKDUP_DIR", "CALLING_DIR", "ANNOT_DIR"]
dir_names = ["refs", "logs", "benchmarks", "qc", "trimming", "alignment", "mark_duplicates", "variant_calling", "annotation"]
dirs_dict = dict(zip(dir_list, dir_names))

import os
import pandas as pd
# getting the samples information (names, path to r1 & r2) from samples.txt
samples_information = pd.read_csv("samples.txt", sep='\t', index_col=False)
# get a list of the sample names
sample_names = list(samples_information['sample'])
sample_locations = list(samples_information['location'])
samples_dict = dict(zip(sample_names, sample_locations))
# get number of samples
len_samples = len(sample_names)


# Singularity with conda wrappers

singularity: "docker://continuumio/miniconda3:4.5.11"

# Rules -----------------------------------------------------------------------

rule all:
    input:
        "resources/vep/plugins",
        "resources/vep/cache",
        expand('{QC_DIR}/{QC_TOOL}/before_trim/{sample}_{pair}_fastqc.{ext}', QC_DIR=dirs_dict["QC_DIR"], QC_TOOL=config["QC_TOOL"], sample=sample_names, pair=['R1', 'R2'], ext=['html', 'zip'])

rule download_vep_plugins:
    output:
        directory("resources/vep/plugins")
    params:
        release=100
    resources:
        mem=1000,
        time=30
    wrapper:
        "0.66.0/bio/vep/plugins"

rule get_vep_cache:
    output:
        directory("resources/vep/cache")
    params:
        species="caenorhabditis_elegans",
        build="WBcel235",
        release="100"
    resources:
        mem=1000,
        time=30
    log:
        "logs/vep/cache.log"
    cache: True  # save space and time with between workflow caching (see docs)
    wrapper:
        "0.66.0/bio/vep/cache"

def getHome(sample):
  return(list(os.path.join(samples_dict[sample],"{0}_{1}.fastq.gz".format(sample,pair)) for pair in ['R1','R2']))

rule qc_before_trim_r1:
    input:
        r1=lambda wildcards: getHome(wildcards.sample)[0]
    output:
        html=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R1_fastqc.html"),
        zip=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R1_fastqc.zip"),
    params:
         dir=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim")
    log:
        os.path.join(dirs_dict["LOG_DIR"],config["QC_TOOL"],"{sample}_R1.log")
    resources:
        mem=1000,
        time=30
    singularity:
        "https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0"
    threads: 1
    message: """--- Quality check of raw data with FastQC before trimming."""
    wrapper:
         "0.66.0/bio/fastqc"

rule qc_before_trim_r2:
    input:
        r1=lambda wildcards: getHome(wildcards.sample)[1]
    output:
        html=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R2_fastqc.html"),
        zip=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R2_fastqc.zip"),
    params:
         dir=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim")
    log:
        os.path.join(dirs_dict["LOG_DIR"],config["QC_TOOL"],"{sample}_R2.log")
    resources:
        mem=1000,
        time=30
    singularity:
        "https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0"
    threads: 1
    message: """--- Quality check of raw data with FastQC before trimming."""
    wrapper:
        "0.66.0/bio/fastqc"

nohup.out

Building DAG of jobs...
Pulling singularity image https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0.
CreateCondaEnvironmentException:
The 'conda' command is not available inside your singularity container image. Snakemake mounts your conda installation into singularity. Sometimes, this can fail because of shell restrictions. It has been tested to work with docker://ubuntu, but it e.g. fails with docker://bash 
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/deployment/conda.py", line 247, in create
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/deployment/conda.py", line 381, in __new__
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/deployment/conda.py", line 394, in __init__
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/deployment/conda.py", line 417, in _check

使用shell:而不是wrapper:

我将包装器改回 shell 命令：

这是我用``提交时得到的错误：

orkflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
        1   qc_before_trim_r2
        1

[Mon Sep 21 16:32:54 2020]
Job 0: --- Quality check of raw data with FastQC before trimming.

Activating singularity image /home/moldach/wrappers/SUBSET/VEP/.snakemake/singularity/6740cb07e67eae01644839c9767bdca5.simg
^[[33mWARNING:^[[0m Skipping mount /var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "en_CA.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Skipping '/home/moldach/wrappers/SUBSET/MTG324_SUBSET/MTG324_R2.fastq.gz' which didn't exist, or couldn't be read
Waiting at most 60 seconds for missing files.
MissingOutputException in line 84 of /home/moldach/wrappers/SUBSET/VEP/Snakefile:
Job completed successfully, but some output files are missing. Missing files after 60 seconds:
qc/fastQC/before_trim/MTG324_R2_fastqc.html
qc/fastQC/before_trim/MTG324_R2_fastqc.zip
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 544, in handle_job_success
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 231, in handle_job_success
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

错误Skipping '/home/moldach/wrappers/SUBSET/MTG324_SUBSET/MTG324_R2.fastq.gz' which didn't exist, or couldn't be read是误导性的，因为文件确实存在...

更新 2

按照Manavalan Gajapathy的建议，我消除了在两个不同级别（全局 + 按规则）定义奇点的问题。

现在我只在全局级别使用奇点容器，并通过--use-conda使用包装器，它在容器内部创建 conda 环境：

# Directories------------------------------------------------------------------
configfile: "config.yaml"

# Setting the names of all directories
dir_list = ["REF_DIR", "LOG_DIR", "BENCHMARK_DIR", "QC_DIR", "TRIM_DIR", "ALIGN_DIR", "MARKDUP_DIR", "CALLING_DIR", "ANNOT_DIR"]
dir_names = ["refs", "logs", "benchmarks", "qc", "trimming", "alignment", "mark_duplicates", "variant_calling", "annotation"]
dirs_dict = dict(zip(dir_list, dir_names))

import os
import pandas as pd
# getting the samples information (names, path to r1 & r2) from samples.txt
samples_information = pd.read_csv("samples.txt", sep='\t', index_col=False)
# get a list of the sample names
sample_names = list(samples_information['sample'])
sample_locations = list(samples_information['location'])
samples_dict = dict(zip(sample_names, sample_locations))
# get number of samples
len_samples = len(sample_names)


# Singularity with conda wrappers

singularity: "docker://continuumio/miniconda3:4.5.11"

# Rules -----------------------------------------------------------------------

rule all:
    input:
    "resources/vep/plugins",
        "resources/vep/cache",
        expand('{QC_DIR}/{QC_TOOL}/before_trim/{sample}_{pair}_fastqc.{ext}', QC_DIR=dirs_dict["QC_DIR"], QC_TOOL=config["QC_TOOL"], sample=sample_names, pair=['R1', 'R2'], ext=['html', 'zip'])

rule download_vep_plugins:
    output:
    directory("resources/vep/plugins")
    params:
    release=100
    resources:
    mem=1000,
        time=30
    wrapper:
    "0.66.0/bio/vep/plugins"

rule get_vep_cache:
    output:
    directory("resources/vep/cache")
    params:
    species="caenorhabditis_elegans",
        build="WBcel235",
        release="100"
    resources:
    mem=1000,
        time=30
    log:
        "logs/vep/cache.log"
    cache: True  # save space and time with between workflow caching (see docs)
    wrapper:
    "0.66.0/bio/vep/cache"

def getHome(sample):
  return(list(os.path.join(samples_dict[sample],"{0}_{1}.fastq.gz".format(sample,pair)) for pair in ['R1','R2']))

rule qc_before_trim_r1:
    input:
    r1=lambda wildcards: getHome(wildcards.sample)[0]
    output:
    html=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R1_fastqc.html"),
        zip=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R1_fastqc.zip"),
    params:
    dir=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim")
    log:
        os.path.join(dirs_dict["LOG_DIR"],config["QC_TOOL"],"{sample}_R1.log")
    resources:
    mem=1000,
    threads: 1
    message: """--- Quality check of raw data with FastQC before trimming."""
    wrapper:
    "0.66.0/bio/fastqc"

rule qc_before_trim_r2:
    input:
    r1=lambda wildcards: getHome(wildcards.sample)[1]
    output:
    html=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R2_fastqc.html"),
        zip=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim","{sample}_R2_fastqc.zip"),
    params:
    dir=os.path.join(dirs_dict["QC_DIR"],config["QC_TOOL"],"before_trim")
    log:
        os.path.join(dirs_dict["LOG_DIR"],config["QC_TOOL"],"{sample}_R2.log")
    resources:
    mem=1000,
        time=30
    threads: 1
    message: """--- Quality check of raw data with FastQC before trimming."""
    wrapper:
    "0.66.0/bio/fastqc"

并通过以下方式提交：

但是，我仍然收到错误消息：

Workflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
        1   qc_before_trim_r2
        1

[Tue Sep 22 12:44:03 2020]
Job 0: --- Quality check of raw data with FastQC before trimming.

Activating singularity image /home/moldach/wrappers/SUBSET/OMG/.snakemake/singularity/d7617773b315c3abcb29e0484085ed06.simg
Activating conda environment: /home/moldach/wrappers/SUBSET/OMG/.snakemake/conda/c591f288
Skipping '/work/mtgraovac_lab/MATTS_SCRATCH/rep1_R2.fastq.gz' which didn't exist, or couldn't be read
Skipping ' 2> logs/fastQC/rep1_R2.log' which didn't exist, or couldn't be read
Failed to process qc/fastQC/before_trim
java.io.FileNotFoundException: qc/fastQC/before_trim (Is a directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at uk.ac.babraham.FastQC.Sequence.FastQFile.<init>(FastQFile.java:73)
        at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)
        at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)
        at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:159)
        at uk.ac.babraham.FastQC.Analysis.OfflineRunner.<init>(OfflineRunner.java:121)
        at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)
Traceback (most recent call last):
  File "/home/moldach/wrappers/SUBSET/OMG/.snakemake/scripts/tmpiwwprg5m.wrapper.py", line 35, in <module>
    shell(
  File "/mnt/snakemake/snakemake/shell.py", line 205, in __new__
    raise sp.CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'set -euo pipefail;  fastqc qc/fastQC/before_trim --quiet -t 1 --outdir /tmp/tmps93snag8 /work/mtgraovac_lab/MATTS_SCRATCH/rep1_R2.fastq.gz ' 2> logs/fastQC/rep1_R$
[Tue Sep 22 12:44:16 2020]
Error in rule qc_before_trim_r2:
    jobid: 0
    output: qc/fastQC/before_trim/rep1_R2_fastqc.html, qc/fastQC/before_trim/rep1_R2_fastqc.zip
    log: logs/fastQC/rep1_R2.log (check log file(s) for error message)
    conda-env: /home/moldach/wrappers/SUBSET/OMG/.snakemake/conda/c591f288

RuleException:
CalledProcessError in line 97 of /home/moldach/wrappers/SUBSET/OMG/Snakefile:
Command ' singularity exec --home /home/moldach/wrappers/SUBSET/OMG  --bind /home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages:/mnt/snakemake /home/moldach/wrappers/SUBSET/OMG/.snakemake$
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2189, in run_wrapper
  File "/home/moldach/wrappers/SUBSET/OMG/Snakefile", line 97, in __rule_qc_before_trim_r2
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 529, in _callback
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/concurrent/futures/thread.py", line 57, in run
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 515, in cached_or_run
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2201, in run_wrapper
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

再现性

要复制它，您可以下载这个小数据集：

git clone https://github.com/CRG-CNAG/CalliNGS-NF.git
cp CalliNGS-NF/data/reads/rep1_*.fq.gz .
mv rep1_1.fq.gz rep1_R1.fastq.gz 
mv rep1_2.fq.gz rep1_R2.fastq.gz

更新 3：绑定坐骑

根据安装时共享的链接：

“默认情况下，Singularity 在运行时将/home/$USER 、 /tmp和$PWD绑定到您的容器中。”

因此，为简单起见（也因为我使用--singularity-args时出错），我已将所需文件移至/home/$USER并尝试从那里运行。

(snakemake) [~]$ pwd
/home/moldach


(snakemake) [~]$ ll
total 3656
drwx------ 26 moldach moldach    4096 Aug 27 17:36 anaconda3
drwx------  2 moldach moldach    4096 Sep 22 10:11 bin
-rw-------  1 moldach moldach     265 Sep 22 14:29 config.yaml
-rw-------  1 moldach moldach 1817903 Sep 22 14:29 rep1_R1.fastq.gz
-rw-------  1 moldach moldach 1870497 Sep 22 14:29 rep1_R2.fastq.gz
-rw-------  1 moldach moldach      55 Sep 22 14:29 samples.txt
-rw-------  1 moldach moldach    3420 Sep 22 14:29 Snakefile

并运行bash -c "nohup snakemake --profile slurm --use-singularity --use-conda --jobs 4 &"

但是，我仍然收到这个奇怪的错误：

Activating conda environment: /home/moldach/.snakemake/conda/fdae4f0d
Skipping ' 2> logs/fastQC/rep1_R2.log' which didn't exist, or couldn't be read
Failed to process qc/fastQC/before_trim
java.io.FileNotFoundException: qc/fastQC/before_trim (Is a directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at uk.ac.babraham.FastQC.Sequence.FastQFile.<init>(FastQFile.java:73)
        at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)
        at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)
        at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:159)
        at uk.ac.babraham.FastQC.Analysis.OfflineRunner.<init>(OfflineRunner.java:121)
        at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)
Traceback (most recent call last):

为什么它认为它被赋予了一个目录？

注意：如果您仅使用--use-conda conda 提交，例如bash -c "nohup snakemake --profile slurm --use-conda --jobs 4 &" fastqc规则没有错误。 但是，单独的--use-conda参数不是 %100 可重现的，这个例子在我测试过的另一个 HPC 上不起作用

使用nohup.out时--printshellcmds的完整日志可以在这个要点上找到

1 个解决方案

解决方案1
 2 已采纳  2020-09-22 15:02:55

TLDR：

qc 规则中使用的 fastqc 奇点容器可能没有可用的conda ，这不满足 snakemake 的--use-conda期望。

解释：

您在两个不同的级别定义了奇点容器 - 1. 将用于所有规则的全局级别，除非它们在规则级别被覆盖； 2. 将在规则级别使用的每个规则级别。

# global singularity container to use
singularity: "docker://continuumio/miniconda3:4.5.11"

# singularity container defined at rule level
rule qc_before_trim_r1:
    ....
    ....
    singularity:
        "https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0"

当您同时使用--use-singularity和--use-conda时，作业将在奇点容器内的 conda 环境中运行。 因此conda命令需要在奇点容器内可用才能实现。 虽然您的全局级容器显然满足了此要求，但我很确定（虽然尚未测试）您的fastqc容器并非如此。

如果提供--use-conda标志，snakemake 的工作方式将根据--use-singularity标志的提供在本地或容器内创建 conda 环境。 由于您将 snakemake-wrapper 用于 qc 规则，并且它带有预定义的 conda env 配方，因此这里最简单的解决方案是仅对所有规则使用全局定义的 miniconda 容器。 也就是说，不需要为 qc 规则使用 fastqc 特定容器。

如果你真的想使用 fastqc 容器，那么你不应该使用--use-conda标志，但这当然意味着所有必要的工具都可以从全局或每个规则定义的容器中获得。

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何在snakemake的奇异容器内加载环境变量脚本 Snakemake - 如何设置 conda 环境路径 强制snakemake在没有--use-conda的规则中运行python2 Conda Snakemake安装问题：UnsatisfiableError 在 snakemake 中激活现有的 conda 环境 SnakeMake不会激活conda环境 Pyboof：如何在Java中使用python包装器？ 如何通过 mac 使用 dtm 模型/包装器 dtm？ jupyter nbconvert 在 snakemake --use-conda 创建的环境中不起作用 从 dockerfile 激活奇异容器中的 conda 环境
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM