Instantiating a Bazel macro twice with same generated output file

Question

. 。 Answers to this question are eligible for a +50 reputation bounty. Francesco Boi wants to to this question. 。

Suppose I have a Bazel macro that is using a generator rule to generate an output file given an input file:

def my_generator(
        name,
        input_file,
        output_file,
        **kwargs):
    args = []
    args.extend(["--arg1", "$(location %s)" % output_file])
    args.extend(["arg2", "$(locations %s)" % input_file])

    cmd_params = " ".join(args)

    native.genrule(
        name = name,
        srcs = [input_file],
        outs = [output_file],
        cmd = "python $(location //path/to:target_generator) %s" % cmd_params,
        tools = ["/path/to/tool:mytool"],
    )

Then I was previously using this macro as:

my_generator(
    name = "gen1",
    input_file = ":targetToGeneratetextFile",
    output_file = "outputfile.txt",
    visibility = ["//myproject/oath/to/current/package/test:__subpackages__"],
)

where a target is passed as input_file . This was working.

Then I wanted to reuse it with a different input but to generate the same output, where the input is now a file within the project but in another folder.

my_generator(
    name = "gen2",
    input_file = "//path/to/the/file/realFile.txt",
    output_file = "outputfile.txt",
    visibility = ["//myproject/oath/to/current/package/test:__subpackages__"],
)

I am getting two errors in this way:

1. For how it is, Bazel cannot find the realFile.txt : it tries to read it as a target:

no such package '//path/to/the/file/realFile.txt': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package

If I copy the file in the current package folder, it is able to read it.

2. Bazel is complaining that gen1 and gen2 are writing/overwriting the same output file outputfile.txt :

Error in genrule: generated file 'outputfile.txt' in rule 'gen2' conflicts with existing generated file from rule 'gen1', defined at...

How can I solve these issues?

I think that the problem is that these two calls are both executed, whereas I would like them to be executed depending on some target, ie, target A needs only run gen1 and target B gen2 exclusively. I do not if that is possible but for example moving each of these call inside the target they belong to might be a solution that avoids this issue.

EDIT I was thinking as solution to do something like:

my_generator(
    name = "gen2",
    input_file = select({
       ":opt1": [":targetToGeneratetextFile"],
       ":opt2": ["realTextFile.txt"],
       "//conditions:default": [":targetToGeneratetextFile"],
    }),
    output_file = "outputfile.txt",
    visibility = ["//myproject/oath/to/current/package/test:__subpackages__"],
)

with proper config_setting and then call it from the target with the proper flag but I am getting the error:

expected value of type 'string' for element 0 of attribute 'srcs' in 'genrule' rule, but got select({":opt1": [":targetToGeneratetextFile"], ":opt2": ["realTextFile.txt"],"//conditions:default": [":targetToGeneratetextFile"], })

Answer 1

The label //path/to/the/file/realFile.txt is shorthand for //path/to/the/file/realFile.txt:realFile.txt , aka <repository root>/path/to/the/file/realFile.txt/realFile.txt . Depending on where the deepest-nested folder with a BUILD file is (which determines the package), you're looking for something like //path/to/the/file:realFile.txt or //path/to:the/file/realFile.txt instead.

You can't have two rules which write the same file, because then Bazel can't tell which way to build it if you bazel build the file. Some alternatives:

• Put them in separate packages (aka separate folders with BUILD files)
• Name them differently, like gen1_outputfile.txt and gen2_outputfile.txt , or gen1/outputfile.txt and gen2/outputfile.txt . You could automate this in the macro like srcs = [name + '/outputfile.txt'] .
• Use a single rule to generate it with an appropriate select, like your edit.

With the select , you're trying to create something like this:

genrule(
    srcs = select({..., "//conditions:default": [":targetToGeneratetextFile"]}),
    ...
)

but as written you have this instead:

genrule(
    srcs = [select({..., "//conditions:default": [":targetToGeneratetextFile"]})],
    ...
)

Effectively, between the list in the select 's value and the macro body, you're creating a nested list. I would change your macro argument to input_files and then do srcs = input_files in the body, so the caller of the macro can bundle things into lists as desired.