使用 genrule 获取 ISPC 的 output

Question

I am trying to make use of ISPC ( Implicit SPMD Program Compiler ) using Bazel . Therefore, I started to implement rules_ispc .

Unfortunately, I run into an issue with generating files using ispc .

You can reproduce my issue by:

git clone https://github.com/Vertexwahn/rules_ispc.git
cd rules_ispc
cd tests
bazel build //example:main

The following error will be generated:

ERROR: /home/vertexwahn/rules_ispc/tests/example/BUILD.bazel:3:16: declared output 'example/square.o' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely) ERROR: /home/vertexwahn/rules_ispc/tests/example/BUILD.bazel:3:16: declared output 'example/square.h' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely) ERROR: /home/vertexwahn/rules_ispc/tests/example/BUILD.bazel:3:16: Executing genrule //example:square_ispc_gen failed: not all outputs were created or valid

I see the same error on Ubuntu 20.04 , Ubuntu 22.04 , Windows Server 2019 , Windows Server 2022 , macOS 11 and macOS 12 . For details see my CI workflow .

My BUILD.bazel file looks like this:

load("@rules_ispc//:ispc.bzl", "ispc_cc_library")

ispc_cc_library(
    name = "square",
    srcs = ["square.ispc"],
)

cc_binary(
    name = "main",
    srcs = ["main.cpp"],
    deps = [":square"],
)

ispc_cc_library calls ISPC internal to generate a header file and an o file:

def ispc_cc_library(name, srcs,  target_compatible_with = [], **kwargs):
    for ispc_source_file in srcs:
        generted_header_filename = name + ".h"
        native.genrule(
            name = "%s_ispc_gen" % name,
            srcs = [ispc_source_file],
            outs = [name + ".o", generted_header_filename],
            cmd = select({
                "@platforms//os:linux": "$(location @ispc_linux_x86_64//:ispc) --target=avx2 --arch=x86-64 $(locations %s) --header-outfile=%s -o %s.o" % (ispc_source_file, generted_header_filename, name),
                "@platforms//os:osx": "$(location @ispc_osx_x86_64//:ispc) --target=avx2 --arch=x86-64 $(locations %s) --header-outfile=%s -o %s.o" % (ispc_source_file, generted_header_filename, name),
                "@platforms//os:windows": "$(location @ispc_windows_x86_64//:ispc) --target=avx2 --target-os=windows --arch=x86-64 $(locations %s) --header-outfile=%s -o %s.o" % (ispc_source_file, generted_header_filename, name),
            }),
            tools = select({
                "@platforms//os:linux": ["@ispc_linux_x86_64//:ispc"],
                "@platforms//os:osx": ["@ispc_osx_x86_64//:ispc"],
                "@platforms//os:windows": ["@ispc_windows_x86_64//:ispc"],
            }),
            target_compatible_with = target_compatible_with,
        )
        native.cc_library(
            name = name,
            srcs = [name + ".o"],
            hdrs = [name + ".h"],
            target_compatible_with = target_compatible_with,
            **kwargs
        )

I wonder how to fetch the generated files of ISPC correctly with the genrule . Any hints welcome!

More details:

square.ispc :

export void square(uniform float vin[], uniform float vout[],
                   uniform int count) {
    foreach (index = 0 ... count) {
        float v = vin[index];
        v = v * v;
        vout[index] = v;
    }
}

The header generated by ISPC should look this:

square.h

//
// C:/rules_ispc/tests/example/square.h
// (Header automatically generated by the ispc compiler.)
// DO NOT EDIT THIS FILE.
//

#pragma once
#include <stdint.h>



#ifdef __cplusplus
namespace ispc { /* namespace */
#endif // __cplusplus

#ifndef __ISPC_ALIGN__
#if defined(__clang__) || !defined(_MSC_VER)
// Clang, GCC, ICC
#define __ISPC_ALIGN__(s) __attribute__((aligned(s)))
#define __ISPC_ALIGNED_STRUCT__(s) struct __ISPC_ALIGN__(s)
#else
// Visual Studio
#define __ISPC_ALIGN__(s) __declspec(align(s))
#define __ISPC_ALIGNED_STRUCT__(s) __ISPC_ALIGN__(s) struct
#endif
#endif


///////////////////////////////////////////////////////////////////////////
// Functions exported from ispc code
///////////////////////////////////////////////////////////////////////////
#if defined(__cplusplus) && (! defined(__ISPC_NO_EXTERN_C) || !__ISPC_NO_EXTERN_C )
extern "C" {
#endif // __cplusplus
    extern void square(float * vin, float * vout, int32_t count);
#if defined(__cplusplus) && (! defined(__ISPC_NO_EXTERN_C) || !__ISPC_NO_EXTERN_C )
} /* end extern C */
#endif // __cplusplus


#ifdef __cplusplus
} /* namespace */
#endif // __cplusplus

If I search for the generated header file square.h in the directories generated by Bazel it seems that this file does not exist, ie is not generated.

Answer 1

You need to perform $(location) expansion on the outputs, too, since Bazel will assign them paths in the bazel-out/ output tree.

diff --git a/ispc.bzl b/ispc.bzl
index 1fca9b4..a1b9450 100644
--- a/ispc.bzl
+++ b/ispc.bzl
@@ -6,9 +6,9 @@ def ispc_cc_library(name, srcs,  target_compatible_with = [], **kwargs):
             srcs = [ispc_source_file],
             outs = [name + ".o", generted_header_filename],
             cmd = select({
-                "@platforms//os:linux": "$(location @ispc_linux_x86_64//:ispc) --target=avx2 --arch=x86-64 $(locations %s) --header-outfile=%s -o %s.o" % (ispc_source_file, generted_header_filename, name),
-                "@platforms//os:osx": "$(location @ispc_osx_x86_64//:ispc) --target=avx2 --arch=x86-64 $(locations %s) --header-outfile=%s -o %s.o" % (ispc_source_file, generted_header_filename, name),
-                "@platforms//os:windows": "$(location @ispc_windows_x86_64//:ispc) --target=avx2 --target-os=windows --arch=x86-64 $(locations %s) --header-outfile=%s -o %s.o" % (ispc_source_file, generted_header_filename, name),
+                "@platforms//os:linux": "$(location @ispc_linux_x86_64//:ispc) --target=avx2 --arch=x86-64 $(locations %s) --header-outfile=$(location %s) -o $(location %s.o)" % (ispc_source_file, generted_header_filename, name),
+                "@platforms//os:osx": "$(location @ispc_osx_x86_64//:ispc) --target=avx2 --arch=x86-64 $(locations %s) --header-outfile=$(location %s) -o $(location %s.o)" % (ispc_source_file, generted_header_filename, name),
+                "@platforms//os:windows": "$(location @ispc_windows_x86_64//:ispc) --target=avx2 --target-os=windows --arch=x86-64 $(locations %s) --header-outfile=$(location %s) -o $(location %s.o)" % (ispc_source_file, generted_header_filename, name),
             }),
             tools = select({
                 "@platforms//os:linux": ["@ispc_linux_x86_64//:ispc"],
diff --git a/tests/example/main.cpp b/tests/example/main.cpp
index 44f7feb..1883757 100644
--- a/tests/example/main.cpp
+++ b/tests/example/main.cpp
@@ -1,4 +1,4 @@
-#include "square.h"
+#include "example/square.h"
 
 #include <stdio.h>