Building a Yocto bitbake recipe, which depends on grpc, grpc-native fails saying the the grpc_cpp_plugin does not exist

Question

I am developing a C++ component that uses gRPC and use the following definitions in my bitbase recipe (Yocto dunfell ) to build it with CMake:

DEPENDS += "\
    grpc \
    grpc-native \
    ...
    "

inherit cmake pkgconfig

I included grpc-native to be able to use the protoc compiler to generate stubs and grpc to link my component with the gRPC libraries for the target host.

In my CMakeLists.txt I use the following CMake functions to find the libraries/cpp-plugin

find_package(Protobuf REQUIRED)
find_package(gRPC CONFIG REQUIRED)
find_program(GRPC_CPP_PLUGIN grpc_cpp_plugin REQUIRED)

However, building the recipe fails with the following error:

| CMake Error at /home/jayjay/build-master/build/tmp/work/corei7-64-mbient-linux/mycomponent/git-r0/recipe-sysroot/usr/lib/cmake/grpc/gRPCTargets.cmake:197 (message):
|   The imported target "gRPC::grpc_cpp_plugin" references the file
|
|      "/home/jayjay/build-master/build/tmp/work/corei7-64-mbient-linux/mycomponent/git-r0/recipe-sysroot/usr/bin/grpc_cpp_plugin"
|
|   but this file does not exist.

And it is true, the grpc_cpp_plugin does not exist in this directory, as well as all the other plugins like grpc_node_plugin , grpc_php_plugin , etc. However, these plugins exist in the corresponding "recipe-sysroot -native /usr/bin/" directory, which is correct as they must be compiled for the build-host (rather than the target host), so that they can be used with protoc to generate stubs.

What I do not understand is, that why the existence of these plugin files are checked, although they are not created?

If I remove grpc from DEPENDS then everything works fine, unless the build host architecture is different from the target host architecture (ie for cross compiling). In this case I get a linker error "file in wrong format", which is OK, as I used the libraries compiled for the build host (grpc-native) and not the target host.

As a workaround I simply create all required plugin files in the desired directory like so:

do_configure_prepend() {
  touch ${STAGING_BINDIR}/grpc_cpp_plugin
  touch ${STAGING_BINDIR}/grpc_csharp_plugin
  touch ${STAGING_BINDIR}/grpc_node_plugin
  ...
}

This works, as only the existence of the files is checked. They are not used.

To be able to find the correct location of the directory with the "native" plugins I pass the following extra parameter to CMake:

EXTRA_OECMAKE = "\
    -DSTAGING_BINDIR_NATIVE=${STAGING_BINDIR_NATIVE} \
    "

Needless to say that this does not feel right.

I do not really grasp what am I doing wrong here (sorry for the long explanation)?

Answer 1

I have a workaround for Yocto Gatesgarth and gRPC version 1.24.x.

Basically I symlink the plugin binaries from recipe-sysroot-native/ to recipe-sysroot/ to make the CMake file happy.

Excerpt from my recipe file for a custom software project with CMake and grpc:

DEPENDS += "grpc"
DEPENDS += "grpc-native"

do_configure_prepend() {
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_cpp_plugin ${STAGING_BINDIR}/grpc_cpp_plugin
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_csharp_plugin ${STAGING_BINDIR}/grpc_csharp_plugin
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_node_plugin ${STAGING_BINDIR}/grpc_node_plugin
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_objective_c_plugin ${STAGING_BINDIR}/grpc_objective_c_plugin
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_python_plugin ${STAGING_BINDIR}/grpc_python_plugin
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_ruby_plugin ${STAGING_BINDIR}/grpc_ruby_plugin
    ln -f -s ${STAGING_BINDIR_NATIVE}/grpc_php_plugin ${STAGING_BINDIR}/grpc_php_plugin
}

Note that PROBABLY this will no longer work on later gRPC versions, as they seem to have fixed a lot of things regarding cross compilation in gRPC with version 1.35 and above (see oe-core changelog for grpc_*.bb).

Thanks goes out to all these sources!

Yocto Lists

Another Yocto Mailinglist

gRPC Issue on GitHub

gRPC Pull request with one solution

gRPC Pull request with another solution

Answer 2

Solution is very simple: use kirkstone

On dunfell, the gRPC recipe does not include https://github.com/grpc/grpc/pull/31525 which I believe fixes the build on Yocto. So technically you could also just update/patch/make your own gRPC recipe.