How to avoid nested imports when compiling proto files from a java project into python modules

Question

eg in a java project

myapp
└── src
    └── main
        └── proto
            └── com
                └── abc
                    └── myapp
                        └── api
                            ├── common.proto
                            └── myapp.proto

$ cat myapp/src/main/proto/com/abc/myapp/api/common.proto

syntax = "proto3";

package com.abc.myapp.api;

message Metadata {

  string key = 1;
}

$ cat myapp/src/main/proto/com/abc/myapp/api/myapp.proto

syntax = "proto3";

import "com/abc/myapp/api/common.proto";

package com.abc.myapp.api;

message Request {
    string name = 1;

    Metadata metada = 2;
}

when compiling the protos into python modules

$ protoc -I myapp/src/main/proto --python_out=tmp  com/abc/myapp/api/common.proto 
$ protoc -I myapp/src/main/proto --python_out=tmp  com/abc/myapp/api/myapp.proto

the output structure is like

tmp
└── com
    └── abc
        └── myapp
            └── api
                ├── common_pb2.py
                └── myapp_pb2.py

and inside myapp_pb2.py , the import is very nested and like

from com.abc.myapp.api import common_pb2 as com_dot_abc_dot_myapp_dot_api_dot_common__pb2

Is there a recommended practice to make the compiled pb2 files flat in structure? eg

tmp
├── common_pb2.py
└── myapp_pb2.py

and myapp_pb2.py has import like

from . import common_pb2 as ...

instead of

from com.abc.myapp.api import common_pb2 com.abc.myapp.api ...

Answer 1

You could create an init file in the directory with your protobufs and then add the following code:

# pb/__init__.py
import sys
from pathlib import Path
sys.path.append(str(Path(__file__).parent))

This should allow you to import the files the way you want.

Please notice, that what you're asking is not a feature that is currently available and my proposed solution is just one of many being used.

This is a known irritation source for many python users of protobuf, see: https://github.com/protocolbuffers/protobuf/pull/7470 and https://github.com/protocolbuffers/protobuf/issues/1491