Python : Process calling GRPC server gets stuck and terminates unexpectedly

Question

I have one GRPC server running. say: localhost, 46001

I have a python program which is creating a Process Pool of 4 processes. Here the Process Pool is custom:

Process_Pool(Process): have a process dict with 4 running process context.

I have maintained this process pool for my use case. Note : this functionality is working fine as i tested without GRPC involvement.

GRPC Client implementation is like:

class Client(object):

def __init__(self, host, port):
    self.host = host
    self.server_port = port
    self.channel = grpc.insecure_channel("{}:{}".format(self.host, self.server_port))
    self.stub = some_server_pb2_grpc.SomeInterfaceStub(self.channel)

Which is working fine when i run it from command line:

python

client = Client("localhost", 46001)

my_dict= {'timestamp': 1555421947, 'additional_info': {}}

client.set_status(my_dict)

The problem i am facing is, while creating a GRPC client call to server from those 4 processes like:

client = Client("localhost", 46001)
client.set_status(some_dict)

The process terminates itself unexpectedly. No logs gets printed further. I have handled every exception, even around the above code call.

My only inference till now is: GRPC does not work with newly spawned Processes.

Can anyone help me in solving this?

Answer 1

gRPC Python client supports multi-processing with environment variable GRPC_ENABLE_FORK_SUPPORT=1 .

However, the fork support story for gRPC Python server is more complicated (see https://github.com/grpc/grpc/issues/18321 ).