chttp2_server.cc:298：断言失败：*port_num == port_temp 在同一台机器上运行服务器和客户端时（仅在 Citrix 上）

Question

We have trouble finding the cause of a problem that only occurs when running our application on a citrix server.

The application is a windows GUI application starting 2 grpc services on dynamic ports - one c++ service and a the second service is a.NET component implemented in c#. Both services run on the same machine in the same process. The c++ service starts first and runs fine using 127.0.0.1:0 or localhost:0. When the c# service starts using 127.0.0.1:0 it works on our machines and some of the customers machines but fails on their citrix servers and I have now idea what can cause this.

We are using grpc v1.12.0 . The assertion at the end of this code block fires and kills our process.

grpc_error* grpc_chttp2_server_add_port(grpc_server* server, const char* addr,
                                        grpc_channel_args* args,
                                        int* port_num) {
  grpc_resolved_addresses* resolved = nullptr;
  grpc_tcp_server* tcp_server = nullptr;
  size_t i;
  size_t count = 0;
  int port_temp;
  grpc_error* err = GRPC_ERROR_NONE;
  server_state* state = nullptr;
  grpc_error** errors = nullptr;
  size_t naddrs = 0;

  *port_num = -1;

  /* resolve address */
  err = grpc_blocking_resolve_address(addr, "https", &resolved);
  if (err != GRPC_ERROR_NONE) {
    goto error;
  }
  state = static_cast<server_state*>(gpr_zalloc(sizeof(*state)));
  GRPC_CLOSURE_INIT(&state->tcp_server_shutdown_complete,
                    tcp_server_shutdown_complete, state,
                    grpc_schedule_on_exec_ctx);
  err = grpc_tcp_server_create(&state->tcp_server_shutdown_complete, args,
                               &tcp_server);
  if (err != GRPC_ERROR_NONE) {
    goto error;
  }

  state->server = server;
  state->tcp_server = tcp_server;
  state->args = args;
  state->shutdown = true;
  gpr_mu_init(&state->mu);

  naddrs = resolved->naddrs;
  errors = static_cast<grpc_error**>(gpr_malloc(sizeof(*errors) * naddrs));
  for (i = 0; i < naddrs; i++) {
    errors[i] =
        grpc_tcp_server_add_port(tcp_server, &resolved->addrs[i], &port_temp);
    if (errors[i] == GRPC_ERROR_NONE) {
      if (*port_num == -1) {
        *port_num = port_temp;
      } else {
        GPR_ASSERT(*port_num == port_temp);
      }
      count++;
    }
  }

Even with GRPC_VERBOSITY=debug and GRPC_TRACE=all I can't see any error nor useful hints.

The log then only shows this:

I0727 10:10:51.045681 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\init.cc:155: grpc_init(void)
I0727 10:10:51.045681 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\server.cc:944: grpc_server_create(0000009E5A5AA470, 0000000000000000)
I0727 10:10:51.045681 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\server.cc:930: grpc_server_register_completion_queue(server=0000009E5A40CAA0, cq=0000009E59D52FD0, reserved=0000000000000000)
I0727 10:10:51.045681 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\server.cc:930: grpc_server_register_completion_queue(server=0000009E5A40CAA0, cq=0000009E59D53490, reserved=0000000000000000)
I0727 10:10:51.045681 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\server.cc:930: grpc_server_register_completion_queue(server=0000009E5A40CAA0, cq=0000009E59D52D70, reserved=0000000000000000)
I0727 10:10:51.045681 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\server.cc:930: grpc_server_register_completion_queue(server=0000009E5A40CAA0, cq=0000009E59D536F0, reserved=0000000000000000)
I0727 10:10:51.061313 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\ext\transport\chttp2\server\insecure\server_chttp2.cc:34: grpc_server_add_insecure_http2_port(server=0000009E5A40CAA0, addr=localhost:0)
E0727 10:10:51.061313 0 T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\ext\transport\chttp2\server\chttp2_server.cc:298: assertion failed: *port_num == port_temp

Any hint of a possible cause would be very welcome.

After adding some more log messages to grpc I can see that there are actually 2 different addresses with different ports involved ([::1]:20530 and [::ffff:127.0.0.1]:20532)

I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\insecure\server_chttp2.cc:34: grpc_server_add_insecure_http2_port(server=000000DA0EF73120, addr=localhost:0)
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\chttp2_server.cc:291: WARNING: Zk, grpc_chttp2_server_add_port: naddrs = 2
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\chttp2_server.cc:298: WARNING: Zk, before grpc_tcp_server_add_port: port_temp = 484827504, resolved->addrs[0] = [::1]:0
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:501: Warning: Zk: grpc_sockaddr_to_v4mapped returned false
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:513: Warning: Zk: grpc_sockaddr_is_wildcard returned false
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:194: Warning: Zk: before grpc_tcp_prepare_socket
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:200: Warning: Zk: before bind
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:207: Warning: Zk: before listen
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:213: Warning: Zk: before getsockname
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:222: Warning: Zk: before grpc_sockaddr_get_port
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:226: Warning: Zk: sockname_temp: [::1]:20530
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\chttp2_server.cc:306: WARNING: Zk, after grpc_tcp_server_add_port: port_temp = 20530, errors[i] = 0
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\chttp2_server.cc:298: WARNING: Zk, before grpc_tcp_server_add_port: port_temp = 20530, resolved->addrs[1] = 127.0.0.1:0
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:496: Warning: Zk: grpc_sockaddr_to_v4mapped returned true
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:513: Warning: Zk: grpc_sockaddr_is_wildcard returned false
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:194: Warning: Zk: before grpc_tcp_prepare_socket
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:200: Warning: Zk: before bind
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:207: Warning: Zk: before listen
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:213: Warning: Zk: before getsockname
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:222: Warning: Zk: before grpc_sockaddr_get_port
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\lib\iomgr\tcp_server_windows.cc:226: Warning: Zk: sockname_temp: [::ffff:127.0.0.1]:20532
I0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\chttp2_server.cc:306: WARNING: Zk, after grpc_tcp_server_add_port: port_temp = 20532, errors[i] = 0
E0727 14:22:16.875600 0 E:\grpc-v1.12.0\grpc\src\core\ext\transport\chttp2\server\chttp2_server.cc:313: assertion failed: *port_num == port_temp

Answer 1

The reason was that we were using localhost for the .NET grpc service instead of 127.0.0.1 .

ports = new ServerPort("localhost", 0, ServerCredentials.Insecure);

Grpc_blocking_resolve_address thus provided an address for ipv4 and ipv6. grpc_tcp_server_add_port was then called for both addresses and delivered 2 different ports that led to the assertion. Unfortunately there were no diagnostic options in Grpc here.

After changing localhost to 127.0.0.1 out application works fine.

ports = new ServerPort("127.0.0.1", 0, ServerCredentials.Insecure);