堆棧內存溢出
  簡體   English   中英

從 gRPC 服務到 RESTful 客戶端的響應返回 stream 時出現問題

[英]Problem returning stream of responses from gRPC service to RESTful client

 原文  2019-09-27 14:20:37  6   2    java/ rest/ spring-boot/ grpc-java

問題描述

我正在嘗試編寫一組服務，這些服務將為外部客戶端提供 REST 接口，同時在內部使用 gRPC/protobuf 進行服務間通信。 Spring Boot 用於幫助支持后台基礎架構。 大約一周前，我才開始學習和使用 Google protobuf 和 gRPC。

我已經能夠成功地獲得更簡單的端到端處理，但是這些返回單個 object （即沒有流）。 我在調用 stream 響應對象集合時遇到問題。 另一件需要注意的是，到目前為止我所做的一切都是 GET 請求。

下面的示例僅顯示了我遇到問題的相關部分。

細節：

Protobuf 定義：

syntax = "proto3";

package experiment;

message BOM {
    PartSpec part_spec = 1;
    string description = 2;
    string category = 3;
}

message PartSpec {
    string part_number = 1;
    string cage_code = 2;
}

service BOMService {
    rpc getBOMList(google.protobuf.Empty) returns (stream BOM) {}
}

“域”服務：

實現 .proto 文件定義的服務。 響應來自“網關”服務的請求。

@GrpcService
public class BOMDomainService extends BOMServiceImplBase {

  private final Logger logger = LogManager.getLogger();

  private Random       rand   = new Random(1);

  @Override
  public void getBOMList(Empty request, StreamObserver<BOM> responseObserver) {
    for (int i = 1; i <= rand.nextInt(40); i++) {
      PartSpec part = PartSpec.newBuilder().setPartNumber("part-1").setCageCode("cage-1").build();
      BOM bom = BOM.newBuilder().setPartSpec(part).setDescription("some part").build();
      // Stream BOM response.
      responseObserver.onNext(bom);
    }

    // Complete server response.
    responseObserver.onCompleted();
  }
}

“網關”

網關服務：

通過 gRPC 調用（阻塞存根）與“域”服務交互。

@Service
public class BOMGatewayService {

  private final Logger           logger = LogManager.getLogger();

  @GrpcClient("bom-service")
  private BOMServiceBlockingStub bomSvcStub;

  public List<BOM> getBOMList() {
    List<BOM> result = new ArrayList<>();

    try {
      Iterator<BOM> iter = bomSvcStub.getBOMList(Empty.getDefaultInstance());

      while (iter.hasNext()) {
        result.add(iter.next());
      }
    } catch (StatusRuntimeException e) {
      logger.warn("RPC failed: {}", e.getStatus());
    }

    return result;
  }
}

網關 controller：

@RestController
@RequestMapping("/alisn/bom")
public class BOMController {

  @Autowired
  private BOMGatewayService service;

  @GetMapping(value = "/getBOMList", produces = { MediaType.APPLICATION_JSON_VALUE })
  public DeferredResult<ResponseEntity<List<BOM>>> getBOMList() {
    DeferredResult<ResponseEntity<List<BOM>>> result = new DeferredResult<>();
    List<BOM> response = service.getBOMList();
    result.setResult(ResponseEntity.ok(response));
    return result;
  }
}

外部“客戶”服務：

通過 REST 調用與“網關”服務交互。 不使用 gRPC，但使用從 .proto 文件生成的消息類。

注意：我不想在這里獲取“BOM”列表。 如果我嘗試使用ResponseEntity<List<BOM>> ， getForEntity()方法會引發編譯器錯誤。 我不確定我應該在這里做什么，也許這是問題的一部分。

@Service
public class ClientService {

  private final Logger logger = LogManager.getLogger();

  @Autowired
  private RestTemplate restTemplate;

  @Scheduled(fixedDelay = 5000)
  public void getBOMList() {
    ResponseEntity<BOM> response = restTemplate
        .getForEntity("http://localhost:7090/alisn/bom/getBOMList", BOM.class);
    logger.info("Received:\n{}", response.getBody());
  }
}

我試圖通過兩種方式來練習：

  1. 從客戶端 Java Spring 使用RestTemplate進行 REST 調用的引導服務（上面顯示的“客戶端”）。
  2. 從 Postman 發出 HTTP GET 請求。

兩種方法的反應似乎或多或少相同。

Java客戶端方法：

客戶端控制台：

[2019-09-27 10:01:16.373] scheduling-1 ERROR: support.TaskUtils$LoggingErrorHandler:96 - Unexpected error occurred in scheduled task.
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 null
    at org.springframework.web.client.HttpServerErrorException.create(HttpServerErrorException.java:79) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:124) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:102) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:778) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:736) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:670) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.getForEntity(RestTemplate.java:338) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at edu.mit.ll.alisn.exp.client.service.ClientService.getBOMList(ClientService.java:26) ~[classes/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_201]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_201]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_201]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

“網關”控制台：

[2019-09-27 10:01:16.356] http-nio-7090-exec-3 ERROR: [/].[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] threw exception
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["defaultInstanceForType"])
    at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContentsUsing(CollectionSerializer.java:171) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:116) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[jackson-databind-2.9.9.jar:2.9.9]
    at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:287) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:103) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:290) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProcessor.handleReturnValue(HttpEntityMethodProcessor.java:225) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:82) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:122) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) ~[spring-boot-actuator-2.1.6.RELEASE.jar:2.1.6.RELEASE]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) ~[spring-boot-actuator-2.1.6.RELEASE.jar:2.1.6.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:633) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:601) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.AsyncContextImpl$AsyncRunnable.run(AsyncContextImpl.java:566) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.AsyncContextImpl.doInternalDispatch(AsyncContextImpl.java:355) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:235) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:241) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:53) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

Postman：

{
    "timestamp": "2019-09-27T14:03:38.530+0000",
    "status": 500,
    "error": "Internal Server Error",
    "message": "Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM[\"unknownFields\"]->com.google.protobuf.UnknownFieldSet[\"defaultInstanceForType\"])",
    "path": "/alisn/bom/getBOMList"
}

任何關於我做錯了什么的建議，或關於更好地執行此類過程的建議將不勝感激。

更新：

我還包括客戶端和網關服務的 bean 配置。

客戶：

@Configuration
@EnableScheduling
@ComponentScan(XXXXXX)
public class Configurer {
  @Bean
  ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufJsonFormatHttpMessageConverter();
  }

  @Bean
  RestTemplate restTemplate(ProtobufHttpMessageConverter hmc) {
    return new RestTemplate(Arrays.asList(hmc));
  }
}

網關：

@Configuration
@ComponentScan(XXXXXX)
public class Configurer {
  @Bean
  ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufJsonFormatHttpMessageConverter();
  }
}

更新：

我做了一個更簡單的實驗，我創建了一個只返回一個BOM object 的新服務。 這工作得很好。

我不明白為什么這行得通，而List<BOM>案例卻不行。 拋出的異常中的信息似乎指向生成的BOM class 中的內部結構。

網關控制台：

[2019-10-02 08:37:25.960] http-nio-7090-exec-3 ERROR: [/].[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet$Parser]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])] with root cause
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])

對我的參考： java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"]表明BOM class 的內部結構存在問題，但我可以毫無問題地返回單個BOM 。

2 個解決方案

解決方案1
 0  2019-09-27 16:57:20

根據網關日志，看起來jackson無法將 proto 轉換為 json。

你能試試com.googlecode.protobuf.format.JsonFormat嗎？ 我不是 spring 用戶，但應該有 API 用於自定義序列化。 或者，您可以使用JsonFormat向 jackson 添加自定義序列化，但這聽起來有點奇怪。

解決方案2
 0  2020-12-05 22:17:46

在可能的情況下，我有： .... response.setMessageCode(0).setResponseMessage("string message"); ....永遠不會返回給客戶。

我將其更改為： ... APIResponse response = APIResponse.newBuilder().setResponseMessage(localidad.toString()).build(); ...

沒有“.setMessageCode(0)”......它可以工作！

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 從 gRPC 服務到 REST 客戶端的 Stream 響應 從 gRPC 服務中調用 gRPC 客戶端時不工作 從RESTful服務檢索ArrayList到Jersey客戶端 RESTful Java 客戶端將 JSON 響應轉換為對象 為RESTful服務創建客戶端 使用Jersey從客戶端調用RESTFul Web Service的問題 如何從Enterprise Application Client使用靜態Web服務？ 是否可以從EJB2客戶端調用Restful Web服務 寧靜的服務沒有返回正確的對象 Jersey客戶端調用RESTful服務
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM