Spring webflux webclient 在评估第一次呼叫的响应时再次拨打电话

Question

I am calling an endpoint with webclient and want to map the response that I get to another object. While mapping that object I want to make additional calls for some parameters of the response.

My first call returns the following object

{
  "type": "Collection",
  "key": "some.key",
  "settings": [
    {
      "type": "Struct",
      "key": "steps",
      "value": [
        {
          "type": "Struct",
          "key": "1",
          "value": [
            {
              "type": "String",
              "key": "headline",
              "value": "someheadline"
            },
            {
              "type": "String",
              "key": "subheadline",
              "value": "somesubheadline"
            },
            {
              "type": "Link",
              "key": "link.to.another.object",
              "value": {
                "linkType": "Boilerplate",
                "key": "configurabletextkey"
              }
            }
          ]
        }
      ]
    },
    {
      "type": "Struct",
      "key": "commons",
      "value": [
        {
          "type": "String",
          "key": "mandatory.fields.text",
          "value": "Pflichtfelder"
        }
      ]
    }
  ]
}

I map that response like so:

webClient
        .get()
        .uri(
            uriBuilder ->
                uriBuilder
                    .path(headlessConfig.getEndpoint())
                    .pathSegment(contentId)
                    .build())
        .retrieve()
        .bodyToMono(Collection.class)
        .map(response -> {
                  return getCollectionContent(response);
                })

In the getCollectionContent method I iterate over the settings array and I extract data from the response and map it to my PageContent Object.

public class PageContent {

  private String pageId;

  private List<Message> messages;
}

public class Message {

  @NonNull private String key;

  @NonNull private String text;

  private Boolean containsHtml = false;
}

If the response contains the type "String" i'll just add the data to an Message Object and add it to the List of the PageContent.

Now to the problem. If the type is "Link" I want to make another call with webclient like above to the same endpoint to get the key and text for that object, create a Message Object out of it and add it to my existing List.

The corresponding code looks like so:

webClient
        .get()
        .uri(
            uriBuilder ->
                uriBuilder
                    .path(headlessConfig.getEndpoint())
                    .pathSegment(contentKey)
                    .build())
        .retrieve()
        .bodyToMono(ConfigurableText.class)
        .map(
                configurableTextResponse -> {
                  messages.add(
                      new Message(
                          prefix + configurableTextResponse.getKey(),
                          configurableTextResponse.getText(),
                          true));
                  return Mono.empty();
                })

Now when I try to do that nothing happens and I just receive the PageContent object without the message for the link.

In a blocking way with resttemplate this logic should work but I would like to get it to work with webclient.

Edit:

Code to iterate through the list and extract the message data:

private PageContent getCollectionContent(Collection response) {

    PageContent pageContent = new PageContent();
    pageContent.setPageId(response.getKey());

    List<Message> messages = new ArrayList<>();

    response
        .getSettings()
        .forEach(
            settingsItemsArray -> {
              var settingsItemList = (List<?>) settingsItemsArray.getValue();
              String prefix = settingsItemsArray.getKey() + ".";

              extractMessageText(prefix, (LinkedHashMap<?, ?>) settingsItemList.get(0), messages);
            });

    pageContent.setMessages(messages);

    return pageContent;
  }

Code to extract the MessageText, iterate further or get the missing text for a link type.

private void extractMessageText(
      String prefix, LinkedHashMap<?, ?> settingsItem, List<Message> messages) {

    String itemKey = (String) settingsItem.get(KEY);
    String itemType = (String) settingsItem.get(TYPE);

    switch (itemType) {
      case "String":
        messages.add(new Message(prefix + itemKey, (String) settingsItem.get(VALUE)));
        break;
      case "Struct":
        ((List<?>) settingsItem.get(VALUE))
            .forEach(
                structItems ->
                    extractMessageText(
                        prefix + settingsItem.get(KEY) + ".",
                        (LinkedHashMap<?, ?>) structItems,
                        messages));
        break;
      case "Link":
        webClient
        .get()
        .uri(
            uriBuilder ->
                uriBuilder
                    .path(headlessConfig.getEndpoint())
                    .pathSegment(contentKey)
                    .build())
        .retrieve()
        .bodyToMono(ConfigurableText.class)
        .map(
                configurableTextResponse -> {
                  messages.add(
                      new Message(
                          prefix + configurableTextResponse.getKey(),
                          configurableTextResponse.getText(),
                          true));
                  return Mono.empty();
                })
        break;
      default:
        break;
    }
  }

Answer 1

I have changed some of your code to make it more compatible with the reactor pattern. I have changed the recursion into expandDeep and also used Jackson to parse the JSON. I hope this will give you some ideas how to solve your issue.

List<Message> messages = Flux
                .fromIterable(jsonNode.get("settings"))
                //expand the graph into a stream of flat data and track the address of the node with 'prefix'
                //expand/exapndDeep operators are alternatives of recursion in project reactor
                .expandDeep(parent -> {
                    String parentPrefix = Optional.ofNullable(parent.get("prefix")).map(JsonNode::asText)
                            .orElse(parent.get("key").asText());
                    String type = parent.get("type").asText();
                    if (type.equals("Struct")) {
                        return Flux.fromIterable(parent.get("value"))
                                .cast(ObjectNode.class)
                                .map(child -> child.put("prefix", parentPrefix + ":" + child.get("key").asText()));
                    }
                    return Mono.empty();
                })
                //we have to choose only leaf nodes aka String and Link nodes
                .filter(node -> Arrays.asList("String", "Link").contains(node.get("type").asText()))
                //now process expanded leaf nodes
                .flatMap(leaf -> {
                    if ("String".equals(leaf.get("type").asText())) {
                        return Mono.just(new Message(leaf.get("prefix").asText(), leaf.get("value").asText(), true));
                    }
                    if ("Link".equals(leaf.get("type").asText())) {
                        return webClient
                                .get()
                                .uri(
                                        uriBuilder ->
                                                uriBuilder
                                                        .pathSegment(leaf.get("key").asText())
                                                        .build())
                                .retrieve()
                                .bodyToMono(JsonNode.class)
                                .map(configurableTextResponse -> new Message(
                                        leaf.get("prefix") + configurableTextResponse.get("key").asText(),
                                        configurableTextResponse.get("text").asText(),
                                        true));
                    }
                    return Mono.empty();
                })
                // at this point we are getting stream of the Message objects from the Link/String nodes
                //collect them into a list
                .collectList()
                //we have to subscribe()/block() the mono to actually invoke the pipline.
                .block();

The main reason your code did nothing was that you were not subscribing to your WebClient pipeline.

EDIT:

change

  .map(response -> {
                  return getCollectionContent(response);
                })

to

.flatMap(response -> {
                      return getCollectionContent(response);
                    })

and return from getCollectionContent(response) Mono<PageContent> page

something like:

     // at this point we are getting stream of the Message objects from the Link/String nodes
                    //collect them into a list
                    .collectList()
                    .map(messages -> {
                        PageContent pageContent = new PageContent();
                        pageContent.setPageId(response.get("pageId").asText());
pageContent.setMessages(messages);
                        return pageContent;
                    });

after these changes, your getCollectionContent() will return a publisher Mono<PageContent> which will be subscribed from the flatMap operator.