Spring Boot: Control if null fields present in serialized response per request

Question

Introduction

We have a REST API implemented in Spring Boot. Currently it returns all fields when serialising. So it returns something like

{
    "foo": "A",
    "bar": null,
    "baz": "C",
}

We would like the option to not return the null fields, so it would just return

{
    "foo": "A",
    "baz": "C",
}

for that case - but still (if bar had a value)

{
    "foo": "A",
    "bar": "B",
    "baz": "C",
}

I know you can steer it it to not return nulls via the application properties, but this is an existing AI and some applications implemented against it might fail on deserialization if the fields were missing. We'd therefore like to let the calling client steer this. Our idea is to have a header you can send in: X-OurCompany-IncludeNulls; false X-OurCompany-IncludeNulls; false . This would allow the client to choose and we would initially default to true but might change the default in a managed way over time.

The nearest I could find was this which was steering pretty-printing via a query parameter. When I try doing something similar it works for pretty-printing. However, for the inclusion it works for the first request after I start the API, but after that every other request gets the value from the first request. I can see it is setting it, via a breakpoint, and also I added pretty-print against the same parameter just for diagnostic purposes.

Details of What I Tried

Our API is based off one generated using the Swagger Codegen server stub. We use the delegate pattern, so it generates a controller which just has an auto-wired delegate and a getDelegate

@Controller
public class BookingsApiController implements BookingsApi {

    private final BookingsApiDelegate delegate;

    @org.springframework.beans.factory.annotation.Autowired
    public BookingsApiController(BookingsApiDelegate delegate) {
        this.delegate = delegate;
    }

    @Override
    public BookingsApiDelegate getDelegate() {
        return delegate;
    }
}

The delegate is an interface which includes a function per endpoint. These return CompletableFuture<ResponseEntity<T>> (where T is the type of that response). It also a getObjectMapper() which I assume is what is then used by Spring to serialise the response?

public interface BookingsApiDelegate {

    Logger log = LoggerFactory.getLogger(BookingsApi.class);

    default Optional<ObjectMapper> getObjectMapper() {
        return Optional.empty();
    }

    default Optional<HttpServletRequest> getRequest() {
        return Optional.empty();
    }

    default Optional<String> getAcceptHeader() {
        return getRequest().map(r -> r.getHeader("Accept"));
    }
    
    // Functions per endpoint here.  By default returns Not Implemented.
}

We have an object we call ApiContext . This is scoped to a custom scope we call ApiCallScoped - basically per-request but it handles async and copies over to the created threads. We already had something implementing HandlerInterceptorAdapter (although ours is @Component instead of @Bean like in the pretty-print example above). We create the context in preHandle so I thought to add it there to set the object mapper properties. Leaving aside some clean-up this looks like:

@Component
public class RestContextInterceptor extends HandlerInterceptorAdapter {

  @Autowired
  private ContextService apiContextService;
  @Autowired
  private RestRequestLogger requestLogger;
  @Autowired
  private ObjectMapper mapper;
  @Autowired
  private Jackson2ObjectMapperBuilder objectMapperBuilder;
  @Autowired
  private MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter;

  @Override
  public boolean preHandle(final HttpServletRequest request, final HttpServletResponse response,
      final Object handler) throws Exception {
    requestLogger.requestStart(request);

    if (request.getAttribute("apiCallContext") == null) {
      ApiCallContext conversationContext;
      ApiContext apiContext = readApiContext(request, mapper, objectMapperBuilder, mappingJackson2HttpMessageConverter);
      if (apiContext == null) {
        conversationContext = new ApiCallContext("local-" + UUID.randomUUID().toString());
      } else {
        conversationContext = new ApiCallContext(apiContext.getTransId());
      }
      ApiCallContextHolder.setContext(conversationContext);
      request.setAttribute("apiCallContext", conversationContext);

      if (apiContext != null) {
        apiContextService.setContext(apiContext);
      }
    } else {
      ApiCallContextHolder.setContext((ApiCallContext) request.getAttribute("apiCallContext"));
    }

    return true;
  }

  private static ApiContext readApiContext(
      final HttpServletRequest request,
      final ObjectMapper mapper,
      final Jackson2ObjectMapperBuilder objectMapperBuilder,
      final MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter) {
    if (request.getHeader(ApiContext.SYSTEM_JWT_HEADER) != null) {
      return new ApiContext(Optional.of(request), mapper, objectMapperBuilder, mappingJackson2HttpMessageConverter);
    }
    return null;
  }
}

In the ApiContext we look at the header. I tried

public final class ApiContext implements Context {
  private final ObjectMapper mapper;
  public ApiContext(
      final Optional<HttpServletRequest> request,
      final ObjectMapper mapper,
      final Jackson2ObjectMapperBuilder objectMapperBuilder,
      final MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter
  ) {
    if (!request.isPresent()) {
      throw new InvalidSessionException("No request found");
    }

    if (getBooleanFromHeader(request, NULLMODE_HEADER).orElse(DEFAULT_INCLUDE_NULL_FIELDS_IN_OUTPUT)) {
      objectMapperBuilder.serializationInclusion(JsonInclude.Include.ALWAYS);
      objectMapperBuilder.indentOutput(false);
      mappingJackson2HttpMessageConverter.getObjectMapper().setPropertyInclusion(
          JsonInclude.Value.construct(JsonInclude.Include.ALWAYS, JsonInclude.Include.ALWAYS));
      mapper.setPropertyInclusion(
          JsonInclude.Value.construct(JsonInclude.Include.ALWAYS, JsonInclude.Include.ALWAYS));
    } else {
      objectMapperBuilder.serializationInclusion(JsonInclude.Include.NON_EMPTY);
      objectMapperBuilder.indentOutput(true);
      mappingJackson2HttpMessageConverter.getObjectMapper().setPropertyInclusion(
          JsonInclude.Value.construct(JsonInclude.Include.NON_EMPTY, JsonInclude.Include.NON_EMPTY));
      mapper.setPropertyInclusion(
          JsonInclude.Value.construct(JsonInclude.Include.NON_EMPTY, JsonInclude.Include.NON_EMPTY));
    }
    objectMapperBuilder.configure(mapper);
    this.mapper = objectMapperBuilder.build().copy();
  }

  @Override
  public ObjectMapper getMapper() {
    return mapper;
  }

  private static Optional<Boolean> getBooleanFromHeader(final Optional<HttpServletRequest> request, final String key) {
    String value = request.get().getHeader(key);
    if (value == null) {
      return Optional.empty();
    }

    value = value.trim();
    if (StringUtils.isEmpty(value)) {
      return Optional.empty();
    }

    switch (value.toLowerCase()) {
      case "true":
        return Optional.of(true);
      case "1":
        return Optional.of(true);
      default:
        return Optional.of(false);
    }
  }
}

I have tried setting it on the injected ObjectMapper , via the Jackson2ObjectMapperBuilder and also via the Jackson2ObjectMapperBuilder . I've tried (I think) all the various combinations. What happens is that the prettify part works per request, but the null-inclusion only works on the first request and after that it stays at that value. The code is running (prettify works, gone through in debugger and seen it) and not throwing errors when I try and set the inclusions properties but it's not using them.

We then have an @Component that implements the delegate interface. The getObjectMapper returns the mapper from our ApiContext

@Component
public class BookingsApi extends ApiDelegateBase implements BookingsApiDelegate {

  private final HttpServletRequest request;

  @Autowired
  private ContextService contextService;

  @Autowired
  public BookingsApi(final ObjectMapper objectMapper, final HttpServletRequest request) {
    this.request = request;
  }

  public Optional<ObjectMapper> getObjectMapper() {
    if (contextService.getContextOrNull() == null) {
      return Optional.empty();
    }
    return Optional.ofNullable(contextService.getContext().getMapper());
  }

  public Optional<HttpServletRequest> getRequest() {
    return Optional.ofNullable(request);
  }

  // Implement function for each request
}

The ContextServiceImpl is @ApiCallScoped and @Component . The ApiContext obtained via that is per-request in all other ways, but the mapper isn't behaving as I expected.

What it Produces

Eg If my first request has the header set to false (pretty-print, do not include nulls) then I get response

{
    "foo": "A",
    "baz": "C"
}

(which is correct). Sending a subsequent request without the header (do not pretty print, do include nulls) returns

{"foo": "A","baz": "C"}

which is wrong - it does not have the nulls - although pretty-printing has been correctly turned off. Subsequent requests with/without header return the same as the above two examples, depending on the header value.

On the other hand, if my first request did not include the header (do not pretty print, do include nulls) I get

{"foo": "A","bar":null,"baz": "C"}

(which is correct). But subsequent requests with the header on return

{
    "foo": "A",
    "bar": null,
    "baz": "C"
}

which is wrong - it does have the nulls - although pretty-printing has been correctly turned on. Subsequent requests with/without header return the same as the above, depending on the header value.

My Question

Why is it respecting the pretty-print but not the Property Inclusion and is there a way to make it work like I want?

Update

I think the problem is that Jackson caches the serialiser it uses per object. I guess this is by design - they are probably generated using reflection and fairly expensive. If I call one endpoint (first time after starting API) with the header it return without nulls. Fine. Subsequent calls are all without nulls, whether header is present or not. Not so fine. However, if I then call another, related, endpoint (first time after starting API) without the header, it returns with nulls for the main object (fine) but without nulls for some sub-objects that are common to the two responses (as the serialisers for those objects have been cached - not so fine).

I see the object mapper has some concept of views. Is there any way to use these to solve this? So it had two cached serialisers per object, and chose the right one? (I will try and look into this, haven't had time yet, but if someone knows I'm on the right or wrong track it would be good to know!)

Answer 1

You make it too complex.

Also ObjectMapper should not be initialized or reconfigured per request like you did. Check out the reason here

Note: The following configuration does not depend on your ApiContext or ApiScope at all, remove all your ObjectMapper customization in those classes before using this code. You can create a bare spring boot app to test the code.

First need a way to detect your request is null inclusion or exclusion

import javax.servlet.http.HttpServletRequest;

import org.springframework.web.context.request.RequestAttributes;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;

public class RequestUtil {

    public static boolean isNullInclusionRequest() {
        RequestAttributes requestAttrs = RequestContextHolder.currentRequestAttributes();
        if (!(requestAttrs instanceof ServletRequestAttributes)) {
            return false;
        }
        HttpServletRequest servletRequest = ((ServletRequestAttributes)requestAttrs).getRequest();
        return "true".equalsIgnoreCase(servletRequest.getHeader(NULLMODE_HEADER));
    }

    private RequestUtil() {

    }
}

Second, declare your custom message serializer

import java.lang.reflect.Type;

import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.http.MediaType;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;

@Order(Ordered.HIGHEST_PRECEDENCE) // Need this to be in the first of the serializers
public class NullExclusionMessageConverter extends MappingJackson2HttpMessageConverter {

    public NullExclusionMessageConverter(ObjectMapper nullExclusionMapper) {
        super(nullExclusionMapper);
    }

    @Override
    public boolean canRead(Type type, Class<?> contextClass, MediaType mediaType) {
        // Do not use this for reading. You can try it if needed
        return false;
    }

    @Override
    public boolean canWrite(Class<?> clazz, MediaType mediaType) {
        return super.canWrite(clazz, mediaType) && !RequestUtil.isNullInclusionRequest();    }
}

import java.lang.reflect.Type;

import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.http.MediaType;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;

@Order(Ordered.HIGHEST_PRECEDENCE)
public class NullInclusionMessageConverter extends MappingJackson2HttpMessageConverter {

    public NullInclusionMessageConverter(ObjectMapper nullInclusionMapper) {
        super(nullInclusionMapper);
    }

    @Override
    public boolean canRead(Type type, Class<?> contextClass, MediaType mediaType) {
        // Do not use this for reading. You can try it if needed
        return false;
    }

    @Override
    public boolean canWrite(Class<?> clazz, MediaType mediaType) {
        return super.canWrite(clazz, mediaType) && RequestUtil.isNullInclusionRequest();
    }
}

Third, register the custom message converter:

import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder;

@Configuration
public class JacksonConfiguration {

    @Bean
    public NullInclusionMessageConverter nullInclusionMessageConverter(Jackson2ObjectMapperBuilder builder) {
        ObjectMapper objectMapper = builder.build();
        objectMapper.setSerializationInclusion(JsonInclude.Include.ALWAYS);
        objectMapper.enable(SerializationFeature.INDENT_OUTPUT);
        return new NullInclusionMessageConverter(objectMapper);
    }

    @Bean
    public NullExclusionMessageConverter nullExclusionJacksonMessageConverter(Jackson2ObjectMapperBuilder builder) {
        ObjectMapper objectMapper = builder.build();
        objectMapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
        objectMapper.disable(SerializationFeature.INDENT_OUTPUT);
        return new NullExclusionMessageConverter(objectMapper);
    }
}