当对象足够大时，如何告诉杰克逊序列化器停止继续序列化和写入

Question

in my recent project, there are a lot of log statement like below:

Bar bar = foo();
logger.info("something happened: param:{}", JSON.toJSONString(bar));

as you can see, just json serialize the object into log. sometimes bar is very big, and serializing the bar object consumes too much time, and log file expands a lot. the json serializing framework is fasterxml-jackson

So my question is: is there a way ,like customizing JsonSerializer to implement:

1. count the chars/bytes that has already been serialized(by invoking method #toJSONString())
2. if the count has exceed the MAX_SIZE, say 2000, then stop
3. return just the 2000 char String and print it in log file

Answer 1

If we do not want to fall into Jackson and how it works we can just limit Writer where objects are serialised into. It is simple approach which does not require to implement custom serialisers with limits checked inside. Downside of this approach is we do not skip serialisation process we just ignore it's result.

You can write your own Writer with a maximum buffer size. Simple implementation based on StringBuilder :

class LimitedStringBuilderWriter extends Writer {

    private final StringBuilder buffer;
    private int remaining;

    public LimitedStringBuilderWriter(int limit) {
        if (limit <= 0) {
            throw new IllegalArgumentException("Limit must be positive number!");
        }

        this.remaining = limit;
        this.buffer = new StringBuilder(limit);
    }

    @Override
    public void write(char[] cbuf, int off, int len) {
        if (len == 0 || this.remaining <= 0) {
            return;
        }
        if ((off < 0) || (off > cbuf.length) || (len < 0) ||
                ((off + len) > cbuf.length) || ((off + len) < 0)) {
            throw new IndexOutOfBoundsException();
        }
        final int size = len - off;
        if (this.remaining >= size) {
            this.remaining -= size;
            buffer.append(cbuf, off, len);
            return;
        }
        buffer.append(cbuf, off, this.remaining);
        this.remaining = 0;
    }

    @Override
    public void write(int c) {
        if (this.remaining <= 0) {
            return;
        }
        this.remaining--;
        this.buffer.append(c);
    }

    @Override
    public void flush() {
    }

    @Override
    public void close() {
    }

    @Override
    public String toString() {
        return this.buffer.toString();
    }
}

Usage:

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.json.JsonMapper;

import java.io.IOException;
import java.io.Writer;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.function.Supplier;

public class LimitJsonLogApp {

    public static void main(String[] args) {
        Map<String, Object> value = new LinkedHashMap<>();
        value.put("array", Arrays.asList(1, 2, 3));
        value.put("string", "Value");
        value.put("int", 23);

        for (int limit = 11; limit < 30; limit += 3) {
            System.out.println(limit + " => " + JSON.toJSONString(value, limit));
        }
    }
}

class JSON {

    private static final ObjectMapper mapper = JsonMapper.builder().build();

    public static String toJSONString(Object value) {
        return toJSONStringSupplier(value).get();
    }

    public static String toJSONString(Object value, int limit) {
        return toJSONStringSupplier(value, limit).get();
    }

    public static Supplier<String> toJSONStringSupplier(Object value) {
        return toJSONStringSupplier(value, 1000);
    }

    public static Supplier<String> toJSONStringSupplier(Object value, int limit) {
        if (value == null) {
            return () -> "null";
        }
        return () -> {
            try {
                LimitedStringBuilderWriter writer = new LimitedStringBuilderWriter(limit);
                mapper.writeValue(writer, value);
                return writer.toString();
            } catch (IOException e) {
                throw new IllegalArgumentException(e);
            }
        };
    }
}

Above code prints:

11 => {"array":[1
14 => {"array":[1,2,
17 => {"array":[1,2,3],
20 => {"array":[1,2,3],"st
23 => {"array":[1,2,3],"strin
26 => {"array":[1,2,3],"string":
29 => {"array":[1,2,3],"string":"Va

Notice, some methods in JSON class returns Supplier<String> . You should use them if your logger allows to provide suppliers. It allows to postpone serialisation to the moment when it is really needed. And in case you disable INFO log in configuration it would really disable execution of this code.

Answer 2

after doing some research, the below thought comes up. this is not that elegant. wish someone could improve this!

import java.io.IOException;
import java.io.Writer;
import java.util.stream.IntStream;

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.json.WriterBasedJsonGenerator;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.MappingJsonFactory;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationConfig;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.jsontype.TypeSerializer;
import com.fasterxml.jackson.databind.ser.BeanSerializerModifier;
import com.fasterxml.jackson.databind.ser.SerializerFactory;
import com.fasterxml.jackson.databind.ser.Serializers;
import org.jetbrains.annotations.NotNull;

public class JSON {

    static int MAX_SIZE = 10;
    final static ObjectMapper OBJECT_MAPPER;
    static {
        OBJECT_MAPPER = new ObjectMapper(new WrappedJsonFactory(new MappingJsonFactory()), null, null);
        OBJECT_MAPPER.setSerializerFactory(new WrapperSerialFactory(OBJECT_MAPPER.getSerializerFactory()));
    }

    static String toJSONString(Object value) {
        try {
            return OBJECT_MAPPER.writeValueAsString(value);
        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }


    public static void main(String... args) throws Exception {
        int[] array = IntStream.range(1, 100).toArray();
        System.out.println(JSON.toJSONString(array));
        MAX_SIZE = 20;
        System.out.println(JSON.toJSONString(array));
        MAX_SIZE = 30;
        System.out.println(JSON.toJSONString(array));
    }

    static class WrapperSerialFactory extends SerializerFactory {

        private SerializerFactory wrapped;

        public WrapperSerialFactory(SerializerFactory wrapped) {
            this.wrapped = wrapped;
        }

        @Override
        public SerializerFactory withAdditionalSerializers(Serializers additional) {
            return wrapped.withAdditionalSerializers(additional);
        }

        @Override
        public SerializerFactory withAdditionalKeySerializers(Serializers additional) {
            return wrapped.withAdditionalKeySerializers(additional);
        }

        @Override
        public SerializerFactory withSerializerModifier(BeanSerializerModifier modifier) {
            return wrapped.withSerializerModifier(modifier);
        }

        @Override
        public JsonSerializer<Object> createSerializer(SerializerProvider prov, JavaType baseType)
            throws JsonMappingException {
            JsonSerializer<Object> serializer = wrapped.createSerializer(prov, baseType);
            return new WrappedTypeSerializer(serializer);
        }

        @Override
        public TypeSerializer createTypeSerializer(SerializationConfig config, JavaType baseType)
            throws JsonMappingException {
            return wrapped.createTypeSerializer(config, baseType);
        }

        @Override
        public JsonSerializer<Object> createKeySerializer(SerializationConfig config, JavaType type,
            JsonSerializer<Object> defaultImpl) throws JsonMappingException {
            return wrapped.createKeySerializer(config, type, defaultImpl);
        }
    }



    static class WrappedJsonFactory extends JsonFactory {
        private JsonFactory wrapped;

        public WrappedJsonFactory(JsonFactory wrapped) {
            this.wrapped = wrapped;
        }

        @Override
        public JsonGenerator createGenerator(Writer w) throws IOException {
            w = new LimitingWriter(w);
            return wrapped.createGenerator(w);
        }
    }

    static class LimitingWriter extends Writer {
        private int count;
        private boolean finished = false;

        private Writer delegate;

        public LimitingWriter(Writer delegate) {
            this.delegate = delegate;
        }

        private void finish() throws IOException {
            if (!finished) {
                delegate.write("...");
                finished = true;
            }
        }

        @Override
        public void write(int c) throws IOException {
            if (!finished) {
                if (count >= MAX_SIZE) {
                    finish();
                } else {
                    delegate.write(c);
                    ++count;
                }
            }
        }

        @Override
        public void write(@NotNull char[] cbuf) throws IOException {
            if (!finished) {
                if ((count + cbuf.length) > MAX_SIZE) {
                    delegate.write(cbuf, 0, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(cbuf);
                    count += cbuf.length;
                }
            }
        }

        @Override
        public void write(@NotNull String str) throws IOException {
            if (!finished) {
                if (count + str.length() > MAX_SIZE) {
                    delegate.write(str, 0, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(str);
                    count += str.length();
                }
            }
        }

        @Override
        public void write(@NotNull String str, int off, int len) throws IOException {
            if (!finished) {
                if (count + len > MAX_SIZE) {
                    delegate.write(str, 0, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(str, off, len);
                    count += len;
                }
            }
        }

        @Override
        public Writer append(CharSequence csq) throws IOException {
            if (!finished) {
                if (count + csq.length() > MAX_SIZE) {
                    count = MAX_SIZE;
                    delegate.append(csq, 0, MAX_SIZE - count);
                    finish();
                } else {
                    count += csq.length();
                    delegate.append(csq);
                }
            }
            return this;
        }

        @Override
        public Writer append(CharSequence csq, int start, int end) throws IOException {
            if (!finished) {
                if (count + end - start > MAX_SIZE) {
                    delegate.append(csq, start, start + MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    count += (end - start);
                    delegate.append(csq, start, end);
                }
            }
            return this;
        }

        @Override
        public Writer append(char c) throws IOException {
            if (!finished) {
                if (count >= MAX_SIZE) {
                    finish();
                } else {
                    ++count;
                    delegate.append(c);
                }
            }
            return this;
        }

        @Override
        public void write(@NotNull char[] cbuf, int off, int len) throws IOException {
            if (!finished) {
                if (count + len > MAX_SIZE) {
                    delegate.write(cbuf, off, MAX_SIZE - count);
                    count = MAX_SIZE;
                    finish();
                } else {
                    delegate.write(cbuf, off, len);
                    count += len;
                }
            }
        }

        @Override
        public void flush() throws IOException {
            delegate.flush();
        }

        @Override
        public void close() throws IOException {
            delegate.close();
        }
    }

    static class WrappedTypeSerializer extends JsonSerializer {
        private JsonSerializer wrapped;

        public WrappedTypeSerializer(JsonSerializer wrapped) {
            this.wrapped = wrapped;
        }

        @Override
        public void serialize(Object value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
            WriterBasedJsonGenerator writerBasedJsonGenerator = (WriterBasedJsonGenerator)gen;
            LimitingWriter writer = (LimitingWriter)writerBasedJsonGenerator.getOutputTarget();
            if (writer.finished) {
                return;
            }
            wrapped.serialize(value, gen, serializers);
        }
    }
}

output:

[1,2,3,4,5...
[1,2,3,4,5,6,7,8,9,1...
[1,2,3,4,5,6,7,8,9,10,11,12,13...