需要在 spring 引导项目中从 JSON 响应中过滤 null 值

Question

public class APIService {

private List<CoinDTO> coinList = new ArrayList<>();

private final TrackerConfigProperties trackerConfig;

//Injecting configuration properties into the constructor.
public APIService(TrackerConfigProperties trackerConfig) {
    this.trackerConfig = trackerConfig;
}

public List<CoinDTO> getCoinList() {
    return coinList;
}

public void setCoinList(List<CoinDTO> coinList) {
    this.coinList = coinList;
}

@PostConstruct
@Scheduled(cron = "* * 1 * * *")
public void getAPIData() throws IOException, JSONException{     
    List<CoinDTO> newData = new ArrayList<>();
    OkHttpClient client = new OkHttpClient();
    
    JSONObject obj = null;
    JSONObject data = null;

    //Use request builder to configure api 
    Request request = new Request.Builder()
        .url(trackerConfig.apiUrl())
        .get()
        .addHeader("X-RapidAPI-Key", trackerConfig.apiKey())
        .addHeader("X-RapidAPI-Host", trackerConfig.apiHost())
        .build();
    
    Response response = null;
    
    try {
        response = client.newCall(request).execute();
                    
        String jsonResponse = response.body().string();
        System.out.println(jsonResponse);

        
        obj = new JSONObject(jsonResponse);
        data = obj.getJSONObject("data");
        JSONArray arr = data.getJSONArray("coins");
                    
        System.out.println(arr);

        for(int i=0; i<arr.length(); i++) {
            CoinDTO coinRecords = new CoinDTO();
            coinRecords.setName(arr.getJSONObject(i).getString("name"));
            coinRecords.setSymbol(arr.getJSONObject(i).getString("symbol"));
            coinRecords.setPrice(arr.getJSONObject(i).getFloat("price"));
            coinRecords.setRank(arr.getJSONObject(i).getInt("rank"));
            coinRecords.setChange(arr.getJSONObject(i).getFloat("change"));
            coinRecords.setUrl(arr.getJSONObject(i).getString("iconUrl"));
            coinRecords.setMarketCap(arr.getJSONObject(i).getString("marketCap"));
            newData.add(coinRecords);
        }
        this.coinList = newData;
        
    }catch(IOException e) {
        e.printStackTrace();
    }
    
    // Requesting response.body().string() more than once will cause illegal state exception.   
}

}

The JSON response from crypto API looks very fishy. 1-60 results, there is no problem in getting the data. But when I increase the response count to 100 coins. A few of them have null values and this is giving me number format exception. As shown below,

Caused by: org.json.JSONException: JSONObject["change"] is not a number.
    at org.json.JSONObject.getFloat(JSONObject.java:655) ~[json-20180130.jar:na]
    at com.vish.trackerapp.service.APIService.getAPIData(APIService.java:86) ~[classes/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:424) ~[spring-beans-6.0.2.jar:6.0.2]
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:368) ~[spring-beans-6.0.2.jar:6.0.2]
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:192) ~[spring-beans-6.0.2.jar:6.0.2]
    ... 37 common frames omitted

Caused by: java.lang.NumberFormatException: For input string: "null"
    at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054) ~[na:na]
    at java.base/jdk.internal.math.FloatingDecimal.parseFloat(FloatingDecimal.java:122) ~[na:na]
    at java.base/java.lang.Float.parseFloat(Float.java:476) ~[na:na]
    at org.json.JSONObject.getFloat(JSONObject.java:653) ~[json-20180130.jar:na]
    ... 45 common frames omitted

Json response is as follows,

{
                "uuid": "aQx_vW8s1",
                "symbol": "XEC",
                "name": "eCash",
                "color": null,
                "iconUrl": "https://cdn.coinranking.com/wdqRaUEhn/xec.png",
                "marketCap": "458134192",
                "price": "0.000023888579923343",
                "listedAt": 1634887994,
                "tier": 1,
                "change": "0.07",
                "rank": 73,
                "sparkline": [
                    "0.000023888455899517",
                    "0.000023873825475088",
                    "0.00002383507280166",
                    "0.000023738913764452",
                    "0.000023705304867018",
                    "0.000023643911925818",
                    "0.000023626155269768",
                    "0.000023699523887097",
                    "0.000023745220955252",
                    "0.000023820571142281",
                    "0.000023801617704237",
                    "0.000023816460854558",
                    "0.000023795169626986",
                    "0.000023858079419847",
                    "0.000023892553157129",
                    "0.000023849240281632",
                    "0.000023857584839752",
                    "0.000024048737766589",
                    "0.00002392004515143",
                    "0.000023808725016789",
                    "0.000023751508927767",
                    "0.000023764344487179",
                    "0.000023826878245569",
                    "0.000023885406955056",
                    "0.000023888537567965"
                ],
                "lowVolume": false,
                "coinrankingUrl": "https://coinranking.com/coin/aQx_vW8s1+ecash-xec",
                "24hVolume": "5918517",
                "btcPrice": "1.416302521e-9"
            }

I tried this idea, but couldn't figure it out how to include in my code.

            newData.stream().filter(v -> v != null);

Greatly appreciate any suggestions or ideas.

Answer 1

In my opinion, a good solution would be to use a mapper and a DTO to deal with the json, since there you could use the @JsonProperty tag that allows you to make the json field not mandatory.

Here's how to use a mapper:

 ObjectMapper om = new ObjectMapper();
        om.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
        JSONObject battle= new JSONObject(text);
        JSONObject battleObject = battle.getJSONObject(JsonObject);
        try{
            Battle battleDto =  om.readValue(battleObject.toString(), Battle.class);
            return battleDto;
        }catch (JsonProcessingException e ){
            throw new NoStarPlayerException("No starPlayer for this battle ="+battle.toString());
        }

And here the DTO:

public class Battle {
    @JsonProperty("battleTime")
    String battletime;
    @JsonIgnore()
    String map;
    @JsonProperty("mode")
    String mode;
    @JsonProperty("type")
    String type;
    @JsonProperty("result")
    String result;
    @JsonProperty("duration")
    int duration;
    @JsonProperty("trophyChange")
    int trophyChange;
    @JsonProperty("starPlayer")
    StarPlayer starPlayer;
    @JsonProperty("teams")
    ArrayList<ArrayList<Player>> teams;
    @JsonProperty("players")
    ArrayList<Player>players;
    @JsonProperty("rank")
    int rank;

To work around a missing Json field, you can use @JsonIgnore(If you are not interested in the field ever) or @JsonProperty(value = "battleTime",required = false) if it may or may not be null.

I hope I have helped you, Regards

Answer 2

json.org library that you use is a good training tool for JSON but not very good for practical use. The best libraries available are JSON-Jackson and Gson (of Google). I personally prefer JSON-Jackson. Use one of those to parse your JSON and you will solve your problem easily.

Also, For simplistic case like this I wrote my own thin wrapper over JSON-Jackson library that allows you to serialize and parse and serialize JSON to/from java class. Here is Javadoc for JsonUtils class . Your code may look something like this:

   HashMap<String, Object> map;
    try {
        map = JsonUtils.readObjectFromJsonString(jsonStr, Map.class);
    } catch (IOException e) {
        ...
    }

Instead of Map you can write your own DTO class and than substitute Map.class with your own class instance. JsonUtils class comes with Open Source MgntUtils library written and maintained by me. The library can be obtained from Maven Central here or from Github