JSON.NET serialize JObject while ignoring null properties

Question

I have a JObject which is used as a template for calling RESTful web services. This JObject gets created via a parser and since it's used as a template telling the user what the endpoint schema looks like, I had to figure out a way to preserve all properties, which is why I'm defaulting their values to null . As as example, this is what the object originally looks like:

{  
   "Foo":{  
      "P1":null,
      "P2":null,
      "P3":null,
      "P4":{  
         "P1":null,
         "P2":null,
         "P3":null,
      },
      "FooArray":[  
         {  
            "F1":null,
            "F2":null,
            "F3":null,
         }
      ]
   },
   "Bar":null
}

The user is then able to fill in individual fields as they need, such as Foo.P2 and Foo.P4.P1 :

{  
   "Foo":{  
      "P1":null,
      "P2":"hello world",
      "P3":null,
      "P4":{  
         "P1":1,
         "P2":null,
         "P3":null,
      },
      "FooArray":[  
         {  
            "F1":null,
            "F2":null,
            "F3":null,
         }
      ]
   },
   "Bar":null
}

meaning they only care about those two fields. Now I want to serialize this template ( JObject ) back to a JSON string, but want only those fields that are populated to show up. So I tried this:

string json = JsonConvert.SerializeObject(template,
    new JsonSerializerSettings
    {
        NullValueHandling = NullValueHandling.Ignore
    });

Unfortunately, this didn't work. I came across this question and realized that a null value in the object is an actual JToken type and not really a null , which makes sense. However, in this very particular case, I need to be able to get rid of these "unused" fields. I tried manually iterating over nodes and removing them but that didn't work either. Note that the only managed type I'm using is JObject ; I don't have a model to convert the object to or define attributes on, since this "template" gets resolved at runtime. I was just wondering if anyone has encountered a problem like this and has any insights. Any help is greatly appreciated!

Answer 1

You can use a recursive helper method like the one below to remove the null values from your JToken hierarchy prior to serializing it.

using System;
using Newtonsoft.Json.Linq;

public static class JsonHelper
{
    public static JToken RemoveEmptyChildren(JToken token)
    {
        if (token.Type == JTokenType.Object)
        {
            JObject copy = new JObject();
            foreach (JProperty prop in token.Children<JProperty>())
            {
                JToken child = prop.Value;
                if (child.HasValues)
                {
                    child = RemoveEmptyChildren(child);
                }
                if (!IsEmpty(child))
                {
                    copy.Add(prop.Name, child);
                }
            }
            return copy;
        }
        else if (token.Type == JTokenType.Array)
        {
            JArray copy = new JArray();
            foreach (JToken item in token.Children())
            {
                JToken child = item;
                if (child.HasValues)
                {
                    child = RemoveEmptyChildren(child);
                }
                if (!IsEmpty(child))
                {
                    copy.Add(child);
                }
            }
            return copy;
        }
        return token;
    }

    public static bool IsEmpty(JToken token)
    {
        return (token.Type == JTokenType.Null);
    }
}

Demo:

string json = @"
{
    ""Foo"": {
        ""P1"": null,
        ""P2"": ""hello world"",
        ""P3"": null,
        ""P4"": {
            ""P1"": 1,
            ""P2"": null,
            ""P3"": null
        },
        ""FooArray"": [
            {
                ""F1"": null,
                ""F2"": null,
                ""F3"": null
            }
        ]
    },
    ""Bar"": null
}";

JToken token = JsonHelper.RemoveEmptyChildren(JToken.Parse(json));
Console.WriteLine(token.ToString(Formatting.Indented));

Output:

{
  "Foo": {
    "P2": "hello world",
    "P4": {
      "P1": 1
    },
    "FooArray": [
      {}
    ]
  }
}

Fiddle: https://dotnetfiddle.net/wzEOie

Notice that, after removing all null values, you will have an empty object in the FooArray , which you may not want. (And if that object were removed, then you'd have an empty FooArray , which you also may not want.) If you want to make the helper method more aggressive in its removal, you can change the IsEmpty function to this:

    public static bool IsEmpty(JToken token)
    {
        return (token.Type == JTokenType.Null) ||
               (token.Type == JTokenType.Array && !token.HasValues) ||
               (token.Type == JTokenType.Object && !token.HasValues);
    }

With that change in place, your output would look like this instead:

{
  "Foo": {
    "P2": "hello world",
    "P4": {
      "P1": 1
    }
  }
}

Fiddle: https://dotnetfiddle.net/ZdYogJ

Answer 2

You can prevent the null tokens from being created to begin with by specifying the JsonSerializer with its NullValueHandler set to NullValueHandler.Ignore . This is passed in as a parameter to JObject.FromObject as seen in an answer to the same question you linked to: https://stackoverflow.com/a/29259032/263139 .

Answer 3

Brian's answer works. I also came up with another (yet still recursive) way of doing it shortly after posting the question, in case anyone else is interested.

private void RemoveNullNodes(JToken root)
{
    if (root is JValue)
    {
        if (((JValue)root).Value == null)
        {
            ((JValue)root).Parent.Remove();
        }
    }
    else if (root is JArray)
    {
        ((JArray)root).ToList().ForEach(n => RemoveNullNodes(n));
        if (!(((JArray)root)).HasValues)
        {
            root.Parent.Remove();
        }
    }
    else if (root is JProperty)
    {
        RemoveNullNodes(((JProperty)root).Value);
    }
    else
    {
        var children = ((JObject)root).Properties().ToList();
        children.ForEach(n => RemoveNullNodes(n));

        if (!((JObject)root).HasValues)
        {
            if (((JObject)root).Parent is JArray)
            {
                ((JArray)root.Parent).Where(x => !x.HasValues).ToList().ForEach(n => n.Remove());
            }
            else
            {
                var propertyParent = ((JObject)root).Parent;
                while (!(propertyParent is JProperty))
                {
                    propertyParent = propertyParent.Parent;
                }
                propertyParent.Remove();
            }
        }
    }
}

Answer 4

Here's what I was able to come up with. It removes properties that contain only null values. This means that it will handle the case where the property is a scalar value that is null and will also handle the case where there is an array that is all null values. It also removes properties that have no values. This handles the case where the property contains an object that has no child properties. Note, mine uses a JObject which has a Descendents() method which is what made the implementation easy. JToken doesn't have that. My implementation mutates the JObject itself rather than creating a copy of it. Also, it continues removing properties until there aren't any more occurrences. It's a bit more succinct than the other implementations. I don't know how it compares performance-wise.

using Newtonsoft.Json.Linq;
using System;
using System.IO;
using System.Linq;

namespace JsonConsoleApplication
{
    class Program
    {
        static void Main(string[] args)
        {
            var jo = JObject.Parse(File.ReadAllText(@"test.json"));
            Console.WriteLine($"BEFORE:\r\n{jo}");
            jo.RemoveNullAndEmptyProperties();
            Console.WriteLine($"AFTER:\r\n{jo}");
        }
    }

    public static class JObjectExtensions
    {
        public static JObject RemoveNullAndEmptyProperties(this JObject jObject)
        {
            while (jObject.Descendants().Any(jt => jt.Type == JTokenType.Property && (jt.Values().All(a => a.Type == JTokenType.Null) || !jt.Values().Any())))
                foreach (var jt in jObject.Descendants().Where(jt => jt.Type == JTokenType.Property && (jt.Values().All(a => a.Type == JTokenType.Null) || !jt.Values().Any())).ToArray())
                    jt.Remove();
            return jObject;
        }
    }
}

The following is the program output:

BEFORE:
{
  "propertyWithValue": "",
  "propertyWithObjectWithProperties": {
    "nestedPropertyWithValue": "",
    "nestedPropertyWithNull": null
  },
  "propertyWithEmptyObject": {},
  "propertyWithObjectWithPropertyWithNull": {
    "nestedPropertyWithNull": null
  },
  "propertyWithNull": null,
  "emptyArray": [],
  "arrayWithNulls": [
    null,
    null
  ],
  "arrayWithObjects": [
    {
      "propertyWithValue": ""
    },
    {
      "propertyWithNull": null
    }
  ]
}
AFTER:
{
  "propertyWithValue": "",
  "propertyWithObjectWithProperties": {
    "nestedPropertyWithValue": ""
  },
  "arrayWithObjects": [
    {
      "propertyWithValue": ""
    },
    {}
  ]
}

Answer 5

Using JsonPath we can have a more elegant solution:

 jObject.SelectTokens("$..*")
            .OfType<JValue>()
            .Where(x=>x.Type == JTokenType.Null)
            .Select(a => a.Parent)
            .ToList()
            .ForEach(a => a.Remove());

With a working example here: https://dotnetfiddle.net/zVgXOq