Splunk: Not all field values are extracted for long JSON files

Question

I am trying to ingest long JSON files into my Splunk index, where a record could contain more than 10000 characters. To prevent long records from getting truncated, I added a TRUNCATE=0 into my props.conf , and the entire record was ingested into the index. All events are forwarded and stored in the index, but I'm having problems with fields that appear towards the end of the JSON records.

I'm currently testing with 2 files:

• File A has 382 records, of which 166 are long records.
• File B has 252 records, of which all are long records.

All 634 events are returned with a simple search of the index, and I can see all fields in each event, regardless of how long the event is.

However, not all fields are extracted and directly searchable. For example, one of the fields is called name , and it appears towards the end of each JSON record. On the Interesting fields pane, under name , it shows only a count of 216 events from File A, and none of the remaining 166 + 252 long events in Files A and B. This is the same for other fields that appear towards the end of each JSON record, but fields towards the beginning of the record show all 634 events.

If I negate the 216 events, then these fields do not appear on the Fields pane at all.

Also, while I'm not able to directly search for name=<name in File B> , I can still select the field from the event and add to search , and all 252 events would be returned.

I'm not sure why these fields are not properly extracted even though they did not appear to be truncated. How can I extract them properly?

Answer 1

If this is a one-time use case, I'd suggest using Indexed Extractions. You'll make sure all fields are available; I bet you're hitting a limit somewhere in terms of search time fields per event. But be warned indexed extractions blow out your storage.

If it's a repeated task, I'd try to troubleshoot. Anything in _internal giving an error? Maybe a character set issue. Could also be a limit in [kv]. I'd try to set limit=0 in limits.conf in this stanza https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf#.5Bkv.5D