How to split text file into multiple files and extract filename from line prefix?

Question

I have a simple log file with content like:

1504007980.039:{"key":"valueA"}
1504007990.359:{"key":"valueB", "key2": "valueC"}
...

That I'd like to output to multiple files that each have as content the JSON part that comes after the timestamp. So I would get as a result the files:

1504007980039.json
1504007990359.json
...

This is similar to How to split one text file into multiple *.txt files? but the name of the file should be extracted from each line (and remove an extra dot), and not generated via an index

Preferably I'd want a one-liner that can be executed in bash.

Answer 1

Since you aren't using GNU awk you need to close output files as you go to avoid the "too many open files" error. To avoid that and issues around specific values in your JSON and issues related to undefined behavior during output redirection, this is what you need:

awk '{
    fname = $0
    sub(/\./,"",fname)
    sub(/:.*/,".json",fname)
    sub(/[^:]+:/,"")
    print >> fname
    close(fname)
}' file

You can of course squeeze it onto 1 line if you see some benefit to that:

awk '{f=$0;sub(/\./,"",f);sub(/:.*/,".json",f);sub(/[^:]+:/,"");print>>f;close(f)}' file

Answer 2

awk solution:

awk '{ idx=index($0,":"); fn=substr($0,1,idx-1)".json"; sub(/\./,"",fn); 
       print substr($0,idx+1) > fn; close(fn) }' input.log 

• idx=index($0,":") - capturing index of the 1st :

• fn=substr($0,1,idx-1)".json" - preparing filename

---

Viewing results (for 2 sample lines from the question):

for f in *.json; do echo "$f"; cat "$f"; echo; done

The output ( filename -> content ):

1504007980039.json
{"key":"valueA"}

1504007990359.json
{"key":"valueB"}