convert pandas df to json

Question

I have data in pandas df that looks like this:

Date    current_temperature
2020-06-29 14:04:21.000000  60.06
2020-06-29 14:19:26.000000  61.47
2020-06-29 14:34:30.000000  62.15
2020-06-29 14:49:35.000000  62.65
2020-06-29 15:04:39.000000  62.20
2020-06-29 15:19:44.000000  62.51
2020-06-29 15:34:48.000000  63.09
2020-06-29 15:49:53.000000  63.19
2020-06-29 16:04:58.000000  64.17
2020-06-29 16:20:02.000000  64.40
2020-06-29 16:35:07.000000  64.78
2020-06-29 16:50:11.000000  63.30
2020-06-29 17:05:16.000000  63.27
2020-06-29 17:20:20.000000  63.88
2020-06-29 17:35:25.000000  65.82
2020-06-29 17:50:30.000000  66.15
2020-06-29 18:05:34.000000  66.20
2020-06-29 18:20:39.000000  66.00

How would I convert this to a json format like this?:

    chart.data = [{
  "Date": "2012-07-27",
  "current_temperature": 13
}, {
  "Date": "2012-07-28",
  "current_temperature": 11
}, {
  "Date": "2012-07-29",
  "current_temperature": 15
}, {
  "Date": "2012-07-30",
  "current_temperature": 16
}, {
  "Date": "2012-07-31",
  "current_temperature": 18
}]

If I use data = df.to_json() this creates:

'{"Date":{"0":"2020-06-29 14:04:21.000000","1":"2020-06-29 14:19:26.000000","2":"2020-06-29 14:34:30.000000","3":"2020-06-29 14:49:35.000000","4":"2020-06-29 15:04:39.000000","5":"2020-06-29 15:19:44.000000","6":"2020-06-29 15:34:48.000000","7":"2020-06-29 15:49:53.000000","8":"2020-06-29 16:04:58.000000","9":"2020-06-29 16:20:02.000000","10":"2020-06-29 16:35:07.000000","11":"2020-06-29 16:50:11.000000","12":"2020-06-29 17:05:16.000000","13":"2020-06-29 17:20:20.000000","14":"2020-06-29 17:35:25.000000","15":"2020-06-29 17:50:30.000000","16":"2020-06-29 18:05:34.000000","17":"2020-06-29 18:20:39.000000"},"current_temperature":{"0":60.06,"1":61.47,"2":62.15,"3":62.65,"4":62.2,"5":62.51,"6":63.09,"7":63.19,"8":64.17,"9":64.4,"10":64.78,"11":63.3,"12":63.27,"13":63.88,"14":65.82,"15":66.15,"16":66.2,"17":66.0}}'

Any tips greatly appreciated... Its almost like to go to javascipt notation, I need nested dictionary's of each pd dataframe row???

EDIT

if I use df3 = df2.to_dict(orient='index')

Its gets closure, but is this json format? Ultimetely I am trying to figure out a way to pass data from a Flask app to the front end for web development, create charts with javascript. (still learning here...)

{0: {'Date': '2020-06-29 14:04:21.000000', 'current_temperature': 60.06},
 1: {'Date': '2020-06-29 14:19:26.000000', 'current_temperature': 61.47},
 2: {'Date': '2020-06-29 14:34:30.000000', 'current_temperature': 62.15},
 3: {'Date': '2020-06-29 14:49:35.000000', 'current_temperature': 62.65},
 4: {'Date': '2020-06-29 15:04:39.000000', 'current_temperature': 62.2},
 5: {'Date': '2020-06-29 15:19:44.000000', 'current_temperature': 62.51},
 6: {'Date': '2020-06-29 15:34:48.000000', 'current_temperature': 63.09},
 7: {'Date': '2020-06-29 15:49:53.000000', 'current_temperature': 63.19},
 8: {'Date': '2020-06-29 16:04:58.000000', 'current_temperature': 64.17},
 9: {'Date': '2020-06-29 16:20:02.000000', 'current_temperature': 64.4},
 10: {'Date': '2020-06-29 16:35:07.000000', 'current_temperature': 64.78},
 11: {'Date': '2020-06-29 16:50:11.000000', 'current_temperature': 63.3},
 12: {'Date': '2020-06-29 17:05:16.000000', 'current_temperature': 63.27},
 13: {'Date': '2020-06-29 17:20:20.000000', 'current_temperature': 63.88},
 14: {'Date': '2020-06-29 17:35:25.000000', 'current_temperature': 65.82},
 15: {'Date': '2020-06-29 17:50:30.000000', 'current_temperature': 66.15},
 16: {'Date': '2020-06-29 18:05:34.000000', 'current_temperature': 66.2},
 17: {'Date': '2020-06-29 18:20:39.000000', 'current_temperature': 66.0}}

Answer 1

• As you can see to_json() returns a string
• If you want a list of dicts use ast.literal_eval to convert the string to an object
• If you want Date formatted like 2012-07-31 , then you can format it prior to using to_json

from ast import literal_eval

data = literal_eval(df.to_json(orient='records'))

print(data)
[{'Date': '2020-06-29 14:04:21.000000', 'current_temperature': 60.06},
 {'Date': '2020-06-29 14:19:26.000000', 'current_temperature': 61.47},
 {'Date': '2020-06-29 14:34:30.000000', 'current_temperature': 62.15},
 {'Date': '2020-06-29 14:49:35.000000', 'current_temperature': 62.65},
 {'Date': '2020-06-29 15:04:39.000000', 'current_temperature': 62.2},
 {'Date': '2020-06-29 15:19:44.000000', 'current_temperature': 62.51},
 {'Date': '2020-06-29 15:34:48.000000', 'current_temperature': 63.09},
 {'Date': '2020-06-29 15:49:53.000000', 'current_temperature': 63.19},
 {'Date': '2020-06-29 16:04:58.000000', 'current_temperature': 64.17},
 {'Date': '2020-06-29 16:20:02.000000', 'current_temperature': 64.4},
 {'Date': '12020-06-29 16:35:07.000000', 'current_temperature': 64.78},
 {'Date': '2020-06-29 16:50:11.000000', 'current_temperature': 63.3},
 {'Date': '2020-06-29 17:05:16.000000', 'current_temperature': 63.27},
 {'Date': '2020-06-29 17:20:20.000000', 'current_temperature': 63.88},
 {'Date': '2020-06-29 17:35:25.000000', 'current_temperature': 65.82},
 {'Date': '2020-06-29 17:50:30.000000', 'current_temperature': 66.15},
 {'Date': '2020-06-29 18:05:34.000000', 'current_temperature': 66.2},
 {'Date': '2020-06-29 18:20:39.000000', 'current_temperature': 66.0}]

Alternate format

• In terms of Flask or JS, I'm not certain of the requirement
• The following code will get you a dict of lists
• Use dd = dict(dd) to get rid of the defaultdict and class part of the result.

from collection import defaultdict

# using data from above
dd = defaultdict(list)
for d in data:
    for k, v in d.items():
        dd[k].append(v)

# print(dd)

defaultdict(<class 'list'>,
            {'Date': ['2020-06-29 14:04:21.000000',
                      '2020-06-29 14:19:26.000000',
                      '2020-06-29 14:34:30.000000',
                      '2020-06-29 14:49:35.000000',
                      '2020-06-29 15:04:39.000000',
                      '2020-06-29 15:19:44.000000',
                      '2020-06-29 15:34:48.000000',
                      '2020-06-29 15:49:53.000000',
                      '2020-06-29 16:04:58.000000',
                      '2020-06-29 16:20:02.000000',
                      '12020-06-29 16:35:07.000000',
                      '2020-06-29 16:50:11.000000',
                      '2020-06-29 17:05:16.000000',
                      '2020-06-29 17:20:20.000000',
                      '2020-06-29 17:35:25.000000',
                      '2020-06-29 17:50:30.000000',
                      '2020-06-29 18:05:34.000000',
                      '2020-06-29 18:20:39.000000'],
             'current_temperature': [60.06,
                                     61.47,
                                     62.15,
                                     62.65,
                                     62.2,
                                     62.51,
                                     63.09,
                                     63.19,
                                     64.17,
                                     64.4,
                                     64.78,
                                     63.3,
                                     63.27,
                                     63.88,
                                     65.82,
                                     66.15,
                                     66.2,
                                     66.0]})