动态地将python字典存储在JSON文件中

Question

I have a python dictionary with keys in the format Aa123 and each value is a list. Now I need to store this data permanently in some format or in any database so that I can retrieve it easily. I thought JSON would be better for this and tried to store all the data in JSON format and then some application could use this file. My JSON format should be like,

[
  {  "firstLetter" : "A",
     "remaining" : [
        {
           "secondLetter" : "a",
           "ID" : [
              {"id" : "Aa123", "listOfItems" : ["ABC123","ASD100"]},
              {"id" : "Aa100", "listOfItems" : ["ABC123","COD101"]}
           ]
        },
        {
           "secondLetter" : "b",
           "ID" : [
              {"id" : "Ab100", "listOfItems" : ["ABC123","ASD100"]}
           ]
        }
     ]
  },
  {  "firstLetter" : "B",
     "remaining" : [
        {
           "secondLetter" : "a",
           "ID" : [                  
              {"id" : "Ba106", "listOfItems" : ["AUD123","CML101"]}
           ]
        },
        {
           "secondLetter" : "b",
           "ID" : [
              {"id" : "Bb153", "listOfItems" : ["AER113","ASD100"]},
              {"id" : "Bb100", "listOfItems" : ["ATC123","ASD500"]}
           ]
        }
     ]
  }
]

I know how to dump python dictionary into JSON, but that doesn't provide easy data query. My question is "how to store this python dictionary(that I obtained by running a python program)in required format(like the one shown above) that makes data query easy. Thanks!

Answer 1

It sounds like you may benefit from tinydb . It stores values directly in a JSON file and provides methods for querying.

So to store your values, we do

from tinydb import TinyDB, Query

db = TinyDB('test.json')
values = [{'firstLetter': 'A',
  'remaining': [{'ID': [{'id': 'Aa123', 'listOfItems': ['ABC123', 'ASD100']},
     {'id': 'Aa100', 'listOfItems': ['ABC123', 'COD101']}],
    'secondLetter': 'a'},
   {'ID': [{'id': 'Ab100', 'listOfItems': ['ABC123', 'ASD100']}],
    'secondLetter': 'b'}]},
 {'firstLetter': 'B',
  'remaining': [{'ID': [{'id': 'Ba106', 'listOfItems': ['AUD123', 'CML101']}],
    'secondLetter': 'a'},
   {'ID': [{'id': 'Bb153', 'listOfItems': ['AER113', 'ASD100']},
     {'id': 'Bb100', 'listOfItems': ['ATC123', 'ASD500']}],
    'secondLetter': 'b'}]}]
for value in values:
    db.insert(value)

To query, we do

>>> Q = Query()
>>> db.search(Q.firstLetter == "A")

[{'firstLetter': 'A',
  'remaining': [{'ID': [{'id': 'Aa123', 'listOfItems': ['ABC123', 'ASD100']},
     {'id': 'Aa100', 'listOfItems': ['ABC123', 'COD101']}],
    'secondLetter': 'a'},
   {'ID': [{'id': 'Ab100', 'listOfItems': ['ABC123', 'ASD100']}],
    'secondLetter': 'b'}]}]

Answer 2

Other than a NoSQL database like MongoDB and Cassandra , if you are using PostgreSQL, have a look at the hstore data type, specifically psycopg2's hstore functionality. This may be of help if you want to be able to query your data more easily.

Answer 3

In the example below, I output the data to a JSON , and also store the dict using Shelves .

import json
import codecs
import shelve

formatted_item = {
    "data": [
      {  "firstLetter" : "A",
         "remaining" : [
            {
               "secondLetter" : "a",
               "ID" : [
                  {"id" : "Aa123", "listOfItems" : ["ABC123","ASD100"]},
                  {"id" : "Aa100", "listOfItems" : ["ABC123","COD101"]}
               ]
            },
            {
               "secondLetter" : "b",
               "ID" : [
                  {"id" : "Ab100", "listOfItems" : ["ABC123","ASD100"]}
               ]
            }
         ]
      },
      {  "firstLetter" : "B",
         "remaining" : [
            {
               "secondLetter" : "a",
               "ID" : [                  
                  {"id" : "Ba106", "listOfItems" : ["AUD123","CML101"]}
               ]
            },
            {
               "secondLetter" : "b",
               "ID" : [
                  {"id" : "Bb153", "listOfItems" : ["AER113","ASD100"]},
                  {"id" : "Bb100", "listOfItems" : ["ATC123","ASD500"]}
               ]
            }
         ]
      }
    ]
}


###
# JSON

# *** Store

output = json.dumps(dict(formatted_item), sort_keys=True, indent=4, separators=(',', ': '))
json_file_path = 'temp.json'
with codecs.open(json_file_path, 'wb', encoding='utf8') as file:
    file.write(output)

# *** Read

with open(json_file_path) as json_file:
    json_data = json.load(json_file)

# *** Print

print json_data
print json_data['data'][0]['firstLetter']


###
# Shelves

# *** Store

shelf_file_path = 'temp.log'
shelf = shelve.open(shelf_file_path)
shelf.update(formatted_item)
shelf.close()

# *** Read

loaded_shelf = shelve.open(shelf_file_path)

# *** Print

print loaded_shelf
print loaded_shelf['data'][0]['firstLetter']

Answer 4

You can save Python Dict objects directly on MongoDB

For more information, refer to this: http://api.mongodb.com/python/current/tutorial.html

Answer 5

DBMS doesn't deal with JSON query out of the box very well.

You can now store JSON file in supported RDBMS, NoSQL for quick query. Nevertheless, DBMS that support JSON data type, try workaround by skimming the json structure and index them massively. NoSQL is no better than RDBMS when dealing with json data type.

So you must always read particular RDBMS/NoSQL documentation before storing json file, especially when you have tons of json data.