使用 Python 将 JSON 数据从 API 发送到 Firestore 的最佳方式

Question

I have received JSON data with results for an Amazon search through an API, and i am trying to send it to my firestore database. I would like it to be a collection of products, that has been searched for.

This means; for each position (product) in the search result, would i like to store it as a document. I have made the collection manually to show how it could look like.

I have converted the JSON into readable json with the code below.

json_api = api_result.json()

def convert_json_to_dict(json_api):
    with open(json_api) as JSON:
        json_dict = json.load(JSON)
    return json_dict


def convert_dict_to_df(json_api):
    return pd.json_normalize(convert_json_to_dict(json_api))

This gives me the following data, that i would like to handle. It is the metadata and the two first positions.

{'request_info': {'success': True,
  'credits_used': 46,
  'credits_remaining': 54,
  'credits_used_this_request': 1},
 'request_metadata': {'created_at': '2021-02-25T10:17:35.565Z',
  'processed_at': '2021-02-25T10:17:39.793Z',
  'total_time_taken': 4.23,
  'amazon_url': 'https://www.amazon.com/s/?k=beetle+chair&ref=nb_sb_noss_2'},
 'request_parameters': {'amazon_domain': 'amazon.com',
  'type': 'search',
  'search_term': 'beetle chair'},
 'search_results': [{'position': 1,
   'title': 'CosmoLiving by Cosmopolitan Astor Dining Chair Green',
   'asin': 'B07P6T73Y3',
   'link': 'https://www.amazon.com/dp/B07P6T73Y3',
   'categories': [{'name': 'All Departments', 'id': 'search-alias=aps'}],
   'image': 'https://m.media-amazon.com/images/I/81F4zscz2OL._AC_UL320_.jpg',
   'is_prime': True,
   'is_amazon_fresh': False,
   'is_whole_foods_market': False,
   'rating': 4.1,
   'ratings_total': 232,
   'sponsored': False,
   'prices': [{'symbol': '$',
     'value': 99.99,
     'currency': 'USD',
     'raw': '$99.99'}],
   'price': {'symbol': '$',
    'value': 99.99,
    'currency': 'USD',
    'raw': '$99.99'},
   'delivery': {'tagline': 'Get it as soon as Fri, Mar 5',
    'price': {'raw': 'FREE Shipping by Amazon',
     'symbol': '$',
     'currency': 'USD',
     'value': 0,
     'is_free': True}}},
  {'position': 2,
   'title': 'Guyou Modern Plush Velvet Accent Chairs, Upholstered Round Back Dining Chairs Living Room Decor Set of 2 with Brass Legs (Emerald Green)',
   'asin': 'B085NKFJ2Y',
   'link': 'https://www.amazon.com/dp/B085NKFJ2Y',
   'categories': [{'name': 'All Departments', 'id': 'search-alias=aps'}],
   'image': 'https://m.media-amazon.com/images/I/51Evpn0iw-L._AC_UL320_.jpg',
   'is_prime': False,
   'is_amazon_fresh': False,
   'is_whole_foods_market': False,
   'rating': 4.5,
   'ratings_total': 38,
   'sponsored': False,
   'prices': [{'symbol': '$',
     'value': 159.83,
     'currency': 'USD',
     'raw': '$159.83'}],
   'price': {'symbol': '$',
    'value': 159.83,
    'currency': 'USD',
    'raw': '$159.83'},
   'delivery': {'price': {'raw': 'FREE Shipping',
     'symbol': '$',
     'currency': 'USD',
     'value': 0,
     'is_free': True}}},

The intention is to use this on thousands of products.

Answer 1

You could create a for loop to create a document for each search_result and use the asin for the document name.

Using your data set i had to fix the data set you showed because it was invalid JSON, here is the data set for testing:

data = {'request_info': {'success': True,
  'credits_used': 46,
  'credits_remaining': 54,
  'credits_used_this_request': 1},
 'request_metadata': {'created_at': '2021-02-25T10:17:35.565Z',
  'processed_at': '2021-02-25T10:17:39.793Z',
  'total_time_taken': 4.23,
  'amazon_url': 'https://www.amazon.com/s/?k=beetle+chair&ref=nb_sb_noss_2'},
 'request_parameters': {'amazon_domain': 'amazon.com',
  'type': 'search',
  'search_term': 'beetle chair'},
 'search_results': [{'position': 1,
   'title': 'CosmoLiving by Cosmopolitan Astor Dining Chair Green',
   'asin': 'B07P6T73Y3',
   'link': 'https://www.amazon.com/dp/B07P6T73Y3',
   'categories': [{'name': 'All Departments', 'id': 'search-alias=aps'}],
   'image': 'https://m.media-amazon.com/images/I/81F4zscz2OL._AC_UL320_.jpg',
   'is_prime': True,
   'is_amazon_fresh': False,
   'is_whole_foods_market': False,
   'rating': 4.1,
   'ratings_total': 232,
   'sponsored': False,
   'prices': [{'symbol': '$',
     'value': 99.99,
     'currency': 'USD',
     'raw': '$99.99'}],
   'price': {'symbol': '$',
    'value': 99.99,
    'currency': 'USD',
    'raw': '$99.99'},
   'delivery': {'tagline': 'Get it as soon as Fri, Mar 5',
    'price': {'raw': 'FREE Shipping by Amazon',
     'symbol': '$',
     'currency': 'USD',
     'value': 0,
     'is_free': True}}},
  {'position': 2,
   'title': 'Guyou Modern Plush Velvet Accent Chairs, Upholstered Round Back Dining Chairs Living Room Decor Set of 2 with Brass Legs (Emerald Green)',
   'asin': 'B085NKFJ2Y',
   'link': 'https://www.amazon.com/dp/B085NKFJ2Y',
   'categories': [{'name': 'All Departments', 'id': 'search-alias=aps'}],
   'image': 'https://m.media-amazon.com/images/I/51Evpn0iw-L._AC_UL320_.jpg',
   'is_prime': False,
   'is_amazon_fresh': False,
   'is_whole_foods_market': False,
   'rating': 4.5,
   'ratings_total': 38,
   'sponsored': False,
   'prices': [{'symbol': '$',
     'value': 159.83,
     'currency': 'USD',
     'raw': '$159.83'}],
   'price': {'symbol': '$',
    'value': 159.83,
    'currency': 'USD',
    'raw': '$159.83'},
   'delivery': {'price': {'raw': 'FREE Shipping',
     'symbol': '$',
     'currency': 'USD',
     'value': 0,
     'is_free': True}}}]}

With this sample you could do:

import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore

# Use a service account
cred = credentials.Certificate('path/to/serviceAccount.json')
firebase_admin.initialize_app(cred)

db = firestore.client()
for result in data["search_results"]:
    doc_ref = db.collection(u'amazon').document(result["asin"])
    doc_ref.set(result)

This would create a document using the asin, but it could be any other id you want or even the position on search result by using db.collection(u'amazon').document("position: " + str(result["position"]))

Here is how'd it look if you used asin for the document name.

Here is more information on quick starting python firestore by setting up a service account and some code snippets