使用新索引更新完整的 dynamodb 表

Question

I am using dynamo db having around 15000 items. Each item has 4 indexes "url","date","html_data","org". I added one more index named "base_url" to this table.

Here the index "url" contains links of websites like https://stackoverflow.com/questions/ask , https://www.goal.com/en/news/neville-calls-man-utd-bunch-of-whingebags-ronaldo-seen/blt71f0824c3e8eaf1e etc.

and I need to update the new index "base_url" with the base urls of links present in index "url".

Here I am referring base url like https://stackoverflow.com/ , https://www.goal.com/

I can update each of these individually but how can it be done for the complete 15000 items. I found there is batchwrite item but didnt find anything like batchupdate.

I am using python boto3 for doing this.

response = table.get_item(
    Key={
        'url': 'https://stackoverflow.com/questions/ask',
        "date" : "2021-12-28"
    }
)
item = response['Item']
print(item)

and

table.update_item(
    Key={
        'url': 'https://stackoverflow.com/questions/ask',
        "date" : "2021-12-28"
    },
    UpdateExpression='SET base_url = :val1',
    ExpressionAttributeValues={
        ':val1': "https://stackoverflow.com/"
    }
)

Base url can be obtained using

url = "https://stackoverflow.com/questions/ask"
print(requests.urllib3.util.parse_url(url)).path)

Answer 1

useawswrangler to insert your data either as a dataframe using put_df or a list of dict using put_items