uploading Csv file to S3 using boto3

Question

am trying to uplaod a csv file to S3 Bucket "ebayunited" using Boto3

1. i fetched json products from dummy data as json
2. Convert it to Csv and save it in the same folder
3. i used Boto3 to uplaod it

Code

    import requests
import pandas as pd
import boto3 as bt



from API_keys import access_key, secret_access_key
client = bt.client(
    's3', aws_access_key_id=access_key,
          aws_secret_access_key=secret_access_key
)

r = requests.get("https://dummyjson.com/products")
json_products = pd.DataFrame(r.json()["products"])
file_s3 = json_products.to_csv("Orders.csv", index=False)


bucket = "ebayunited"
path = "Lysi Team/ebayapi/"+str(file_s3)

client.upload_file(str(file_s3), bucket, path)

and i get this Error

    Traceback (most recent call last):
  File "c:\Users\PC\Desktop\S3 Import\s3_import.py", line 18, in <module>
    client.upload_file(str(file_s3), bucket, path)
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\boto3\s3\inject.py", line 143, in upload_file
    return transfer.upload_file(
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\boto3\s3\transfer.py", line 288, in upload_file
    future.result()
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\s3transfer\futures.py", line 103, in result
    return self._coordinator.result()
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\s3transfer\futures.py", line 266, in result
    raise self._exception
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\s3transfer\tasks.py", line 269, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\s3transfer\upload.py", line 585, in _submit
    upload_input_manager.provide_transfer_size(transfer_future)
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\s3transfer\upload.py", line 244, in provide_transfer_size
    self._osutil.get_file_size(transfer_future.meta.call_args.fileobj)
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\site-packages\s3transfer\utils.py", line 247, in get_file_size
    return os.path.getsize(filename)
  File "C:\Users\PC\AppData\Local\Programs\Python\Python310\lib\genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'None'

Answer 1

The upload_file(filename, bucket, key) command expects the name of a file to upload from your local disk.

Your program appears to be assuming that the to_csv() function returns the name of the resulting file, but the to_csv() documentation says:

If path_or_buf is None, returns the resulting csv format as a string. Otherwise returns None.

Therefore, you will need to pass the actual name of the file:

key = 'Lysi Team/ebayapi/test.csv' # Change desired S3 Key here
client.upload_file('test.csv', bucket, key)