S3 to IPFS from Pinata

Question

I am trying to upload a lot of files from S3 to IPFS via Pinata. I haven't found in Pinata documentation something like that.

This is my solution, using the form-data library. I haven't tested it yet (I will do it soon, I need to code some things).

Is it a correct approach? anyone who has done something similar?

async uploadImagesFolder(
    items: ItemDocument[],
    bucket?: string,
    path?: string,
  ) {

    try {
      const form = new FormData();
      for (const item of items) {
        const file = getObjectStream(item.tokenURI, bucket, path);
        form.append('file', file, {
          filename: item.tokenURI,
        });
      }
      console.log(`Uploading files to IPFS`);
      const pinataOptions: PinataOptions = {
        cidVersion: 1,
      };
      const result = await pinata.pinFileToIPFS(form, {
        pinataOptions,
      });
      console.log(`Piñata Response:`, JSON.stringify(result, null, 2));
      return result.IpfsHash;
    } catch (e) {
      console.error(e);
    }
  }

Answer 1

I had the same problem

So, I have found this: https://medium.com/pinata/stream-files-from-aws-s3-to-ipfs-a0e23ffb7ae5

But in the article If am not wrong, is used a different version to the JavaScript AWS SDK v3 (nowadays the most recent: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html ).

This is for the Client side with TypeScript:

If you have this version, for me works this code snippet:

export const getStreamObjectInAwsS3 = async (data: YourParamsType) => {
    
    try {

        const BUCKET = data.bucketTarget
        const KEY = data.key

        const client = new S3Client({
            region: 'your-region',
            credentials: {
                accessKeyId: 'your-access-key',
                secretAccessKey: 'secret-key'
            }
        })

        const resource = await client.send(new GetObjectCommand({
            Bucket: BUCKET,
            Key: KEY
        }))

        const response = resource.Body

        if (response) {
            return new Response(await response.transformToByteArray()).blob()
        }

        return null
        
    } catch (error) {

        return null
        
    }

}

With the previous code, you can get the Blob Object for pass it to the File object with this method and get the URL resource using the API :

export const uploadFileToIPFS = async(file: Response) => {
    
    const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`
    
    const data = new FormData()
    data.append('file', file)

    try {

        const response = await axios.post(url, data, {
            maxBodyLength: Infinity,
            headers: {
                pinata_api_key: 'your-api',
                pinata_secret_api_key: 'your-secret'
            },
            data: data
        })

        return {
            success: true,
            pinataURL: `https://gateway.pinata.cloud/ipfs/${ response.data.IpfsHash }`
        }
        
    } catch (error) {

        console.log(error)

        return null
        
    }

}

I have found this solution from this nice article and you can explore other implementations (including the Node.js side)

Answer 2

Take a look at Filebase, which offers an S3-Compatible API for storing data on IPFS!

https://filebase.com