Scrapy - How save the json file in S3 and local file system - at same time

Question

I'm already have my settings set to save the json in a S3 bucket. But I want to save also in my local machine, if this is possible.

I tried the config below, but Scrapy save only in the local machine.

FEED_URI = 's3://bucket/scraped/file.jl'
FEED_URI = 'file:///tmp/file.jl'

I dont understand very well the settings explained by the Scrapy docs here

Answer 1

Scrapy's feed exports extension doesn't support sending items to two places at the time.

FEED_URI in your settings is just a Python variable, so the reason it's only saving in your local machine is because it's being overwritten the second time.

You can work around that by using FEED_URI to send items to S3 and writing a pipeline that saves your items locally.