部署在 Heroku 上时更新 Django web 应用程序内容的最佳实践

Question

I'm finally at the stage of deploying my Django web-app on Heroku. The web-app retrieves and performs financial analysis on a subset of public companies. Most of the content of the web-app is stored in.csv and.xslx files. There's a master.xslx file that I need to open and manually update on a daily basis. Then I run scripts which based on info in that.xsls file retrieve financial data, news, etc and store the data in.csv files. Then my Django views and html templates are referring to those files (and not to a sql/postgress database). This is my setup in a nutshell.

Now I'm wondering what is the best way to make this run smoothly in production.

1. Shall I store these.xslx and.csv files on AWS S3 and have Django access them and update them from there? I assume that this way I can easily open and edit the master.xslx file anytime. Is that a good idea and do I run any performance or security issues?
2. Is it better to convert all these data into a postgress database on Heroku? Is there a good guideline in terms of how I can technically do that? In that scenario wouldn't it be much more challenging for me to edit the data in the master.xsls file?
3. Are there any better ways you would suggest to handle this?

I'd highly appreciate any advice on this matter.

Answer 1

You need to perform a trade-off between easy of use (access/update the source XSLX file) and maintainability (storing safely and efficiently the data).

Option #1 is more convenient if you need to quickly open and change the file using your Excel/Numbers application. On the other hand your application needs to access physical files to perform the logic and render the views.

BTW some time ago I have created a repository Heroku Files to present some options in terms of using external files.

Option #2 is typically better from a design point of view: the data is organised in the database and can be more efficiently queried and manipulated. The challenge in this case is that you need a way to view/edit the data, and this normally requires more development (creating new screens, etc..)

Involving a database is normally the preferred approach as you can scale up to large dataset without problems (which is not the case with files). On the other hand if the XLS file stays small and you only need simple (quick) updates your current architecture can work.