What is the best approach for storing for Email History in CRM App databases

Question

We are building a CRM with a fairly standard database model. Users, Permissions, Contacts, Leads etc.

We also want to maintain all email communication, so that it can be stored, searched, and used as business processes backup.

I imagine the amount of email traffic will be quite heavy - leading to a large database really really quick.

Is it safe to bundle this with the CRM app or should it be isolated in its own unique database (probably on its own server).

Answer 1

You could use the mysql database as quick lookup for some metadata like date sent, email address, subject or similar as well as a unique id and store everything else related to the mail (like content/attachments) in a nosql database like redis, since you can access those really fast with the unique key.

we are using this approach to store > 8.000.000 emails and the lookup by email-address and/or date is blazing fast due to indexing.

(if you really need to search the mail content (like fulltext search) I would expect other engines like elasticsearch to be more search speed friendly than mysql fulltext searches)