保存Web应用程序的动态图结构

Question

I have been searching for an answer to this question for a while, however haven't found a satisfying solution yet.

I am trying to store a dynamic undirected graph structure for a web application. It should store user "subscriptions" between each other, which can change quite frequently.

A traditional database solution makes no sense. JSON files for each user doesn't seem like the best solution either, for the same reason why the database solution isn't ideal.

Any other ideas for an optimal solution for my problem?

Thanks in advance!

Answer 1

The type of structure you are talking about makes most sense in a relational database (which is what I assume you meant by "traditional"). Because you have the subscriptions between users, this is a relationship, thus, a relational database makes most sense. Relational databases allow explicit connections between different tables.

A document database (ie one that holds JSON documents) is a very bad idea for this type of data . Document databases can be very good at certain things, but data that is highly interdependent (like some sort of subscription system) is a poor use of a document database. I'll explain more as we go along.

You say that your graph edges are undirected, but that the fact that you call them "subscriptions" tells me that they are actually directed: one user subscribes to another. If it was undirected, it would be more like friending someone on facebook or connecting on LinkedIn: if I am friends with you, then you must be friends with me. In a subscription system (say like Google+ or Twitter), even if I am subscribed to you, you don't necessarily need to be subscribed to me. If we are both subscribed to each other, then it is actually two directed edges: one to you from me, and one to me from you.

Thus, the best solution is to have at least two tables: a primary " users " table, and a secondary " subscriptions " table. The " users " table would have columns like uid , name , email and so forth. The " subscriptions " table has only two columns: subscriber and subscription . Both hold a uid value from the " users " table and every pair of values must be unique in the table.

You asked if this would "bloat up" with so many subscriptions. First off, you are assuming you are going to be the next facebook and will need to deal with millions or billions of users. Don't worry, you won't have that problem, at least at first. Second, most relational databases are logarithmic in their performance for retrieving and inserting records, which scales very nicely as your numbers of users increases. For the type of behavior you are expecting from a document database or JSON files on disk, your behavior will either be of linear time complexity , since you will need to iterate over every single document in the database to be sure you have checked for all subscriptions (linear behavior scales far worse than logarithmic), or you will need to duplicate subscriber/subscription information in all records. The second solution would indeed would be bloated since you are duplicating tons of data and, more importantly, it runs the huge risk of getting out of sync quite easily. It is far easier to get out of sync in this scenario than you might think.

To show you how to do this, I will use the sqlite3 dialect of SQL . It is what I prototype in most, so I am most familiar with it. Converting it to something like MySQL or PostgreSQL should be fairly trivial. Here is the statements to make the databases:

# since `uid` is the primary key, just pass it a
# null value on insertion and the database will 
# generate a unique integer and use that automatically.
# it might also be good to make more than just the uid unique,
# such as their email.
CREATE TABLE users (uid INTEGER PRIMARY KEY,
                    name TEXT,
                    email TEXT);

# we will use the uid for the foreign key reference since this should
# never change, even if the user changes their name or email.
CREATE TABLE subrs (subscriber INTEGER,
                    subscription INTEGER,

                    # make sure each entry of pairs is unique
                    CONSTRAINT uc_edges UNIQUE (subscriber,subscription),

                    # be sure subscribers can only be created for users that exist
                    CONSTRAINT fk_subr FOREIGN KEY (subscriber) REFERENCES users(uid),

                    # be sure subscription can only be created for users that exist
                    CONSTRAINT fk_subee FOREIGN KEY (subscription) REFERENCES users(uid)
                   ); 

This usually has the nice added benefit that you can't delete users who have subscriptions to them until you first delete those subscriptions. Depending on the database you choose, YMMV, so check the docs for your DB of choice. Almost all SQL databases also support the behavior with foreign keys that you can not create a record with a value for a foreign key that does not yet exist. With JSON files or a document database, it is easy to leave dangling subscriptions or to have user deletion take a long time since you need to modify every user document that references a given user. A relational SQL database can simplify a lot of things that would otherwise be done in your code. Handling this logic in your application code introduces many more opportunities for mistakes and bugs with your data handling. A bit of advice: what ever work you can offload onto your database, you should offload onto your database. Professional databases are far better tested than your code and already have the logic for many of the common sorts of actions you could want to do to data.

To find a user's subscriptions, you would do a query something like the following:

SELECT * FROM subrs WHERE subscriber=some_uid;

To get all the subscribers to a given user, the query is similarly simple:

SELECT * FROM subrs WHERE subscription=some_uid;

To delete a user record would only be about three lines long:

DELETE FROM subrs WHERE subscription=some_uid;
DELETE FROM subrs WHERE subscriber=some_uid;
DELETE FROM users WHERE uid=some_uid;

In a document database you would have far more application code to do something pretty similar and you run the risk of your application code having poor logic and breaking your data.

TL;DR

Use a relational SQL database. You can create explicit relationships between records. Thus, it won't be as easy to shoot yourself in the foot as it would be with a document database (since all relationships are merely implied). SQL databases like MySQL also tend to scale better, both vertically (ie with more users records) and horizontally (ie with more replica servers).