How to store smart-list rules in a relational database

Question

The system I'm building has smart groups. By smart groups, I mean groups that update automatically based on these rules:

1. Include all people that are associated with a given client.
2. Include all people that are associated with a given client and have these occupations.
3. Include a specific person (ie, by ID)

Each smart groups can combine any number of these rules. So, for example, a specific smart list might have these specific rules:

1. Include all people that are associated with client 1
2. Include all people that are associated with client 5
3. Include person 6
4. Include all people associated with client 10, and who have occupations 2, 6, and 9

These rules are OR'ed together to form the group. I'm trying to think about how to best store this in the database given that, in addition to supporting these rules, I'd like to be able to add other rules in the future without too much pain.

The solution I have in mind is to have a separate model for each rule type. The model would have a method on it that returns a queryset that can be combined with other rules' querysets to, ultimately, come up with a list of people. The one downside of this that I can see is that each rule would have its own database table. Should I be concerned about this? Is there, perhaps, a better way to store this information?

Answer 1

Why not use Q objects ?

rule1 = Q(client = 1)
rule2 = Q(client = 5)
rule3 = Q(id = 6)
rule4 = Q(client = 10) & (Q(occupation = 2) | Q(occupation = 6) | Q(occupation = 9))

people = Person.objects.filter(rule1 | rule2 | rule3 | rule4)

and then store their pickled strings into the database.

rule = rule1 | rule2 | rule3 | rule4
pickled_rule_string = pickle.dumps(rule)
Rule.objects.create(pickled_rule_string=pickled_rule_string)

Answer 2

Here are the models we implemented to deal with this scenario.

class ConsortiumRule(OrganizationModel):
    BY_EMPLOYEE = 1
    BY_CLIENT = 2
    BY_OCCUPATION = 3
    BY_CLASSIFICATION = 4
    TYPES = (
        (BY_EMPLOYEE, 'Include a specific employee'),
        (BY_CLIENT, 'Include all employees of a specific client'),
        (BY_OCCUPATION, 'Include all employees of a speciified  client ' + \
            'that have the specified occupation'),
        (BY_CLASSIFICATION, 'Include all employees of a specified client ' + \
            'that have the specified classifications'))

    consortium = models.ForeignKey(Consortium, related_name='rules')
    type = models.PositiveIntegerField(choices=TYPES, default=BY_CLIENT)
    negate_rule = models.BooleanField(default=False,
        help_text='Exclude people who match this rule')


class ConsortiumRuleParameter(OrganizationModel):
    """ example usage: two of these objects one with "occupation=5" one
    with "occupation=6" - both FK linked to a single Rule
    """

    rule = models.ForeignKey(ConsortiumRule, related_name='parameters')
    key = models.CharField(max_length=100, blank=False)
    value = models.CharField(max_length=100, blank=False)

At first I was resistant to this solution as I didn't like the idea of storing references to other objects in a CharField (CharField was selected, because it is the most versatile. Later on, we might have a rule that matches any person whose first name starts with 'Jo'). However, I think this is the best solution for storing this kind of mapping in a relational database. One reason this is a good approach is that it's relatively easy to clean hanging references. For example, if a company is deleted, we only have to do:

ConsortiumRuleParameter.objects.filter(key='company', value=str(pk)).delete()

If the parameters were stored as serialized objects (eg, Q objects as suggested in a comment), this would be a lot more difficult and time consuming.