Django - Replace None by an empty string on the CharField - best practices

Question

I've decided to review my Django App and I found some issues in my models.

My issue is related to the CharField blank and null options.

I was reading about best practices in Django and I've just found out that I was doing everything wrong. Many posts here explain that we should not use blank and null options on CharField at the same time. To follow the rule I deleted the null=True and kept just blank=True

Here comes my doubt: I'm populating my DB from a web scrape code. Sometimes the CharField options are returned as None from the website and when my code tried to send it to the DB (I am not doing it in Forms) I get the Invalid Format error telling me that I shouldn't use None - because I took of the null=True.

I've read some stuff about it and figure out one solution, which I'm presenting here to hear from you, guys/girls. Is this the best approach? I feel like Django might have something out of the box for this very common situation.

class Empresa(models.Model):
    #I've chosen just some fields to keep it simple

    userstelegram = models.ManyToManyField(UserTelegram, blank=True)
    nome = models.CharField(max_length=150, blank=True)
    denominacao_social = models.CharField(max_length=150, blank=True)
    setor_atividade = models.CharField(max_length=150, blank=True)
    categoria_registro = models.CharField(max_length=100, blank=True)
    cnpj = models.CharField(max_length=100, blank=True)
    coberta = models.BooleanField(default=True)

    def save(self, *args, **kwargs):
        self.full_clean(
            exclude = [
                'nome',
                'denominacao_social', 
                'setor_atividade',
                'categoria_registro',
                'cnpj',
            ]
        ) # performs regular validation then clean()

        super(Empresa, self).save(*args, **kwargs)

    def clean(self):
        
        self.cnpj = str(self.cnpj or '').strip()
        self.categoria_registro = str(self.categoria_registro or '').strip()
        self.setor_atividade = str(self.setor_atividade or '').strip()
        self.denominacao_social = str(self.denominacao_social or '').strip()
        self.nome = str(self.nome or '').strip()

    def __str__(self):
        return self.nome if self.nome != None else self.denominacao_social

Two extra questions:

1 - I'm using self.full_clean() with the exclude param so the validator does not trigger an error on CharField with None. Is this the correct way?

2 - Does the full_clean function use strip() on all string fields? As I'm excluding my CharFields from the full_clean, should I use the strip on the clean function?

Thank you, folks!

Answer 1

Your workaround doesn't look clean to me, feels too hack-ish and wrong.

I don't know where you read that that it's a bad idea to set null=True and blank=True at the same time. I don't think there is a general rule for this and you should proceed from what you're trying to achieve and what will be most convenient for you in your specific situation. If you don't really care if some of your char fields could be null and some of them could be blank, and your code will handle both cases just fine then perhaps there is no reason to remove null=True setting? Seems like that decision caused you some troubles and made your code less clean.

Anyway, clean method is for validating data, not transforming it. So, if you set validation to forbid null values in your fields why do you then pass null data? If you do want to forbid storing null data then the right approach would be to convert null values that your scraper receives to empty strings before you pass it to your model. This will eliminate the need to implement dirty workarounds with validators.

Does the full_clean function use strip() on all string fields?

It doesn't. It only performs a validation. So if you want to have your stings stripped before you save them to DB then you should manually stip them, otherwise don't.