django.db.utils.IntegrityError: NOT NULL constraint failed: unesco_site.category_id

Question

I am trying to populate database from data in the csv file using a python script. I did some research but couldn't find a relevant example rather I found python packages that would load csv. And there some articles guiding on how to upload csv file which isn't what I wanted.

Following is a glimpse of load.csv file. There are 11 columns as you can see.

# models.py
from django.db import models


class Category(models.Model):
    category = models.CharField(max_length=128)

    def __str__(self):
        return self.name

class State(models.Model):
    state = models.CharField(max_length=25)

    def __str__(self):
        return self.name


class Region(models.Model):
    region = models.CharField(max_length=25)

    def __str__(self):
        return self.region

class Iso(models.Model):
    iso = models.CharField(max_length=5)

    def __str__(self):
        return self.iso

class Site(models.Model):
    name = models.CharField(max_length=128)
    year = models.IntegerField(null=True)
    area = models.FloatField(null=True)
    describe = models.TextField(max_length=500)
    justify = models.TextField(max_length=500, null=True)
    longitude = models.TextField(max_length=25, null=True)
    latitude = models.TextField(max_length=25, null=True)

    #one to many field
    category = models.ForeignKey(Category, on_delete=models.CASCADE)
    state = models.ForeignKey(State, on_delete=models.CASCADE)
    region = models.ForeignKey(Region, on_delete=models.CASCADE)
    iso = models.ForeignKey(Iso, on_delete=models.CASCADE)

    def __str__(self):
        return self.name

I'm not being able to make the relationship among the models while inserting data. Earlier, I had used get_or_create() method but I was recommended to not to use it as I had no default value to be given and used create() method. Somewhere I had found that, until a data is saved Primary key isn't generated so, I tried saving data to the field each time after reading data but that also didn't work. . Following is the code in many_load.py script. There's no problem in reading data.

import csv 

#python3 manage.py runscript many_load

from unesco.models import Site, Category, Iso, Region, State

def run():
    fhand = open('unesco/load.csv')
    reader = csv.reader(fhand)
    next(reader)

    
    # Category.objects.all().delete()
    # Iso.objects.all().delete()
    # Region.objects.all().delete()
    # State.objects.all().delete()

    # Site.objects.all().delete()


    for row in reader:
        # print(row)
        # print (len(row))

        nm= Site.objects.create(name=row[0])
        dsc = Site.objects.create(describe=row[1])
        jst = Site.objects.create(justify=row[2])
        yr = Site.objects.create(year=row[3])
        lng = Site.objects.create(longitude=row[4])
        lat = Site.objects.create(latitude=row[5])
        area = Site.objects.create(area=row[6])

        
        st = Site(category=row[7], state=row[8], region=row[9], iso=row[10], 
                    name=nm, area=area, describe=dsc, justify=jst, longitude=lng, latitude=lat)
        st.save()

But when I try running the script python3 manage.py runscript many_load it gives me the following error. Help please. You could also refer me some good articles for understanding more on this.

Traceback (most recent call last):
  File "/home/bu113t/envs/djpro/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/home/bu113t/envs/djpro/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.IntegrityError: NOT NULL constraint failed: unesco_site.category_id

Answer 1

This error is occurring because you are passing a string value to the category field of Site but it is expecting a Category object as it is a ForeignKey field.

Not only that you have other problems in many_load.py

The following code has the correction of all of the problems. Hope it will work for you.

import csv

# python3 manage.py runscript many_load

from unesco.models import Site, Category, Iso, Region, State


def run():
    fhand = open('unesco/load.csv')
    reader = csv.reader(fhand)
    next(reader)

    # Category.objects.all().delete()
    # Iso.objects.all().delete()
    # Region.objects.all().delete()
    # State.objects.all().delete()

    # Site.objects.all().delete()

    for row in reader:
        # print(row)
        # print (len(row))

        nm = row[0]
        dsc = row[1]
        jst = row[2]
        yr = row[3]
        lng = row[4]
        lat = row[5]
        area = row[6]

        category = Category.objects.create(category=row[7])
        state = State.objects.create(state=row[8])
        region = Region.objects.create(region=row[9])
        iso = Iso.objects.create(iso=row[10])

        st = Site(category=category, state=state, region=region, iso=iso,
                  name=nm, area=area, describe=dsc, justify=jst, longitude=lng, latitude=lat)
        st.save()