Importing csv file in mysql

Question

I have a database with tables: person, player, coach, and team. All the tables have an auto-increment id field as the primary key. Person has id, firstname, lastname. Player and coach both have the id field, as well as person_id and team_id as foreign keys to tie them to a team.id or person.id field in the other tables. I have one master csv file, from that I want import all the values in MySql different tables with ids. And I want to check the value also in the data base. If the value is in database then do not import that value. I have used CSV parsing and indexing function. But I am not able to do that. Can any one help me in that My sql table below

mysql> describe person;
+-----------+-------------+------+-----+---------+----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------+-------------+------+-----+---------+----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| firstname | varchar(30) | NO   |     | NULL    |                |
| lastname  | varchar(30) | NO   |     | NULL    |                |
+-----------+-------------+------+-----+---------+----------------+
mysql> describe player;
+-----------+---------+------+-----+---------+----------------+
| Field     | Type    | Null | Key | Default | Extra          |
+-----------+---------+------+-----+---------+----------------+
| id        | int(11) | NO   | PRI | NULL    | auto_increment |
| person_id | int(11) | NO   | MUL | NULL    |                |
| team_id   | int(11) | NO   | MUL | NULL    |                |
+-----------+---------+------+-----+---------+----------------+
mysql> describe team;
+-----------+-------------+------+-----+---------+----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------+-------------+------+-----+---------+----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| teamname  | varchar(25) | NO   |     | NULL    |                |
| location  | varchar(40) | NO   |     | NULL    |                |
| city      | varchar(25) | NO   |     | NULL    |                |
| state     | varchar(2)  | NO   |     | NULL    |                |
| venue     | varchar(35) | NO   |     | NULL    |                |
| league_id | int(11)     | NO   | MUL | NULL    |                |
+-----------+-------------+------+-----+---------+----------------+

My Csv file is First Name Last Name teamname Location city state |venue
abc cdf india csv bng kar abc

After importing

I have a database with tables: person, player, coach, and team. All the tables have an auto-increment id field as the primary key. Person has id, firstname, lastname. Player and coach both have the id field, as well as person_id and team_id as foreign keys to tie them to a team.id or person.id field in the other tables. I have one master csv file, from that I want import all the values in MySql different tables with ids. And I want to check the value also in the data base. If the value is in database then do not import that value. I have used CSV parsing and indexing function. But I am not able to do that. Can any one help me in that My sql table below

mysql> describe person;

+-----------+-------------+------+-----+---------+----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------+-------------+------+-----+---------+----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| firstname | varchar(30) | NO   |     | NULL    |                |
| lastname  | varchar(30) | NO   |     | NULL    |                |
+-----------+-------------+------+-----+---------+----------------+
mysql> describe player;
+-----------+---------+------+-----+---------+----------------+
| Field     | Type    | Null | Key | Default | Extra          |
+-----------+---------+------+-----+---------+----------------+
| id        | int(11) | NO   | PRI | NULL    | auto_increment |
| person_id | int(11) | NO   | MUL | NULL    |                |
| team_id   | int(11) | NO   | MUL | NULL    |                |
+-----------+---------+------+-----+---------+----------------+
mysql> describe team;
+-----------+-------------+------+-----+---------+----------------+
| Field     | Type        | Null | Key | Default | Extra          |
+-----------+-------------+------+-----+---------+----------------+
| id        | int(11)     | NO   | PRI | NULL    | auto_increment |
| teamname  | varchar(25) | NO   |     | NULL    |                |
| location  | varchar(40) | NO   |     | NULL    |                |
| city      | varchar(25) | NO   |     | NULL    |                |
| state     | varchar(2)  | NO   |     | NULL    |                |
| venue     | varchar(35) | NO   |     | NULL    |                |
| league_id | int(11)     | NO   | MUL | NULL    |                |
+-----------+-------------+------+-----+---------+----------------+

My Csv file is

First Name  Last Name   teamname    Location    city    state   |venue  
abc cdf india   csv bng kar abc

After importing

id First Name   Last Name   teamname    Location    city    state   |venue  coment
1   1   1   1   1   1   1 abc abc

I am trying with some small code

 # initialize with empty ints and dicts
        name,cities,countries,states=[],[],[],[]

        with open('ind.csv','rb') as csvfile:
            reader = csv.reader(csvfile, delimiter=',')
            reader.next() #skip header
            for row in reader:
                name.append(row[0])
                cities.append(row[2])
                states.append(row[3])
                countries.append(row[4])
        cl = list(set(countries))
        sl = list(set(states))
        citl = list(set(cities))
        inf1 = list(set(name)) 



        with open('countries.csv','w') as cfile:
            writer = csv.writer(cfile, delimiter=',')
            writer.writerow(['country_id','name'])
            for i,x in enumerate(cl):
                writer.writerow([i,x])

        with open('state.csv','w') as cfile:
            writer = csv.writer(cfile, delimiter=',')
            writer.writerow(['state_id','country_id','state'])
            for i,x in enumerate(sl):
                writer.writerow([i,x,cl.index(countries[states.index(x)])])

        with open('cities.csv','w') as cfile:
            writer = csv.writer(cfile,delimiter=',')
            writer.writerow(['city_id','city','st_id','country_id'])
            for i,x in enumerate(citl):
                writer.writerow([i,x,sl.index(states[cities.index(x)]),
                                 cl.index(countries[cities.index(x)])
                                 ])


        with open('inf123.csv','w') as cfile:
            writer = csv.writer(cfile,delimiter=',')
            writer.writerow(['Name_id', 'Name','city_id','st_id','country_id'])
            for i,x in enumerate(inf1):
                writer.writerow([i,x,
                                citl.index(cities[name.index(x)]),
                                sl.index(states[name.index(x)]),
                                cl.index(countries[name.index(x)])

                                 ])

        import MySQLdb 
        import csv
        mydb = MySQLdb.connect(host="localhost", # The Host 
        user="root", # username 
        passwd="root", # password 
        db="abcm") # name of the data base


        cursor = mydb.cursor()

        csv_data = csv.reader(file('countries.csv'))
        for row in csv_data:

            cursor.execute('INSERT INTO country(id, \
                  name )' \
                  'VALUES("%s", "%s")', 
                  row)
        #close the connection to the database.
        mydb.commit()
        cursor.close()
        print "Done"


        cursor = mydb.cursor()

        csv_data = csv.reader(file('state.csv'))
        for row in csv_data:

            cursor.execute('INSERT INTO state(id, \
                  country, name )' \
                  'VALUES("%s", "%s", "%s")', 
                  row)
        #close the connection to the database.
        mydb.commit()
        cursor.close()
        print "Done"    

Answer 1

I have one master csv file, from that I want import all the values in MySql different tables with ids.

This is not possible because the import routine doesn't know where you want to put the data.

If your master csv file contained a column containing the table name you could then

• import your csv file into a temporary
• use different sql statements to move the data into the appropriate tables