简体   繁体   中英

Adding timestamp index column with a column header to existing csv (python)

#gets rid of spaces in existing csv headers
def getColumns(readCSV):    
    return [column.replace(' ','') for column in next(readCSV)]

#insert format used to avoid hard-coded headers in script
def insertData(tableName, columns, readCSV):
    print("Inserting Data")
    query = 'INSERT INTO {}({}) VALUES ({})'.format(
            tableName,
            ','.join(columns),
            ','.join('?' * len(columns))
            )
    for data in readCSV:
        cursor.execute(query, data)
    con.commit()

def updateTable(csvPath, tableName):
    print("Updating table...")
    print("Reading file contents and uploading into db table...")

    ## insert timestamp column into existing csv. Does not incorporate header correctly for timestamp
    rows = []
    with open(csvPath, 'r', newline='') as csvFile:
        readCSV = csv.reader(csvFile, delimiter=',')
        for row in readCSV:
            rows.append(row)
        with open(csvPath, 'w', newline='')as writeFile:
            file_write = csv.writer(writeFile)
            for val in rows:
                timestamp = datetime.now()
                val.insert(0, timestamp)
                file_write.writerow(val)
                
    with open(csvPath) as csvFile:
        readCSV = csv.reader(csvFile, delimiter=',')
        columns = getColumns(readCSV)
        insertData(tableName, columns, readCSV)
    print ("Upload complete")

Above is a snippet of the code I'm working on. I am gathering data from a csv to insert into a SQL database. Currently, the csv does not have a timestamp column and without that, the import wont work as it sees duplicate data. I found a solution at https://www.geeksforgeeks.org/how-to-add-timestamp-to-csv-file-in-python/ for adding a timestamp column and have incorporated it into the code, but it does not add a header for the column. I'm sure it's an easy fix, but I am new to python and cant find the solution anywhere else. Also, if you see something inefficient with the coding in updateTable, let me know so that I can recognize it and learn a better way.

I ended up using a different method with import pandas as pd

def updateTable(csvPath, tableName):
    print("Updating table...")
    print("Reading file contents and uploading into db table...")

    timestamp = datetime.now()        
    df = pd.read_csv(csvPath)
    df.insert(0, 'timestamp',timestamp)    
    df.to_csv(csvPath, index=False)

    with open(csvPath) as csvFile:
       readCSV = csv.reader(csvFile, delimiter=',')
       columns = getColumns(readCSV)
       insertData(tableName, columns, readCSV)                
    print ("Upload complete")

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM