Read data from a serial device using Python and save it to PostgreSQL using a Raspberry Pi

Question

Im trying read data from a smartmeter that has FTDI chip. Wrote a simple Python serial program to pass the commands to meter and the meter replies back. the data from meter is then converted to float and stored in dictionary.

now i want to store the dictionary to DB, here is the code to put data in the table.

import psycopg2

conn = psycopg2.connect(database="smartmeter", user="postgres", password="12345", host="localhost", port="5432")

cur = conn.cursor()

cur.execute('''CREATE TABLE metertable (ID      SERIAL  PRIMARY KEY     NOT NULL,meter  TEXT    NOT NULL,temperature    TEXT    NOT NULL,freq   TEXT    NOT NULL,penergy        TEXT    NOT NULL,qenergy    TEXT    NOT NULL,senergy        TEXT    NOT NULL,cospi  TEXT    NOT NULL,irms   TEXT    NOT NULL,ppower TEXT    NOT NULL,qpower TEXT    NOT NULL,spower TEXT    NOT NULL);''')
while 1:
    data=dict[]
    data={
                'time':timestamp,
                'meter':m0_data,
                'temperature':m1_data,
                'freq':m2_data,
                'penergy':m3_data,
                'qenergy':m6_data,
                'senergy':m7_data,
                'cospi':m11_data,
                'irms':m15_data,
                'vrms':m16_data,
                'ppower':realpower,
                'qpower':reactivepower,
                'spower':apparentpower
        }
        cur.executemany ("""INSERT INTO metertable(time,meter,temperature,freq,penergy,qenergy,senergy,cospi,irms,vrms,ppower,qpower,spower) VALUES (%(time)s, %(meter)s), %(temperature)s), %(freq)s), %(penergy)s), %(qenergy)s), %(senergy)s), %(cospi)s), %(irms)s), %(vrms)s), %(ppower)s) %(qpower)s) %(spower)s)""", data)

I get an error like this

Traceback (most recent call last):
  File "metercmd.py", line 97, in <module>
    cur.executemany("""INSERT INTO metertable(time,meter,temperature,freq,penergy,qenergy,senergy,cospi,irms,vrms,ppower,qpower,spower) VALUES (%(time)s, %(meter)s), %(temperature)s), %(freq)s), %(penergy)s), %(qenergy)s), %(senergy)s), %(cospi)s), %(irms)s), %(vrms)s), %(ppower)s) %(qpower)s) %(spower)s)""", data)
TypeError: string indices must be integers, not str

Am I going on the right direction to enter data to the DB? Please suggest some best methods.

Answer 1

Error says: Some of fields you try to push into database are string fields but you push integer values.

Make sure that your data values matches your database table fields.

Answer 2

You have extra closing parenthesis and missing commas:

cur.executemany ("""
    INSERT INTO metertable(
        time,meter,temperature,freq,penergy,qenergy,senergy,cospi,irms,vrms,ppower,qpower,spower
    ) VALUES (
        %(time)s, %(meter)s, %(temperature)s, %(freq)s, %(penergy)s, %(qenergy)s, %(senergy)s,
        %(cospi)s, %(irms)s, %(vrms)s, %(ppower)s, %(qpower)s, %(spower)s
    )
""", data)

Answer 3

Thanks for the help . The error was with the executemany. i used type(variable) to get the variable types and changed in those table I changed that to cur.execute. But when I run the code. I see no data saving in my database. here is the code

import serial
import psycopg2

port=serial.Serial("/dev/ttyUSB0",baudrate=9600,timeout=.1)

conn =psycopg2.connect(database="smartmeternode",user="postgres",password="amma",host="localhost",port="5432")
cur=conn.cursor
data2=dict()
while 1:
   port.write('m0\r')
   meter=port.read()
   port.write('m2\r')
   temp=port.read()
   ph=meter*temp
   time=timestamp
   data={'meter':meter,'temp':temp,'ph':ph,'time':time}
   cur.execute("INSERT INTO METERDATATEST(meter,temp,ph,time) VALUES(%(meter)s,%(temp)s,%(ph)s,%(time)s;",data)
   cur.commit
cur.close
conn.close

the program will run for ínfinite time and i stop the program using ctrl+Z. when I check the database its still empty. what am i missing? I want the data to be stored in the PSQL database as soon as it is read from my smartmeter

psql smartmeternode -U postgres
smartmeternode=# SELECT * FROM meterdatatest;
id | meter | temp | ph | time 
----+-------+------+----+------
(0 rows)