Using ConfigParser in Python - tests OK but wipes file once deployed

Question

I am running my own pool control system and I wanted to implement copying certain system parameters to a flat file for processing by a web interface I am working on. Since there are just a few entries I liked ConfigParser for the job.

I built a test setup and it worked great. Here is that code:

import ConfigParser

config = ConfigParser.ConfigParser()
get_pool_level_resistance_value = 870


def read_system_status_values(file, section, system):
    config.read(file)
    current_status = config.get(section, system)
    print("Our current {} is {}.".format(system, current_status))


def update_system_status_values(file, section, system, value):
    cfgfile = open(file, 'w')
    config.set(section, system, value)
    config.write(cfgfile)
    cfgfile.close()
    print("{} updated to {}".format(system, value))


def read_test():
    read_system_status_values("current_system_status", "system_status",
                              "pool_level_resistance_value")


def write_test():
    update_system_status_values("current_system_status", "system_status",
                                "pool_level_resistance_value",
                                get_pool_level_resistance_value)


read_test()
write_test()
read_test()

This is my config file "current_system_status":

[system_status]
running_status = True
fill_control_manual_disable = False
pump_running = True
pump_watts = 865
pool_level_resistance_value = 680
pool_level = MIDWAY
pool_is_filling = False
pool_is_filling_auto = False
pool_is_filling_manual = False
pool_current_temp = 61
pool_current_ph = 7.2
pool_current_orp = 400
sprinklers_running = False
pool_level_sensor_battery_voltage = 3.2
pool_temp_sensor_battery_voltage = 3.2
pool_level_sensor_time_delta = 32
pool_temp_sensor_time_delta = 18

When I run my test file I get this output:

ssh://root@scruffy:22/usr/bin/python -u /root/pool_control/V3.2/system_status.py
Our current pool_level_resistance_value is 350.
pool_level_resistance_value updated to 870
Our current pool_level_resistance_value is 870.

Process finished with exit code 0

This is exactly as expected. However when I move it to my main pool_sensors.py module, anytime I run it I get the following error:

Traceback (most recent call last):
  File "/root/pool_control/V3.2/pool_sensors.py", line 58, in update_system_status_values
    config.set(section, system, value)
  File "/usr/lib/python2.7/ConfigParser.py", line 396, in set
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'system_status'

Process finished with exit code 1

I then debugged (using PyCharm) and as I was walking through the code as soon as it gets to this line in the code:

cfgfile = open(file, 'w')

it wipes out my file completely, and hence I get the NoSectionError. When I debug my test file and it gets to that exact same line of code, it opens the file and updates it as expected.

Both the test file and the actual file are in the same directory on the same machine using the same version of everything. The code that opens and writes the files is an exact duplicate of the test code with the exception of a debug print statement in the "production" code.

I tried various methods including:

cfgfile = open(file, 'w')
cfgfile = open(file, 'r')
cfgfile = open(file, 'wb')

but no matter which one I use, once I include the test code in my production file, as soon as it hits that line it completely wipes out the file as opposed to updating it like my test files does.

Here is the pertenent lines of the code where I call it:

import pooldb  # Database information
import mysql.connector
from mysql.connector import errorcode
import time
import notifications
import logging
import ConfigParser

DEBUG = pooldb.DEBUG
config = ConfigParser.ConfigParser()

def read_system_status_values(file, section, system):
    config.read(file)
    current_status = config.get(section, system)
    if DEBUG:
        print("Our current {} is {}.".format(system, current_status))


def update_system_status_values(file, section, system, value):
    cfgfile = open(file, 'w')
    config.set(section, system, value)
    config.write(cfgfile)
    cfgfile.close()
    if DEBUG:
        print("{} updated to {}".format(system,value))

def get_pool_level_resistance():
    """ Function to get the current level of our pool from our MySQL DB. """
    global get_pool_level
    try:
        cnx = mysql.connector.connect(user=pooldb.username,
                                      password=pooldb.password,
                                      host=pooldb.servername,
                                      database=pooldb.emoncms_db)
    except mysql.connector.Error as err:
        if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
            logger.error(
                'Database connection failure: Check your username and password')
            if DEBUG:
                print(
                    "Database connection failure: Check your username and "
                    "password")
        elif err.errno == errorcode.ER_BAD_DB_ERROR:
            logger.error('Database does not exist. Please check your settings.')
            if DEBUG:
                print("Database does not exist. Please check your settings.")
        else:
            logger.error(
                'Unknown database error, please check all of your settings.')
            if DEBUG:
                print(
                    "Unknown database error, please check all of your "
                    "settings.")
    else:
        cursor = cnx.cursor(buffered=True)
        cursor.execute(("SELECT data FROM `%s` ORDER by time DESC LIMIT 1") % (
            pooldb.pool_resistance_table))

        for data in cursor:
            get_pool_level_resistance_value = int("%1.0f" % data)
            cursor.close()
            logger.info("Pool Resistance is: %s",
                        get_pool_level_resistance_value)
            if DEBUG:
                print(
                    "pool_sensors: Pool Resistance is: %s " %
                    get_pool_level_resistance_value)
                print(
                    "pooldb: Static critical pool level resistance set at ("
                    "%s)." %
                    pooldb.pool_resistance_critical_level)
                print(
                    "pooldb: Static normal pool level resistance set at (%s)." %
                    pooldb.pool_resistance_ok_level)
        cnx.close()

    print("We made it here with a resistance of (%s)" %
          get_pool_level_resistance_value)
    update_system_status_values("current_system_status",
                                "system_status",
                                "pool_level_resistance_value",
                                get_pool_level_resistance_value)

    if get_pool_level_resistance_value >= pooldb.pool_resistance_critical_level:
        get_pool_level = "LOW"
        update_system_status_values("current_system_status",
                                    "system_status",
                                    "pool_level",
                                    get_pool_level)
        if DEBUG:
            print("get_pool_level_resistance() returned pool_level = LOW")
    else:
        if get_pool_level_resistance_value <= pooldb.pool_resistance_ok_level:
            get_pool_level = "OK"
            update_system_status_values("current_system_status",
                                        "system_status",
                                        "pool_level",
                                        get_pool_level)
            if DEBUG:
                print("get_pool_level_resistance() returned pool_level = OK")

    if DEBUG:
        print("Our Pool Level is %s." % get_pool_level)

    return get_pool_level

I suspect that it might have something to do with another import maybe conflicting with the open(file,'w').

My main module is pool_fill_control.py and it has these imports:

import pooldb  # Configuration information
import datetime
import logging
import os
import socket
import subprocess
import threading
import time
import RPi.GPIO as GPIO  # Import GPIO Library
import mysql.connector
import requests
import serial
from mysql.connector import errorcode
import notifications
import pool_sensors
import ConfigParser

Within a function in that module, it then calls my pool_sensors.py module shown above using this line of code:

get_pool_level = pool_sensors.get_pool_level_resistance()

Any informtion or help as to why it works one way and not the other would be greatly appreciated.

Answer 1

Well, interesting enough after doing more research and looking line by line through the code I realized that the only thing different I was doing in my test file was reading my file first before writing to it.

So I changed my code as follows:

Old Code:

def update_system_status_values(file, section, system, value):
    cfgfile = open(file, 'w')
    config.set(section, system, value)
    config.write(cfgfile)
    cfgfile.close()
    if DEBUG:
        print("{} updated to {}".format(system,value))

New Code (thanks @ShadowRanger)

def update_system_status_values(file, section, system, value):
    config.read(file)
    cfgfile = open('tempfile', 'w')
    config.set(section, system, value)
    config.write(cfgfile)
    cfgfile.close()
    os.rename('tempfile', file)
    if DEBUG:
        print("{} updated to {}".format(system, value))

and now it works like a charm!!

These are the steps now:

1) Read it

2) Open a temp file

3) Update it

4) Write temp file

5) Close temp file

6) Rename temp file over main file