How to copy fields from one table to another

Question

I have searched and tried everything but have failed and was wondering if someone could help with this slightly unusual twist to copying data from one table to another.

I am collecting data from my own weather station with a piece of software written by someone else in C. I have had a look at this open source code but it is very difficult to read as it uses a lot of single character variables etc. It writes the data to a MySQL database table in a sequential format like this:-

CREATE TABLE readings (
    `date` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    `name` TEXT NOT NULL,
    `value` DOUBLE NOT NULL,
    `id` INT(11) NOT NULL AUTO_INCREMENT
)

Where each sensor reading is stored on a seperate row. Name denotes a sensor name such as a temperature sensor or raingauge, windspeed sensor etc, etc. All the sensors are updated every minute on the minute. The problem I am finding is that this way of storing the data makes it extremely difficult to perform queries on it as I have to include a lot of conditionals in the query, such as checking that the 'name' field is "RGC1" for the raingauge counter etc. This really complicates things as the raingauge sensor is actually just a counter and not an instant rainfall rate. I have to perform a query on the database and subtract the previous raingauge counter value from the current one to get the number of counts between the two occasions that these were read. Each count is 0.01 inches of rainfall, so multiplying the count difference by 0.01 gives the rainfall rate as inches/interval. Temperature sensors are a little easier as they are instant temperature values but it is still complicated making comparances between different occasions.

I either need a way of calculating this data leaving the database structure as it is (which I have failed to do dismally) or to copy the new data every minute as it comes in to another database where all the sensor readings are stored on the same row, to something like this:-

CREATE TABLE `readings2` (
    `id` BIGINT(20) NOT NULL AUTO_INCREMENT,
    `readdate` DATETIME NULL DEFAULT NULL,
    `MS-TH-TEMP` FLOAT NULL DEFAULT NULL,
    `MS-TH-HUM` FLOAT UNSIGNED NULL DEFAULT NULL,
    `UTEMP` FLOAT NULL DEFAULT NULL,
    `UV` FLOAT UNSIGNED NULL DEFAULT NULL,
    `WSPD` SMALLINT(5) UNSIGNED NULL DEFAULT '0',
    `WDIR` TINYINT(3) UNSIGNED NULL DEFAULT '0',
    `WS-TEMP` FLOAT NULL DEFAULT '0',
    `RGC0` BIGINT(20) UNSIGNED NULL DEFAULT '0',
    `RGC1` BIGINT(20) UNSIGNED NULL DEFAULT '0',
    `LC0` BIGINT(20) UNSIGNED NULL DEFAULT '0',
    `LC1` BIGINT(20) UNSIGNED NULL DEFAULT '0',
    `PRS0` FLOAT UNSIGNED NULL DEFAULT '0',
    `TAI8570-TEMP` FLOAT UNSIGNED NULL DEFAULT '0',
    PRIMARY KEY (`id`)
)

I am running all of this on Ubuntu Server 11.04 and copying the data could be done on a cron job every minute with a ruby script or something similar. Can anyone help? I have searched and tried a lot of suggestions from this site but have failed in my efforts to come up with a solution.

Further to my original post above, I have been doing a lot of messing around but am getting persistent syntax errors with the following code but I am damned if I can see anything wrong. If I comment out the INSERT INTO statement, the syntax error goes away. I can't see anything wrong with the syntax in the INSERT, though:-

DELIMITER $$

DROP TRIGGER /*!50033 IF EXISTS */ `copyFields`$$

CREATE TRIGGER `copyFields` AFTER UPDATE on `readings`
FOR EACH ROW
BEGIN
    DECLARE mychanged INT DEFAULT 0;
    DECLARE newDATE DATETIME DEFAULT NULL;
    DECLARE newMSTHTEMP FLOAT DEFAULT NULL;
    DECLARE newMSTHHUM FLOAT DEFAULT NULL;
    DECLARE newUTEMP FLOAT DEFAULT NULL;
    DECLARE newUV FLOAT DEFAULT NULL;
    DECLARE newWSPD SMALLINT DEFAULT NULL;
    DECLARE newWDIR TINYINT DEFAULT NULL;
    DECLARE newWSTEMP FLOAT DEFAULT NULL;
    DECLARE newRGC0 BIGINT DEFAULT NULL;
    DECLARE newRGC1 BIGINT DEFAULT NULL;
    DECLARE newLC0 BIGINT DEFAULT NULL;
    DECLARE newLC1 BIGINT DEFAULT NULL;
    DECLARE newPRS0 FLOAT DEFAULT NULL;
    DECLARE newTAI8570TEMP FLOAT DEFAULT NULL;
    IF NEW.name = "UTEMP" THEN
        SET mychanged = 1;
    END IF;
    IF mychanged = 1 THEN
        SET newDATE = NEW.date;
    IF NEW.name = 'MS-TH-TEMP' THEN
        SET newMSTHTEMP = NEW.value;
    END IF;
    IF NEW.name = 'MS-TH-HUM' THEN
        SET newMSTHHUM = NEW.value;
    END IF;
    IF NEW.name = 'WSPD' THEN
        SET newWSPD = NEW.value;
    END IF;
    IF NEW.name = 'WDIR' THEN
        SET newWDIR = NEW.value;
    END IF;
    IF NEW.name = 'WS-TEMP' THEN
        SET newWSTEMP = NEW.value;
    END IF;
    IF NEW.name = 'RGC0' THEN
        SET newRGC0 = NEW.value;
    END IF;
    IF NEW.name = 'RGC1' THEN
        SET newRGC1 = NEW.value;
    END IF;
    IF NEW.name = 'UTEMP' THEN
        SET newUTEMP = NEW.value;
    END IF;
    IF NEW.name = 'UV' THEN
        SET newUV = NEW.value;
    END IF;
    INSERT INTO readings2 
        (readdate,
        ms-th-temp,
        ms-th-hum,
        utemp,
        uv,
        wspd,
        wdir,
        ws-temp,
        rgc0,
        rgc1,
        lc0,
        lc1,
        prs0,
        tai8570-temp)
        VALUES (newDATE,
        newMSTHTEMP,
        newMSTHHUM,
        newUTEMP,
        newUV,
        newWSPD,
        newWDIR,
        newWSTEMP,
        newRGC0,
        newRGC1,
        newLC0,
        newLC1,
        newPRS0,
        newTAI8570TEMP);
END IF; 

END$$

DELIMITER;

The error I am getting above is:-

SQL Error (1064): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-th-temp, ms-th-hum, utemp, uv, wspd, wdir, ws-temp, ' at line 53

It looks more complicated than it really is but if anyone could help point me in the right direction with this, I would be grateful.

Thanks in advance.

Answer 1

I'd change your table definition a tiny bit, so that all fields are null (as in unknown) until they're explicitly filled in. Other than that, use an insert trigger and an update trigger, eg:

create trigger readings_insert
  after insert on readings
  for each row
begin
  -- upsert into readings2
end;

http://dev.mysql.com/doc/refman/5.0/en/create-trigger.html