How can I read a large set of data from a file into either a pointer to a structure or array of a structure in C

Question

I have a data file with a known key, that is, it has many entries (devices) with the same properties and I have this structure in code to capture it.

struct deviceData{
  int id;
  char serial[10];
  float temperature;
  float speed; 
  long timestamp; 
}

struct deviceData fileItems;

It's 4 bytes for the ID, 10 bytes for the serial code, 4 bytes for both the temperature and speed and 8 bytes for the timestamp. 30 bytes in total.

What I would like to achieve is to be able to read all those entries and run a calculation in the quickest way I can.

What I initially thought of doing was to simply create a giant array to capture all the entries but that causes errors.
Secondly I thought of allocating space from a pointer to that structure and reading the whole file to that. That worked in execution but I had trouble processing the data. Possibly a gap in fundamentals on my part.

The way I'm currently looking at is to loop through readings where I capture a single entry using fread(), process that and then move the file to put the next entry into the buffer. Something like this:

fread(&fileItems, 30, 1, filename)

What happens though is that when I view what actually gets read I see that the ID and the serial code were read correctly but the following data points are garbage. Reading a little bit about it I came across something about padding which I don't fully understand but the fix seems to be to make my char array 100 which seems to work for the first entry but I suspect it's causing problems with subsequent readings because it's throwing my calculations off.

I'm kind of at a wall here because every strategy I try seems to have something that works strangely. If I could at least be pointed in the right direction I'll at least know I'm putting effort in the right thing.

Answer 1

If you are just wanting to process a group of data record by record, you probably can utilize the methodology of defining a structure, then reading the data from a file into the structure, and then processing the data. To make things consistent, it would make sense to store the data as a structure in a binary file. Following is a code snippet creating some sample data and then processing it following the spirit of your project.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct deviceData{
  int id;
  char serial[10];
  float temperature;
  float speed;
  long timestamp;
};

struct deviceData fileItems;

void create_data(char * file_name)
{
    FILE* fp = fopen(file_name, "wb");
    if(!fp)
    {
        printf("\n\tFile open error\n");
        return;
    }

    for (int i = 0; i < 10; i++)
    {
        fileItems.id = i + 20000 + i * 3;
        sprintf(fileItems.serial, "SN%d", fileItems.id);
        fileItems.temperature = 166.0 + i;
        fileItems.speed = 2400.0;
        fileItems.timestamp = 20220830;
        fwrite(&fileItems, sizeof(struct deviceData), 1, fp);
    }

    fclose(fp);
}

void read_data(char *file_name)
{
    FILE* fp = fopen(file_name, "rb");

    if(!fp)
    {
        printf("\n\tFile open error\n");
        return;
    }

    while(fread(&fileItems, sizeof(struct deviceData), 1, fp))
    {
        printf("ID. . .  . . . .: %d\n", fileItems.id);
        printf("Serial number. .: %s\n", fileItems.serial);
        printf("Temparature. . .: %f\n", fileItems.temperature);
        printf("Speed. . . . . .: %f\n", fileItems.speed);
        printf("Timestamp. . . .: %ld\n", fileItems.timestamp);
    }
}

int main()
{
    create_data("device.dat");      /* Create some sample data to be read      */
    read_data("device.dat");        /* Read and print the data to the terminal */
    return 0;
}

The storage of device data probably would occur in some other monitor program, but for this snippet a one-off function was included to produce some data to process.

Analyze that code and see if it meets the spirit of your project.