Resizing struct / char array (to reduce memory usage)

Question

This is my first project on Arduino/C++/ESP32. I wrote a fairly big program and got almost everything working - except that in the end I realized that the device would run out of breath (memory) periodically and go for a reboot. The reboot is because I configured a watchdog to do so.

There is one area where I think there's a chance to reduce the memory usage but my experience on c++ is "not there yet" for me to be able to write this by myself. Any pointers (no pun intended) please? I have been on this since yesterday and getting rid of one error only results in another new error popping up. Moreover I don't want to come up with something that is hacky or might break later. It should be a quick answer for the experienced people here.

Let me explain the code that I prefer to refactor/optimize.

I need to store a bunch of records that I would need to read/manipulate later. I declared a struct ( because they are related fields ) globally. Now the issue is that I may need to store 1 record, 2 records or 5 records which I would only know later once I read the data from the EEPROM. And this has to be accessible to all the functions so it has to be a global declaration.

To summarize

Question 1 - how to set " NumOfrecs " later in the program once the data is read from the eeprom.

Question 2 - The size( sizeOfUsername ) of the char array username can also change depending upon the length of the username read from the eeprom. At times it might be 5 characters long, at times it could be 25. I can set it to a max 25 and solve this problem but then wouldn't I be wasting memory if many usernames were just 4-5 characters long? So in short - just before copying over the data in eeprom into the "username" char array, is it possible to set it's size to the optimal size required for holding that data ( which is the data size + 1 byte for null termination ).

struct stUSRREC {
  char username[sizeOfUsername];
  bool online;
}; 

stUSRREC userRecords[NumOfrecs];

I familiarized myself with a whole bunch of functions like strcpy, memset, malloc etc but now I have run out of time and need to keep the learning part for another day.

I can try to do this in a slightly different manner where I don't use the struct and instead use individual char arrays ( for each field like username ). But then again I'll have to resize the arrays as I read the data from the eeprom.

I can explain all the things I have tried but that will make this question unnecessarily long and perhaps result in losing some clarity. Greatly appreciate any help.

While responding to Q&A on SO I was trying some random stuff and at least this little piece of code below seems to work ( in terms of storing smaller/bigger values )

struct stUSRREC {
  char username[];        
  bool online;                   
}; 

stUSRREC userRecords[5];

Then manipulate it this way

strcpy(userRecords[0].username, "MYUSERNAME");
strcpy(userRecords[0].username, "test");
strcpy(userRecords[0].username, "MYVERYBIGUSERNAME");

I have been able to write/rewrite different lengths (above) and can read all of them back correctly. Resizing "userRecords" might be a different game but that can wait a little

One thing I forgot to mention was that I will need to size/resize the array ( holding username ) ONLY ONCE. In the setup() itself I can read/load the required data into those arrays. I am not sure if that opens up any other possibility. The rest of the struct/array I need to manipulate during the running are only boolean and int values. This is not an issue at all because there is no resizing required to do so.

On a side note I am pretty sure I am not the only one who faced this situation. Any tips/clues/pointers could be of help to many others. The constraints on little devices like ESP32 become more visible when you really start loading them with a bunch of things. I had it all working with "Strings" (the capital S) but the periodic reboot (cpu starvation?) required me to get rid of the Strings. Even otherwise I hear that using Strings (on ESP, Arduino and gang) is a bad idea.

Answer 1

You tagged this question as C++, so I'll ask:

Can you use vector and string in your embedded code?

#include <string>
#include <vector>

struct stUSRREC {
  std::string username;
  bool online;
  stUSRREC(const char* name, bool isOnline) :
       username(name), 
       online(isOnline)
  {
  }
};

std::vector<stUSRREC> userRecords;

The use of string as the username type means you only allocate as many characters needed to hold the name instead of allocated an assumed max size of sizeOfUsername . The use of vector allows you to dynamically grow your record set.

Then to add a new record:

stUSRREC record("bob", true);
userRecords.push_back(record);

And you may not need NumOfrecs anymore. That's covered by userRecrods.size()