How to dynamically store address of a QVector<T> in another QVector<QVector<T>*>?

Question

I am working with large amounts of data, something like 100,000 double values, being gathered every 100 milliseconds or so. I have to store 25 or even more of these generated data in my software at any given time. Both, the data length on each acquisition as well as the number of acquisitions varies depending on the current situation. I do not want to store my data as QVector<QVector<double>> because QVector stores data in adjacent memory locations and each addition of QVector<double> to QVector<QVector<double>> results in the entire thing being copied to a new location given, resulting in huge lags (especially if there is already 20 QVector<double> present and I am adding the 21st one to it).

My solution is to store the given data as QVector<QVector<double>*> ie each data acquisition I am just storing the address to the QVector for the current acquisition. However, I want it to be stored in QVector<QVector<double>*> independently ie as long as I don't clear the pointer I am able to access the data. I am unable to figure out how to do this. I have tried multiple methods. Let's say I declare QVector<QVector<double>*> myVec; in my .h file and I use a function to new add data to it on each acquisition like this (below function is just for testing, that's why I am creating data inside the function):

void MainWindow::addData()
{
    myVec << &QVector<double>(0) // Creating a new empty vector and storing it's location
    *myVec[0] << 2.7 << 3.4 << 4.5;
}

This doesn't work, most of the time it throws an exception, and a few times it just shows some garbage value.

void MainWindow::addData()
{
    QVector<double> tempVec; // Create a temprory vector
    tempVec << 2.7 << 3.4 << 4.5;
    myVec << &tempVec;
}

This also doesn't work, of course, because tempVec is destroyed as soon as addData() is exited.

void MainWindow::addData()
{
    QVector<double> tempVec; // Create a temprory vector
    tempVec << 2.7 << 3.4 << 4.5;
    myVec << &QVector(tempVec);
}

I thought this would work as I am not storing the address of tempVec but rather copying tempVec in a new QVector and assigning its address to myVec , but it is also throwing an exception.

Is there any way I can store QVector purely as pointers inside another QVector ?

Answer 1

You're storing addresses to objects that are then destroyed. That means your vector of pointers contains dangling pointers:

void MainWindow::addData()
{
    myVec << &QVector<double>(0);
    // the temporary vector is destroyed here and myVec contains a
    // dangling pointer
    *myVec[0] << 2.7 << 3.4 << 4.5;
}

I'm surprised your compiler even accepts this code, since taking the address of a temporary should be a compilation error. Naming the temporary allows it to compile, but it's not any better, since you still get a dangling pointer:

void MainWindow::addData()
{
    QVector<double> tempVec(0);
    myVec << &tempVec;
    *myVec[0] << 2.7 << 3.4 << 4.5;
} // <-- tempVec is destroyed here, so you get a dangling pointer

If you store a pointer to an object, you need to make sure this object still exists whenever you access that pointer.

One way around this would be to allocate each vector with new before storing its address. But then you need to remember to delete it when you no longer need the pointers. Instead of that, you can use a smart pointer instead. std::unique_ptr in this case would do the job, but unfortunately QVector cannot store non-copyable elemens. So you'd need std::shared_ptr . This way, you don't need to store the pointers anywhere else but myVec :

#include <memory>

QVector<std::shared_ptr<QVector<double>>> myVec;

// ...

void MainWindow::addData()
{
    myVec << std::make_shared<QVector<double>>(0);
    *myVec[0].get() << 2.7 << 3.4 << 4.5;
}

If you don't absolutely require QVector and can do with std::vector instead, I'd recommend that, since it can store std::unique_ptr elements just fine:

#include <memory>
#include <vector>

std::vector<std::unique_ptr<QVector<double>>> myVec;

// ...

void MainWindow::addData()
{
    myVec.push_back(std::make_unique<QVector<double>>(0));
    *myVec[0].get() << 2.7 << 3.4 << 4.5;
}

If you really need to store raw pointers and can't use smart pointers, then I'm afraid you need to manage the pointed-to objects yourself by allocating them with new and then once you're done with myVec and no longer need it, clean up with delete :

QVector<QVector<double>*> myVec;

// ...

void MainWindow::addData()
{
    myVec << new QVector<double>(0);
    *myVec[0] << 2.7 << 3.4 << 4.5;
}

// Cleanup code you need to call when you no longer need myVec
for (auto ptr : myVec) {
    delete ptr;
}

Keep in mind that you also need to delete a pointer if you remove it from myVec or override it with a different pointer. This is tedious and error prone. When you forget to do it, you leak memory. So I recommend going the unique_ptr route instead, which will automatically delete the pointer for you.

Finally, and this might be preferable in your case, you can avoid pointers altogether and move the vector into myVec by casting it to an rvalue using std::move() . Not sure how well QVector works with move semantics, but std::vector does the job just fine, in which case you'd use std::vector for both "inner" and "outer" vectors. As an optimization, if you know the amount of doubles you need to store, you can pre-allocate with reserve() to avoid costly re-allocations when you add elements:

#include <memory>
#include <vector>

std::vector<std::vector<double>> myVec;

// ...

void MainWindow::addData()
{
    std::vector<double> tmpVec;
    tmpVec.reserve(amount_of_elements);
    
    tmpVec.push_back(2.4);
    tmpVec.push_back(3.7);
    // etc until filled

    myVec.push_back(std::move(tmpVec));
}

With moving, no allocations or copies are performed, if the objects involved support move semantics. As mentioned, std::vector does.

Very important: after an object has been "moved from" ( tmpVec in this case), it's in a valid but unspecified state (usually empty.)