Declaring two large 2d arrays gives segmentation fault

Question

i'm trying to declare and allocate memory for two 2d-arrays. However when trying to assign values to itemFeatureQ[39][16816] I get a segmentation vault. I can't understand it since I have 2GB of RAM and only using 19MB on the heap. Here is the code;

double** reserveMemory(int rows, int columns)
{
    double **array;
    int i;
    array = (double**) malloc(rows * sizeof(double *));
    if(array == NULL)
    {
        fprintf(stderr, "out of memory\n");
        return NULL;
    }
    for(i = 0; i < rows; i++)
    {
        array[i] = (double*) malloc(columns * sizeof(double *));
        if(array == NULL)
        {
            fprintf(stderr, "out of memory\n");
            return NULL;
        }
    }

    return array;

}

void populateUserFeatureP(double **userFeatureP)
{
    int x,y;

    for(x = 0; x < CUSTOMERS; x++)
    {
        for(y = 0; y < FEATURES; y++)
        {
            userFeatureP[x][y] = 0;
        }
    }
}

void populateItemFeatureQ(double **itemFeatureQ)
{
    int x,y;

    for(x = 0; x < FEATURES; x++)
    {
        for(y = 0; y < MOVIES; y++)
        {
            printf("(%d,%d)\n", x, y);
            itemFeatureQ[x][y] = 0;
        }
    }
}

int main(int argc, char *argv[]){

    double **userFeatureP = reserveMemory(480189, 40);
    double **itemFeatureQ = reserveMemory(40, 17770);

    populateItemFeatureQ(itemFeatureQ);
    populateUserFeatureP(userFeatureP);

    return 0;
}

Answer 1

You have a couple of typos - change:

    array[i] = (double*) malloc(columns * sizeof(double *));
    if(array == NULL)

to:

    array[i] = (double*) malloc(columns * sizeof(double));
    if(array[i] == NULL)

Answer 2

A better way to deal with memory allocation for 2D stuff if it's large is to do something like this:

// small fix to this, added pointer de-reference
// if you iterate over x in your outer loop you should 
// change the index part to (a)->h*(x) + (y)
// but only do one or the other
#define IDX_DBL2D(a, x, y) (*((a)->d[(a)->w*(y) + (x)]))

struct dbl2d {int w,h; double[] d; };

struct dbl2d *alloc_2d_dbl(int w, int h) {
    struct dbl2d *r = malloc(sizeof(struct dbl2d) + sizeof(double)*w*h);
    r->w = w, r->h = h;
    return r;
}

this is kind of like what the compiler is doing when you declare a 2d array (but it would have the indices the other way 'round though)

This will use less memory (don't need to store all those row pointers) and keep everything together

When the compiler creates a fixed size 2d array it knows how big it is each way so if you write bar = foo[x][y]; and foo is nxm then the compiler turns that into bar = *(foo + x*m + y) however this only works for stuff where the compiler knows what shape your array is, so what I defined above is basically an array that carries it's dimensions around with it.

The index calculation works since if you think of foo as an array of arrays each row has size sizeof(*foo)*width (depending on weather you make x or y the first co-ordinate, here I use x since I'm used to images and that's the convention used there) so if you multiply your y co-ordinate by the number of elements in a row you skip over 'y' rows then you can get the element within the row by adding x.

Usage example:

void populateUserFeatureP(struct dbl2d *userFeatureP)
{
    int x,y;

    for(x = 0; x < CUSTOMERS; x++)
    {
        for(y = 0; y < FEATURES; y++)
        {
            IDX_DBL2D(userFeatureP, x, y) = 0;
        }
    }
}

void populateItemFeatureQ(struct dbl2d *itemFeatureQ)
{
    int x,y;

    for(x = 0; x < FEATURES; x++)
    {
        for(y = 0; y < MOVIES; y++)
        {
            printf("(%d,%d)\n", x, y);
            IDX_DBL2D(itemFeatureQ, x, y) = 0;
        }
    }
}

int main(int argc, char *argv[]){

    struct dbl2d *userFeatureP = alloc_2d_dbl(480189, 40);
    struct dbl2d *itemFeatureQ = alloc_2d_dbl(40, 17770);

    populateItemFeatureQ(itemFeatureQ);
    populateUserFeatureP(userFeatureP);

    return 0;
}

Answer 3

When allocating 2d arrays, a nice trick is that you can only use 2 mallocs.

double** reserveMemory(int rows, int columns) {
    double **array, *data;
    int i;
    array = (double**) malloc(rows * sizeof(double *));
    data = (double*) malloc(rows * columns * sizeof(double));
    for(i = 0; i < rows; i++){
       array[i] = data + i * columns;
    }    
    return array;    
}

This can also be used to treat the 2d array as 1d array