Why do these 4 different random number generator functions produce the same series of numbers?

Question

A neuroevolution program I am in the process of debugging does not produce random values every time it is called. In the program, a vector of Network objects are initialized with the following statement:

vector<Network> population(POPULATION_SIZE, Network(sizes, inputCount));

Why I believe the program not to be converging to an optimal solution is that, always, the first 100 of the population are the same. When a network is initialized in this manner, the connection weights and neuron biases are (each) initialized with the following class function:

double Network::randDouble(double low, double high) {
    /*default_random_engine generator(std::chrono::system_clock::now().time_since_epoch().count());
    uniform_real_distribution<double> distribution(low, high);
    return distribution(generator);*/

    /*srand(time(NULL));
    double temp;
    if (low > high) {
        temp = low;
        low = high;
        high = temp;
    }
    temp = (rand() / (static_cast<double>(RAND_MAX) + 1.0)) * (high - low) + low;
    return temp;*/

    /*mt19937 rgn(std::chrono::system_clock::now().time_since_epoch().count());
    uniform_real_distribution<double> gen(low, high);
    return gen(rgn);*/

    default_random_engine rd;
    uniform_real_distribution<double> gen(low, high);
    auto val = std::bind(gen, rd);
    return val();
}

The 3 commented-out sections are previously attempted means of generating the functionality required. In each case, they produce the same numbers for each network (differing from 1 weight to another, but not 1 network to another). The methods attempted are based on answers from here:

1. c++-default_random_engine creates all the time same series of numbers
2. http://en.cppreference.com/w/cpp/numeric/random/uniform_real_distribution

In addition, the second method produces the same results with or without the seed. I must be missing something.

Another, albeit potentially irrelevant concern, is that functions using this may be parallel-ized using OpenMP, and that when called in parallel, the results could be the same.

Answer 1

Your problem is that you are initializing (seeding) the random generator every time you generate a number. In the simple srand() case, you should call srand() just once during program start, then call rand() every time you need one number. In the more complex cases, you should construct the generator just once (in the entire program run), and use it as many times as you need.

Answer 2

The C++11 standard random-number engines (and most other random generators) are in fact generators of pseudo -random sequences of numbers. Pseudo-random means that the sequences are repeatable. Every time a given pseudo-random generator is seeded with the same seed, it will always produce the same sequence. (But this is not exactly what is happening in your code. Read on.)

In C++11, the seeding happens at the time the random-number engine is instantiated. This means that you need to instantiate the engine once per pseudorandom sequence. The way your code seeds the engine in every call to the Network::randDouble() method, you cannot expect to get the pseudorandom sequence that the engine is designed to produce. Instead, you will get a series of the first numbers from sequences seeded by the call to the system_clock::... or the time() methods.

The call to the system_clock::now().time_since_epoch().count() returns time in integer number of periods . The period refers to the specialization of the template class std::chrono::duration which is returned by time_since_epoch(). The period may be seconds by default, which could explain why all your Network objects were getting the same seed in every call to Network::randDouble().

If you want a different sequence for each of the Networks, you should better instantiate the pseudorandom engine in the c-tor of the Network class, and seed it with a different seed for each object of the Network class. This means that the engine, or a pointer to the engine object should be member of the class.

Example:

class Network {

...
protected:
    mt19937 rd;
...
}

Network::Network(int rndseed) :
    rd(rndseed)
{
...
}

double Network::randDouble(double low, double high) {

    uniform_real_distribution<double> gen(low, high);
    auto val = gen(rd);
    return val;
}

To make sure that each instance of the pseudorandom engine is getting a different seed, you may use something as simple as consequent integer numbers. If you want to use the system clock, it is far more tricky to guarantee that the seeds are different every time, even if you use std::chrono::high_resolution_clock. CPUs are simply very fast and you need to take special care to make sure that the count of the clock that you are using has actually changed between two calls.