Numpy随机选择多个循环

Question

I am trying to perform multiple simulations many times to get a desired simulation distribution. I have a dataset that looks like the one below.

fruit_type, reading, prob
Apple, 12,.05
apple, 15, .5
orange 18, .99

An example of my code is below.

def sim(seconds):
    output = pd.DataFrame()
    current = []
    #output = pd.DataFrame()
    for i in range(1, 100000000):
        if data2['fruit_type'].all() == 'Apple':
            hostrecord1 = np.random.choice(data2['reading'], size=23, replace=True, p=data2['prob'])
            current = hostrecord1.sum() + 150

        if data2['fruit_type'].all() == 'Orange':
            hostrecord2 = np.random.choice(data2['reading'], size=23, replace=True, p=data2['prob'])
            current = hostrecord2.sum() + 150

        if data2['fruit_type'].all() == 'Peach':
            hostrecord3 = np.random.choice(data2['reading'], size=20, replace=True, p=data2['prob'])
            current = hostrecord3.sum() + 150

    #put all records in one array
    #return all records 
    output = pd.concat(current)
    return output

I am trying to figure out how to perform multiple simulations with different conditions varying by fruit_type , but currently can't figure out the logic. Each simulation should select specific rows in relation to the fruit_type so the simulations are specified by fruit_type so part of it. The size of each sample is different by design as each fruit_type has different conditions.

My expected output is an array of all the simulation values. I also want to append all the results into one pandas dataframe.

Answer 1

Your explanation is pretty unclear, but here's a guess:

# initialize data
In [1]: fruits = ['apple', 'peach', 'orange']
In [2]: data = np.vstack((np.random.choice(fruits, size=10), 
                          np.random.randint(0, 100, size=10), 
                          np.random.rand(10))).T
In [3]: df = pd.DataFrame(data, columns=['fruit_type', 'reading', 'prob'])

The key is indexing df such that df[df.fruit_type == fruit_of_interest] . Here is a sample function:

def simulate(df, N_trials):
    # replace with actual sizes for ['apple', 'peach', 'orange'] respectively
    sample_sizes = [N1, N2, N3]
    fruits = ['apple', 'peach', 'orange']

    results = np.empty((N_trials, len(fruits))
    for i in xrange(N_trials): # switch to range if using python3
        for j, (fruit, size) in enumerate(zip(fruits, sample_sizes)):
            sim_data = df[df.fruit_type == fruit]
            record = np.random.choice(sim_data.reading, size=size, p=sim_data.prob)
            # do something with the record
            results[i, j] = record.sum()

Note that the results array may be too big to fit in memory if you're doing 100 million trials. It may also be faster if you swap the for loops so the fruit/size one is the outermost for loop.

---

It's also worth noting that instead of for -looping, you could always generate a huge sample with np.random.choice and then reshape:

np.random.choice([0, 1], size=1000000).reshape(10000, 100)

would give you 10000 trials with 100 samples each. This could be useful if your 100 million trials is taking too long -- you could split that into 100 loops with choice doing 1 million samples at once. An example could be

def simulate(df, N_trials, chunk_size=10000):
    # replace with actual sizes for ['apple', 'peach', 'orange'] respectively
    sample_sizes = [N1, N2, N3]
    fruits = ['apple', 'peach', 'orange']

    for i in xrange(N_trials/chunk_size): # switch to range if using python3
        chunk_results = np.empty((chunk_size, len(fruits))
        for j, (fruit, size) in enumerate(zip(fruits, sample_sizes)):
            sim_data = df[df.fruit_type == fruit]
            record = np.random.choice(sim_data.reading, size=(chunk_size, size), 
                                      p=sim_data.prob)
            chunk_results[:, j] = record.sum(axis=1)

        # do something intermediate with this chunk